{"name":"Uplink newsletter archive","description":"All articles from Uplink, the newsletter of the National Institute for Computer-Assisted Reporting (NICAR), October 1990 - March 2007.","publisher":"National Institute for Computer-Assisted Reporting (NICAR)","source":"https://github.com/dwillis/uplink","documentation":"https://dwillis.github.io/uplink/data.html","generated_at":"2026-07-12T03:49:39.644Z","count":1838,"articles":[{"id":"1990-october-ghostbusting-in-east-st-louis","slug":"1990-october-ghostbusting-in-east-st-louis","url":"https://dwillis.github.io/uplink/article/1990-october-ghostbusting-in-east-st-louis.html","year":1990,"month":"October","monthNum":10,"headline":"GHOSTBUSTING IN EAST ST. LOUIS","kicker":"","author_name":"George Landau","author_title":"Computer-assisted reporting specialist","affiliation":"St. Louis Post-Dispatch","summary":"George Landau and Tim Novak investigated voter fraud in East St. Louis using computer-assisted methods, uncovering hundreds of names of deceased or fictitious voters registered to vote.","keywords":["voter fraud","East St. Louis","computer-assisted reporting"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["PC","magnetic tape"],"full_text":"A man named Admiral Wherry, an army veteran who owned a barbecue pit and tire repair shop in East St. Louis, died more than two years ago. But that didn't stop him from voting in the Illinois Democratic primary on March 20. That's how reporter Tim Novak and I began our Page One account of voter fraud and inflated voter roles in East St. Louis, III., an impoverished city straight across the Mississippi river from downtown St. Louis. In less than a month, using an IBM-compatible PC, two sets of computer data and printed listings of regis- tered voters, we found dozens of ballots cast in the names Head people. And by sampling 10 percent of the city's precincts, we found more than 100 people registered to vote from vacant lots, 16 of whom had voted in the March primary. Reporters, politicians and state election officials have heard allegations of voter fraud in East St. Louis for at least a decade, but no one had produced hard evidence. Novak and I decided to give it a try after finishing our computer-assisted analysis of property ownership in East St. Louis. For that project, we acquired from the county assessor a magnetic tape listing the owners and address ees of every parcel in the city. Of 27,000 lots, nearly 5,000 were vacant. \"Of those 1,000 dead people, 270 were still registered to vote. Some had been dead for 11 years.\" One of our colleagues had suggested that we use the computer to match the list of vacant lots against the addresses of registered voters. A great idea; the only problem was that the state and local election boards were prohibited by law from releasing the list of voters in machine-readable form. Our lawyer said our prospects of getting the data on magnetic tape were slim, so we got printouts listing the city's 30,000 registered voters by precinct (there are 50) and alphabetically for the entire city. We then decided to enter only the addresses of voters from five small precincts into the PC. This took two days. We then edited the list of street names so that abbreviations (such as \"ave\" and \"st\") were in the same style as the assessor's data. We matched the two sets of data and came up with a list of several hundred properties valued at less than $200 where voters were registered. We then visited each of the properties to make sure they were vacant or abandoned. This property review only took two days because we organized the list by street name and were able to check several properties at the same time. Reasoning that many East St. Louis residents might die in the large medical centers on this side of the river, we searched for their death records among the 550,000 Missouri death records spanning 11 years. At the same time, I received on magnetic tape a set of all Missouri Death Certificates from 1979-89. (Our efforts to obtain Illinois Death Certificates have not yet succeeded). We came up with more than 1,000 people's names, addresses, social security numbers and dates of birth and death. We alphabetized this list and compared it by hand with the citywide alphabetical listing of voters. This sounds onerous but took only one afternoon. We were surprised to learn that the assessor's rec- ords were often inaccurate, to the point that we found well-kept 20-year-old homes on lots listed as vacant. This, of course, became another story. Of those 1,000 dead people, 270 were still regis- tered to vote. Some had been dead for 11 years. And some, we discovered later after looking at their reg- istration cards, hadn't begun to vote until after they died. We'r still following up the story, which imme- diately prompted investigations by the U.S. Justice Department, the Illinois Attorney General and the state election board. Questions may be directed to George Landau at the City Desk, St. Louis Post-Dispatch, 900 North Tucker, St. Louis, MO 63101. Or call him at (314) 622-7205.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-october-plugging-in-the-hartford-connection","slug":"1990-october-plugging-in-the-hartford-connection","url":"https://dwillis.github.io/uplink/article/1990-october-plugging-in-the-hartford-connection.html","year":1990,"month":"October","monthNum":10,"headline":"PLUGGING IN: THE HARTFORD CONNECTION","kicker":"","author_name":"Brant Houston and Kenton Robinson","author_title":"Reporter and Editor","affiliation":"The Hartford Courant","summary":"The article provides eight practical tips for journalists introducing computer-assisted reporting to newsrooms, including learning technical language, understanding hardware/software needs, educating editors and colleagues, securing funding, and working collaboratively.","keywords":["reporting","computers","journalists"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["mainframe"],"full_text":"1) Learn the language. \"The people who give you advice on computers speak in a foreign tongue Learn enough of the lingo to know when data processing people (inside and outside the newsroom) are trying to snow you. Getting a computer-assisted reporting pro- gram started in a traditional newsroom is typi- cally the first and toughest step for the trained reporter. Brant Houston is a reporter for The Hartford Courant. (Conn.) who. along with Kenton Robinson, has braved the challenge of introducing computer-assisted reporting to new territory. The above are some of their experiences and survival tips. If you have any of your own words of wisdom, please send them to the Missouri In- stitute for Computer-Assisted Reporting, and we TIpass them along. 2) Know your hardware and software require- ments. \"Expert computer people, usually from land of the mainframe, want to give you a sledgehammer to hit a nail Those people will do a-special program for you when a store-bought piece of software will do.\" 3) Educate your editors. \"Once you score your first major story, editors will want a similar story every day and not understand why you can't hit a few buttons and run out a blockbuster every 24 hours. You need to work on convincing editors that many stories take time, and end up'being much better in content, much wider in scope and have greater impact.\" 4) Educate other reporters. \"Reporters will want to stick to their tried and true ways, talking to people for anecdotes and slowly sifting through thousands of pages of documents. They also have computer or keyboard phobia. Get them to sit still for 15 minutes and show them time-saving tricks.\" 5) Know how to fight for funds. \"Once you have bought your equipment, no one may see any reason that you need more. Some people won't understand that the computer business is developing rap- idly. Become a superior memo writer and use graphics and documentation to back up the memos. Also, keep a list of every major and minor success.\" 6) Learn how to get government data. \"With constant self-education, you can learn to discern the amount of manure you are being served by government owners of computer data. If they say they can't technically give you something, offer to get your own data processing people to help them doi it. Also, scoff at their first purchase price figure (unless it's zero).\" Do even the boring work yourself. You will spend time on janitorial work such as saving and backing up files of information, cleaning your computer files and installing new software. You probably will want to get someone else to do this. This is a bad idea because you lose track of your own information and how to move it around and use it.\" 8) Work with a partner. \"If you are a reporter your production will suffer and thus your stature with production minded managers. If possible pursue this with another editor or reporter be- cause the buddy system works well. At The Courant, my editor Kenton Robinson and I work together, teaching each other, with me concentrating on data tapesand diskettes, and with him nailing down getting information through the phone lines.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-november-ohios-toxic-avenger","slug":"1990-november-ohios-toxic-avenger","url":"https://dwillis.github.io/uplink/article/1990-november-ohios-toxic-avenger.html","year":1990,"month":"November","monthNum":11,"headline":"Ohio's Toxic Avenger","kicker":"","author_name":"Dave Davis","author_title":"","affiliation":"","summary":"A journalist discovers how the Toxic Release Inventory (TRI) reveals hidden industrial pollution like benzene emissions from an Ohio steel plant that residents never knew about, and how this data can change the relationship between polluters and the press.","keywords":["pollution","benzene","inventory"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["PC"],"full_text":"Fifty-three year old Clyde King remembered with a laugh the time he came home and found his next-door neighbor mowing the lawn in a protective moonsuit.\n\nAs he watched from his backyard, King couldn't help but wonder if something had happened on the other side of the chain-link fence that he should know about. But when he asked his neighbor the Armco Steel works in Middletown, Ohio - the reason for the rubber suit and respirator, company officials simply shrugged and pretended not to know what he was talking about.\n\nClyde King didn't press that matter. Some of his relations worked at Armco and he had already learned how many questions were enough.\n\nWhen I asked King if the plant's releases were riting his family's health, he smiled and said yes, the llution was probably killing them. When I asked what was killing them, King pointed to a thin stream of gray smoke coming from a tall stack in the distance.\n\nKing was wrong.\n\nThough the black soot collected everywhere and made life unpleasant, the true source of concern was much closer - inside two tar-storage tanks about 20 yards from King's home.\n\nNo one told King that the tanks released benzene, a gas he could not see or smell. No one told him that the benzene was the reason for the moonsuit, or that it caused cancer such as leukemia.\n\nThe U.S. Environmental Protection Agency had estimated that Armco's release of one million pounds of benzene a year would cause two cancer deaths a year in King's neighborhood, which included an elementary school at the end of the block.\n\nClyde King did not know these things and he was not alone. Though generations of families have grown up next to the sprawling steel mills, paper plants and auto makers of Ohio, no one knew what toxics come from their industrial neighbors.\n\nThe Toxic Release Inventory is changing this. TRI is a pollution inventory that contains estimated releases of more than 300 toxic chemicals by most companies. The information, required by the EPA since 1987, is presented by release for each company. The numbers include the total amount of toxic in pounds that escaped into the air, was discharged into our waterways or was buried in the ground for the reporting year.\n\nWith TRI, Armco Steel's benzene release figure was calculated in about an hour or two.\n\nTRI is not perfect: the numbers are self-reported and I've often wondered how accurate they are. But TRI is a pretty good way to look at pollution in an area.\n\nWhen I used it as the basis for a series on pollution in Ohio, I found it could help change the long-standing relationship between polluters and the press. Typically, companies had been tight-lipped about what came out of their plants. But when they found out we were going to publish the numbers they had given the federal government, they treated us to plant tours, interviews with company engineers and press conferences where pledges to lower emissions were made.\n\nUsing a PC, we were able to track down previously undetectable emissions, adding up the total releases for each county in the state. Then we looked at each city in the state, and each zip code in our area. We put together a pollution map for our area, which looked like a multi-color weather map.\n\nThe \"Every Breath You Take\" series showed that some of the area's neighborhoods were among the dirtiest in the nation. Millions of pounds of toxic chemicals had been released and many of the most polluted neighborhoods were inhabited by workers like King and their families who have accepted pollution as necessary to their paychecks. Also, some companies have avoided reporting their releases by playing a complicated toxics shell game that hides their waste from public view.\n\nThe series from start to finish took a couple of months. The most time consuming process was cleaning up the data. For example, we wanted to break down the information by zip code, but the codes were often wrong. Sometimes, the reported zip code referred to the corporate headquarters rather than the manufacturing plants.\n\nSince coming to Cleveland, I've found TRI helpful with daily stories, too. Daily stories usually take just a couple of days and they don't require a lot of PC sophistication. For example, a local unpopular polluter, a plastics company that proposed to be allowed to triple its size, fudged its emissions numbers on the environmental impact study presented to a local town council. A quick check of TRI showed that the amount of toxics released by the firm already was more than double what was predicted in the impact study for the new, larger company.\n\nThe lesson Clyde King taught me was a valuable one: that when it comes to toxic releases, what you see often isn't what you're getting.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1990-november-when-the-smoke-cleared-it-was-not-a-pretty-sight","slug":"1990-november-when-the-smoke-cleared-it-was-not-a-pretty-sight","url":"https://dwillis.github.io/uplink/article/1990-november-when-the-smoke-cleared-it-was-not-a-pretty-sight.html","year":1990,"month":"November","monthNum":11,"headline":"When the Smoke Cleared, It Was Not a Pretty Sight","kicker":"Ohio's Toxic Avenger","author_name":"Dave Davis","author_title":"","affiliation":"The Plain Dealer","summary":"The Toxic Release Inventory reveals hidden dangers like benzene emissions from industrial plants near residential areas, forcing companies to become more transparent about their pollution.","keywords":["pollution","transparency","benzene"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["PC"],"full_text":"Fifty-three year old Clyde King remembered with a laugh the time he came home and found his next-door neighbor mowing the lawn in a protective moonsuit.\n\nAs he watched from his backyard, King couldn't help but wonder if something had happened on the other side of the chain-link fence that he should know about. But when he asked his neighbor the Armco Steel works in Middletown, Ohio - the reason for the rubber suit and respirator, company officials simply shrugged and pretended not to know what he was talking about.\n\nClyde King didn't press that matter. Some of his relations worked at Armco and he had already learned how many questions were enough.\n\nWhen I asked King if the plant's releases were ruining his family's health, he smiled and said yes, the pollution was probably killing them. When I asked what was killing them, King pointed to a thin stream of gray smoke coming from a tall stack in the distance.\n\nKing was wrong.\n\nThough the black soot collected everywhere and made life unpleasant, the true source of concern was much closer - inside two tar-storage tanks about 20 yards from King's home.\n\nNo one told King that the tanks released benzene, a gas he could not see or smell. No one told him that the benzene was the reason for the moonsuit, or that it caused cancer such as leukemia.\n\nThe U.S. Environmental Protection Agency had estimated that Armco's release of one million pounds of benzene a year would cause two cancer deaths a year in King's neighborhood, which included an elementary school at the end of the block.\n\nClyde King did not know these things and he was not alone. Though generations of families have grown up next to the sprawling steel mills, paper plants and auto makers of Ohio, no one knew what toxics come from their industrial neighbors.\n\nThe Toxic Release Inventory is changing this. TRI is a pollution inventory that contains estimated releases of more than 300 toxic chemicals by most companies. The information, required by the EPA since 1987, is presented by release for each company. The numbers include the total amount of toxic in pounds that escaped into the air, was discharged into our waterways or was buried in the ground for the reporting year.\n\nWith TRI, Armco Steel's benzene release figure was calculated in about an hour or two.\n\nTRI is not perfect: the numbers are self-reported and I've often wondered how accurate they are. But TRI is a pretty good way to look at pollution in an area.\n\nWhen I used it as the basis for a series on pollution in Ohio, I found it could help change the long-standing relationship between polluters and the press. Typically, companies had been tight-lipped about what came out of their plants. But when they found out we were going to publish the numbers they had given the federal government, they treated us to plant tours, interviews with company engineers and press conferences where pledges to lower emissions were made.\n\nThe lesson Clyde King taught me was a valuable one: that when it comes to toxic releases, what you see often isn't what you're getting.\n\nUsing a PC, we were able to track down previously undetectable emissions, adding up the total releases for each county in the state. Then we looked at each city in the state, and each zip code in our area. We put together a pollution map for our area, which looked like a multicolor weather map.\n\nThe \"Every Breath You Take\" series showed that some of the area's neighborhoods were among the dirtiest in the nation. Millions of pounds of toxic chemicals had been released and many of the most polluted neighborhoods were inhabited by workers like King and their families who have accepted pollution as necessary to their paychecks. Also, some companies have avoided reporting their releases by playing a complicated toxics shell game that hides their waste from public view.\n\nThe series from start to finish took a couple of months. The most time consuming process was cleaning up the data. For example, we wanted to break down the information by zip code, but the codes were often wrong. Sometimes, the reported zip code referred to the corporate headquarters rather than the manufacturing plants.\n\nSince coming to Cleveland, I've found TRI helpful with daily stories, too. Daily stories usually take just a couple of days and they don't require a lot of PC sophistication. For example, a local unpopular polluter, a plastics company that proposed to be allowed to triple its size, fudged its emissions numbers on the environmental impact study presented to a local town council. A quick check of TRI showed that the amount of toxics released by the firm already was more than double what was predicted in the impact study for the new, larger company.\n\nTRI data is available on floppy disk for 1987 from the National Technical Information Service at (703) 487-4650 for $75 per state, or $1100 for the entire country. (It's available for 1987 and 1988 for $726 from the Missouri Institute for Computer-Assisted Reporting at (314) 882-0684.)\n\nHappy hunting and if I can help anyone just ring me at The Plain Dealer, 216-344-4808.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1990-november-if-you-want-to-use-the-toxic-release-inventory","slug":"1990-november-if-you-want-to-use-the-toxic-release-inventory","url":"https://dwillis.github.io/uplink/article/1990-november-if-you-want-to-use-the-toxic-release-inventory.html","year":1990,"month":"November","monthNum":11,"headline":"If You Want To Use The Toxic Release Inventory","kicker":"","author_name":"","author_title":"","affiliation":"Missouri Institute for Computer-Assisted Reporting","summary":"This article discusses pricing and availability options for 1988 and 1987 TRI (Toxic Release Inventory) data, comparing costs between NTIS and the Missouri Institute for Computer-Assisted Reporting.","keywords":["data","pricing","inventory"],"topics":["Data & Databases","Health & Environment"],"technologies":["9-track tape"],"full_text":"The 1988 volume is available in 4 Reels, ASCII, 6250 bpi, 32450 block-size. Record length is 550. NTIS sells TRI for $1100. The Missouri Institute for Computer-Assisted Reporting will provide the entire volume for $726. MICAR also can provide 1987 TRI data for the same price:","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1990-november-washingtons-secret-service","slug":"1990-november-washingtons-secret-service","url":"https://dwillis.github.io/uplink/article/1990-november-washingtons-secret-service.html","year":1990,"month":"November","monthNum":11,"headline":"Washington's Secret Service","kicker":"","author_name":"Michele Ward","author_title":"Crime reporter","affiliation":"The Herald","summary":"Crime reporter Scott North of The Herald used computer-assisted reporting and database analysis to investigate drug cases in Snohomish County, uncovering that drug offenders were demographically similar to typical residents and often received light sentences despite high conviction rates.","keywords":["drugs","database","conviction"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Steve Dutton was not your typical under- cover police officer. When accusations flew out of the mouths of the cocaine dealers he allied himself with, Dutton would silence them with the thrust of a knife deep into his leg. The leg was wooden. \"Cops don't have one leg,\" he' d calmly tell them. Dutton was the first amputee to graduate from Washington's police training academy. Crime reporter Scott North from The Herald in Everett, Wash., found this interesting character through the use of his database. (Dutton's) name kept coming up,\" North said. A year ago, North went to his editors with a story idea. He wanted to report how the war on drugs was being fought in Snohomish County. North and reporter Dale Folkerts began their reporting by scouring 1,134 drug cases on file with the county. This data was entered by hand into their computers and information about the defendant, charges, depositions and names of the officials involved were examined. That's how North found Dutton. After they had completed the process of entering the data, they discovered that they had hundreds of files in their computers, but no way to manipulate the files. Last April, North attended the Missouri Institute for Computer-Assisted Reporting's TRI/DART (training re- porters in data acquisition retrieval technologies) seminar to learn how to manipulate the data already entered into his database. The trianing resulted in the Herald's \"War Without Victory\" series. The reporting team responsible for the series consisted of reporters from the county courthouse, political and crime beats plus a reporter who specialized in demo- graphics. They found that the drug offender wasn't so very different from a typical Snohomish County resident. The demographic description of both were quite similar in relation to age, race and educational level. This kind of activity could be found in any neighborhood; this kind of criminal was often one's neighbor. \"Myths were unmasked our numbers showed that these cases are happening in your own back yard,\" North said. Unfortunately, the sentence doesn't always fit the crime. The reporters found that 76% of the cases resulted in convictions with 93% of those plea-bargained for a lesser sentence. Of those who were sentenced to serve time, the median jail sentence was only two and a half months. In 30% of the cases the offender served a month or less, often serving nonconsecutive days. Prosecutors tend to focus on getting convictions at the expense of reducing punishment. Prosecutors are working against time - they put together a case quickly because they don't want it postponed, often up to two years, before it gets to court. One police officer noted that a person with a suspended driver's license is apt to serve more time than someone convicted of a drug offense. The Herald will continue to use this drug database in police reporting, especially homicides where there is drug involvement. \"We often beat our competition with stories on crime,\" North commented.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-november-tridart-seminar-news","slug":"1990-november-tridart-seminar-news","url":"https://dwillis.github.io/uplink/article/1990-november-tridart-seminar-news.html","year":1990,"month":"November","monthNum":11,"headline":"TRI/DART Seminar News","kicker":"","author_name":"","author_title":"","affiliation":"MICAR","summary":"MICAR is offering a week-long seminar on computer-assisted reporting starting January 27, teaching journalists how to obtain, download, and analyze government computer data. The quarterly training program covers technical skills, program administration, and legal considerations for using government tapes.","keywords":["training","journalism","computers"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["9-track tape"],"full_text":"MICAR will hold its next week-long seminar on computer-assisted reporting on January 27. These seminars, which are held every three months, train people with little or no experience with computers how to negotiate for computer tapes, download the data on those tapes to a personal computer using a nine-track tape drive, verify that the information has not been tam- pered with, and analyze it using database software. The seminar also discusses how to administer a computer- assisted reporting program and the legal implications of using government computer tapes. Last year, MICAR trained 41 people representing 25 newspapers, the Associated Press and 2 universities. Tuition for the print media is based on ciruclation size and on market share for broadcasters. For more information call MICAR at (314) 882 0684.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-november-boston-reeling-in-stories","slug":"1990-november-boston-reeling-in-stories","url":"https://dwillis.github.io/uplink/article/1990-november-boston-reeling-in-stories.html","year":1990,"month":"November","monthNum":11,"headline":"Boston Reeling-In Stories","kicker":"","author_name":"Paul Martin","author_title":"","affiliation":"Boston Herald","summary":"The Boston Herald's computer-assisted reporting program has produced investigative studies on Massachusetts polluters and defense contractors, including findings that the state ranks third nationally in Freon use with increasing amounts being released despite known ozone-damaging effects.","keywords":["reporting","Massachusetts","environment"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["XDB"],"full_text":"Since the Boston Herald got its computer-as- sisted reporting program going in April, reporters David Armstrong and Alan Levin have produced computer stud- ies on Massachusetts polluters, dwindling defense con- tracts and suprising sources of campaign money. The Herald got the toxic-release inventory tapes for Massachusetts companies last spring. Levin, who had attended the Missouri Institute for Computer-Assisted Reporting's TRI/DART seminar in April, downloaded the tapes and analyzed the data with XDB software. He found that Massachusetts companies were using inreasing amounts of Freon 113, a chloroflourocar- bon known to damage the ozone layer. Levin and reporter Nick Tate put togeher a three-story package that ran on page 1 of the July 2 issue. Among the Herald's findings: Massachusetts industry ranked third in the nation for Freon use in 1987 and 1988. - Thirty-one of 48 Freon-using plants increased usage Perhaps the most sobering fact in the package was that Freon rele. today will still bc destroying ozone 140 years from now, wiping out up to 300 times its own weight of the protective gas before it's done. from 1987 to 1988. Increases by small and mid-sized companies nullified reductions made by the state's largest users, General Electric Co. and Raytheon. Overall, Freon use went up. One of the package's smaller stories explained how Freon destroys ozone, allowing the sun's ultraviolet rays to reach the Earth's surfa and wreak havoc on food chains and increase incidences of skin cancer. Another side-bar explained new ways to replace Freon, which is used to clean circuit boards and electronic equipment. Digital Equipment Corp., for example, \"pioneered a new cleaning method\" that isn't so new to most of us: soap and water. Defense Department tapes were the source for Armstrong's study showing how Massachusetts manu- facturers and their workers are suffering from defense cuts. The effect of the cuts was not news when the three day series was published in August. Contractors were al- ready laying off hundreds of employees and announcing plans to lay off thousands more. But Armstrong's articles put the problem in perspective. Day one's stories explained why the end of the Cold War and Reagan-style deficit spending could mean a dim future for Massachussettes' defense-reliant economy. The Reagan Administration's defense buildup had been a major force behind the 1980s boom in the Massachusetts economy, Armstrong's study showed. The second day's stories were about congress- men's attempts to bring home a piece of the shrinking defense-contract pie. Armstrong grouped the state's con- tracts by congressional districts to show exactly which congressmen had garnered the biggest pieces. The series finale reported how government- dependent contractors in Massachusetts were trying to wean themselves by looking for contracts with civilian companies at home and overseas. The Herald reporters has a major obstacle to clear before they can attempt a computer-assisted study of the campaign contributions in the Massachusetts governor's race: The state's contribution records are not computer- ized. Herald management decided to ship the records to a data-entry company for the considerable task of cre- ating a computer database of contributions. So far, the paper has spent more than $10,000 competerizing the records. Armstrong said it's well worth it. The Herald also bought complete records of contribu- tions to the 1988 presidential race. By comparing these contributions to those of the governor's candidaties, Armstrong and reporter Jonathan Wells were able to show that candidates who were making a campaign of Dukakis- bashing were also relying on the Duke's former contribu- tors for campaign money. Armstrong's study also found that a surprising amount of contributions had crossed party lines. Both Democratic primary candidates had tapped former Bush supporters for more than GOP candidate Steven Pierce. And the other Republican candidate, William Weld, had received $61,500 from 1988 Dukakis supporters, nearly two-thirds as much as he had gotton from Bush contribu- tors. Lately, Armstrong and Levin have had to help out with the Herald's campaign coverage, and computer- assisted projects have taken the back seat. But not for long, Armstrong said. \"After November, we're really going to go bonkers on this thing,\" he said.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-november-micar-library-news","slug":"1990-november-micar-library-news","url":"https://dwillis.github.io/uplink/article/1990-november-micar-library-news.html","year":1990,"month":"November","monthNum":11,"headline":"MICAR Library News","kicker":"","author_name":"","author_title":"","affiliation":"MICAR","summary":"MICAR has expanded its data library with three new datasets from federal agencies, including FAA aircraft malfunction reports and FDA medical device incident records, all available on 9-track computer tapes for journalists.","keywords":["data","reports","databases"],"topics":["Data & Databases","Public Safety","Health & Environment"],"technologies":["9-track tape"],"full_text":"MICAR makes available to reporters a library of data stored on 9-track computer tapes. A complete list of these tapes can be promptly mailed toanyone upon request. This month, MICAR has attained three new sets of data. From the Federal Aviation Association: - a report on \"service difficulty,\" which includes a list of all malfunctions and defects on commercial aircraft flown in the United States ($36). From the Food and Drug Administration: - a medical device report, which lists all injuries, malfunctions and deaths caused by medical devices ($336). -a problem-reporting program record, which lists all problems logged by the FDA by doctors from hospitals concerning medical equipment ($204).","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-december-fatal-flaws-death-by-device","slug":"1990-december-fatal-flaws-death-by-device","url":"https://dwillis.github.io/uplink/article/1990-december-fatal-flaws-death-by-device.html","year":1990,"month":"December","monthNum":12,"headline":"Fatal Flaws, Death by Device","kicker":"","author_name":"David Hinchman","author_title":"","affiliation":"MICAR","summary":"The investigative report reveals a significant number of malfunctions in medical devices, particularly defibrillators manufactured by Physio Control Corp., leading to patient deaths and injuries.","keywords":["medical devices","defibrillators","malfunctions"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["9-track tape"],"full_text":"Two paramedics went to help a person suffering a heart attack. They used a high-tech medical device designed to save the suffering victim. Not only did the device malfunction, allowing the patient to die, but it also severely jolted both paramedics putting them in the hospital. Chris Szechenyi, an investigative reporter at WRC-TV in Washington, D.C., called MICAR with news of a medical device malfunctions database. He knew the Medical Device Report (MDR) tape was more than just useful; it would be a blockbuster tape. The tape includes every malfunction associated with medical devices in the past five years, including any medical device used in the United States. Even those devices manufactured overseas but used in the U.S. are reported on this tape. If a medical device is used on a patient and it fails, resulting in the death or injury of that patient, the responsible device must be reported to the Food and Drug Administration. For example, if a hip implant is used for a patient who has broken a hip and that implant fractures during normal activity, resulting in the death of the patient, the device must then be reported to the FDA. Szechenyi, a veteran medical reporter, knew of this sort of data. Once he heard of MICAR, he put two and two together. He sent the tape to Elliot Jaspin at MICAR. The first thing Jaspin did with the data was determine which company had the most number of deaths associated with products. Physio Control Corp., a manufacturer of defibrillators, floated to the top. Defibrillators are devices that deliver an electric shock returning normal heart beat during cardiac arrest. If a defibrillator fails, the patient will almost certainly die. Physio Control products have been associated with more deaths than any other group of devices reported in the database. Physio Control defibrillators have contributed to the deaths of more than 400 patients over the past five years. The FDA records show that the corporation is responsible for the most deaths directly due to defibrillators. One product in particular, the LIFEPAK 5 defibrillator, has been associated with more deaths than any other device. Once Szechenyi filed a FOIA request for all Physio Control records through the FDA, he discovered that the company had 13 recalls and three safety alerts during the past five years. Some of the recalls were voluntary, but the FDA still had to demand that defects be fixed. The FDA record showed however, that Physio Control resisted these demands. Using the MDR database as a roadmap, Szechenyi discoverd that Physio Control was not the only defibrillator company with a lot of recalls and safety alerts. He also discovered the FDA has failed to set market standards for defibrillators. Neither the FDA nor Physio Control would discuss the matter with Szechenyi. But that didn't bother Tom Brokaw. A small portion of Szechenyi's story aired on NBC Nightly News, and Szechenyi's entire four-part series \"Fatal Flaws\" aired on all NBC owned-and-operated stations in New York, Los Angeles, Denver, Miami, Cleveland and Chicago. There are more than 100,000 records on the MDR and defibrillators represent only a small portion. There's plenty more to be done with the record for national and local stories. David Hinchman was able to broadcast a three- part version of the defibrillator story for KOMU-TV in Columbia. Hinchman is available through MICAR to answer any questions concerning MDR. If you want to use MDR. The volume is 3 reels, 6250 bpi, EBCDIC and avalible through NTIS (# pb90-591610) NTIS phone number is 703-487-4848. But at the time this issue of Uplink was printed, the NTIS sales desk was closed due to a fire in their computer room. MICAR can provide the volume (updated through Sept. 1990) for $300.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-december-if-you-want-to-use-mdr","slug":"1990-december-if-you-want-to-use-mdr","url":"https://dwillis.github.io/uplink/article/1990-december-if-you-want-to-use-mdr.html","year":1990,"month":"December","monthNum":12,"headline":"If you want to use MDR.","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Information about accessing MDR data available through NTIS on 3 reels of tape, with alternative availability through MICAR for $300 due to temporary NTIS closure.","keywords":["data","NTIS","availability"],"topics":["Data & Databases","Technology"],"technologies":["9-track tape","EBCDIC"],"full_text":"If you want to use MDR.\n\nThe volume is 3 reels, 6250 bpi, EBCDIC and avalible through NTIS (# pb90-591610) NTIS phone number is 703-487-4848. But at the time this issue of Uplink was printed, the NTIS sales desk was closed due to a fire in their computer room. MICAR can provide the volume (updated through Sept. 1990) for $300.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1990-december-national-weather-dis-service","slug":"1990-december-national-weather-dis-service","url":"https://dwillis.github.io/uplink/article/1990-december-national-weather-dis-service.html","year":1990,"month":"December","monthNum":12,"headline":"National Weather Dis-Service","kicker":"","author_name":"Valerie Brunell","author_title":"","affiliation":"","summary":"Cleveland Plain Dealer reporters Bill Sloat and Keith Epstein investigated the National Weather Service's accuracy in issuing severe weather warnings after a deadly Ohio flood, discovering that the service fails to warn of most dangerous weather events by analyzing government databases and records. The pair found that the weather service misses more than two-thirds of tornadoes, three-fifths of flash floods, and 40 percent of severe thunderstorms annually.","keywords":["weather","warnings","accuracy"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["magnetic tape"],"full_text":"Ever wonder how often weather forecasters make mistakes? Bill Sloat and Keith Epstein of Cleveland's The Plain Dealer did after a flood in Shadyside, Ohio, killed 26 people in June. The National Weather Service issued no warning after the heavy rains. \"They said the radar could not spot this kind of storm,\" Sloat said. \"I found it odd that in 1990 that we would not be able to issue a radar warning.\" After asking a few questions, the team started searching through individual disaster reports. They discovered that the weather service kept records of everything they did. Sloat said computers played an integral part in making the data available. \"The story would have been impossible without the computer era because of the way the government examines itself and stores information,\" he said. With the help of some key sources, they were able to determine the nation's best and worst weather service stations based on the accuracy of issuing warnings for severe storms, tornadoes and floods. \"The National Weather Service - meant 'to provide weather and flood warnings for the protection of life and property' - fails to warn of most of the savage weather that each year claims more than 400 lives and accounts for more than $6 billion in losses,\" they wrote in the first of a four-day series that began Dec. 2. \"You can get all kinds of information if you want to examine the offices in your area,\" Epstein said. The weatherservice, for example, keeps data on every forecast made by every person in the bureau. Although Epstein and Sloat did not do the tape analysis themselves, they said the weather service will sell tapes for $60. They discovered that \"the weather service misses more than two-thirds of all tornadoes, more than three-fifths of all flash floods and 40 percent of severe thunderstorms. Severe storms strike without warning more than 4,000 times each year and without even a preliminary 'watch' 5,000 times.\" The pair agreed the most difficult part was deciphering the formulas and acronyms. \"They have scores for their forecasts that are measured somewhat like a batting average,\" Epstein said. \"One of the keys with any government agency is to cut through the gobbledegook and find meaning,\" Epstein said. \"The computer can cover more ground and makes it easier to look at subsets of information.\" Crunching numbers was necessary to rank the top and bottom 15 weather service offices for accuracy in issuing warnings during a three-year period, but Sloat said the pair also relied heavily on old-fashioned reporting and did some data analysis by hand. \"I don't like computers very much,\" Sloat said. \"I still like to use paper, pencils and a calculator.\" Epstein and Sloat suggested that reporters interested in similar stories should look for available databases. \"Use your imagination,\" Epstein said. \"There's all kinds of data out there. I never really thought all of this information would be there. This (computer databases) is a burgeoning area for journalists.\" Epstein said reporters should find out what data is available and then get beneath the acronyms and formulas to find some meaning. People are what make the news, regardless of how significant the numbers are, Epstein stressed. \"It's good to remember that sometimes you can get lost in facts, names and conclusions,\" Epstein said. \"You need to give your information meaning and find out why something is happening. You have to get people in the story. That's what makes the numbers what they are.\" For Sloat and Epstein, it was the people - both those who controlled the issuance of warnings and those who saw the effects of their work - who were the victims or survivors of the natural disasters.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-december-plugging-in","slug":"1990-december-plugging-in","url":"https://dwillis.github.io/uplink/article/1990-december-plugging-in.html","year":1990,"month":"December","monthNum":12,"headline":"Plugging In","kicker":"","author_name":"Susan Drumheller","author_title":"","affiliation":"MICAR","summary":"Greg Reeves of the Kansas City Star shares his experiences implementing computer-assisted reporting, discussing the software and hardware challenges he faced, the efficiency gains achieved through better data processing tools, and the importance of verifying data accuracy before publication.","keywords":["software","database","verification"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Nine Track Express","XDB","Datatrieve","9-track tape"],"full_text":"Every other month, Uplink asks an individual or organization that is just beginning to use computer- assisted reporting to share their problems and victories for \"Plugging In.\" If you have any advice or words of wisdom to share, please call us at MICAR. The Kansas City Star began it's commitment to computer-assisted reporting in March 1989 when reporter Greg Reeves pursued a half million Kansas and Missouri property assessment records on computer tape. Since then, Reeves attended MICAR's TRI/DART seminar and has helped to firmly establish computer-assisted reporting at his organization. The Star has purchased 44 database tapes in the past 18 months. Reeves shared some of his experiences with Uplink: Software and Hardware: Before Reeves attended the MICAR seminar, the Star invested in two tape drives, Nine Track Express and XDB software. Though NineTrack Express was a great help, XDB presented several new problems for Reeves and the Star. For Reeves, the problem was that he had already become accustomed to another database management program called Datatrieve by Digital Equipment Corporation. For the Star, the problem was that XDB required a great deal of memory. Because Reeves' computer is hooked \"I was exposed and discovered by data-processing and forced to delete files I had become emotionally attached to.\" into a network, the Star's computer people have had trouble dealing with XDB's large size. Reeves required 10 gigabytes of memory for XDB and file space. \"It was the first time anyone saw a file bigger than 100 megabytes,\" Reeves said. For a while, he tried stashing files on other reporters' unused disk space. \"I was discovered by data processing and forced to delete files I had become emotionally attached to,\" Reeves said. He will be running some \"acid tests\" to compare XDB with Datatrieve in the near future. Data: Life became much easier for Star reporters once they started using NineTrack Express, Reeves explained. \"We used to get computer tapes, take them down to data-processing and have the data in usable form in about 6 to 8 weeks,\" Reeves said. \"Now, with decent software, it takes days or even hours.\" NineTrack was the \"single biggest help in this whole thing,\" Reeves said. \"It frees you from computer programmers.\" Reporters at the Star are using drivers' license records regularly, now that Reeves was able to download the data into the network. Reporting: Making sure that information is correct must be a priority, Reeves said. For example, the drivers' license records Reeves attained were missing a crucial code that distinguished one John Smith born of Jan. 1 from another with the same name and birthdate. Reeves said he uses data from computer tape simply as a starting block for further investigation. He will, for example, double-check electronically stored records with hard-copy. The potential for error is too huge, Reeves said.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-december-taxing-investigation-in-palm-beach","slug":"1990-december-taxing-investigation-in-palm-beach","url":"https://dwillis.github.io/uplink/article/1990-december-taxing-investigation-in-palm-beach.html","year":1990,"month":"December","monthNum":12,"headline":"Taxing Investigation in Palm Beach","kicker":"","author_name":"Michele Ward","author_title":"","affiliation":"Palm Beach Post","summary":"The Palm Beach Post's investigation of over 22,000 property records revealed systemic inequities in how the county appraiser assessed homes, with expensive properties undervalued and cheaper homes overtaxed. The database-driven series prompted widespread challenges to assessments and led to a state audit that confirmed numerous errors in the appraisal office.","keywords":["assessments","taxation","investigation"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":["9-track tape"],"full_text":"Thanks to a database and a good investigation by the Palm Beach Post, a record number of challenges against the Palm Beach County property appraisers was filed this year. The Post examined more than 22,000 single-family- home records and found that, for some reason, the more expensive the house the lower its tax assessment. This meant that people in less expensive neighborhoods were carrying a heavier tax burden, or the \"bad luck tax.\" This past September, The Post ran the series called \"Property Taxes: An Unfair Burden.\" Amy Driscoll, a Post staff writer, said, \"We had been writing stories about the taxes in Palm Beach County for years; we wanted a detailed study about the inconsistencies in the way people were taxed.\" With the aid of the computer, Driscoll, Staff Writer Paul Martin and Special Projects Editor Rich Gordon (now with The Miami Herald) compiled this series which revealed the inconsistencies in the way property owners in Palm Beach County were taxed in 1989. The Post investigators, after a year of discussion with the Property Appraiser's Office, purchased a key computer tape, the Name-Address-Legal file (known as NAL). \"We bought the NAL file, which basically includes (for each property) the owner, the legal description, information on assessed values and information on sales prices,\" Martin said in a memo regarding the series. To create an accurate sample of properties to analyze, The Post reporters had to organize the information they retrieved from the NAL tape. The Post limited the study to warranty-deed sales and eliminated insignificant property types such as property that sold for less than $10,000. Also, on the advice of an assessment expert, the paper cut 1 percent of the sample with the highest and lowest assessment ratio. The assessment-sales ratio is calculated by dividing the property assessment by the sale price. This is the yardstick by which comparisons are made to determine the fairness of assessments. The ratio is valid if the selling price represents the true market value of the home. If everyone's home were assessed at the same percent (of market value), assessments and taxes would be fair. After calculating the weighted mean assessment-sales ratio, the Post reporters looked for any unfair pattern of taxation. Inequitable assessments became apparent. These assessments resulted in unfair tax burdens on the owners of less expensive houses and tax breaks for owners of more expensive houses. This phenomenon is not unknown in the business of property appraisers. Nor is it unknown among the media. The primary precedent to The Post series was New York Newsday's \"Property Taxes/The Unbalanced Burden\" published in May 1985 about unfair property appraisals. But The Post study had its own local impact. A state audit in the spring of 1990 uncovered numerous errors and failures by the department as it double-checked local appraisal offices, Driscoll's article stated. The Post had started investigating this problem months before the audit. Nearly four weeks after the series ran, the Florida Grand Jury backed up the newspaper's findings. \"(The grand jury) clearly bore out our original observations,\" Driscoll said. According to Driscoll, a record number of notices of intent to challenge the tax appraisers' findings were filed after publication. Since the grand jury's findings, the state will address this problem and attempt to rewrite the code tax appraisers use.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1990-december-datacore-news","slug":"1990-december-datacore-news","url":"https://dwillis.github.io/uplink/article/1990-december-datacore-news.html","year":1990,"month":"December","monthNum":12,"headline":"Datacore News","kicker":"","author_name":"","author_title":"","affiliation":"MICAR","summary":"Datacore, launching in January, is an online database service that enables reporters to access government records including Census data and Federal Elections Commission records through a modem connection. The service allows journalists to obtain exclusive access to computer tapes containing campaign contribution records and other publicly available information.","keywords":["database","government","reporters"],"topics":["Data & Databases","Journalism & Newsroom","Government & Politics"],"technologies":["modem/BBS","CD-ROM"],"full_text":"Beginning in January, Datacore will finally be ready for use. Datacore is an on-line database service that can be used by reporters to access government records. The service is designed to provide a powerful link to electronically stored records for reporters who do not have adequate PC computing power. With a modem and software provided by Datacore, a reporter can access Datacore's information library, which includes Census data and Federal Elections Commission records. Datacore has more than 960,000 campaign contribution records for all federal races since 1983 on-line. In addition to this, Datacore can put on-line any information that is sent to MICAR Reporters can acquire their own computer tapes and have exclusive access to the information for a year. If Datacore sounds interesting to you, call MICAR and we will send you all the details.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-january-newsday-report-pays-off","slug":"1991-january-newsday-report-pays-off","url":"https://dwillis.github.io/uplink/article/1991-january-newsday-report-pays-off.html","year":1991,"month":"January","monthNum":1,"headline":"Newsday Report Pays Off","kicker":"","author_name":"Tom Braden","author_title":"Graduate student","affiliation":"Missouri School of Journalism","summary":"An investigation by New York Newsday revealed that the city owes $275 million in tax overpayments to property owners, highlighting the ease of uncovering bureaucratic errors using computer-assisted reporting.","keywords":["tax refunds","bureaucracy","investigation"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Nine Track Express","XDB"],"full_text":"When New York Newsday reporter Penny Loeb called Aly Dadras to tell him that the city owed him at least $5,000, Dadras reacted with disbelief. And for good reason. Several months earlier, the New York City Department of Finance had told Dadras it was preparing to take over a house he owns in Queens for nonpayment of taxes.\n\nFor Dadras, an architect and professor at the New York University Institute of Technology who takes pride in his fastidious attention to bills, the Finance Department proceedings had come as a great shock.\n\nLoeb's computer analysis of city finance records set the record straight - for Dadras as well as hundreds of thousands of other property owners to whom, she discovered, the city owes as much as $275 million in overpayments on their real-estate, water and sewer taxes. So read the front page of Newsday on Jan. 7. Now Dadras awaits a check from the city, which has ceased pursuing his house. And city officials assure they will repay all other residents owed money. Loeb said that to date their progress has been slow. The city says it will take a month just to straighten out its records.\n\nBut it only took an instant for Loeb to know that she had a dynamite story. \"It just came so fast,\" she recalled recently. \"I remember looking at the result and thinking, wow, there it is. I was dumbfounded.\"\n\nSo were the property owners she called to notify of the city's malfeasance. They never knew they were owed money because the city isn't required to tell them. Thanks to Newsday, they are beginning to find out.\n\nNew York City residents have reacted to the story with outrage; and city officials, already smarting because of the sorry state of city finances, are scratching their heads and wringing their hands.\n\n\"I was disturbed by learning about it,\" said Carol O'Cleireacain, the city's finance commissioner, when she discovered the secrets contained in her department's records.\n\nAsked why the city never notified the taxpayer to whom it owes money, one city official replied: \"The simple answer to that is we are not required by law to let anybody know about credits.\"\n\nIn his State of the City address, given shortly after the story appeared on the streets, Mayor David N. Dinkins took a conciliatory tone: \"Our people have a right to expect that their government will be both honest enough to refund tax overpayments and caring enough to do it quickly.\"\n\nLoeb, meanwhile, expressed less surprise at this most recent example of the city's bureaucracy gone awry than at the ease with which the story unfolded using a personal computer. Once she had obtained the computer tape for $225 from the Finance Department under the New York State Freedom of Information Law, it took her just one month to analyze the information and report and write the story.\n\nThe computer tape contains information on credits for all affected properties in New York City's five boroughs - a total of nearly 1.5 million records. The tape also has a code for the type of credit owed (whether real estate, sewer or water) and the type of building for which owners are deserving of money. If the credit was tied to vacant land, the computer tape identifies it as such.\n\nLoeb used NineTrack Express to filter in just a half hour only those records with credits for property owners, leaving more than 500,000 records to analyze in XDB. The file consumed 40 megabytes of memory.\n\nAside from identifying the properties' locations, giving the borough, block and lot, the computer tape has two open balance fields, each of which represent credits and delinquencies for two different payment periods.\n\nThese two fields contain figures that either represent credits the city owes residents or delinquencies owed to the city. Queries in XDB took anywhere from a few minutes to a half hour.\n\nLoeb expressed less surprise at the city bureaucracy gone awry than at the ease with which the story unfolded using a personal computer.\n\nThe tape does not identify property owners by name, so Loeb tracked them down, matching the borough, block and lot to owners listed in real estate books.\n\nMany of the bilked taxpayers were skeptical when Loeb reached them by telephone. \"They didn't believe me,\" she said. \"Others thanked me, but they didn't want their names in the paper.\"\n\nIn producing the story Loeb confronted more formidable challenges than publicity-weary city residents.\n\nShe was asked near deadline by her editors to find out how many property owners were owed money. Since more than one credit can accrue to a single property, this was impossible to tell by looking at the tape. So Loeb took a 10 percent sample, and counted the records - about 1,000 of them - by hand.\n\n\"I couldn't believe I was doing a computer-assisted story on deadline,\" she said.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-january-software-success","slug":"1991-january-software-success","url":"https://dwillis.github.io/uplink/article/1991-january-software-success.html","year":1991,"month":"January","monthNum":1,"headline":"Software Success","kicker":"","author_name":"George Landau","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"A journalist at the St. Louis Post-Dispatch shares his technical toolkit and software recommendations for efficiently managing and analyzing data on personal computers, including database managers, remote access tools, and utility programs.","keywords":["software","database","tools"],"topics":["Journalism & Newsroom","Technology","Data & Databases"],"technologies":["XDB","SQL","FoxPro","dBase","Nine Track Express","PCanywhere","DESQview","XyWrite","PC Tools","Norton Utilities","PKZIP","PKUNZIP"],"full_text":"What this newsletter needs is a little more technical shop talk. Because beyond the hassles of getting data and the thrills of analyzing it, there's the matter of making efficient use of the PC.\n\nIn hope of getting a dialogue going, I'll go first. Here, with the emphasis on software, is how we do it at the Post-Dispatch:\n\nPersonnel: One guy so far (me).\n\nSoftware that I rely on:\n\nXDB: a database manager that uses Structured Query Language (SQL), a very powerful and relatively easy to learn language for analyzing and maintaining tables of data. On my PC, XDB gets annoyingly slow working with tables of more than about 50,000 records.\n\nFoxPro: a database manager compatible with the popular dBASE standard. FoxPro is four times faster than XDB at manipulating data, but dBASE language requires actual programming to do anything fancy.\n\nNine Track Express: this Elliot Jaspin production, buggy in its earlier versions, has become an indispensable tool for getting data off a tape.\n\nPCanywhere: this very handy program gives me full remote control, by modem, of the office PC. With a laptop at home, I can cook dinner while working the bugs out of long complicated queries.\n\nDESQview: for efficient multitasking, this beats Windows 3.0. DESQview allows me to run several full-size DOS applications at once. I can edit an XDB batch file in XyWrite and then, in a single keystroke, jump to XDB to run what I've just written.\n\nVarious Utilities: If you don't already use PC Tools, buy it. Its disk cache, to which I grant one megabyte of RAM, can double the effective speed of your hard disk drive without putting your data at the slightest risk.\n\nI use The Norton Utilities to recover files that I or my colleagues have mistakenly erased. I've used its Disk Doctor to diagnose and repair the occasional bad floppy.\n\nPKZIP: a shareware program available on many PC bulletin boards, saves disk space by compressing files you don't use regularly. PKZIP compresses most database tables by 80 to 90 percent; PKUNZIP, a companion to PKZIP, decompresses even large tables for use in less than a minute.\n\nQuestions or comments? Call (314) 622-7205 or write to George Landau, St. Louis Post-Dispatch, 900 North Tucker Blvd., St. Louis, MO 63105.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-january-plugging-in-the-new-york-times","slug":"1991-january-plugging-in-the-new-york-times","url":"https://dwillis.github.io/uplink/article/1991-january-plugging-in-the-new-york-times.html","year":1991,"month":"January","monthNum":1,"headline":"Plugging In The New York Times","kicker":"","author_name":"Valerie Brunell","author_title":"","affiliation":"The New York Times","summary":"The New York Times uses computer-assisted reporting to analyze large datasets, producing two major investigative stories: one examining campaign contributions to Gov. Cuomo and another analyzing the proposed NYPD expansion to determine if resources were being allocated based on crime rates.","keywords":["computer","reporting","analysis"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":[],"full_text":"Computer-assisted reporting is still relatively new at The New York Times, but after less than six months in the works, the staff has generated two major stories that would have been otherwise impossible - one on the state's campaign contributions and the other on the proposed expansion of the New York City police force.\n\nNew York Times Database Editor John C. Freed describes computer-assisted reporting as having 10,000 monkeys sorting information for a reporter. \"The human then has to look through (the information) and say this is good and this is not,\" he says. \"It's almost an art rather than a science. You have to think about what might be a link.\"\n\nFreed's first major project involved an examination of more than $20 million in campaign contributions over the past eight years collected by New York Gov. Mario M. Cuomo. Freed updated information that had already been compiled on computer disks by a private company for the State Government Integrity Commission. To this, Freed said, he only needed to add the contributions that had been made since the commission did its report. Then his analysis began.\n\n\"Wells, Rich, Greene, the advertising agency, has a $30 million state contract for the 'I Love New York' campaign and hopes to renew the contract next March. The agency has contributed at least $42,000 to (Gov. Cuomo's) campaigns over the last eight years, including $20,000 this year,\" read the lead in the Times' Nov. 1 story written by reporter Frank Lynn.\n\nThe story also reported that \"64 of 68 engineering firms on a Cuomo list of prospects to be solicited ended up contributing to the campaign and that all but 11 of the 64 had contracts with (the State).\"\n\nAnother Times story concerning the proposed expansion of the New York police force became the newspaper's second major investigative project using computer-assisted reporting.\n\nFreed used two hard-copy reports released by the city. One report, which detailed the precincts to which the additional officers would be assigned, was scanned into the computer. The other reported types and numbers of arrests in each area and was typed-in by staff within a week.\n\nThe reports were then cross-referenced to see whether the areas with higher crime rates and more violent crimes were getting increases in officers. This was not always the case, as the computer analysis showed.\n\nThe report required six weeks' work - about three for Freed's analysis and three for interviews by Times' reporter Ralph Blumenthal. The story ran Dec. 15, the day after the State Legislature postponed approving the expansion plan at its last session because members didn't know exactly where the 9,700 additional patrol officers were to be placed.\n\n\"One Staten Island precinct, the 123d, which already enjoys the city's best police coverage for violent crime - about 85 officers for every 100 reported violent crimes - would move to an even higher ratio.\"\n\n\"At the same time, the Brooklyn North command, with some of the city's highest-crime precincts, would wind up with the lowest percentage increase of officers,\" the story read.\n\nAlthough the number of ideas for future computer-assisted investigative stories at the Times are increasing, Freed said the major projects don't go unless there's evidence that something is there. \"Computer analyses are not good for fishing expeditions.\"\n\nBut if reporters are handed a pile of documents, they need to know to ask for the tapes instead, Freed said.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-january-access-to-info-in-california","slug":"1991-january-access-to-info-in-california","url":"https://dwillis.github.io/uplink/article/1991-january-access-to-info-in-california.html","year":1991,"month":"January","monthNum":1,"headline":"Access to Info in California","kicker":"FREEDOM OF THE PRESS","author_name":"Sandy Davidson Scott","author_title":"Professor of Communications Law","affiliation":"The Missouri School of Journalism","summary":"Following actress Rebecca Schaeffer's murder by a stalker who obtained her address through public records, California restricted access to driver's license information, exemplifying how security fears can lead to barriers on information access for everyone.","keywords":["privacy","security","records"],"topics":["Government & Politics","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Chain link fences and cement barricades surrounded the stadium at Tampa. Metal detectors stood ready at the gates. It was the Super Bowl! Missing were the familiar radios held by fans in the stands. Why? Because some terrorist might have stuffed his or her radio with a bomb.\n\nThere is an analog to this rush for security caused by the Persian Gulf war - the clamor to restrict information caused by unfortunate uses of the information. All it takes is one lunatic, or even the threat of one, to result in barriers for us all.\n\nThe murder of 21-year-old actress Rebecca Schaeffer, costar of the television series \"My Sister Sam,\" is a case in point. She lost her life in Los Angeles on July 18, 1989, when she answered a knock at her apartment door. She opened her security door, and a deranged, jealous admirer, 19-year-old Robert Bardo, shot her in the chest. Bardo, who lived in Tucson, Arizona, had paid $250 to a Tucson private investigating firm, Anthony Agency, to get Schaeffer's address.\n\nSchaeffer was an old friend, Bardo had told the firm, and he wanted her current address. He carried a picture of Schaeffer which bore her signature and a personal, handwritten message.\n\nThe firm got her address from driver's license records in California public records available at the time to anyone virtually on demand. The requestor did have to fill out a form that asked the reason for the request, but only addresses of judges and peace officers were automatically restricted.\n\nA furor followed Schaeffer's death.\n\nThe clampdown on records held by the California Department of Motor Vehicles became almost as inevitable as war in the Persian Gulf.\n\nThe Los Angeles Times quoted California legislator Mike Roos as saying, \"Tragically, government records have become the personal research archives of thieves, kooks and killers. The fact is we are all sitting ducks for any criminal or kook who wants to track us down.\"\n\n\"Kooks and killers\" - and most sane persons, for that matter - can no longer use California driver's license records to learn residential addresses. On October 1, 1989, California's governor approved legislation sponsored by Roos. The law says, in part:\n\nIn order for individuals to be able to exercise their right to privacy, they must be able to choose when to release personal information, and to whom, and reasonable laws requiring the individual to surrender control should be enacted only when it is deemed absolutely necessary for society's welfare.\n\nThe personal privacy and security of one's home is fundamental to this right of privacy. Now California law says that residential addresses maintained by Department of Motor Vehicle records are \"confidential and shall not be disclosed to any person,\" with basically four exceptions: (1) a court; (2) a law enforcement agency; (3) a financial institution licensed by California or the federal government to do business in California and which has a written waiver signed by the person whose address the institution requests; and (4) an insurance company with a California license which is seeking the address of a motorist involved in an accident with someone insured by the company.\n\nNote that journalists have no right to gain residential addresses through California's Department of Motor Vehicles. Anyone who smuggles the information to a journalist (or any other unauthorized person) faces a fine of up to $5,000, imprisonment in a county jail for up to a year, or both.\n\nUnfortunately, the California law is placing a roadblock in the path of a journalist who would like to do a story, say, on school bus safety. If the journalist can get traffic violation records through the Department of Motor Vehicles but has no residential addresses, the journalist may well have difficulty trying to determine if the \"Bob Smith\" with 20 violations is the same Bob Smith in whose hands school children's lives are placed.\n\nSurely \"society's welfare\" is improved by aiding journalists in writing such stories, not in placing a roadblock with no exceptions in their way.\n\nAnd so the erosion continues, chipping away here and there at what we can do as society creates a padded cell for us all - because some are \"kooks and killers.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-january-tridart-seminar-news","slug":"1991-january-tridart-seminar-news","url":"https://dwillis.github.io/uplink/article/1991-january-tridart-seminar-news.html","year":1991,"month":"January","monthNum":1,"headline":"TRI/DART Seminar News","kicker":"","author_name":"","author_title":"","affiliation":"Missouri Institute for Computer-Assisted Reporting","summary":"MICAR is offering a week-long computer-assisted reporting seminar on May 5 that trains journalists with minimal computer experience how to obtain, transfer, verify, and analyze government data using database software. The seminar covers practical skills and legal considerations, with tuition based on media outlet size.","keywords":["training","journalism","data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["9-track tape"],"full_text":"TRI/DART Seminar News\n\nMICAR will hold its next week-long seminar on computer-assisted reporting on May 5.\n\nThese seminars, which are held every three months, train people with little or no experience with computers how to negotiate for computer tapes, download the data on those tapes to a personal computer using a nine-track tape drive, verify that the information has not been tampered with, and analyze it using database software. The seminar also discusses how to administer a computer-assisted reporting program and the legal implications of using government computer tapes.\n\nLast year, MICAR trained 41 people representing 25 newspapers, the Associated Press and 2 Universities.\n\nTuition for the print media is based on circulation size and on market share for broadcasters.\n\nFor more information, call the Missouri Institute for Computer-Assisted Reporting at (314) 882-0684","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-february-plain-dealers-overnight-success","slug":"1991-february-plain-dealers-overnight-success","url":"https://dwillis.github.io/uplink/article/1991-february-plain-dealers-overnight-success.html","year":1991,"month":"February","monthNum":2,"headline":"Plain Dealer's Overnight Success","kicker":"","author_name":"Dave Davis","author_title":"Reporter","affiliation":"Cleveland Plain Dealer","summary":"The article describes the challenges faced by Plain Dealer reporters in covering the historical crash of an Emery Worldwide Airlines jet at Cleveland's Hopkins Airport, and how computer-assisted reporting contributed to the story.","keywords":["plane crash","DC9","reporting"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Imagine this: it's early Sunday morning, about 12:20 a.m., 40 minutes before the newspaper is put to bed, and a jetliner crashes at the airport. You hear it on the police scanner.\n\nFrantically, you call the rounds, the police dispatcher, the fire dispatcher, airport security. You manage to get a few details confirmed a DC9 cargo jet crashed during takeoff at Cleveland's Hopkins International Airport. The two crew members on-board were killed. You even get a few paragraphs onto the front page.\n\nBut there's not another edition for nearly 24 hours, and the first full day you have to chase the story is a Sunday, a day of rest, a day when officials at the Federal Aviation Administration are not in their offices to look up the vital statistics on the aircraft for you. Then imagine that the next day is a holiday.\n\nThat's the situation Plain Dealer reporters and editors found themselves in when an Emery Worldwide Airlines jet on February 17 became the first plane to crash in the history of Hopkins Airport.\n\nBy Monday, the day after the crash, we managed to print stories that included comments from the families of the victims - David M. Reay, a pilot who went to work for the cargo outfit just three weeks earlier after being laid off by USAir, and 28-year-old Richard J. Duney Jr., the first officer. We managed to publish eye-witness accounts that described the jet taking off in a snow squall, rising briefly and then toppling into a fireball 2,500 feet from the end of the runway. The photographs showed the wreckage laying on its back.\n\nAnd by Monday, with the assistance from Adam Berliant and David Hinchman at the Missouri Institute for Computer-Assisted Reporting, we managed to tell our readers that the aging DC9 had a recent history of parts defects and malfunctions that could have hindered the ability to operate safely.\n\nWith the tail number that our reporter at the crash scene had gotten early Sunday, Adam and Dave searched the 75,391 FAA computer records of defects and malfunctions in commercial airplanes. Bingo!\n\nThey found 32 of the so-called Service Difficulty Reports filed for the DC9 that crashed in Cleveland, including one problem that received the FAA's worst severity rating, a \"5.\" The FAA assigns a severity level rating of \"1\" to \"5\" for the reported problems.\n\nThe database covers a two-year period ending on October 1990. While the records did not provide a complete picture of the aircraft, but they did provide a good one.\n\nFor example, during a landing at Jacksonville on Nov. 28, 1989, the aircraft's landing gear malfunctioned. Said the report: \"Upon gear extension, (the) flight crew heard a loud noise as the gear came down.\"\n\nAn investigation revealed that the right, main landing-gear rod end had separated from the wheel. The DC9 was ferried in for repairs and returned to service the report stated. The FAA reports detailed cracks and corrosion in the aircraft's body structure. Some of the cracks measured five inches, while the corrosion was sometimes described as \"very heavy.\" They told of parts that were loose, cracked, stripped or frozen in the plane's landing gear, warning systems and cabin and cargo doors.\n\nCraft had history of problems\n\nThough the majority of reports did not detail life-threatening problems, they did tell of the wear and tear on an aging plane that had been in service since 1968.\n\nAfter the Sunday night analysis, Dave FAXed me about 30 pages of the most important information. It was detailed and clearly presented, so I had the confidence to put it in my story.\n\nI've since bought the FAA tape from the Institute (at $37 I wish I had done so sooner) and I'm looking at other DC9s used by Emery to haul mail for the U.S. Post Office. I'm hoping that the results will make for an even better piece.\n\nMy advice: Don't wait for a plane crash to get the FAA tape. It will be too late. And feel free to call me at the Plain Dealer if you have any questions, (216) 344-4808.\n\nThe FAA service difficulty records are continually updated by the Administration. New updated copies are available through Myriad Systems, 3750 N 144, Oklahoma City, OK 73112 (405) 946-3395. Copies are also available through MICAR.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-february-census-data-seminar-news","slug":"1991-february-census-data-seminar-news","url":"https://dwillis.github.io/uplink/article/1991-february-census-data-seminar-news.html","year":1991,"month":"February","monthNum":2,"headline":"Census Data Seminar News","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Missouri Institute of Computer-Assisted Reporting and the U.S. Census Bureau are hosting a one-day seminar on April 15, 1991 at the University of Missouri-Columbia to teach reporters how to obtain and analyze census data using various formats and tools.","keywords":["census","seminar","reporters"],"topics":["Demographics & Census","Journalism & Newsroom","Technology"],"technologies":["GIS"],"full_text":"The Missouri Institute of Computer-Assisted Reporting and the U.S. Census Bureau are holding a conference for reporters on obtaining and using census data in early April.\n\nThe one-day seminar will be held Monday, April 15, 1991 at the University of Missouri-Columbia.\n\nSpeakers from the Census Bureau, MICAR and the University of Missouri, plus software and on-line systems experts will discuss ways to obtain census data. Featured are on-line systems, magnetic tapes as well as traditional paper records. Plus, ways to analyze census data, such as using mapping software, will be discussed. Panelists will also share information concerning how the media uses census data and notable demographic trends on the national, state and local levels.\n\nThe cost of this conference is $30, transportation, accommodations and meals not included. Attendance will be limited to 200.\n\nIf you're interested in attending the seminar, please call Michele Ward at MICAR, (314) 882-0684.\n\nCensus Seminar Schedule appears on Page 4","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-february-contribute-this-space-should-be-yours","slug":"1991-february-contribute-this-space-should-be-yours","url":"https://dwillis.github.io/uplink/article/1991-february-contribute-this-space-should-be-yours.html","year":1991,"month":"February","monthNum":2,"headline":"CONTRIBUTE! This space should be yours","kicker":"","author_name":"","author_title":"","affiliation":"MICAR, University of Missouri School of Journalism","summary":"Uplink invites journalism professionals to contribute stories, experiences, and insights about computer-assisted reporting to their forum at the University of Missouri.","keywords":["journalism","reporting","collaboration"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"CONTRIBUTE! This space should be yours\n\nAfter all, Uplink is supposed to be a forum for computer-assisted reporting. Tell us about your success stories, your defeats, your problems, your ideas, your insights anything! If it concerns computer-assisted reporting, it is material for Uplink.\n\nSend stories and ideas to: MICAR, 120 Neff Hall, University of Missouri School of Journalism, Columbia, MO 65211. Or call us at (314) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-february-blaming-the-bug-computer-negligence","slug":"1991-february-blaming-the-bug-computer-negligence","url":"https://dwillis.github.io/uplink/article/1991-february-blaming-the-bug-computer-negligence.html","year":1991,"month":"February","monthNum":2,"headline":"Blaming the Bug, Computer Negligence","kicker":"","author_name":"Lisa Touye","author_title":"","affiliation":"","summary":"When faulty database software causes reporters to publish incorrect information, liability depends on whether the case falls under contract or tort law, with reporters bearing responsibility for their reliance on software despite potential claims against developers.","keywords":["software","liability","negligence"],"topics":["Technology","Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Your database management software omits data and gives you the wrong information for your story. But it's not until you start receiving angry phone calls that you learn there's a bug in your software. What happens if one of the angry callers turns to the legal system?\n\nWho's to blame? You or the people who supposedly debugged the software?\n\nReporters who rely too heavily on software are to blame - bug or no bug. But as the legal system evolves to account for our computerized society, there are some strategies you can take to defend yourself, even if the software has a disclaimer on its label.\n\nThere is such a thing as computer negligence. If your software really is to blame, you might want to consult a lawyer. But here are some of the basics.\n\nIt used to be that computer negligence cases fell specifically under contract law, but today, because of the prevalence of computers in the workplace and their use by people other than the buyer, computer negligence cases have fallen in the realm of tort law.\n\nWhat's the difference?\n\n\"Say you have car insurance,\" said John Jay Fossett an attorney at Strauss and Troy in Covington, Ky., who has published journal articles concerning computer law. \"That's a contract between you and your insurance company. You pay them money and they provide coverage for you.\"\n\n\"If you're in your car and you run into somebody, well that's tort,\" Fossett said. \"You've committed a tort because if you acted negligently and should have been watching where you're going, you've committed a tort against the person you hit.\"\n\nFossett said that as computers are used more in society, especially in manufacturing, more computer negligence cases will move toward tort law especially, in cases that involve physical injury.\n\nFor example, if a glitch in a computer program causes hot molten steel to be poured on a mill worker instead of where it was supposed to be poured, the worker has recourse under tort law. The worker does not have recourse under contract law because he or she was not one of the members who drew up the original contract.\n\n\"If there is an error caused by the computer, and whoever developed the program or machinery was negligent in developing that, they should be held liable for someone who is injured,\" Fossett said.\n\nMost of the computer negligence cases that have been decided under tort law so far have involved physical injury, not the loss of information, Fossett said. Loss of information would most likely come under contract law.\n\nEven though most cases have fallen under tort law, a case where a reporter has used faulty software would be a breach of contract between the reporter and the creator of the software he or she used.\n\n\"Then you'd have to prove how you or your business was damaged because of the loss of information,\" Fossett said.\n\nIf the story causes damages to your organization, say you're sued for libel, you may then have a contract case against the software company. You will still pay damages for your libel case, but you may have recourse against the software company.\n\nEven disclaimers on software packaging can not prevent you from this kind of recourse Fossett said.\n\n\"For negligence you need to prove a duty to someone - a breach of that duty. That breach must be the cause of injury, and that injury has to result in damages,\" Fossett said. \"The hardest question to answer is duty,\" he said. \"If some software you've used was not completely debugged and you lose information by using it, part of the case depends on the effort that went into debugging the program. The programmers have a duty to create a product that is relatively safe.\"\n\n\"They put in warranties and disclaimers,\" he said. \"If I had a good case, I'd try to get around those limited disclaimers.\"\n\nBut the disclaimers are carefully written, so recovery would be limited, he added.\n\nClaiming computer negligence is a tricky business. Do not expect that a bug in your software will relieve you from responsibility.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-february-census-bureau-media-seminar-schedule","slug":"1991-february-census-bureau-media-seminar-schedule","url":"https://dwillis.github.io/uplink/article/1991-february-census-bureau-media-seminar-schedule.html","year":1991,"month":"February","monthNum":2,"headline":"Census Bureau Media Seminar Schedule","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Census Bureau hosted a media seminar at the University of Missouri School of Journalism covering how journalists use census data, including presentations on data trends, products, and online demonstrations of mapping and data access tools.","keywords":["census","journalism","data"],"topics":["Demographics & Census","Journalism & Newsroom","Data & Databases"],"technologies":["Cendata","TIGER","Strategic Mapping","9-track tape","CD-ROM"],"full_text":"Census Bureau Media Seminar Schedule:\n\n8:30 - 9:00 Registration\n9:00 - 9:15 Welcome\nDean Mills, Dean of MU School of Journalism\n- Marvin Postma, Director of Census Bureau Regional Office, Kansas City\n\n9:15 - 10:15 How the Media Uses Census Data\n- Ray Bancroft, PIO for the Census Bureau, Washington, DC\n- Byron Scott, Chairman of the Editorial Dept., MU School of Journalism\n- Bill Elder, Research Associate of the MU Office of Social and Economic Data Analysis\n\n10:15 - 10:30 Break\n\n10:30 - 11:15 Subject Content\n- John Kavaliunas, Chief of the User Training Branch, Census Bureau, Washington, DC\n\n11:15 - 12:00 Trends to Track\nNational Trends: Pete Bounpane, Asst. Director of the Decennial Census, Washington, DC\nState and Local Trends: Rex Campbell, MU Sociology Dept.\n\n12:00 - 1:00 Census Products Availability and Support\n- John Kavaliunas\n- Marlys Davis, Census Data Center, Missouri\n\n1:00 - 2:30 Lunch\n\n2:30 - 3:00 On-line panel\n- Elliot Jaspin, DATACORE, MU\n- Cendata Representative\n\n3:00 - 3:30 Breakout Sessions\nQ & A Session on media uses of census data:\nByron Scott, Ray Bancroft and Bill Elder\n\nOn-Line Demonstration\n- Elliot Jaspin\n- Cendata Representative\n\nMapping demonstration\n- John Kavaliunas, Tiger Mapping System\n- Tod Everett, Strategic Mapping\n- Tim Haithcoat, MU Dept. of Geography\n\n3:30 - 3:45 Break\n\n3:45 - 4:15 Tape Panel\n- Elliot Jaspin, NineTrack Express\n- Tod Everett, Strategic Mapping\n- John Kavaliunas, Census Bureau\n\n4:15 - 4:45 Breakout Sessions\nHow census data is obtained and used\n- Emmett Morris, Census Bureau Regional Office, Kansas City\n- Ryan Burson, Census Data Center, Public Law Data File, MO\n\nCD-ROM demonstration\n- John Kavaliunas\n\nTape Demonstration\n- Elliot Jaspin, Tod Everett\n\n4:45 Closing Session","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-march-the-count-of-san-francisco","slug":"1991-march-the-count-of-san-francisco","url":"https://dwillis.github.io/uplink/article/1991-march-the-count-of-san-francisco.html","year":1991,"month":"March","monthNum":3,"headline":"The Count of San Francisco","kicker":"","author_name":"Judy Miller","author_title":"Special Projects Editor","affiliation":"San Francisco Chronicle","summary":"The San Francisco Chronicle achieved detailed reports on the 1990 census data through advanced preparation and computer-assisted techniques, despite unexpected challenges.","keywords":["census","San Francisco","journalism"],"topics":["Demographics & Census","Investigative Methods","Data & Databases"],"technologies":["9-track tape","XDB"],"full_text":"The San Francisco Chronicle's reporting on the 1990 census adhered to the finest traditions of computer-assisted journalism. Sometimes it seemed that everything that could go wrong did go wrong.\n\nThe Census Bureau surprised the California Media by releasing the data a month before it was expected. The magnetic tape containing the data arrived right on deadline while the presses were running. And then the tape drive broke down.\n\nBut despite problems we published better and more detailed reports on census data than any other paper in California, mostly because we had programmed our software to do its work before the census data arrived.\n\nWith the support of top editors, the paper plans to use computers to improve both projects and daily reporting.\n\nThe census project got off the ground when Editor Tim Schreiner and I teamed up to prepare for the release of the 1990 census data for California. Schreiner was a whiz at understanding the census and crunching numbers, and I was a graduate of MICAR's weeklong workshop and knew how to import 9-track magnetic tape information into a personal computer and manipulate with the database management program, XDB. We also recruited Library Director Richard Geiger and Systems Specialists Gerald Jampol for assistance with research and technical matters. The team's combined experience became key when the U.S. Census Bureau, which had promised to release its California data \"no earlier than mid-March,\" instead released the information on February 25, with only two weeks notice.\n\nThe first wave of data consisted of total population figures, racial and ethnic population, and numbers of housing units broken down by county, census track and block. In California, where minorities are expected to outnumber whites within 10 to 15 years, the growth in population by racial and ethnic groups between 1980 and 1990 was the obvious lead to our first-day story.\n\nIn preparation for C-Day, the paper purchased a 9-track tape from the Census Bureau that gave sample census data for portions of Washington state. The purpose of this so-called \"Dress Rehearsal\" tape was to familiarize journalists with the contents of the 1990 census tape.\n\nThe record layout for this tape closely matched the layout of the 1990 California release. Thus, it was possible to import 1980 California census data in the precise format and layout of the 1990 data before it arrived so that the two files easily could be merged.\n\nWe then figured out what XDB commands we wanted to perform on the merged 1980 and 1990 data and saved them. For example, we stored queries in XDB that asked the computer to calculate the total percentage of minorities versus whites in the state, in each county and in each city.\n\nWe stored commands that asked the computer to calculate percentage growth for each minority group between 1980 and 1990. We even figured out the commands for our second- and third-day stories, which gave racial and ethnic snapshots of all nine Bay Area counties.\n\nThe preparation paid off. On the day the census information was to arrive, the tape drive broke down. Fortunately, we had logged enough practice hours with it to get it going again.\n\nThe census data didn't reach the paper until 9 p.m., nine hours after originally promised. By 9:30 p.m., about 15 minutes before the second-edition deadline, we were punching out numbers for our demographics writer and the graphics artists.\n\nThey were able to produce stories and graphics showing the percentage of minorities in each of the state's 58 counties, the growth in minority population between 1980 and 1990 and population totals for every city in the state.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-march-hartfords-toxic-front-page","slug":"1991-march-hartfords-toxic-front-page","url":"https://dwillis.github.io/uplink/article/1991-march-hartfords-toxic-front-page.html","year":1991,"month":"March","monthNum":3,"headline":"Hartford's Toxic Front Page","kicker":"","author_name":"Brant Houston and Kenton Robinson","author_title":"reporter","affiliation":"Hartford Courant","summary":"Connecticut journalists used the federal Toxic Release Inventory database to investigate unreported toxic emissions by manufacturers and significant gaps in state environmental protection reporting, revealing millions of pounds of undocumented chemical releases and regulatory failures.","keywords":["emissions","manufacturers","oversight"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Toxic Release Inventory","Nine Track Express","XDB","Powerbase"],"full_text":"You call this environmental protection?\n\n- Connecticut manufacturers are failing to report millions of pounds of toxic emissions to the state as required by law.\n- Connecticut manufacturers are actually emitting more ozone-eating Freon into the atmosphere than they were three years ago, when they pledged to reduce emissions.\n- Some manufacturers are seriously damaging municipal sewer treatment plants by dumping toxic chemicals into the plants, thus causing sewage to pour into Connecticut water and costing taxpayers tens of thousands of dollars in repairs.\n- Connecticut's Department of Environmental Protection has for three years left out tens of millions of pounds of toxic chemical emissions in its annual reports to the public; thus significantly downplaying the amount of poisons dumped and stored in local landfills.\n\nThese are some of the facts we've been able to reveal over the past six months with just a rudimentary use of the Toxic Release Inventory (TRI). The inventory is composed of annual reports from certain manufacturers on how much of 300 or so toxic chemicals they emit. While the reports have their shortcomings, they are good for giving an indication of how bad pollution is in your state and for cross-referencing with other files.\n\nWe began our work by getting computer tapes of the inventory for 1988 and with the use of Nine-Track Express and XDB setting up eight different tables of information on what toxic chemicals manufacturers in Connecticut reported emitting.\n\nWe thought of doing a story from those tapes last summer, but realized that the paper reports for 1989 had just been filed - or should have been - with the state Department of Environmental Protection.\n\nWe knew that the state would take months to type those reports into a database, so we decided to enter the information onto our own database using laptops. This had been done by several reporters (including Dave Davis at the Cleveland Plain Dealer, who gave us the idea) and environmentalists with good results, so we plunged in and created our own 1989 database of 1,500 reports and verified it over a period of three weeks by ourselves and with the help of a researcher. Our database was an abbreviated version, quite similar to the summaries issued on diskette by the National Technical Information Service. (Data entry is really a bother, but we actually got a lot of story ideas while looking at the reports and discovered many flaws in the reports.) We totaled the emissions, made a top-ten list of manufacturers and chemicals, etc., so we could put out a report six months ahead of the state's annual report. But while comparing our database with the 1988 tapes, which were from the federal EPA, we found that some manufacturers had neglected to send their reports to the state over the past years, meaning the state had underestimated emissions in its annual reports. We also found many shortcomings in how the state cracked-down on some manufacturers.\n\nFor data entry, we used an inexpensive, off the shelf software called Powerbase. It has a great function key for repeating duplicate fields in records, and it was small enough to fit on a single diskette and still leave half the diskette open for data.\n\nOver the past six months, The Hartford Courant's environmental reporter, Dan Jones, and I have produced several front page stories and a bunch of others by using TRI. At this point, I think we've just started to appreciate the possibilities.\n\nOver the past few months, we've matched the database with another state database on pollution from sewer treatment plants, used our database for daily stories and for a longer story on ozone eating chemicals, given information from it to many bureau reporters, and used it to analyze transfers of chemicals to landfills and private disposal treatment companies. The state finally came out with its report this month and we immediately did a story on how it downplayed emissions by leaving out 14 million pounds of toxic chemicals shipped offsite by manufacturers. We intend to get the 1989 federal EPA tapes because it will have more detailed information that we didn't have time to type in.\n\nIf you want to use TRI: The 1988 federal 4-volume set is available through both the National Technical Information Service (703) 487-4848, and MICAR (314) 882-0684. 1989 data is expected soon. State-wide data is also available through NTIS at $50 per state. State environmental protection agencies sometimes sell abbreviated data on diskette.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-march-plugging-in-the-dallas-morning-news","slug":"1991-march-plugging-in-the-dallas-morning-news","url":"https://dwillis.github.io/uplink/article/1991-march-plugging-in-the-dallas-morning-news.html","year":1991,"month":"March","monthNum":3,"headline":"Plugging In The Dallas Morning News","kicker":"","author_name":"Allen Pusey","author_title":"Assistant Projects Editor","affiliation":"Dallas Morning News","summary":"The Dallas Morning News is implementing a computer-assisted reporting program with a $15,000 budget, combining centralized database research functions with decentralized reporter training through hands-on projects. The initiative, inspired by an MICAR seminar, aims to integrate computer technology into the newsroom workflow despite infrastructure challenges.","keywords":["databases","newsroom","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["XDB","9-track tape","PC","Nine Track Express","Macintosh"],"full_text":"It would be an exaggeration to say I was pistol-whipped into attending a Missouri Institute for Computer-Assisted Reporting seminar in January.\n\nBut not much. That may have been an advantage, however, in our efforts to start up a computer-assisted reporting program here at The Dallas Morning News. Although we have yet to produce a single story - or download a single database - I think our program is going to be successful. Here's why:\n\nA visit here last year by Elliot Jaspin laid the groundwork. It created such enthusiasm throughout the reporting staff that databases and computers became a routine topic of discussion. There was no need to sell the program to the grunts. Elliot's enthusiasm caught the attention of several key editors. But, like other newspapers, the Dallas Morning News has always had trouble matching its computer capabilities with the needs of the staff. Management wanted an evaluation. I was the skeptic. And after a week at MICAR, my evaluation was simple: this stuff can work.\n\nBut there is more to it than that. I am convinced that computer-assisted reporting is not just a program, but rather a research and reporting technology that should be with us for a long time. And to that end, I have recommended a comprehensive infusion of computer-assisted reporting into the heart of the newsroom using a two-pronged approach based on a $15,000 budget.\n\nThe first is a centralized, research and maintenance function. Our newsroom library will be responsible for the logging and storage of the tapes, the maintenance of any large-scale databases and the research and development of new and useful computer-based tools. They were already doing that. This will be just one more (very powerful) tool. The library already had several computers: two large capacity 386s and a 486 on order. We have added a 1600/6250 tape drive, NineTrack Express and two copies of XDB. We are awaiting a 670 mb, 8mb RAM computer from a local \"brewery.\"\n\nThe second, and the hardest, is the creative reporting function. The enthusiasm is there, but the development of database skills throughout the newsroom is likely to be slow. We have decided to teach XDB through the development of small-scale computer-assisted reporting projects chosen very selectively. Reporters learn by doing - and such projects will give them both incentive and results.\n\nThere have been problems. For instance, our newsrooms have been converting to MacIntosh based PCs. In light of what we have learned from MICAR, we have had to reevaluate that program. For many reasons, I favor an integration of Apple and IBM.\n\nBut solving those problems has made our program more inclusive, and probably stronger in the bargain. We have pulled expertise from virtually every department to make things right. And when things go wrong, as surely we can expect, I think we can now expect widespread assistance and support.\n\nStarting, you know, can be such sweet sorrow. We'll keep you posted.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-march-startling-population-changes","slug":"1991-march-startling-population-changes","url":"https://dwillis.github.io/uplink/article/1991-march-startling-population-changes.html","year":1991,"month":"March","monthNum":3,"headline":"Startling Population Changes","kicker":"Success Story News","author_name":"","author_title":"investigative reporter","affiliation":"WRC-TV","summary":"Investigative reporter Chris Szechenyi won a National Headliner's Award for his series on faulty defibrillators created through FDA data analysis, but was laid off by WRC-TV on the same day due to budget cuts.","keywords":["defibrillators","investigation","layoffs"],"topics":["Investigative Methods","Health & Environment","Technology"],"technologies":["Food and Drug Administration's Medical Device Report"],"full_text":"Last December, UPLINK featured a story produced by Chris Szechenyi, an investigative reporter at WRC-TV in Washington, D.C.\n\nSzechenyi used a computer analysis of the Food and Drug Administration's Medical Device Report to create a multi-part series on faulty defibrillators, the device used to jump-start hearts that have stopped beating. NBC Nightly News featured his story.\n\nThis month, the story won the National Headliner's Award for investigative reporting by a television station. On the same day Szechenyi learned of the award, he was laid-off by his station due to budget cuts.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-april-new-york-times-audits-us-banks","slug":"1991-april-new-york-times-audits-us-banks","url":"https://dwillis.github.io/uplink/article/1991-april-new-york-times-audits-us-banks.html","year":1991,"month":"April","monthNum":4,"headline":"New York Times Audits U.S. Banks","kicker":"","author_name":"John Freed","author_title":"Database Editor","affiliation":"The New York Times","summary":"John Freed discusses the New York Times' investigative series on U.S. banking stability, utilizing federally available data to assess and report on potentially weak banks.","keywords":["banking","audit","data"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["9-track tape"],"full_text":"It all started routinely enough, with a boilerplate article on regional banking that appeared in our Business Day section. \"Bank losses in Northeast up 4 percent in quarter, or some such,\" we reported. \"Blah, blah, blah,\" I thought to myself as I read the article. \"What I want to know, if I have a checking account in Boston, is whether my bank is about to fail!\" It turns out I wasn't alone in that question. But one problem in an organization as large as The Times is figuring out just who should be asking the question. A few days after the \"bank losses\" story appeared, I went to our Washington Bureau to discuss computer-assisted journalism and how we could apply it. One of the people I spoke with was our Washington banking reporter. \"Wouldn't it be more interesting and informative,\" I asked, \"to list specific banks that might be in trouble? Or, for that matter, on shakiest footing?\" Everyone around the table nodded in agreement. But nobody seemed motivated to follow up. Back in New York, my boss, Dave Jones, an assistant managing editor, framed a series of questions: Are there standards, agreed on in the industry, to rate banks? Are the data on which the standards based publicly available? If we get it, should we print it? The last question was the easiest to answer: Yes. While we feared we might start a \"run\" on a bank, the overarching feeling was that insiders already knew which banks were weak; the only people in the dark were the depositors. At the same time, Business Day appointed a special projects editor. One of his first efforts was to coordinate a series on banking. And we found the right reporter to ask our questions: Mike Quint, our New York-based banking reporter. He quickly provided the other answers. Yes, there are standards that are generally applied. And yes, the data are available. Each bank in the United States falls under the purview of one or more Federal agencies: the Comptroller of the Currency, the Federal Reserve, or the Federal Deposit Insurance Corporation. As a result, four times a year each bank is required to file a detailed balance sheet and income statement with the government called a Call Report. About three months after the end of each quarter, the Call Reports are available on tape from the National Technical Information Service (NTIS) for $600. So in mid-January, we ordered the tape, with a target of starting our banking series in the last week of January. I advised the business section that such a timetable was tight, but I thought I could make it, depending on how \"clean\" the data were. The tape arrived Jan. 21, just six days before we were going to press. The documentation was in two separate books! And the government divides banks up by size, so the location, say, of the assets field would vary depending on the size of the bank. I'd need a few extra days. The series was rescheduled for Feb. 3. By the end of the week, though, I had key financial information available on each of the 13,000 banks in the country. It was a snap, then, to rate each on one of four scales that experts said were meaningful: capital-to-asset ratio; concentration of commercial real estate; percentage of loans in trouble; or insolvency. Since the figures were from Sept. 30, we thought we'd also check to see how many banks on our \"weak\" lists had actually failed in the intervening four months. About a dozen had, besides the dozen or so that were already in receivership on Sept. 30. We decided to make the lists of problem banks the centerpiece of the series. We produced lists of the 100 or so weakest banks by each of the standards, and sorted them by state and region. Then we started looking at banks that made more than one list. It turned out that two banks made all four! We sent our reporter to White River Junction, Vt., where one of the two, Valley Bank, was located. It was both a fascinating study and a warning of what can go wrong with a bank. Valley Bank was established in 1987, in the midst of the real estate boom. But because it was a new bank, the established developers wouldn't deal with them. So they lent to the more speculative ventures. When the market collapsed, a huge proportion of the Valley's loans defaulted. The bank's leadership was forced out. And today the Valley Bank is \"looking everywhere for capital.\" With the help of NineTrack Express to download the data and FoxBase (a clone of dBase) to do rankings, the listings were ready to run on Feb. 3, as planned. But the editing process being what it is you can find in the paper of Feb. 17.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-april-uplink-subscription-news","slug":"1991-april-uplink-subscription-news","url":"https://dwillis.github.io/uplink/article/1991-april-uplink-subscription-news.html","year":1991,"month":"April","monthNum":4,"headline":"UPLINK Subscription News","kicker":"","author_name":"","author_title":"","affiliation":"Missouri Institute for Computer-Assisted Reporting","summary":"UPLINK, a journalism newsletter with a rapidly growing readership, will begin charging a $15 annual subscription fee starting in June due to doubled production costs. Readers must subscribe by late June to continue receiving the newsletter.","keywords":["subscription","newsletter","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Each month, hundreds of journalists across the country are receiving and reading UPLINK. The newsletter's mailing list has almost doubled since its debut. Unfortunately, this means UPLINK's production costs have also doubled The Missouri Institute for Computer-Assisted Reporting can no longer afford to provide UPLINK for free.\n\nIn June, MICAR will require UPLINK's readers to pay a $15 subscription fee. This will entitle the subscriber to 1 year of UPLINK. All of UPLINK's readers will receive next month's issue, but MICAR will cease delivery of the June UPLINK to those who fail to subscribe by the last week of that month.\n\nA subscription form can be found on Page 4 of this issue. If there are any questions, call MICAR at (314) 882-0684. Thank You.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-april-mainframes-cpu-time-and-your-wallet","slug":"1991-april-mainframes-cpu-time-and-your-wallet","url":"https://dwillis.github.io/uplink/article/1991-april-mainframes-cpu-time-and-your-wallet.html","year":1991,"month":"April","monthNum":4,"headline":"Mainframes, C.P.U. Time and Your Wallet","kicker":"","author_name":"Valerie Brunell","author_title":"","affiliation":"MICAR","summary":"This article examines the costs and pricing structures involved in computer-assisted reporting, explaining different fee calculation methods from data processing centers and advising journalists on how to evaluate service options based on their needs and budget.","keywords":["costs","computing","pricing"],"topics":["Investigative Methods","Technology","Business & Economy"],"technologies":["mainframe","PC"],"full_text":"This is part one of a two-part series on costs related to computer-assisted reporting. Next month's article will discuss state Sunshine laws in relation to charges for duplication of records onto computer tapes. Just because you don't have all the equipment to do data analyses from your office doesn't mean that you have to forego all computer-assisted reporting projects. Many data processing centers can do the work for you or provide you with access to the equipment you need. Even if you do have your own system, it's a good idea to know what costs are involved when purchasing a data tape, whether it be through a private company or an agency governed by the state. What you have to consider is what resources you have available, what type of work is being done, and how much you are willing to pay for those services, says Bill Plummer, director of campus computing at the University of Missouri. Many private companies base prices on the type of processing being done, says Mike Dalleska of the Word Processing and Information Services, Inc. in Cincinnati. \"No two projects are alike,\" he says. Making matters worse for price shoppers is that no standardized formula for rates has been established in the data processing field. Some agencies base rates on the amount of use they get from the system and your share of that total use. Others base prices on the amount of information being manipulated, whether it be on a per-record basis or by computer bytes used. Costs based on your share of the system are usually listed as CPU time. The CPU is the computer's central processing unit that directs its logical and mathematical functions. Other charges, such as input and output time or access charges, may be included, but the main cost will be calculated in minutes used. CPUs can be calculated either as computer time or wall clock time, says Jerry Fox, a marketing representative for IBM. With computer time, prices are high but only include the amount of time that a computer is actively running a command. Depending on the type of computer system and the complexity of the command, most computers can complete a task within a matter of seconds. With wall clock time, rates are low; users are charged from the time they sign on to the system until they sign off, regardless of whether or not the computer is active or sitting idle. When contracting out for services, consider the type of system being used and its ability to process information. A project that takes only an hour on a mainframe computer may take days or even months on a personal computer. Most importantly, know specifically what is included in the prices quoted. You may be paying for something you shouldn't.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-april-tape-drive-trouble-youre-not-alone","slug":"1991-april-tape-drive-trouble-youre-not-alone","url":"https://dwillis.github.io/uplink/article/1991-april-tape-drive-trouble-youre-not-alone.html","year":1991,"month":"April","monthNum":4,"headline":"Tape Drive Trouble? You're Not Alone.","kicker":"","author_name":"Adam Berliant","author_title":"Editor","affiliation":"Missouri Institute for Computer-Assisted Reporting","summary":"Reporters using Qualstar 1260 tape drives are experiencing frequent malfunctions characterized by drives rocking back and forth, with the Missouri Institute for Computer-Assisted Reporting receiving numerous complaints about the issue. The problems are typically caused by dirty heads or faulty head connectors, particularly involving defective cable connectors that Qualstar switched to approximately a year ago, with replacement cables reportedly solving about half of the reported cases.","keywords":["tape","drives","problems"],"topics":["Technology","Investigative Methods"],"technologies":["9-track tape","magnetic tape"],"full_text":"During the past several months, the Missouri Institute for Computer-Assisted Reporting has received an increasing number of phone calls from reporters having problems with their tape drives. The complaints have been similar, \"Sometimes the tape drive would do what I asked, but often it would just rock back and forth interminably,\" said Bob Paynter of the Akron Beacon Journal. He had a Qualstar tape drive that \"never seemed to work.\" \"I didn't know what was wrong with it,\" Paynter said. \"It was like asking a kid if his first car was operating properly. I just didn't know.\" Few pieces of electronic equipment could be more foreign to reporters than 9-track tape drives. Fortunately, finding the source of the problem Paynter described is not difficult. Chances are it's either caused by dirty heads or a bad head connector, say the drive's manufacturer, Qualstar, and one of its brokers, Chi Corp. Qualstar is the maker of the model 1260 tape drive, the inexpensive, 6250 bpi-capable drive used by Paynter and other reporters, many of whom purchased their drives through Chi Corp. Chi manufactures the controller card and software that go inside the PC to make the tape drives and computers communicate. Chi's emphasis on the U.S. PC market makes it the company many reporters turn to. Representatives from both of these companies say that rocking problems can be handled. This should be late but great news for Judy Miller, special projects editor at the San Francisco Chronicle, who is now in possession of her fourth Qualstar tape drive. Two of her first three were sent back to the manufacturer because they wouldn't function properly. \"Our first drive didn't work, so we sent it back,\" Miller said. \"Chi Corp. was really good to us, and they sent us another one that worked at first. Then census data arrived, and it started to rock back and forth. 'C3' came up - head error. I called Qualstar and talked to the technician who walked me through cleaning and reseating, and it worked again. But then the next day it just consistently started rocking.\" \"They sent a third one,\" Miller said. \"That one didn't work at all, ever.\" The problems described here are typical of \"read\" and \"write\" problems. Chi's software will call these C0, C1, C2, C3 errors or 40, 41, 42, 43 errors. Chi, too, has been receiving complaints about write and read errors for the past year. \"I think Qualstar switched (cable) brands about a year ago,\" Chi's John Thome said. \"It's not a problem with the head. The green card-edge connector has a ribbon that fastens onto it. That seems to be the problem. We've been sending people new cables, and that has solved 50 percent of the problems.\" The \"green card-edge connector\" lies beneath the head protector on top of the head. One can pull the head cover off and see the connection in question. Look for the vertical green card, and you will see a ribbon of O wires connected to it. If the connector is blue, it is an Ansley connector, one of the most reliable connectors made. If it is not blue, it may be one of the other connectors Qualstar was experimenting with. This doesn't always mean the connector is the problem. It's more likely to be dirty tape heads. \"The most common problem for tape drive malfunctions is dirt accumulating on the heads,\" said Dan Thorlakson, vice president of operations at Qualstar. Heads do need to be cleaned periodically; Thorlakson suggests after every two hours of use. If cleaning fails, both Thome and Thorlakson suggest removing and \"reseating\" the connection. In other words, unplug it and then re-plug it in. If this still does not work, you may have to do what Judy Miller had to do: return the drive so the connectors can be replaced. The Qualstar customer service line is always an excellent first step for solving tape drive difficulties. The number is (818) 882-5822.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-april-cpu-time-cont","slug":"1991-april-cpu-time-cont","url":"https://dwillis.github.io/uplink/article/1991-april-cpu-time-cont.html","year":1991,"month":"April","monthNum":4,"headline":"C.P.U. Time Cont.","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"When purchasing data processing services from private companies or state agencies, organizations must carefully compare pricing models, which vary based on CPU time, system usage, or data manipulation, and understand exactly what services are included in quoted rates.","keywords":["pricing","processing","services"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["mainframe","PC"],"full_text":"purchasing a data tape, whether it be through a private company or an agency governed by the state.\n\nWhat you have to consider is what resources you have available, what type of work is being done and how much you are willing to pay for those services, says Bill Plummer, director of campus computing at the University of Missouri.\n\nMany private companies base prices on the type of processing being done, says Mike Dalleska of the Word Processing and Information Services, Inc. in Cincinnati. ``No two projects are alike,\" he says.\n\nMaking matters worse for price shoppers is that no standardized formula for rates has been established in the data processing field. Some agencies base rates on the amount of use they get from the system and your share of that total use. Others base prices on the amount of information being manipulated, whether it be on a per-record basis or by computer bytes used.\n\nCosts based on your share of the system are usually listed as CPU time. The CPU is the computer's central processing unit that directs its logical and mathematical functions. Other charges, such as input and output time or access charges, may be included, but the main cost will be calculated in minutes used.\n\nCPUs can be calculated either as computer time or wall clock time, says Jerry Fox, a marketing representative for IBM. With computer time, prices are high but only include the amount of time that a computer is actively running a command. Depending on the type of computer system and the complexity of the command, most computers can complete a task within a matter of seconds. With wall clock time, rates are low; users are charged from the time they sign on to the system until they sign off, regardless of whether or not the computer is active or sitting idle.\n\nWhen contracting out for services, consider the type of system being used and its ability to process formation. A project that takes only an hour on a mainframe computer may take days or even months on a personal computer.\n\nMost importantly, know specifically what is included in the prices quoted. You may be paying for something you shouldn't.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-april-letter-to-the-editor","slug":"1991-april-letter-to-the-editor","url":"https://dwillis.github.io/uplink/article/1991-april-letter-to-the-editor.html","year":1991,"month":"April","monthNum":4,"headline":"LETTER TO THE EDITOR","kicker":"","author_name":"Christopher H. Schmitt","author_title":"Reporter","affiliation":"San Jose Mercury News","summary":"The San Jose Mercury News responds to criticism in UPLINK's March issue, defending their superior coverage of California census data with more detailed statistics and faster reporting of demographic shifts than the San Francisco Chronicle.","keywords":["census","journalism","coverage"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Dear UPLINK,\n\nTsk, tsk, tsk. Slighting the Mercury News the way the (March) issue of UPLINK did. Well, the best we can do now is try to set the record straight. It's so dissappointing when the press screws up, no? Just kidding. Anyhow, (enclosed) are the copies of the San Francisco Chronicle and the San Jose Mercury News for the two days following the release of the California census tape. You make the call, as they say, but here are several things we modestly point out:\n\n- We think our statistical material was more detailed and better presented.\n- It took the Chron until the second day to find out San Francisco had lost an ethnic white majority. We reported the first day that four of five California's largest cities had lost majority.\n- The Chron's first-day statistics were 1990 based. We had a 1980-1990 comparison the first day.\n\nAs my editor suggested, we're willing to let this slight pass.\n\nBest Regards, Christopher H. Schmitt, San Jose Mercury News\n\nImagine that. A Californian computer-assisted reporting war fought here in the meek pages of UPLINK. As for making \"the call\" on the Census packages produced by The Chronicle and the Mercury News, UPLINK has decided to take a flimsy but conveniently diplomatic position. They were both great. The editor","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-may-transporting-toxics-through-california","slug":"1991-may-transporting-toxics-through-california","url":"https://dwillis.github.io/uplink/article/1991-may-transporting-toxics-through-california.html","year":1991,"month":"May","monthNum":5,"headline":"Transporting Toxics Through California","kicker":"","author_name":"Christopher Schmitt","author_title":"","affiliation":"San Jose Mercury News","summary":"A four-part series reveals numerous flaws in California's toxic waste monitoring system, including unaccounted waste and regulatory failures.","keywords":["toxic waste","California","environment"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Paradox","PC","XDB"],"full_text":"Cradle to grave. To spare the environment and safeguard public health - that's the way toxic waste is supposed to be monitored, from when it's created until it safely meets its demise. On paper, environmentally conscious California has one of the toughest systems in the nation for keeping track of toxic waste. But in a recent four-part series that drew heavily on computer-assisted reporting, the San Jose Mercury News documented big problems in monitoring and management of toxic waste: - California's system for keeping track of 2 million tons of waste a year is riddled with so many holes that it can't account for where hundreds of thousands of tons of waste go each year. - Although notoriously illegal dumpers might be caught with the help of some of the tracking information, regulators have a policy to ignore tipoffs the system offers. - The biggest violator of waste disposal reporting rules is the state itself. - Recycling of toxics often is a sham, nothing more than disposal in disguise. - Waste generators have roundly failed in efforts to cut down the torrent of waste they produce - even though there's clear evidence such cuts can be made profitably. Key to making our project work were 410,000 computerized records detailing waste shipments for 1989, the latest year for which data were available. These records, not to be confused with TRI [3], were from shipping documents known as manifests. They offered a load-by-load look at waste disposal. This information created big databases - one with about 60 megabytes, another with 30. To slice through it all, we used a 33-megahertz, 80386-based IBM PC clone with a 360 MB hard disk drive. (When we were done, we had pretty much filled up the drive.) We didn't use the MICAR-touted XDB database management software, instead opting for Paradox version 3.5. We can confirm recent highly favorable computer magazine reviews of Paradox: It's quite speedy and offers excellent, easy to learn Query-By-Example system. \"When we matched a list of Superfund sites against waste destinations, we identified dozens of tons of waste that had been illegally shipped.\" Some of the most productive computer work on this project came in matching one set of data against another. California has a law that prohibits transport of hazardous waste on state highways if the destination is a federal Superfund site. When we matched a list of Superfund sites against waste destinations, we identified dozens of tons of waste that had been illegally shipped. When we asked one of the biggest offenders about their practice, they immediately stopped. We also used matching to dispute state officials' claim that their tracking system didn't offer information of value for cracking down on toxic violators. We took a sample of 36 enforcement cases in one year, then looked back to the previous year to see whether the tracking system had pointed out any problems with these firms. It had - 4,978 different times. In one case, we found that one of the biggest hazardous waste cases in years - involving smuggling of waste into Mexico - could have been cracked months earlier if regulators had paid attention to tracking system information. You can do what we did, but outside California, information may be more limited. For any questions about what we did, please feel free to call: (408)-920-5048.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-may-uplink-subscription-news","slug":"1991-may-uplink-subscription-news","url":"https://dwillis.github.io/uplink/article/1991-may-uplink-subscription-news.html","year":1991,"month":"May","monthNum":5,"headline":"UPLINK Subscription News","kicker":"","author_name":"The Editor","author_title":"Editor","affiliation":"Missouri Institute for Computer-Assisted Reporting","summary":"The Missouri Institute for Computer-Assisted Reporting's UPLINK newsletter is transitioning from free to paid distribution, requiring a $15 annual subscription to continue receiving issues.","keywords":["subscription","newsletter","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"THIS IS YOUR LAST ISSUE unless you have paid for your UPLINK subscription. Because of the Missouri Institute for Computer-Assisted Reporting's growing mailing list and high production costs, UPLINK will no longer come to you free of charge. Your June issue will not arrive without a subscription. A year-long subscription costs $15. MICAR believes that this fee will be enough to cover our costs. If you wish to continue receiving UPLINK, fill out the subscription form on Page 4. We hope you will continue to read and contribute to UPLINK. Checks should be made out to the University of Missouri and sent to the address found on the subscription form. Thank you,","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-may-cpu-time-part-two","slug":"1991-may-cpu-time-part-two","url":"https://dwillis.github.io/uplink/article/1991-may-cpu-time-part-two.html","year":1991,"month":"May","monthNum":5,"headline":"C.P.U. Time, Part Two","kicker":"","author_name":"Tom Braden","author_title":"","affiliation":"MICAR","summary":"Government agencies are charging exorbitant fees for electronic public records, and journalists can challenge these costs by analyzing the agencies' own computer cost accounting formulas to determine if prices reflect actual expenses.","keywords":["records","costs","agencies"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["mainframe"],"full_text":"Last year the Dayton Daily News asked the state's Department of Motor Vehicles to provide computer tapes of drivers license records. The tapes contain about seven million records. The state agency said it would provide the tapes at a cost of $3 per record - a mere $21 million. Similarly, the Hartford Courant recently sought death certificate records from the state's Department of Health and Human Services. The agency wanted $10,000 for the records. These are just two examples of a trend that is occurring with increasing frequency throughout the nation: Government agencies, suddenly besieged by journalists seeking computer tapes, are finding ways to keep information from the public by charging absurd amounts for making electronic copies of information. This means reporters need a way to analyze costs and determine if public record custodians are gouging them with outrageous prices. Government agencies using mainframe computers generally use Central Processing Unit (CPU) time to allocate costs. A mainframe computer's CPU performs all the computer's data movement. Agencies charging CPU time do so because it allows them to perform several different jobs at once. These agencies use operating system software that have special utilities, such as accounting programs, to account for and charge different users. To recover the costs associated with operating a mainframe computer, government agencies and data centers first identify these costs, then make a volume estimate, using CPU time, on how much the mainframe is used in a year. They divide the volume estimate into the cost to come up with a rate. This process, called computer cost accounting, allows the mainframe operator to take into account all of the overhead costs, such as the cost of the mainframe itself, labor, utilities and rent, in establishing rates for users. An organization quoted a price of $600 per minute of CPU time, which actually occurred in Boston, can use the agency's own formula to determine if the rate reflects the actual cost to the agency. One would multiply $600 by the number of minutes in a year -- 525,600. Multiplying that figure by $600 gives you $315 million. The question then becomes, how does that match the agency's known operating costs? The news organization requesting computer tapes must match the agency's quoted cost figure against the amount of CPU usage in a given year just as the agency itself does in allocating costs. If the mainframe is active, say, 50 percent of the time, obviously, it does not cost the agency more than $157 million to operate its system in a given year. Journalists must not acquiesce to the outrageous fees for duplicating computer tapes. Case law involving the cost of such records remains scant. But as news organizations accelerate their pursuit of public records stored in electronic format, the number of cases going to court is sure to increase. What is desperately needed is some kind of uniformity among states as well as among different government agencies within a given state as to what electronic records should cost the public. To use computerized records as a means of increasing state revenue obviously contradicts the intent of freedom of information laws, which is to ensure the public access to government information so that it may exercise its duty of self-government.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-may-using-data-at-the-nations-data-capital","slug":"1991-may-using-data-at-the-nations-data-capital","url":"https://dwillis.github.io/uplink/article/1991-may-using-data-at-the-nations-data-capital.html","year":1991,"month":"May","monthNum":5,"headline":"Using Data at the Nation's Data Capital","kicker":"","author_name":"Karl Ross","author_title":"","affiliation":"MICAR","summary":"Washington D.C. news organizations have access to vast government computer databases but underutilize computer-assisted reporting due to staff constraints, reliance on traditional reporting methods, and lack of full-time technical resources, though some outlets like Cox Newspapers and USA Today are beginning to invest more aggressively in the technology.","keywords":["reporting","computers","Washington"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["CD-ROM"],"full_text":"There is no greater concentration of computerized information than in Washington, D.C. But strangely, in this capital of government record keeping, there seems to be a lack of computer-assisted reporting. Scripps-Howard Bureau Chief Al Thompson says that although he senses a growing interest in accessing government computer files, computer-assisted reporting is still a marginal technique at his office. \"We have our hands full just doing what we're doing day-to-day putting out the news, so we don't have a lot of time to discover what's out there and what's available,\" Thompson says. Rich Morin, who heads up the computer-assisted effort at The Washington Post, suggests that because most news organizations in Washington, D.C., are bureaus for out-of-state papers and national magazines, they are geared toward beats using more traditional means. \"There's quicker ways to get some of this information,\" he says, \"though certainly not all of it.\" Tom Moore, formerly a Washington-bureau reporter for Knight-Ridder and now a graduate student at George Washington University, offers a more sociological explanation. He says that for many reporters, a position at the Washington bureau represents a hard-won plum, topping off a successful career in the newsroom. Most Washington bureau reporters, Moore says, are older, more prosperous and more at ease using conventional reporting techniques. \"There are exceptions,\" he says, \"but the last time I looked, reporters in Washington were very far behind in information technology.\" Most major news organizations have appropriate technology at their disposal but don't employ it. Knight Ridder Bureau Chief Clark Hoyt says his office is equipped with computers and they try to keep a running database on campaign finances but they don't have anybody to work with these resources on a full-time basis. If computer assisted reporting has yet to realize its potential in Washington, Hoyt says, \"It's not because it's not a good tool. It's because we as an industry haven't taken full advantage of it. Cox Newspapers' Andrew Glass wants to change this. Cox recently invested in a CD-ROM player for its Washington bureau and has taken a more aggressive approach to the technology. \"It's something we feel the time has come for,\" Glass says. He says that instead of hiring some computer-jock to handle the chores for the office, he would like to see all staffers get involved to the extent that they are able. \"We think it should be part of everybody's job, just like knowing how to type.\" Even though a multitude of think-tanks and public interest groups are providing the media with their own ready-made analyses of government files, Glass questions the sagacity of relying on such \"canned\" sources. \"It's garbage-in, garbage-out if you're not sure how you came by the data,\" Glass says. Another Washington-based organization that is counting on computers to give it a jump on the competition is Gannett's USA Today. The special projects department created a special computer-assisted unit in 1988. Julia Wallace, head of special projects, says the investment has yielded a number of significant stories from dangerous roads to troubled banks. The special unit has been especially busy cranking out customized reports comparing 1980 and 1990 census data. Wallace warns newcomers that it is not as easy as it seems. \"Logically, you think you ought to be able to press a button and get the answer right away,\" she says, \"but the reality is things take twice as long and are twice as hard as you expect them to be.\" \"But we're at the point where we've been through those learning stages. It's a lot more fun because now we're able to do things a lot quicker and things that are a lot more complicated.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-may-tri-working-guide-news","slug":"1991-may-tri-working-guide-news","url":"https://dwillis.github.io/uplink/article/1991-may-tri-working-guide-news.html","year":1991,"month":"May","monthNum":5,"headline":"TRI Working Guide News","kicker":"","author_name":"","author_title":"","affiliation":"MICAR","summary":"A new Reporter's Working Guide to the Toxic Release Inventory provides step-by-step instructions for journalists to download, organize, and analyze EPA pollution data to create meaningful environmental stories. MICAR offers the guide in various package options ranging from $30 to $500, including database access and technical tools.","keywords":["pollution","database","reporters"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["SQL","9-track tape"],"full_text":"The Environmental Protection Agency's Toxic Release Inventory (TRI) has been one of the most popular databases used by the readers of UPLINK. It has been the source of excellent industrial pollution stories in Florida, California, Connecticut, Ohio and elsewhere.\n\nBut many reporters still have not used the database.\n\nIn June, N.T.I.S. will make the 1989 TRI available on computer tape. We have found a way to make using these tapes irresistible. MICAR has produced A Reporter's Working Guide to the Toxic Release Inventory.\n\nThis short, step-by-step guide helps reporters download TRI into their computers, organize the data so that is usable, and manipulate the data to produce meaningful reports.\n\nAnd because specific SQL commands are included, reporters using the guide will be able to: identify the major polluters in their area; list the quantities of toxic chemicals discharged by state, county or zip code; identify the most polluted rivers and streams in their area; find out what toxic chemicals are used for; list the most common types of discharges and the list goes on.\n\nThe guide, which is really all you need if you already own the TRI database, costs $30 by itself. But for $50, MICAR will also include the NineTrack Express record profiles needed to download TRI.\n\nIf you do not own TRI, cannot afford it, or simply do not have a tape drive, for $350 MICAR will also supply TRI data concerning your entire state on 3.5\" diskettes. These diskettes contain unabbreviated data, unlike that which N.T.I.S. sells.\n\nFinally, if you wish to purchase the entire TRI database on computer tape, MICAR can supply it for $500. To order, call MICAR at (314)-882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-may-reporting-position-in-ohio","slug":"1991-may-reporting-position-in-ohio","url":"https://dwillis.github.io/uplink/article/1991-may-reporting-position-in-ohio.html","year":1991,"month":"May","monthNum":5,"headline":"Reporting Position in Ohio","kicker":"","author_name":"","author_title":"","affiliation":"Dayton Daily News","summary":"The Dayton Daily News is hiring a projects reporter with investigative skills and computer-assisted reporting expertise, requiring applicants to submit resumes and clips to the listed address.","keywords":["reporting","investigative","position"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"The Dayton Daily News is accepting applications for a projects reporting position. Candidates must have demonstrated investigative reporting skills and an aptitude for using personal computers as a reporting tool. In the past year we have done computer-assisted reporting projects on discrimination in court sentencing, campaign finance and safety in the workplace. We have sent reporters through courses at the MICAR and have worked with faculty members of the National Institute for Advanced Reporting at Indiana University. Applicants should send resumes and clips to Jim Ripley, executive metro editor at: Dayton Daily News Fourth and Ludlow Streets Dayton, OH 45402","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-may-uplink-subscription-form","slug":"1991-may-uplink-subscription-form","url":"https://dwillis.github.io/uplink/article/1991-may-uplink-subscription-form.html","year":1991,"month":"May","monthNum":5,"headline":"UPLINK SUBSCRIPTION FORM","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This is a membership or registration form for The Missouri Institute for Computer-Assisted Reporting requesting personal information and a $15 payment sent to the University of Missouri School of Journalism.","keywords":["registration","journalism","payment"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Name:\n\nTitle:\n\nOrganization:\n\nAddress:\n\nPlease send this form and $15 dollars to The Missouri Institute for Computer-Assisted Reporting, University of Missouri School of Journalism, 120 Neff Hall, Columbia, MO, 65211. Make Checks Payable to The University of Missouri. Thank you.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-june-the-safe-and-the-sorry-working-in-ohio","slug":"1991-june-the-safe-and-the-sorry-working-in-ohio","url":"https://dwillis.github.io/uplink/article/1991-june-the-safe-and-the-sorry-working-in-ohio.html","year":1991,"month":"June","monthNum":6,"headline":"The Safe and the Sorry, Working in Ohio","kicker":"","author_name":"Mike Casey","author_title":"Reporter","affiliation":"Dayton Daily News","summary":"A series on workers' safety, based on OSHA's database, reveals systemic failures in workplace safety regulations and enforcement in Ohio, highlighting minimal penalties for violations.","keywords":["OSHA","workplace safety","violations"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["9-track tape","mainframe"],"full_text":"In the fall of 1989, an Occupational Safety and Health Administration official in Cincinnati told me that the agency kept its investigations on computer tape.\n\nThat got me thinking about using OSHA's database in a project about workers' safety.\n\nThe Dayton Daily News' workers' safety series ran from June 2 through 6 this year, following a successful 10-month fight with OSHA to release its database.\n\nThe series findings were:\n\n- An employer paid a $2 fine in the deaths of two workers in Georgia, even though the employer was cited with two life-threatening violations.\n- OSHA's median fine in accidents that kill and maim workers was $500 from 1972-1990. When adjusted for inflation, fines in 1972 were more than twice as high as those in 1990.\n- The 10 Dayton-area companies with the most OSHA violations.\n- The nation's worksite with the most fatal and serious accident investigations was Bethlehem Steel Corp.'s mill and shipyard near Baltimore.\n- OSHA has taken years to implement tougher standards despite workers' deaths.\n\nIn December 1989, I sent a Freedom of Information Act request to OSHA for its southwestern Ohio inspection records on tape.\n\nHowever, the agency sent boxes containing a 5-foot high stack of paper records for southwestern Ohio. The agency said the 25,000 paper records of OSHA inspections met the newspaper's request for the computer tape. The Dayton Daily News disagreed and filed a FOIA appeal with the Labor Department's Office of the Solicitor.\n\nThe newspaper also enlisted the aid of Sen. Howard M. Metzenbaum, D-Ohio, who chairs the Senate Labor Subcommittee.\n\nMetzenbaum wrote the agency a letter saying the OSHA's reason for denying the request for the computer tape did not meet the exemptions under the FOIA.\n\nIn May 1990, OSHA agreed to turn over its tapes for southwestern Ohio. Four months later, it agreed to turn over the national database.\n\nI used a Dell desk-top computer at the newspaper to analyze the southwestern Ohio records.\n\nThe Missouri Institute for Computer-Assisted Reporting helped by providing software to read the tape.\n\nOSHA investigations with information such as company names, types of violations and amounts of fines. The records came on 1/2 inch, 6250 bpi tape. The record length was 23,399 and the block-size was 23,403.\n\nLater, I asked for the 1.8 million record national database. OSHA's response to the newspaper's request was that it would not hand over the tapes because the newspaper might come to conclusions different than OSHA's.\n\nTo analyze the national database, the newspaper received help from a local University, Wright State University, which provided its mainframe computer and technical assistance.\n\nThe computer records sketch the findings of OSHA investigations with information such as company names, types of violations and amounts of fines.\n\nThe computer pointed toward problems. Daily News reporter Russell Carollo and I conducted scores of interviews and reviewed thousands of pages of records to flush out the picture of workplace dangers.\n\nSome things I learned from working on this project:\n\n- It's useful to learn about the agency before receiving the computer tapes. That way you understand some of the agency's terminology that are used to describe computer fields.\n- The computer allows medium-sized newspapers to tackle tough national stories.\n- Universities can provide newspapers with the necessary technical assistance in computer projects.\n- The data analysis people at OSHA were helpful in explaining the database. I suspect other agencies have people who will be just as helpful.\n- There is a lot of frustration with using a computer. For example, the first set of national tapes that OSHA sent the newspaper were faulty. We had to reapply to get new tapes, which took an extra six weeks.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-june-connecticut-legislature-cuts-the-red-tape","slug":"1991-june-connecticut-legislature-cuts-the-red-tape","url":"https://dwillis.github.io/uplink/article/1991-june-connecticut-legislature-cuts-the-red-tape.html","year":1991,"month":"June","monthNum":6,"headline":"Connecticut Legislature Cuts the Red Tape","kicker":"","author_name":"Peter Viles","author_title":"","affiliation":"Associated Press","summary":"Connecticut has passed landmark legislation guaranteeing public access to government records stored in computerized form, allowing citizens and news organizations to request data in their preferred format while preventing governments from charging excessive fees or signing exclusive contracts that would restrict access.","keywords":["access","records","government"],"topics":["Government & Politics","Journalism & Newsroom","Data & Databases"],"technologies":["floppy disk","magnetic tape"],"full_text":"Connecticut is poised to become a national leader in ensuring public access to computerized government records, a crucial freedom-of-information issue as governments shift their records from paper to computer tapes and disks.\n\nLegislation given final passage in the state Senate June 3 provided that any citizen or news organization requesting computer-stored public documents of information can get the data in the form desired - for example on floppy disk, computer tape or printout. Further, it would block governing bodies from signing exclusive contracts with private firms that would sell government records to the public. It also requires that governments keep the state Freedom of Information law in mind when they construct computer programs for storing and retrieving data.\n\n\"I think this is real landmark legislation in the United States, if not the western world,\" said Mitchell W. Pearlman, executive director of the state Freedom of Information Commission.\n\nThe state Senate unanimously passed the bill without debate Monday, sending it to Gov. Lowell P. Weicker Jr. The measure had cleared the House, 124-21.\n\n\"It's absolutely critical legislation,\" said Timothy B. Gassert, director and counsel for the Division of Public Records in Massachusetts. \"The Connecticut example is one that should be followed by all the states, and by the federal government too.\"\n\nThe bill addresses problems that have surfaced as governments have increased their reliance on computers. News organizations and other groups increasingly want government records in computerized form, which makes it easier to search through the records without leafing through mounds of paper.\n\nBut governments often don't oblige.\n\n\"If someone wants to search government files to see, for example, whether the government is paying the same person twice for the same job, you can find that easily with fairly simple computer search techniques,\" Pearlman said.\n\nIn some cases, they offer documents only in printed copies - expensive and insurmountable reams of paper. In other cases, they charge artificially high fees for shifting information to a computer format the public can use.\n\n\"We're entering into a new era of government, an electronic bureaucracy,\" said Gassert, the Massachusetts official. \"And ironically, that kind of modernization is being used by many in government to restrict access to information that is supposed to be available to the public.\"\n\nThe bill enforces governments to honor any reasonable request for computer-stored public information. For example, a reporter who wants population statistics on floppy disk could get them, provided he or she was willing to pay a fee for the cost of transferring the data from whatever format the government uses. Governments could charge fees for shifting the data, but only within strict guidelines intended to prevent governments from artificially limiting access.\n\nThe bill would not force governments to increase their reliance on computers, and would not make available any information that is now kept private.\n\nThe most forward-looking aspect of the Connecticut bill requires government agencies to consult with a central state information office before buying new computer systems.\n\nMany existing government programs do not separate public and private information, and the unclassified data cannot be easily pulled out.\n\nThe bill would take effect on July 1, 1992.\n\n\"It's a recognition that computerized records have particular benefits and problems associated with them,\" Pearlman said. \"This bill takes the first steps toward dealing with those problems.\"\n\nBrant Houston, a reporter for the Hartford Courant and frequent contributor to UPLINK, testified in front of the Connecticut legislative committee responsible for Freedom of Information laws earlier this year. Houston did this at the request of a coalition of news organizations. Houston told the legislators the difficulties he had encountered when trying to access electronic records, and he described some of the \"silly excuses\" government record keepers used to withhold information. Judging by the bill described in this article, it seems Houston's testimony paid off.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-june-still-plugging-in-the-palm-beach-post","slug":"1991-june-still-plugging-in-the-palm-beach-post","url":"https://dwillis.github.io/uplink/article/1991-june-still-plugging-in-the-palm-beach-post.html","year":1991,"month":"June","monthNum":6,"headline":"(Still) Plugging In The Palm Beach Post","kicker":"","author_name":"Jimmy Montague","author_title":"","affiliation":"MICAR","summary":"Paul Martin, a young journalist at the Palm Beach Post trained in computer-assisted journalism, finds the newspaper's CAJ program stalled due to recession-driven budget cuts despite early success with an award-winning property tax investigation. Both Martin and management remain optimistic that the program will resume once the economy improves.","keywords":["recession","journalism","budget"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["magnetic tape"],"full_text":"If you ask Paul Martin to describe the problems associated with starting a computer-assisted, investigative journalism program these days, his reply would consist of two words - \"The recession.\"\n\nSince Martin has been assigned to cover city government at the Palm Beach Post bureau in Port St. Lucie, Fla., the paper's computer-assisted journalism program would seem to be dead in the water. \"I wouldn't even call it a program at this point,\" Martin said.\n\nMartin got his bachelor's degree from the University of Missouri's School of Journalism in May, 1990. After graduation, he did an internship at The Post from June until October. The paper hired him full-time in November.\n\nMartin was trained in computer-assisted journalism at M.U. and has already used the techniques he learned there in helping with one award-winning piece for his new employer.\n\nLast year, while still an intern working with former Projects Editor Rich Gordon, Martin says he did the computer search that yielded an outstanding series on property tax appraisals in Palm Beach County, Fla. (See \"Taxing Investigation in Palm Beach,\" UPLINK December 1990)\n\nRecently, that series was awarded third prize by The Florida Society of Newspaper Editors in that organization's public service category for newspapers over 100,000 circulation.\n\nNow comes the tough stuff.\n\nEven before the award was announced, Rich Gordon (also computer-trained at M.U.) left The Post. He was hired away by the rival Miami Herald.\n\nGordon's position is now held by Projects Editor John Bartosek. Bartosek, also an assistant managing editor, now wears two hats.\n\nSince its initial success, the paper has used computer analyses on census stories and drought coverage, but those were not projects that required a relational database, said Martin.\n\nMartin attributes the lack of computer-assisted reporting at The Post to a budgetary crunch induced by plummeting ad sales. He feels lucky because The Post hasn't had as many personnel cuts as some newspapers have.\n\nSaid Martin, \"When you're having to cut staff from the features department it's difficult to convince people to create new positions in the metro department. People started saying 'Oh, gee! $5,000 for a tape drive. $1,000 for a training seminar.'\"\n\nBut Martin is an optimist.\n\n\"I think that when we come out of this recession, computer-assisted journalism is going to be one of the first areas management is willing to put money into,\" he said.\n\nManaging editor Tom O'Hara concurs with Martin's diagnosis. \"I'd like to do more [computer-assisted journalism],\" O'Hara said.\n\n\"Two years ago we had expansion plans. But now it's just not an environment in which you can let someone go for a couple of months to digest a database,\" he said.\n\nHe, too, is cautiously optimistic. Said O'Hara: \"I don't know what the future holds. If things pick up I'd like to get back into it but, for now, everything's on hold here.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-june-new-york-times-stops-playing-catch-up","slug":"1991-june-new-york-times-stops-playing-catch-up","url":"https://dwillis.github.io/uplink/article/1991-june-new-york-times-stops-playing-catch-up.html","year":1991,"month":"June","monthNum":6,"headline":"New York Times Stops Playing \"Catch-up\"","kicker":"","author_name":"John C. Freed","author_title":"Database Editor","affiliation":"The New York Times","summary":"A newsroom used computer analysis to investigate Congressional speaking fees, moving beyond simple totals to identify patterns, top earners, and regional breakdowns that surpassed competitors' reporting.","keywords":["Congress","investigation","computer"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"Last year, as best as we can determine, the total amount received by members of the House in outside speaking fees appeared in only two papers: The Los Angeles Times and The New York Times.\n\nWe decided to use the computer ourselves this year. As far as we know the two Timeses were still the only papers with the grand totals.\n\nThere was a difference, however: The New Yorkers, calculators in hand, merely produced a grand total for publication. The L.A. Times, armed with computers, was able to print a list of receipts by the California delegation and give details on contributions by particular industries, such as defense and tobacco.\n\nBut this year in New York we were able to print a \"Top 10\" list for both the House and the Senate. Also, using the computer, we were able to spot patterns that enriched the story. And we easily produced a sidebar on Congressmen from the New York area. Instead of playing catch-up, we're out in front of the story.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-july-mapping-a-new-future-for-reporting","slug":"1991-july-mapping-a-new-future-for-reporting","url":"https://dwillis.github.io/uplink/article/1991-july-mapping-a-new-future-for-reporting.html","year":1991,"month":"July","monthNum":7,"headline":"Mapping a New Future for Reporting","kicker":"","author_name":"George Landau","author_title":"computer-assisted reporting journalist","affiliation":"St. Louis Post-Dispatch","summary":"George Landau discusses the benefits and uses of geographic information systems (GIS) in journalism, highlighting how they can transform raw census data into insightful stories by using thematic maps.","keywords":["GIS","census data","mapping"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Atlas GIS","MapInfo","TIGER"],"full_text":"About six months have passed since I first started using a geographic information system (GIS) and now I can't remember how I survived without it. If you haven't discovered this relatively new and amazingly powerful species of PC software, allow me to convert you. I've also used a product called MapInfo, which is cheaper but less powerful than Atlas* GIS. PC Magazine just gave a thorough treatment to GIS packages in the July issue; you should turn there for a detailed comparison of prices and features. I'm a big fan of a package called Atlas*GIS, which I got last winter in preparation for this spring's tidal wave of census data. Census counts are by definition geographic, and unless you're very quick with crayons you'll need a program to map the data. For this article, I'll limit myself to describing some uses of a GIS in the newsroom. Our first census story, for example, looked at population changes in Missouri from 1980 to 1990. We got the raw data in the form of an alphabetical listing of the state's 115 counties. It was hard to discern any trends from looking at the list; some counties showed dramatic population gains while others showed equally dramatic losses. But as soon as I brought the data into Atlas* GIS, stories started jumping off the computer screen. By having Atlas color each county to reflect the degree and direction of population change, we could see that northern Missouri - farm country - was dying while southwestern Missouri - lakeside resort country - was thriving. This is called a thematic map, in which a region's color indicates a specific range of data. The story was equally easy to ascertain when we turned our attention to St. Louis, making thematic maps of racial population changes in the area's 300 census tracts. Black families who had the resources were abandoning the city's decaying North Side in droves, moving to surrounding counties and to the city's South Side. At the same time, white families from the South Side were moving farther south into solidly white areas of the city and nearby counties. Because I was using the Census Bureau's complete digital map of the St. Louis area (called a TIGER file), I was able to zoom in on specific tracts and print out a street map showing the tract boundaries. This made it easy for my colleague Bill Smith to hop in his car and get interviews that would illustrate the trends we'd identified. In the midst of all these census stories, we also found Atlas* GIS to be indispensable for an analysis of real estate in St. Louis. We obtained a copy of the assessor's records for all 140,000 parcels in the city and used Atlas to 'geocode' the data, assigning latitude and longitude coordinates to each record based on its street address. This allowed us to generate 'pin maps' showing thousands of abandoned buildings, vacant lots and tax-abated properties. By overlaying a digital map of the city's 28 political wards, we were able to tally all these items by ward. With a great deal of precision, we identified vast areas of the city where the tax base had all but disappeared - the same areas that had been neglected by the mayor's redevelopment initiatives of the 1980s. Atlas* GIS has also proved its worth for smaller projects. For example, the Post-Dispatch has been publishing an intermittent series profiling city neighborhoods. To get a demographic profile of a neighborhood, I first need to spend about 20 minutes building the neighborhood outline from street segments on the TIGER digital map. From there, it's a simple matter to have Atlas*GIS select the census blocks inside the neighborhood and aggregate the data. These are only a few of the uses we've found for our GIS. We have several other projects in the pipeline, some of them quite ambitious, that also will depend on geographic analysis. I promise that if you can just get your hands on a good GIS, you'll never want to let go. - George Landau has been doing computer-assisted reporting for the St. Louis Post-Dispatch since August, 1989.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-july-the-reel-inside-story-some-tips-on-tape","slug":"1991-july-the-reel-inside-story-some-tips-on-tape","url":"https://dwillis.github.io/uplink/article/1991-july-the-reel-inside-story-some-tips-on-tape.html","year":1991,"month":"July","monthNum":7,"headline":"THE REEL INSIDE STORY: Some Tips on Tape","kicker":"","author_name":"Jimmy Montague","author_title":"MICAR","affiliation":"Missouri Institute for Computer-Assisted Reporting","summary":"Jimmy Montague offers advice on choosing and maintaining quality computer tapes, addressing common issues like short tape lengths and data write skips, and discussing the impact of ANSI standards on tape quality.","keywords":["computer tapes","ANSI standards","data backup"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["9-track tape"],"full_text":"Think about the last time you felt like shooting your cabinet full of 2400-foot computer tapes. Did you catch yourself wanting to blow up, say, Brand X Corporation? Tapes can be too short or poor in quality. So, how can you pick a good tape from a bad one? What is the explanation for short tape and write skips? TOO SHORT? TOO BAD. You were trying to back-up data again, and your blank tape, even though it was the same brand as the original, was too short. It seems strange, you think, that the information that fit on one tape, somehow doesn't fit on another. Meanwhile, the only way to know if a tape is long enough is to load one on the drive and attempt a copy again. You can waste a lot of time that way. 'Aren't there some sort of standards for these things?' you wonder. Well, there are, but unfortunately, they are too flexible. Chuck Doty, technical service specialist at 3M Company said there is an agreed-upon, standard length for computer tapes. The American National Standards Institute (ANSI) provides one. But the main cause of inconsistent length is the inexactness of this ANSI standard. ANSI specifies that the distance from the end-of-tape marker to the physical end of the tape must be between 25 and 30 feet. When you add this to other variable factors, such as the block size of your data and the number of write-skips that occur, the range of your tape's working length can vary from 2,352 to 2,411 feet - a 59-foot difference. 'Tape made in accordance with ANSI standards will be about 2,372 feet between markers, but that will vary from manufacturer to manufacturer and from day to day,' Doty said. 'Most of the specs that I've seen from different manufacturers are based on ANSI specs,' said Dave Lubert, a technical services engineer at Verbatim. 'We look at the distance between markers as being 2,370 feet, plus or minus 5 feet,' he said. '2,406 feet (plus or minus 2) between markers,' said BASF's product manager for computer tape, Ken Brown. '3M has tape available on special order that is 2,400 feet (minimum) between markers,' said Doty. Until standards are more stringent, tape length will continue to be unpredictable. GET BRANDED What's more maddening than to get most of a tape copied, then run across a series of write-skips and have the drive reject the tape? Finding quality tape is a task of trial and error among brands and the products they produce. BASF, for example, makes three grades of tape each more reliable than the next. But even they won't be perfect. 'Our upper two grades generally don't have errors. But tape is a wearing medium, so even a high quality tape that is initially free of write errors won't stay that way forever,' BASF's Brown said. The best bet is to find a brand of tape you like and use it exclusively. That won't do away with problems completely. But, over time, it should keep your tape hassles to a minimum. (The Missouri Institute for Computer-Assisted Reporting uses 3M Black Watch No. 700 computer-tapes. This is after several misadventures with other brands of tape. Let us know if you have had any luck, good or bad, with other brands.) RETURN TO SENDER Write-skip errors can result from inconsistencies in the manufacturing process. All tape is made of polyester film coated with a magnetic material on one side. The process starts with a polyester web a few feet wide. The webbing is coated, and then slit into half-inch widths. Inconsistencies in the formulation of the coating and its application to the webbing is one source of error. But the slicing can also be a problem. Doty said maintaining consistent width and a straight line when slicing tape while making sure no debris is generated can be difficult. It's is a matter for plant maintenance, he said. If a tape is bad, call the manufacturer. The problem could have originated there. LIVE AND LET DIE Archive keepers, take note. In days of old, tape was tested to see if it would record 800 bits of data per inch, Verbatim's Lubert said. Today's drives are spewing out 6250 bits per inch and older tapes often can't accept data at that speed. If you have older tapes in your collection, perhaps it is time for a transfer. In the meantime, store your tapes in a safe, cool, dustless place. How tapes are handled and stored before and after you get them, can have an impact on the way the they perform for you. Make sure your supplier is aware of this. Does your supplier store tape in an atmosphere hot enough to damage it? Is the dust in the vendor's storage area so thick that the fork-lift drivers have to navigate by radar? If your vendor seems unable to supply you with inconsistent quality in the same brand of tape, you might look for another place to spend your money.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-july-tax-problems-in-new-york-city-again","slug":"1991-july-tax-problems-in-new-york-city-again","url":"https://dwillis.github.io/uplink/article/1991-july-tax-problems-in-new-york-city-again.html","year":1991,"month":"July","monthNum":7,"headline":"Tax Problems in New York City, Again","kicker":"","author_name":"Penny Loeb","author_title":"Reporter","affiliation":"New York Newsday","summary":"Penny Loeb explores property tax issues in New York City, revealing inefficiencies and the disproportionate tax burden on poorer areas, supported by data analysis from computerized assessments.","keywords":["property tax","New York City","tax assessments"],"topics":["Business & Economy","Investigative Methods","Government & Politics"],"technologies":["XDB","PC"],"full_text":"David Blake ought to get a medal just for managing to own a building and run his pharmacy in crime-infested East Harlem for 21 years. Instead, his only reward from the city was a 28 percent increase in property taxes this year, after they tripled in the past five years. And the assessors never even noticed two of his building's four stories were torn down in 1982. Blake's is just one of the many tales of woe and waste New York Newsday found in its computer-assisted investigation of commercial and apartment assessments over the past six months. The tapes were no piece of cake. Each property had as many as 394 fields of data. We had to break them into smaller tables. The nice part was that the tapes contained much more than assessed values. For the Tax Commission story, I used a field identifying the attorney for the case. It was easy to group and sum by these to find the attorneys who did the most business. The project was done on a NEC 386, 25 mhz computer with XDB software. Providing invaluable assistance was Tom Braden, a University of Missouri Journalism School graduate student who worked with us as an intern. As the city struggled with perhaps its worst budget crisis, we decided to examine assessments and taxes on commercial and apartment properties since they provide 83 percent of the city's $7.6 billion in property taxes. From the start, I wanted to use computer tapes to look at assessments in ways that hadn't been tried much before. Several excellent computer-assisted stories have looked at how close assessments on houses are to their sales prices. Instead, we looked at where the tax hikes would fall, how assessments and thus taxes were reduced and were all exemption warranted. We found: In a year of record tax increases, the poor will bear the brunt. Assessments and thus taxes will increase more in many of the city's poorer areas than in the wealthy ones. The Tax Commission, the tiny politicized agency, that reduces taxes up to $260 million a year, robbed the city of at least $50 million in the past decade through unwarranted cuts. The city lost at least $8 million in tax revenues on buildings owned by one of the city's largest property owners because of a law giving them a specially tailored exemption. The landlord, Alex DiLorenzo, also owns the building where 87 people died in the tra, social club fire last year. The tapes also contain exempt codes for properties, designating them non-profit or one of a multitude of other ways of escaping taxes. By just paging down the properties with one non-profit code, Braden saw DiLorenzo had lots of exemptions and wondered how he could qualify. Since New York City has what may be the most complicated assessment formulas anywhere, I had to do many calculations. I was pleased with XDB's handling of complicated formulas. But it won't easily calculate the median, the statistical standard for comparing assessments in various areas. For the Tax Commission story I faced the additional challenge of finding computerized information on property incomes. Commercial assessments are generally based on how much a building makes. I planned to use the assessors' formula to find when the Tax Commission cut too much. The information wasn't on the tapes and not, at first, readily available. But I found a consulting firm had gotten it from the city. With some insistence, I too had it in a few days. These kinds of stories can be tried in other cities, depending on what data is computerized. Any suggestions? Call me at (718) 575-2571.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-july-uplink-correction","slug":"1991-july-uplink-correction","url":"https://dwillis.github.io/uplink/article/1991-july-uplink-correction.html","year":1991,"month":"July","monthNum":7,"headline":"UPLINK Correction","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A correction regarding leadership of the Palm Beach Post's computer-assisted reporting program, clarifying that Marie Dillon, deputy metro editor, heads the program rather than Special Projects Editor John Bartosek as previously stated.","keywords":["correction","reporting","leadership"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"The June UPLINK incorrectly identified Special Projects Editor John Bartosek as the person in charge of the Palm Beach Post computer-assisted reporting program. It is Marie Dillon, the paper's deputy metro editor, who heads the paper's computer-assisted reporting efforts. We regret the mistake.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-september-hidden-pasts-uncovered-in-times-series","slug":"1991-september-hidden-pasts-uncovered-in-times-series","url":"https://dwillis.github.io/uplink/article/1991-september-hidden-pasts-uncovered-in-times-series.html","year":1991,"month":"September","monthNum":9,"headline":"Hidden pasts uncovered in Times' series","kicker":"","author_name":"Robert Jackson","author_title":"","affiliation":"MICAR","summary":"St. Petersburg Times reporters used computer-assisted techniques to investigate the abuse of a Florida state statute allowing criminal records to be sealed, uncovering widespread misuse benefiting offenders of serious crimes.","keywords":["investigation","crime","database"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["9-track tape","XDB","SQL"],"full_text":"In Florida, it's easy for people with criminal records to hide their past. That's what St. Petersburg Times' reporters David Barstow, Susan Taylor Martin, Chuck Murphy, Bob Port, and editor Richard Bockman discovered after a six month long investigation into the abuse of a state statute which allows people charged with crimes to have their records sealed. What the reporters found was astonishing. The statute created to give first-time marijuana offenders a second chance, was now giving offenders charged with crimes ranging from fraud to murder a second chance and sometimes a third. The list of abuses was lengthy and required the four reporters to be creative in compiling the data, according to Port, who did most of the computer analysis. It was also the Times' first crack at computer-assisted reporting. \"It (the investigation) was a good example of shoe leather reporting combined with information analysis on a computer data base,\" Port said. First, the Times had to obtain the state arrest records and occupational data on magnetic tape, Port said. The Times hit a stone wall in their negotiations with the Florida Department of Law Enforcement, which happens to be \"home\" to the state's Crime Information Computer. The newspaper requested that the FDLE provide them with the state's arrest records on computer tape. Florida is unusual in that certain information on the arrest records is considered a public record, according to Port. \"We requested the records on tape under the stipulation that all of the identifiers - names, addresses, date of birth - be removed from the record.\" The FDLE denied the newspaper's request claiming, that while the arrest records - on paper were considered public record, the tapes were not. Fortunately for the Times, records on magnetic tape fall under the umbrella of Florida's \"sunshine law.\" \"What it boils down to is they were afraid that we, being crafty reporters, could use other records to get a match on the names,\" Port said. The Times finally had to threaten a lawsuit to get the FDLE to honor the request, and although the FDLE finally agreed to provide the two tapes, they made the experience costly - after paying $25,000 in legal fees, the Times spent an additional $5,000 to obtain the tapes. \"That's a good indication of how serious the St. Petersburg Times was in obtaining that information,\" Port said. Once the Times had obtained the tapes, Port imported the information into a PC with two 330 megabyte drives using 9-Track Express. He was then able to manipulate the data using XDB and SQL. Port explained that the FDLE tapes were strictly used for statistical analysis. \"For instance, we were able to use the information on the tapes to show which counties had the most requests for sealed records,\" he said. Ironically, every person who was named in the series as having a sealed record was identified means other than from the arrest records. Port built a data base of 3,000 people who had gotten records sealed. Other public records were then used to find out more about the individuals. \"It was piecing together a big jigsaw puzzle,\" he said. Building the data base required reporters to sit through scores of court hearings where defendants requested that their records be sealed. Then reporters obtained the defendants' police and court records before a judge ordered them to be sealed. Reporters also spent hours in the Times library, where cases reported by the newspaper were checked against current court and arrest records to see if the record had been sealed. \"What it boils down to is they were afraid that we, being crafty reporters, could use other records to get a match on the names.\" Bob Port \"I guess you could say that we were crafty reporters,\" Port said. The various techniques turned up plenty of abuses, and the Times' five-part series entitled \"Hiding the Past.\" may cause Florida lawmakers to repeal the statute, according to Port. \"We expect the law to change as a result of the series,\" Port said. \"Legislators are already talking about either repealing or reforming the law.\" The cost of the endeavor has not turned the Times' top management sour on computer-assisted reporting, according to Port. \"Most of the editors at the St. Petersburg Times would say that they don't have a clue about computer-assisted reporting,\" he said. \"But the power of the computer has started to kick in and they are becoming accustomed to the concept.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-september-mainframe-versus-pcs-its-a-matter-of-time-cost","slug":"1991-september-mainframe-versus-pcs-its-a-matter-of-time-cost","url":"https://dwillis.github.io/uplink/article/1991-september-mainframe-versus-pcs-its-a-matter-of-time-cost.html","year":1991,"month":"September","monthNum":9,"headline":"Mainframe versus PCs: it's a matter of time, cost","kicker":"","author_name":"Lisa Touye'","author_title":"","affiliation":"MICAR","summary":"The Sacramento Bee uses its mainframe computer system with a VAX network hookup and large hard drive for computer-assisted reporting, allowing multiple staff members to access census data simultaneously and quickly identify trends like racial representation discrepancies that would be difficult to spot manually.","keywords":["mainframe","database","reporting"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["9-track tape","mainframe","FoxBase","VAX","Macintosh","Illustrator","DeltaGraph"],"full_text":"As the number of papers doing computer-assisted reporting increases, so does the number of people searching for a system that meets their newspaper's needs. Most papers use tape-drive systems, like 9-Track Express, to download tapes while some use their newspaper's mainframe. The Sacramento Bee is one of the latter. George Schlukbier, computer coordinator for editorial and library director at the Bee, uses the paper's mainframe to download information for their projects. Using a VAX network hookup, Schlukbier downloads the whole tape into the 2.4 gigabyte hard drive attached to his Macintosh FX. The VAX network hookup allows his Mac to import records as quickly as the mainframe downloads them. Then he uses FoxBase Mac to extract information from the database now on his hard drive. The mainframe system helps keep the Bee competitive with the other papers in the area doing computer-assisted reporting, like the Los Angeles Times, San Francisco Chronicle, and San Jose Mercury News, Schlukbier said. When the Census released the Public Law data files earlier this year, the Bee was ready with a survey of minority representation on city and county governments, Schlukbier said. When the Bee got the tape, they checked representation with minority population figures from 1990 and 1980 and had a story out by deadline that showed serious discrepancies in population and representation. For example, one area in Los Angeles that was 92 percent black had no black representation on the city council, Schlukbier said. Analysis also showed that Hispanics were better represented on city and county governments than blacks, said David Jensen, special projects editor at the Bee. \"It's almost impossible to find trends or anomalies by looking at the Census' STF1 file printout,\" Jensen said. \"Those things jump out at you when you use the computer.\" Schlukbier said he went with the mainframe system because it met his paper's needs which are: constant access to all the information in their 9-track tape library from any terminal in the paper; a system that allowed special projects reporters, graphic artists, and editors to access and use information from the same database without passing around hard copies or diskettes containing the information; compatibility with the art department's Macintoshes and their Illustrator and DeltaGraph software. \"I decided from day one not to go with 9-Track Express,\" Schlukbier said. \"I wasn't convinced that loading and reloading 9-track tapes each time I needed different information was the way to go.\" Schlukbier said it takes 15 minutes to download a tape with variable length records and packed fields into FoxBase using the mainframe. He found tape drive hardware for the mainframe to be more reliable and user-friendly than those for standalone personal computer systems. The keys to the newspaper's system are the mainframe and the 2.4 gigabyte hard drive, which allows Schlukbier to download a whole tape with no problem. But the cost of a hard drive alone with that capacity may be prohibitive for most newspapers. The one Schlukbier uses runs about $8,000. Schlukbier said the site license for the VAX network hookup cost $1,000 and $300 for each Mac in the network. The database program he uses, FoxBase, costs $400. The editorial department has an agreement with the computing department that when a tape is acquired it can be loaded immediately. According to Schlukbier, a reporter doesn't come in with a hot tape needing to get some information off it immediately for a story. Acquiring a tape can, and usually is, a lengthy process, he said. Schlukbier works with a reporter when he downloads the tapes and conducts queries. \"There aren't hot tapes,\" Schlukbier said. \"We do this for long-term special projects. If we need quick access to information we go online or to our own databases.\" Many newspapers do not have a mainframe that they can use to quickly download tapes. That's where standalone systems come in. Systems like 9-Track Express were created to make it easier for reporters to do their own computer analyses on something familiar to them - the newsroom PC, said Dan Woods, co-author of 9-Track Express. \"Many of the dinosaurs of the mainframe world have a siege mentality when the fact of the matter is that the future of reporting or any other analysis is on desktop,\" Woods said. \"Analysis is an iterative process by which answers to certain questions lead to other questions,\" Woods said. \"If the reporter is not involved in that process he's not really reporting.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-september-paper-chase-mainframe-helpful-librarian-solve-paper-woes-for-post-tribune-reporter","slug":"1991-september-paper-chase-mainframe-helpful-librarian-solve-paper-woes-for-post-tribune-reporter","url":"https://dwillis.github.io/uplink/article/1991-september-paper-chase-mainframe-helpful-librarian-solve-paper-woes-for-post-tribune-reporter.html","year":1991,"month":"September","monthNum":9,"headline":"Paper chase: Mainframe, helpful librarian solve paper woes for Post-Tribune reporter","kicker":"","author_name":"Robert Jackson","author_title":"Editor","affiliation":"Post-Tribune","summary":"Gary Post-Tribune reporter Joseph Conn used a GEO District computer program and a state librarian's assistance to quickly analyze census data and determine how many residents would be displaced by a proposed airport expansion, completing in hours what would have taken him 40-80 hours manually.","keywords":["census","airport","data"],"topics":["Demographics & Census","Investigative Methods","Technology"],"technologies":["GEO District","Atlas GIS","floppy disk"],"full_text":"Gary, Indiana Post-Tribune reporter Joseph Conn knows his city's census data - intimately.\n\nThat's because Conn spent the better part of 20 hours pouring through reams of the data, trying to figure out how many of Gary's west side residents would be displaced if the city received a multi-billion federal construction project to expand its airport.\n\n\"According to airport boosters the expansion could create about 140,000 jobs and bring billions of dollars into the area,\" Conn said.\n\nSo Conn began with the 1991 Census data. He began pulling block numbers off paper maps with the idea of comparing that information with the proposed \"footprint\" of airport expansion.\n\n\"I had already invested about 20 hours in the task and had not even begun to sort the data and marry it to the list of block numbers,\" Conn said. \"It was like using a spoon to clean a stable.\"\n\nConn figured it would take another 20 to 40 hours just to come up with some useable information. That is until he learned about a redistricting computer program called GEO District, and a librarian named Steve Fisher.\n\n\"I learned that the state had purchased this program, which is similar in some respects to Atlas GIS,\" Conn said. \"There are six terminals on the state's GEO District system but only one allowed public access.\"\n\nSo he made an appointment and drove to Indianapolis. That's where he met Fisher.\n\n\"Basically, I navigated and (Fisher) steered,\" Conn explained. \"I gave directions to him from a street map with the proposed airport 'footprint' drawn on it, and he used a mouse to trace the footprint onto the computer screen.\"\n\nUsing the program's lasso and \"point-and-shoot\" tools, Fisher placed the footprint on a map of U.S. Census Bureau blocks for the area specified by Conn. Once the map was drawn and all footprint blocks were selected and filled in with color by the mouse, Fisher queried the data base for households, total population and population by racial groups within the specified area.\n\nWhat Conn found amazing was that it took very little time.\n\n\"(the data base query) only took five minutes, and copying the reports from that query onto a 3.5-inch diskette took about five minutes more,\" Conn said. \"Making the map took roughly one hour.\"\n\nHe was even able to verify the GEO District-generated block numbers by checking them against the paper sheets he had already developed.\n\n\"I found about 10 block numbers that the program didn't pick up along the footprint margins,\" he said. \"It seems the program had difficulty picking up some of the smaller census blocks or we failed to see that certain blocks were not selected or colored in.\"\n\n\"On the flip side, the program did pick up about six blocks that I'd missed when doing the listing by pen and paper, so one effort aided the other,\" he added.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-october-newsroom-conflict-anything-can-and-possibly-will-go-wrong-when-too-many-dps-work-with-your-data","slug":"1991-october-newsroom-conflict-anything-can-and-possibly-will-go-wrong-when-too-many-dps-work-with-your-data","url":"https://dwillis.github.io/uplink/article/1991-october-newsroom-conflict-anything-can-and-possibly-will-go-wrong-when-too-many-dps-work-with-your-data.html","year":1991,"month":"October","monthNum":10,"headline":"Newsroom conflict: Anything can (and possibly will) go wrong when too many DPs work with your data","kicker":"","author_name":"Jerry Uhrhammer","author_title":"Investigative Reporter","affiliation":"Tacoma Morning News Tribune","summary":"Explores the challenges faced by newsrooms in maintaining control over computer-assisted reporting projects and navigating technical bureaucracy.","keywords":["newsroom","data","conflict"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["9-track tape","VAX","XDB","PC"],"full_text":"I felt vaguely uneasy when I attended MICAR last May and listened to Elliot Jaspin talk about the importance of maintaining newsroom control over computer-assisted reporting projects. Before leaving for Missouri, I'd been talking to our newsroom computer systems technician about the hardware needed for a computer-assisted reporting program. He didn't think that a separate nine-track tape drive was necessary. He was convinced it would make better sense and be less expensive to download nine-track government tapes on the newspaper's VAX business computer, then transfer the data to the newsroom network's servers where it could be accessed with a PC. That's why I was uneasy. Jaspin was telling us that if you're going to use a computer to analyze government data from nine-track tapes, that's a reporting function. And it's important that the reporter have control of the software and hardware involved in the process. I happen to be a computer enthusiast as well as an investigative reporter. I also know my own limitations - I'm a non-programmer and non-technician. And I had no illusions about being given access to the business computer to load my own tapes. I knew that if we used the VAX system, I effectively would lose a good deal of control over what I was doing. As it turned out, Jaspin's words foreshadowed a summer of frustration and delays that effectively stalled my newspaper's venture into computer-assisted reporting for months. Returning from Missouri, I wrote a memo that raised the control issue. But management opted instead to first try the VAX-fileserver-network approach. Why spend $10,000 on new hardware if you can do the job as well, if not better, with existing sources? I soon learned first-hand what Jaspin had been talking about. Just the simple act of loading a nine-track tape involved working through two tiers of bureaucracy and across departmental lines. First I had to give the tapes to the newsroom computer technician, who then took them to the data processing department. The data processing professionals (DPPs) actually loaded the tapes on the VAX. The technicians and DPPs are stretched thin with their own projects and I learned that an assurance that something would be done the next day or the next week was not something to bank on. Days had a way of stretching into weeks. When things did get done, the results were not always predictable. The first tape loaded contained 1990 census data. But when I tried to extract data into my XDB database program, my 386sx computer labored mightily for a long time and brought forth absolutely nothing. I tried again and again. Still nothing. Thus began a recurring pattern. I was the only one who knew about my XDB software. And when something didn't work, it was up to me to find out why. I spent hours on the telephone calling people familiar with XDB and census tapes. Elliot Jaspin provided the clue that solved the first mystery. The VAX computer did not recognize the character/linefeed signal that marked the end of a record. The absence of such a command left the computer huffing and puffing through 170 megabytes of census data without finding the end of the first record. When the VAX was reprogrammed and the census tapes were loaded again, more funny things happened. Asked for specific information on a certain census tract, the computer began spewing out data of all shapes and sizes. I finally realized I hadn't set up the right filters to narrow down my request. But when I did, I still got two 'hits' where I should have gotten only one. After much detective work, I determined that the tapes had been loaded with an incorrect record length. The good news was that all we had to do was reload the tapes using the correct record length. The bad news was that data processing was busy with end-of-the-month business-side tasks. My tapes had to wait. More bad news followed. After end-of-the-month time, the DPP who loaded the tapes left on vacation. Thirteen days elapsed before the tapes were reloaded. In the interim, I learned another lesson about what can happen when you aren't in control of your own data. Earlier, we had loaded a different governmental database into the fileserver. But when I finally examined the file, I found the data unreadable. A couple days later, after getting advice on how to cure the problem, I tried to access the database again and found it missing altogether. I had been moved, without my knowledge, to make room for the anticipated reloading of the census tapes. I was told the data would be restored the next day. Nine days later it still hadn't reappeared. I finally stopped asking about its whereabouts because I had new worries. The 40-megabyte hard drive on my computer didn't have enough memory to handle the loads of data I was pulling into XDB. The solution: reprogram the system so that I would do all my work in the network's G-drive, which had plenty of memory. That sounded like a pretty good idea, but when we tried it, my XDB software refused to work in G-drive. XDB technical support finally figured out what was wrong. The AUTOEXEC batch file written to put XDB on the G-drive contained a command referring to C-drive, which was overriding all subsequent instructions to switch to G-drive. The VAX-fileserver-network approach turned out to be something less than advertised. For much of the summer, it seemed as if getting our computer-assisted reporting project off the ground was operating according to Murphy's Law: Whatever can go wrong will. In retrospect, it illustrated again the basic truth of the adage, too many cooks can spoil the broth. The troubles of summer had a happy ending. My editors agreed that we had to bring all of the program back inside the newsroom, giving us control over our own data. We bought our own hardware for the newsroom, a 486-33 MHz computer with a 330-megabyte hard drive, a Qualstar 1260 tape drive and a printer. Now it's up to me, a reporter, to make it all work. As it should be.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-october-tri-working-guide-available-at-micar","slug":"1991-october-tri-working-guide-available-at-micar","url":"https://dwillis.github.io/uplink/article/1991-october-tri-working-guide-available-at-micar.html","year":1991,"month":"October","monthNum":10,"headline":"TRI working guide available at MICAR","kicker":"","author_name":"","author_title":"","affiliation":"Missouri Institute for Computer-Assisted Reporting (MICAR)","summary":"MICAR has released a practical guide helping reporters use the EPA's Toxic Release Inventory database to investigate industrial pollution, with options to purchase the guide alone or bundled with data and software tools.","keywords":["pollution","database","EPA"],"topics":["Data & Databases","Health & Environment","Investigative Methods"],"technologies":["SQL","9-track tape"],"full_text":"The Environmental Protection Agency's Toxic Release Inventory (TRI) has been one of the most popular databases used by the readers of UPLINK. It has been the source of excellent industrial pollution stories in Florida, California, Connecticut, Ohio and elsewhere.\n\nBut many reporters still have not used the database. In June, N.T.I.S. made the 1989 TRI available on computer tape. We have found a way to make using these tapes irresistible. MICAR has produced A Reporters Working Guide to the Toxic Release Inventory.\n\nThis short, step-by-step guide helps reporters download TRI into their computers, organize the data so that it is usable, and manipulate the data to produce meaningful reports.\n\nAnd because specific SQL commands are included reporters using the guide will be able to:\n\nidentify the major polluters in their area\nlist the quantities of toxic chemicals discharged by state, county or zip code.\nfind out what toxic chemicals are used for\nlist the most common types of discharges.\n\nThe guide, which is really all you need if you already own the TRI database, costs $30 by itself. But for $50, MICAR will also include the NineTrack Express record profiles needed to download TRI. If you do not own TRI, cannot afford it, or simply do not have a tape drive, for $350 MICAR will also supply TRI data concerning your entire state on 3.5\" diskettes. These diskettes contain unabbreviated data, unlike that which N.T.I.S sells.\n\nFinally, if you wish to purchase the entire TRI database on computer tape, MICAR can supply it for $500. To order call MICAR at (314) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-october-daily-news-links-up-with-county-info-access-just-a-few-keystrokes-away","slug":"1991-october-daily-news-links-up-with-county-info-access-just-a-few-keystrokes-away","url":"https://dwillis.github.io/uplink/article/1991-october-daily-news-links-up-with-county-info-access-just-a-few-keystrokes-away.html","year":1991,"month":"October","monthNum":10,"headline":"'Daily News' links up with county: Info access just a few keystrokes' away","kicker":"","author_name":"Jim Ripley","author_title":"Reporter","affiliation":"Dayton Daily News","summary":"The Dayton Daily News gained direct computer access to Montgomery County records, enabling reporters to quickly investigate property taxes, court documents, and other public filings, which proved invaluable for breaking stories about a financially troubled department store and a fraudulent immigration attorney.","keywords":["access","records","investigation"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["modem/BBS","PC","floppy disk"],"full_text":"A personal computer a few feet away from the metro desk is forever changing the way reporters research county records at the Dayton (Ohio) Daily News. The PC provides a direct line into the county's computer With a few keystrokes, reporters can examine current Montgomery County property tax records, deeds, mortgages, auto titles, civil court dockets and file indices, and divorce and domestic violence filings. Incoming weeks we expect to go online with the county jail and criminal court, and we're negotiating for access to the board of election's voter registration files. Making the technological link for the system was far easier than getting past county officeholders. We made a formal request for access last September. The officeholders turned it over to an attorney to examine privacy issues and the applicability of Ohio's Public Records Law. The attorney decided the law was on our side. The Dayton Daily News is paying Montgomery County $125 per month to service and maintain the hookup. There is no hourly or per minute charge. We also have to pay $480 per year for a dedicated telephone line. The equipment, including a special modem and a device to enable our PC to emulate a county dedicated terminal, cost $1,180. One drawback to the system is that we can't download data onto a diskette or hard disk drive. However, we can print copies of the records. The newsroom's computer link to the county showed its potential the same day it was connected. Reporters were responding to tips that a financially troubled department store chain might close its downtown store. With an instruction manual at his side, deputy metro editor John Erickson typed in the codes that enabled us to gain entry into property tax records. Within minutes he determined that the store was $800,000 behind on its taxes. For courts reporter Rob Modic the hook-up meant a page one story. Modic had a letter from a college student who believed her immigration attorney had skipped town with her money. He also had a motion filed by the local bar association seeking to reassign the immigration attorney's clients. An hour on the system with Erickson convinced Modic he had more. The county's electronic records showed the attorney had purchased a $305,000 home with a $275,000 mortgage less than a year ago. They showed that he had taken out a second mortgage for $40,000 nine days before leaving town in July. They showed he had bought a $40,000 Mercedes. They showed federal tax liens of $36,825 had been placed on the man's home and $22,504 on his business. They showed a list of creditors who had sued him along with attorneys' names and telephone numbers. The county's computer database only summarized court records. Modic still needed to read some of the files for details. The newsroom hook-up helped him determine which cases held the most promise. Within minutes of closing time at the courthouse, he telephoned clerks, gave them the case numbers, asked them to pull the files and told them he was on his way. By 7:30 p.m., Modic moved a page one story that reported, in fascinating detail, the shell game this attorney had been playing. Modic estimated it would have taken him at least another day to turn his story had not a few computer commands taken the place of a maze of offices staffed by bureaucrats who hate being bothered. More than 40 reporters and editors, including feature and sports writers, have been trained on the system. One of them is minority affairs reporter Edwina Blackwell Clark. Clark used the PC to tap into county auto title records. She was working a deadline story about a man accused by authorities of being a broker of undocumented workers. The county computer divulged that the man bought seven cars last year and owned two apartments, which authorities later said were for the undocumented workers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-october-bringing-the-classroom-to-the-newsroom","slug":"1991-october-bringing-the-classroom-to-the-newsroom","url":"https://dwillis.github.io/uplink/article/1991-october-bringing-the-classroom-to-the-newsroom.html","year":1991,"month":"October","monthNum":10,"headline":"Bringing the classroom to the newsroom","kicker":"","author_name":"Lisa Touye'","author_title":"","affiliation":"MICAR","summary":"The News and Observer in Raleigh democratized computer-assisted reporting by training all interested staff members, including interns, on database analysis rather than restricting expertise to a single person. The newspaper successfully integrated database research as a routine part of the reporting process across the organization.","keywords":["databases","training","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["FoxPro","Nine Track Express","R&R","MapInfo","PC","Qualstar tape drive"],"full_text":"Most newspapers start and run their computer-assisted reporting program with one trained person; few papers train more than half of their staff. But that's what The News and Observer in Raleigh, North Carolina did.\n\n\"We made a decision early on that we would not restrict our equipment and knowledge,\" said Pat Stith, a reporter at the paper. \"Access to databases would be made available to everyone interested in the newsroom, even interns.\"\n\nManagement knew that not everyone who was trained was going to become an expert, but it wanted to eliminate the mystery surrounding computers and databases.\n\n\"We attempt to instill in our staff that looking for and through databases is a routine part of the job,\" Stith said. \"When you have a story idea you go to the library, interview sources and talk about it with your colleagues. Now you also look for databases with something additional that's important to the story, acquire the database, do the work on it and use the information.\"\n\nTo make databases a more routine part of the job management has held training programs since July of this year for anyone interested. Six people are trained in each class, which is held from 8 to 10 a.m. for two weeks.\n\n\"We start off as if they don't know anything not even DOS,\" Stith said. \"We teach them DOS, FoxPro, tapes, NineTrack Express, R&R (Report Writer) and MapInfo.\"\n\nThe classes are divided into two groups. One group loads the tapes, and the other writes reports, Stith said.\n\nReporters work on the paper's 17-station computer network, which is served by a Hewlett Packard 486. The paper has two additional 486s and one Qualstar tape drive with a total of 2.2 gigabytes of memory. One of the additional 486s is used to load tapes and do mapping, the other is a backup used for projects with large, complex calculations.\n\nFor some newspapers, starting a computer-assisted reporting program often meets with resistance from management or financial capabilities. But that didn't happen at The News and Observer.\n\n\"What's different here is that we were not required to prove that the concept of computer-assisted reporting works to our management,\" Stith said. \"It was accepted as a given. The general outlook is that this is an expected thing to do.\n\n\"Some newspapers, and this is an impression of mine, use database analysis on a few major stories,\" Stith said. \"We intend to use it on any story where the time and expense of acquiring the database is negligible compared to what it adds to the story.\"\n\nFor example, the paper acquired the state's death records for the past 15 years on tape for $18.97 and used information for a story about cremation, he said.\n\nAlong with building the network and teaching the system, The News and Observer has created a manual and a list of 3,000 databases available from the government. The list grows as reporters looking into a subject find what the state has available on it.\n\nSo far, the paper has acquired more than 25 databases-some federal, most state, Stith said. One agency in North Carolina keeps all the computer records for the state. The paper pays State Information Processing Services charges, which is the amount the state pays to get access to the computer records, but it won't pay more than the state does, Stith said. For example, a database of corporate companies and limited partnerships cost The News and Observer a total of $43.\n\nThe public record law in North Carolina is particularly helpful because everything in the state is a public record unless another law specifically says it isn't, Stith said. When some agencies do deny the paper access to public records, The News and Observer goes to court.\n\n\"We have a well deserved reputation for not taking prisoners when people try to bend the law,\" Stith said. \"We're polite and we say 'please,' 'thank you,' and 'may I,' but when we're denied access, we don't stand still or walk away. We insist on making these agencies obey the law. We've done it for years with hard copies and now computer tapes.\"\n\nWhen the paper decided to do a story about what happened when people moved away without a forwarding address and left safe deposit boxes, checking accounts, stock dividends, etc. behind, it ran into trouble getting the tapes it wanted from the state.\n\nIn North Carolina it's against the law for people who find this money to keep it-they must turn it in to the state who then must make an attempt to try to find the people and return it to them, Stith said. The state can't use the money; it must hold it so the people or their heirs can claim it.\n\nWhen the paper first asked for the database, it said it wanted the state tape. It was given the county's tape instead. The tape was a print image and needed to be parsed. The paper asked again for the state tape and was denied. The state said that fortune hunters would read the papers, tell people that the state owed them money and take 25 percent from the recipients just for telling them. Eventually, the state gave in and the newspaper received the tape, Stith said.\n\n\"What we found in our story was that a number of people were owed huge amounts of money and that the state had failed to make even a passing attempt to find these people,\" Stith said. \"One elderly woman had $40,000 being held by the state. She didn't want it back. She felt the state had forced her to sell her land, and she didn't want any part of taking their money.\"\n\nThese are some of the ways that newspaper stories can be improved by using computer-assisted reporting.\n\nThe News and Observer has used computer-assisted reporting to cover more in-depth stories it normally runs and to answer questions that sources won't.\n\nFor example, the paper did a feature story that told people how fast they could drive over the speed limit without getting a ticket from state troopers.\n\n\"The troopers we talked to said they looked for a clear and substantial violation depending on traffic, weather conditions and other things. In other words, they'd never tell you,\" Stith said. \"When we looked at 300,000 violations on the database it was clear that 99 percent of the violations were for driving over 10 mph over the speed limit. It just jumped out at you when we graphed the information.\"\n\nWhen the paper looked at how much lobbyists spend to wine and dine legislators, reporters found that using databases saved them precious time. Instead of spending days hand-copying the records at the Secretary of State's office, reporters got the records on floppy disks and within an hour started analyzing them with Report Writer for the next day's story, Stith said.\n\n\"This was much cheaper and better than the old way,\" Stith said. \"It has to be, or we won't do it.\"\n\nThe News and Observer won't do a story just because it involves the computer, because there's too much out there that needs to be covered, Stith said.\n\n\"Computer-assisted reporting is like a tape recorder-it's another tool for accuracy,\" Stith said. \"But if there's a paper report that will answer your questions better than a computer database, then use it and leave the computer alone.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-november-seven-deadly-sins-readers-react-to-news-and-record-survey-rate-sins-ten-commandments-in-paper","slug":"1991-november-seven-deadly-sins-readers-react-to-news-and-record-survey-rate-sins-ten-commandments-in-paper","url":"https://dwillis.github.io/uplink/article/1991-november-seven-deadly-sins-readers-react-to-news-and-record-survey-rate-sins-ten-commandments-in-paper.html","year":1991,"month":"November","monthNum":11,"headline":"Seven Deadly Sins: Readers react to News & Record survey, rate sins, Ten Commandments in paper","kicker":"","author_name":"Lex Alexander","author_title":"Database editor","affiliation":"Greensboro News & Record","summary":"The Greensboro News & Record conducted a survey on perceptions of sin in the Bible Belt community using computer-assisted reporting, highlighting the integration of modern technology in traditional reporting practices.","keywords":["survey","sins","computer-assisted"],"topics":["Investigative Methods","Journalism & Newsroom","Demographics & Census"],"technologies":["PC","XDB","Census Survey Tape File 1A"],"full_text":"When I attended the MICAR seminar in February, I would have never guessed that the Greensboro News & Record's first computer-assisted reporting project would reach Biblical proportions. But when 1,053 Greensboro citizens responded to a News & Record survey on sin, our computer-assisted reporting program realized its genesis. The Sin Survey and the idea of using the computer on it were the brainchildren of our religion editor, who understands the theory of computer-assisted reporting because she keeps up with the latest trends and is married to me. One thing the News & Record wanted to emphasize was letting our news agenda determine how we would use the computer, not vice versa. The Sin Survey followed naturally from that. Greensboro, N.C. is in the heart of the Bible Belt (home to one of the Presbyterian Church's five largest congregations in the U.S.). The News & Record has covered religion as a standing beat for many years, and we have carried or written many stories this year about mainline religious denominations that are struggling publicly with such issues as whether to ordain acknowledged gay men and lesbians. We emphasize reader participation in our news stories, so the reader survey was a logical idea we've done plenty of them before on other subjects. The new twist was using the computer. As I mentioned, in the two weeks we allowed, we received 1,053 responses. Our most popular previous survey - on soap operas drew only about 700 responses. We got more after the deadline that we didn't use. The survey, originally published on the religion page, and then repeated one week later in our local news section, asked readers whether each of 20 listed behaviors was a sin. It also asked readers to rank the Ten Commandments in order of how important they were to obey, and to rank the seven deadly sins in order of deadliness. Readers also were asked to supply their sex, age bracket and religious denomination, and to say whether or not they were clergy members and whether they regularly attended worship services. We also asked readers to define sin in essay form, but those responses were not tabulated by the computer. Using a Dell 325P and XDB, I set up a table for responses, taking yes or no responses for the 20 items and ranking the commandments and deadly sins using numerical values so that we could determine averages. Asking people to rank the most important commandment as number one meant using golf-type scoring. The lower a commandment's average numerical value, the more important it was considered by respondents. We were looking for relative value of commandments and sins, SO if someone said all commandments were equally important to obey, I gave each a value of one. If someone said it was not important to obey any of them (and some did), I gave each a value of 10. I checked the age brackets and sex against the 1990 Census Survey Tape File 1A data for our 11-county coverage area to determine how our survey respondents deviated from the population at large. I'll be the first to admit that using computer-assisted reporting like this is like driving a Lamborghini to the corner for a six-pack and a bag of Doritos. It was worth the effort though, for several reasons. First, the survey touched our readers like no other we've ever done. Second, because of that response, tabulating the survey by hand would have been a nightmare. Third, it was a way to finally get a computer-assisted project into the paper. After eight months of living Murphy's Law, I'll take it. Finally, getting something into the paper got more of my co-workers interested in the computer - particularly the senior editors with budgeting authority.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-november-the-top-10-sin-list","slug":"1991-november-the-top-10-sin-list","url":"https://dwillis.github.io/uplink/article/1991-november-the-top-10-sin-list.html","year":1991,"month":"November","monthNum":11,"headline":"The Top 10 Sin List","kicker":"","author_name":"","author_title":"","affiliation":"News & Record","summary":"A News & Record reader survey ranks perceived sinful behaviors, with premarital sex deemed most immoral at 84% and church attendance ranked lowest at 40%.","keywords":["morality","survey","sin"],"topics":["Demographics & Census"],"technologies":[],"full_text":"As compiled by the News & Record in a survey of its' readers\n1. Having sex outside of marriage.\t84%\n2. Cursing.\t65%\n3. Gossiping\t64%\n4. Being Homosexual.\t62%\n5. Renting a hotel room for two people, and letting more stay there.\t60%\n6. Not counting all penalty strokes on the golf course.\t49%\n7. Dumping used motor oil in a field\t49%\n8. Not going to church\t40%\n9. Drinking alcohol\t36%\n10. Reading a Playboy magazine\t33%","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-november-scanners-may-eliminate-data-entry-problems","slug":"1991-november-scanners-may-eliminate-data-entry-problems","url":"https://dwillis.github.io/uplink/article/1991-november-scanners-may-eliminate-data-entry-problems.html","year":1991,"month":"November","monthNum":11,"headline":"Scanners may eliminate data entry problems","kicker":"","author_name":"Lisa Touye'","author_title":"","affiliation":"Boston Herald","summary":"The Boston Herald and other newspapers are investing in scanning technology to reduce data entry errors and improve efficiency in converting documents to digital formats. While the systems show promise, they still require significant verification work and human oversight.","keywords":["scanners","data","technology"],"topics":["Data & Databases","Technology"],"technologies":["Hewlett Packard Scan Jet Plus","dBase","Lotus 1-2-3","Wordscan"],"full_text":"Scanners may eliminate data entry problems\n\nSince their experience with data entry, the Boston Herald has spent $8,200 to buy a scanning system including hardware and software, said Manny Korkodilos, P.C. systems manager at the Herald.\n\nThe Herald bought a Hewlett Packard Scan Jet Plus black and white scanner, a 486 PC, a hard drive, and the software version of Wordscan that lets the scanner drop what it's scanning into a variety of formats including dBase and Lotus, Korkodilos said.\n\nThe Courant is using a scanner to put campaign expenditures into a database format to use for a story.\n\n\"If the copy is good, there should be no problem,\" Houston said, \"But nothing so far has been simple and it all takes work and verification.\"\n\nDave Davis, of the (Cleveland) Plain Dealer said his paper is not looking into scanners yet. \"When scanners become more common they will be a great house data entry solution,\" Davis said. \"We'll then have records as accurate as the paper ones.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-november-data-entry-problems-how-do-newspapers-protect-the-integrity-of-information","slug":"1991-november-data-entry-problems-how-do-newspapers-protect-the-integrity-of-information","url":"https://dwillis.github.io/uplink/article/1991-november-data-entry-problems-how-do-newspapers-protect-the-integrity-of-information.html","year":1991,"month":"November","monthNum":11,"headline":"Data entry problems: How do newspapers protect the integrity of information","kicker":"Data file","author_name":"Lisa Touye'","author_title":"","affiliation":"MICAR","summary":"Boston Herald reporters investigating campaign contributions discovered significant challenges with outsourcing data entry, including accuracy issues like duplicate records, incorrect amounts, and mislabeled geographic data that required verification and correction.","keywords":["data","verification","campaign"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"Last year when reporters at the Boston Herald decided to look into state election campaign contributions, they found more than they bargained for.\n\nThey stumbled into the problems associated with data entry that many reporters working on computer-assisted projects have come up against- cost, time and accuracy.\n\nCampaign contribution records in Massachussets, like many other states, are not computerized. They are available only as paper copies.\n\nThis can slow down or put off reporters interested in the data because a large amount of time must be spent putting the information into an easily manageable database format.\n\n\"We considered hiring people internally to do the data entry, but there were too many problems,\" said David Armstrong, a reporter at the Boston Herald. \"We'd need a large number of people, computers and time for them to work on them and someone to watch over them. That wasn't feasible.\n\n\"Not every paper is like the L.A. Times which can hire an army of data entry folks and pay $80,000 to get the job done.\"\n\nAfter speaking with other reporters, the Herald hired Capitol Micro Systems in Ohio, which charged them 13 cents a record, $8,000 total, and no consulting fee.\n\n\"There were numerous problems with the statewide records, but we could fix them ourselves,\" Armstrong said. \"There were multiple entries of the same record, bogus amounts such as $30,000 donations instead of $3,000, and incorrect spellings of businesses and places.\"\n\nHerald reporters did some data entry themselves to determine possible problem areas in the data, Armstrong said.\n\nWhen the reporters looked into the mayoral campaign contributions they encountered a more serious problem, Armstrong said.\n\nAccording to the database there were no contributions for the mayoral campaign from the suburbs or towns surrounding Boston, yet the addresses of some contributors were definitely in surrounding suburbs and towns. The reporters learned that the data entry company had used a zip code program that replaced the actual suburb or town name with Boston. The database was sent back to the company to correct this, but when it came back\n\n\"It's a fool that doesn't verify the data he gets,\" Houston said. \"If you don't enter it into the computer yourself, you need to verify it when you get it.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1991-november-portrait-of-a-slum-lord-database-analysis-helps-oregonian-uncover-a-neighborhood-tragedy","slug":"1991-november-portrait-of-a-slum-lord-database-analysis-helps-oregonian-uncover-a-neighborhood-tragedy","url":"https://dwillis.github.io/uplink/article/1991-november-portrait-of-a-slum-lord-database-analysis-helps-oregonian-uncover-a-neighborhood-tragedy.html","year":1991,"month":"November","monthNum":11,"headline":"Portrait of a slum lord: Database analysis helps Oregonian uncover a neighborhood tragedy","kicker":"Investigation","author_name":"Jon Schmid","author_title":"","affiliation":"MICAR","summary":"The Boston Herald and other newspapers face challenges with data entry for computer-assisted reporting projects, with some looking into scanners as potential solutions to improve accuracy and reduce manual efforts.","keywords":["scanners","data entry","accuracy"],"topics":["Investigative Methods","Data & Databases","Crime & Justice"],"technologies":["mainframe"],"full_text":"At the (Portland, Ore.) Oregonian, computers and databases are becoming regular reporters tools used on a daily basis to ferret out slum lords. 'It's accessible, cheap and ought to be used every day,' said Oregonian reporter Dee Lane about computer-assisted analyses of data bases. One of Lane's articles last March demonstrated how these new techniques, when combined with good old fashioned reporting, can help turn over a story fast. Lane was assigned to do a follow-up story the day after Darius Martin, a Portland man affiliated with the Bloods street gang, allegedly shot off a gun while showing off in front of friends. The bullet entered a neighboring home and killed 5-year-old Charlie Johnson. Lane began her paper - and computer tape - chase by running the address of the house through the Multnomah County Assessment and Taxation database at the Oregonian's library. 'We're trying to get in the habit of finding out who owns a house whenever there is a fire or shooting,' Lane said. 'We're basically looking for slumlords.' The names Gail and Carol Bash came up as the owners of the house where Martin lived. Natural curiosity led Lane to wonder what other properties were listed under those names. With this information just a few keystrokes away, she had her answer in only about 15 minutes. It turned out 25 other properties, mostly in North and Northeast Portland, were owned by the Bashes. Many of the properties had been acquired in 1989 and 1990. 'It's a typical pattern of slum lords that I've seen before,' Lane said. A later check on the same database showed that the value of these properties was on the low end of the scale. Next, Lane turned to the Department of Motor Vehicles for biographical information about the Bashes. DMV records revealed that Gail was a man and that the couple owned a motorcycle and 10 cars, including a Mercedes and a Cadillac. Moving on to City Hall, Lane's sources there said a warning letter regarding the Martin house was issued under the drug-house ordinance in September of 1990. Lane knew she had scored big with this information because warning letters are sent out to landlords when there are repeated allegations of drug dealing or other crimes at a property. What's more, the city attorney's office was looking into prosecution of the Bashes for liability in the death of the little boy. Following the paper trail, Lane found out from the Bureau of Buildings that the Bashes had three code violations, all of which were reportedly being dealt with. Lane checked her own Bureau of Buildings database, which was a year old, but did include the records of two other buildings owned by the Bashes that were 'red-tagged' for violations. The city couldn't find any information on these houses but Lane's suspicions of neglect and mismanagement were reinforced. The Portland Organizing Project put Lane in touch with neighbors, who said the Martin house was a continual problem, even after the drug-house letter was issued. The neighbors said they had repeatedly called the Bashes and complained about drug dealing at the home. Eviction records at the county courthouse showed that the Bashes had regularly evicted tenants for failing to pay rent. The Martins, however, had received no such threat. Other landlords often use drug-house warning letters to get rid of dangerous drug-dealing renters. Furthermore, Police Bureau records showed that there had been disturbances at the home, which supported the neighbors' complaints. For example, Lane explained, a gang member escaped from a school of detention and was captured in the basement of the house by police. Other complaints about Bash properties included a report of suspicious plants at one location and of heavy traffic at a boarded-up house. While these complaints did not involve the Martin house, Lane said, they contributed to the profile of typical slum lords. After only a day of computer-assisted leg work, Lane was ready to call Gail Bash and ask him some tough questions. 'Computer-assisted reporting starts at a newspaper with just a few people. But some things are so easy everyone should be doing it.' Since Lane's story ran, Bash is being sued for wrongful death and neglect in the killing of Johnson. Lane said the suit is seen as an important test case. The story shows that computer-assisted reporting isn't just for lengthy investigative pieces. It can add depth to the 'quick and dirty' stories that make up the bulk of news content.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-december-intoxicating-project-mainframe-director-enterprising-reporters-conduct-high-impact-dui-investigation","slug":"1991-december-intoxicating-project-mainframe-director-enterprising-reporters-conduct-high-impact-dui-investigation","url":"https://dwillis.github.io/uplink/article/1991-december-intoxicating-project-mainframe-director-enterprising-reporters-conduct-high-impact-dui-investigation.html","year":1991,"month":"December","monthNum":12,"headline":"Intoxicating project: Mainframe director, enterprising reporters conduct high-impact DUI investigation","kicker":"","author_name":"Lisa Touye'","author_title":"","affiliation":"MICAR","summary":"Reporters at the Dayton Daily News investigated DUI cases in Ohio, revealing systematic issues in enforcement and the possibility for repeat offenses even after convictions.","keywords":["DUI","Ohio","investigation"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["mainframe"],"full_text":"In Ohio, you can drink, drive, be convicted, serve time and drive drunk again as many times as you want. Your driver's license may be temporarily suspended, but with a little time and $75 you can buy it back.\n\nThis is what reporters at the Dayton Daily News found as they prepared their series on drunken driving in Ohio.\n\nIn their series, Russell Carollo, Sandy Theis and Mizzell Stewart recounted the cases of people in Ohio who had the largest number of Driving Under the Influence violations and how those people said time in jail wouldn't stop them from committing another DUI violation.\n\nOne man who has been convicted at least 19 times of drunken driving says that when he gets out of jail that he's going to have a drink and he knows he's going to drive drunk again.\n\nDaily News reporters encountered a number of roadblocks while working on the series. The first was the Ohio Department of Motor Vehicles refusal to give them computer tapes of the state's driver's license records, and the records of motor vehicle violation convictions.\n\nThe price is right.\n\n\"Then they decided they'd sell it to us for an astronomical price,\" said Daily News Staff Writer Mizzell Stewart. \"We finally worked out a deal to get the tapes for the cost of the tapes themselves-$475.\"\n\nThey also found that the records were divided into two separate databases--one with the key field, names, addresses and so forth, and the other with all the conviction information, Stewart said. \"It could take days to run a simple join,\" he said.\n\nThen the reporters discovered that the DMV didn't send them all the documentation for the tapes.\n\n\"We found a lower-echelon computer guy in the department who helped us find out what it all meant,\" Stewart said.\n\nNext they approached the local university to see if the people in computing would be interested in helping out by downloading the tapes on their mainframe.\n\n\"They said they could write the routines and do it, but they didn't have the time and weren't really interested,\" Stewart said.\n\nAfter that snub, they talked with Jack Gaines, MIS director at the Daily News, who said he'd do the work on the paper's mainframe in his spare time.\n\nSoon after that, Gaines was bringing printouts of his analyses down to the newsroom twice a week, and Carollo and Theis followed up on the computer findings.\n\nThen they got the tapes-all 67 of them.\n\n\"Reporters have to do the computer analysis for it to be good. The key is to keep the analysis in-house and have it done by someone who's trusted,\" Stewart said.\n\nThe mainframe analysis worked so well that now the paper is connecting the newsroom to the mainframe so reporters can access tapes at any time. Two reporters and an editor will be trained to work on the mainframe as well, he said.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-december-is-government-copyright-or-wrong-should-morrela-rockefeller-amendment-allow-government-to-copyright-software","slug":"1991-december-is-government-copyright-or-wrong-should-morrela-rockefeller-amendment-allow-government-to-copyright-software","url":"https://dwillis.github.io/uplink/article/1991-december-is-government-copyright-or-wrong-should-morrela-rockefeller-amendment-allow-government-to-copyright-software.html","year":1991,"month":"December","monthNum":12,"headline":"Is government copyright, or wrong? Should Morrela-Rockefeller amendment allow government to copyright software?","kicker":"","author_name":"Elliot Jaspin Sandra Davidson-Scott","author_title":"","affiliation":"University of Missouri","summary":"A proposed amendment to the Stevenson-Wydler Act that would allow copyright protection for government-developed software raises important questions about journalists' access to electronic records, as the lack of standardized digital formats could hinder press freedom and government transparency.","keywords":["software","government","records"],"topics":["Government & Politics","Investigative Methods","Technology"],"technologies":["mainframe","ASCII","EBCDIC","magnetic tape"],"full_text":"On the face of it, an amendment to the Stevenson-Wydler Act to allow copywriting of government-developed software is not something likely to electrify the next editorial board meeting.\n\nIn fact, most newsrooms would be hard pressed to see any connection between the amendment and their efforts to cover government.\n\nHowever, this obscure proposed amendment to an equally obscure federal law illustrates the new challenges facing the press as government shifts from storing records on paper to storing them on computers.\n\nThe current version of the Stevenson-Wydler Technology Innovation Act makes it profitable for private companies to use discoveries that are jointly developed with government labs - partnerships known as cooperative research and development agreements, or CRADAs.\n\nThe law permits the government to obtain patents on inventions, resulting from CRADAs. Federal employees who help develop the innovations share in the royalties, and private industry receives a license on the government's patents to develop commercial products.\n\nWhile the law allows the government to hold patents, it does not cover copyrights. Current copyright law prohibits the federal government from applying for copyright protection for its works, including software.\n\nOne-sixth of the country's scientists and engineers work at the more than 700 government-run laboratories. Rep. Connie Morella (R-Md.) and Sen. John Rockefeller (D-W.Va.) reasoned that by encouraging the private sector to form CRADAs, this country could protect a valuable lead it has in computer technology. Both have submitted legislation that would extend the Stevenson-Wydler Act to include copyright protection for CRADA-produced software. The identical bills are currently winding their way through Congress.\n\nWhat does this bill have to do with a reporter's access to information?\n\nEverything.\n\nElectronic data is rapidly becoming the dominant form of record keeping in the United States. The Federal government, at last count, had over 30,000 mainframes and mini-computers storing information on everything from crop reports to cancer research. This means that if reporters want to do their jobs, they will have to know how to obtain and use electronic records. While electronic information is similar to its paper-counterpart, there are significant differences - differences which the bills of Morella and Rockefeller, despite their good intentions, do not address.\n\nWhen a reporter requests a paper record from a government agency, there is an assumption that is so well accepted that it goes unstated: the information will be in English. Not so electronic information, which has no accepted form for electronic records.\n\nThere are two different coding schemes (ASCII and EBCDIC), at least four different methods of recording information on magnetic tape, and a seemingly endless number of record formats. What is crucial to note here is that the form in which information is recorded on a tape is controlled by the software. This makes it exceedingly easy for a government agency to keep its records secret by hiding behind the protection of the Morella-Rockefeller amendment.\n\nWith the help of a private company, an agency could develop database software that uses a proprietary scheme for storing information. When reporters ask for a copy of the government's database, the agency would gladly oblige. If reporters complain that they can't decipher the way the information is stored, the agency can say, \"Sorry, the copyrighted instructions for deciphering the information are not public.\"\n\nThis is not to say that Morella and Rockefeller are trying to upend the concept of open government. Proponents of the amendment argue that only the software will be copyrighted, not the data.\n\nSoftware in this view is the mechanical version of a clerk that retrieves, sorts, counts, indexes and stores information as if it were paper records.\n\nThe distinction between software and data is likely to become even more tenuous as increasingly powerful software programs are developed. What do we make of artificial intelligence (AI) software, where the knowledge base is built into the software as a series of general rules? Are the rules software, data, or something else? Even more esoteric is software that \"learns,\" that develops its own rules as it examines information.\n\nIs it likely that government will be working with such exotic software? If the Morella-Rockefeller amendment is passed, that is precisely the kind of software that the bill's proponents envision.\n\nOne way out of this thicket is to copyright the software but require the government to provide public access to the information. This rings hollow. The use will be free only if you are using a government computer loaded with the government's copyrighted software in the government's building. It will be the stand-in-line use, the you-can't-take-it-home-or-to-the-office use. This will be a use devoid of much convenience and solitude.\n\nFor the importance of timeliness, imagine this scenario: The government, shortly before an important election, has released new figures on the \"negative growth incline\" (recession) which show that the economy has turned around to such an extent that the U.S. will have a projected 10-percent positive growth by the end of the year. The government's new, copyrighted software made the analysis possible. Phenomenal figures, you think. Impossible, in fact.\n\nYou want to check the government's figures, but to do so, you have to use the government's software. In fact, you want to check out the software, not just the figures it produces. That's impossible, you are told. The first time the much-in-demand software can be available is - you guessed it - after Election Day.\n\nThe Morella-Rockefeller amendment should be seen as a signal flare in every newsroom in America. If we do not develop a national policy on access to electronic information, we may wake up one morning to a closed society. \"We would love to give you that information,\" the clerk explains, \"but it's on a computer.\"\n\nThis article was reprinted with permission by Editor & Publisher. Jackson and Jen Schmid contributed research for this article.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-december-tri-working-guide-still-available","slug":"1991-december-tri-working-guide-still-available","url":"https://dwillis.github.io/uplink/article/1991-december-tri-working-guide-still-available.html","year":1991,"month":"December","monthNum":12,"headline":"TRI working guide still available","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"MICAR has released a Reporters Working Guide to the Toxic Release Inventory (TRI) that helps journalists download, organize, and analyze EPA pollution data to identify major polluters and track toxic chemical discharges. The guide is available in multiple formats at varying price points, from a $30 guide-only option to a $500 complete TRI database on tape.","keywords":["pollution","database","toxins"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["SQL","9-track tape","CD-ROM"],"full_text":"In June, N.T.I.S. made the 1989 TRI available on computer tape. We have found a way to make using these tapes irresistible. MICAR has produced A Reporters Working Guide to the Toxic Release Inventory.\n\nThe Environmental Protection Agency's Toxic Release Inventory (TRI) has been one of the most popular databases used by the readers of UPLINK. It has been the source of excellent industrial pollution stories in Florida, California, Connecticut, Ohio and elsewhere.\n\nIn June, N.T.I.S. made the 1989 TRI available on computer tape. We have found a way to make using these tapes irresistible. MICAR has produced A Reporters Working Guide to the Toxic Release Inventory.\n\nThis guide helps reporters download TRI into their computers, organize the data so that it is usable, and manipulate the data to produce meaningful reports.\n\nAnd because specific SQL commands are included, reporters using the guide will be able to: identify the major polluters in their area; list the quantities of toxic chemicals discharged by state, county or zip code.\n\nThe guide, which is really all you need if you already own the TRI database, costs $30 by itself. But for $50, MICAR will also include the NineTrack Express record profiles needed to download TRI.\n\nIf you do not own TRI, cannot afford it, or simply do not have a tape drive, for $350 MICAR will also supply TRI data concerning your entire state on 3.5\" diskettes.\n\nFinally, if you wish to purchase the entire TRI database on computer tape, MICAR can supply it for $500. To order call MICAR at (314) 882-0684.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-december-new-company-reaches-epic-proportions","slug":"1991-december-new-company-reaches-epic-proportions","url":"https://dwillis.github.io/uplink/article/1991-december-new-company-reaches-epic-proportions.html","year":1991,"month":"December","monthNum":12,"headline":"New company reaches E.P.I.C. proportions","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Electronic Public Information Consultants, a new Seattle-based company founded by three University of Missouri journalism graduates, offers advanced analysis of government records for news organizations and public sector clients.","keywords":["government","data","journalism"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"Electronic Public Information Consultants, a company geared towards obtaining, analyzing and reporting on electronically stored government records, has begun operations in Seattle.\n\nAccording to co-owner Adam Berliant, E.P.I.C. is targeting news organizations, but expects to work for others in the public sector who utilize public information.\n\n\"We are able to do far more complex analyses of government data than one is able to do online,\" Berliant said.\n\nBerliant is joined by Tom Braden and David Hinchman as owners of the new business. All three are recent graduates of the University of Missouri School of Journalism.\n\nTo contact E.P.I.C write to 1932 1st Ave., Suite 922, Seattle, WA 98101, or call (206) 448-0454.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1991-december-the-cost-of-doing-business","slug":"1991-december-the-cost-of-doing-business","url":"https://dwillis.github.io/uplink/article/1991-december-the-cost-of-doing-business.html","year":1991,"month":"December","monthNum":12,"headline":"The cost of doing business","kicker":"Computer-assisted reporting","author_name":"Jennifer Wynn","author_title":"","affiliation":"MICAR","summary":"Small and medium-sized newspapers face challenges with the cost and complexity of acquiring computerized data for computer-assisted reporting, though some papers have found ways to obtain information inexpensively by working directly with government agencies and using appropriate technology.","keywords":["reporting","data","cost"],"topics":["Investigative Methods","Business & Economy","Technology"],"technologies":["9-track tape","6250 BPI","Hewlett-Packard 3000","mainframe","PC"],"full_text":"The cost of computerized information may be a stumbling block for some medium- and small-circulation papers interested in doing computer-assisted reporting.\n\nThat's the problem that Jane Martin, a reporter at the Corpus Christi Caller-Times, encountered while doing a drunk driving story for the 72,000 circulation daily.\n\n\"I was laboriously working by notebook going over old (criminal court) files when I realized that we ought to buy the tape,\" she said.\n\nMartin talked to her boss and told her to find out how much the nine-track magnetic tape would cost.\n\nThe tape, which contained criminal case filings from 1985 to 1989, cost $784.\n\n\"That's when I ran into a roadblock,\" she said.\n\nBecause the county's computer system is older, additional programming was required before the Caller-Times could obtain the tapes. The extra programming increased the overall cost of the tapes, according to Martin.\n\nBut what happened to Martin doesn't have to happen to reporters at all small papers. Pat Stith, reporter for the News & Observer in Raleigh, N.C., said that most tapes acquired by the paper from state and county governments have cost about $30. \"Our most expensive tapes were the Federal Election Commission tapes for $417,\" he said. \"We've gotten some tapes for nothing,\" Stith said. \"(In) some places we've pulled out a few diskettes, popped them into the computer and had the information in minutes.\n\nAccording to Martin, smaller towns may also be reluctant to provide the information on magnetic tape because they are not familiar with FOIA requests and FOIA law.\n\n\"They say no without knowing what they're saying no to,\" Martin said.\n\nUnder the Texas Open Records Law, a judge decides how and in what format government documents will be released.\n\nIn Martin's case, she met with the district judge, a clerk and the sheriff to talk about her request.\n\n\"I told them I was not looking for subjective information about juveniles or anything else,\" Martin said.\n\nOnce they understood the request, they released the information, she said.\n\nReporters have to go through that process with every agency they approach, Stith said.\n\n\"Each time we've gone to a new agency it's been a new ballgame,\" Stith said. \"We've done very little repeat business so far but when we go back, things go smoothly.\"\n\nStith also advises newspapers to look for a central processing agency that handles the computer chores for the city, county or state, because it saves time and more importantly cuts costs.\n\nNewspapers starting a computer-assisted reporting program should try to get one 486 computer with a gigabyte hard drive, because it will give them the capability to do a large database project and run programs quickly, Stith said.\n\nFor Martin's drunk driving story, two computer systems at the Caller-Times were used. The 6250 BPI tape was downloaded into three smaller tapes at 6250 BPI. Then programmers downloaded these tapes into a database on a Hewlett-Packard 3000 mainframe. After that they downloaded the information onto a 386 PC with a 100 megabyte hard drive.\n\nRay Motta, manager of operations at the Caller-Times, said the newspaper leases its equipment — one mainframe, four word processors, five disk drives and four tape drives.\n\nMartin also ran into a cost problem when she tried to get tapes from the U.S. Army Corps of Engineers about shipping ports. They quoted her a price of $750 to $1250 to get the tapes.\n\n\"As the government puts more and more information on magnetic tape, I'm just afraid the cost is going to knock out the smaller media,\" Martin said.\n\nLisa Touye' of MICAR contributed to this report.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-january-access-denied","slug":"1992-january-access-denied","url":"https://dwillis.github.io/uplink/article/1992-january-access-denied.html","year":1992,"month":"January","monthNum":1,"headline":"Access denied?","kicker":"","author_name":"Robert Jackson","author_title":"Editor","affiliation":"Orange County Register","summary":"Orange County Register reporters James Grimaldi and Ron Campbell spent four months negotiating with a California government agency to obtain a database on magnetic tape, facing bureaucratic runaround and inflated fees before ultimately invoking the California Public Records Act to challenge the agency's legal authority to charge excessive duplication costs.","keywords":["patience","database","bureaucracy"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["9-track tape"],"full_text":"If you were to ask Orange County (Ca.) Register reporter James Grimaldi for advice on how to negotiate with government agencies for databases on nine-track magnetic tape, he'd tell you one word - patience.\n\nGrimaldi, and reporter Ron Campbell, had to display plenty of patience in obtaining a database from a state government agency that gave them a four-month-long runaround.\n\nBecause the Register is currently using the database in preparation for a series of computer-assisted projects, Grimaldi and Campbell requested that the database and its origin remain anonymous. That, however, doesn't make their story less interesting. According to Grimaldi the Register discovered the database in late July, and he formally requested it from the agency on Aug. 1.\n\nTheir troubles began immediately.\n\n\"This is a very large agency and I was bounced back and forth from office to office and was finally told we couldn't have the database,\" Grimaldi said. \"Eventually I contacted the computer (technicians) and found that the agency had sold the information to another agency for $2,870.\"\n\n\"We felt that (the information) was grossly overpriced,\" he added.\n\nGrimaldi and Campbell requested the database, but didn't hear from the agency for a month. When they were finally contacted, they were told that their request would be granted - for a mere $1,712.99.\n\n\"The more I thought about it, the more it sounded as if (the agency) just picked a price right out of the air,\" Grimaldi said.\n\nEven more astounding was the reply they received when he asked about the basis for the high charge. \"They informed us that the records were in the custody of the Stephen P. Teale Data Center and that the proposed charges were based on Teale's published billing rates,\" Grimaldi explained.\n\nThe Teale Data Center is a public computer information agency that stores various California public agency databases by contract, according to Grimaldi. Although Teale is a public agency they can include overhead costs such as building maintenance, in pricing the duplication of databases.\n\nSo Grimaldi and Campbell's next step was to contact Teale Director Chong W. Ha, and request the database. What they found was infuriating. According to Ha, the public agency - Teale - had custody of the database, and that their \"representations were inaccurate.\"\n\n\"Our patience came to an end when we learned that (the agency) had misinformed us about the location of (the database) and the basis for the prohibitive fee it proposed to charge,\" Grimaldi explained. \"Since (Aug. 1) we talked to more than two dozen people and exhausted, several times over, the statutory 10-day limit for finding and duplicating the records.\"\n\nMore importantly, Grimaldi said, was that there was no legal basis for the agency to charge the public what Teale would charge. In a letter to the agency requesting the database, Grimaldi and Campbell based their claim on the California Public Records Act.\n\n\"Teale, as a self-supporting agency, claims considerable latitude in setting fees; it recovers every cost of its operation, no matter how tangential, from fees charged to other state agencies. Under the California Public Records Act, (the agency) does not enjoy such latitude,\" Grimaldi wrote. \"The law is explicit: You may charge only the 'direct cost of duplication.'\"\n\n\"In our letter we attempted to convey that we were operating in the public interest and that access to the information concerning the conduct of the people's business is fundamentally important,\" Grimaldi added.\n\nThe head of the agency agreed, and on Nov. 12, Grimaldi and Campbell received more than they could have hoped for - an $81.18 charge for the database, and an apology.\n\n\"I want to thank you for bringing the matter of my department's overcharges for the sale of (the database) file records to my attention,\" wrote (the head of the agency). \"At the same time, please accept my apology for the improper fees quoted to you.\"\n\nGrimaldi said that because of the questions raised by the Register, (the agency) is reevaluating the methods used for calculating associated costs. They are also revising the fee structure so that it \"accurately reflects actual direct costs incurred to produce either hard copy or computer tape reproduction of the files.\"\n\n\"We realized that we were the first news organization to ever request (the database),\" Grimaldi said. \"It was a situation that the agency had set precedent in pricing the cost of duplicating the database.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-january-news-and-observer-challenges-nc-for-access-to-public-information","slug":"1992-january-news-and-observer-challenges-nc-for-access-to-public-information","url":"https://dwillis.github.io/uplink/article/1992-january-news-and-observer-challenges-nc-for-access-to-public-information.html","year":1992,"month":"January","monthNum":1,"headline":"News & Observer challenges N.C. for access to public information","kicker":"","author_name":"Van Denton","author_title":"","affiliation":"Raleigh News & Observer","summary":"North Carolina state agencies are blocking public access to government databases through denials, restrictions, and high fees, despite state public-records laws that should apply to electronic information the same as printed documents.","keywords":["databases","access","government"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":[],"full_text":"Some North Carolina agencies are throwing up roadblocks to anyone seeking access to the billions of bytes of public information in the state government's vast computer network. In some cases, requests for public information are flatly denied. In others, agencies place restrictions on release of the information or ask extraordinary fees for it. North Carolina's public-records law specifically applies to public information stored in electronic data- bases, just as it does to printed information tucked away in file cabinets. But some top ranking state officials, wary of releas- ing databases, say the law is inadequate to guard against misuse or inadvertent release of confidential electronic information. Here are where some battle lines have been drawn: *The state Department of Motor Vehicles has a database containing driving records for 6.5 million drivers. Want a copy? It will sell you a copy at $5 per record or $32.5 million for the whole database. *Since September, the Office of State Personnel has refused to provide the News & Observer a copy of a database containing public information on state employees such as names, positions and salaries. A legal opinion is currently being reviewed and a final decision will be made by Gov. James G. Martin. At the same time, James Lofton, secretary of the state Department of Administration reversed his stance in allowing the News & Observer access to the the department's database.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-january-morning-news-tackles-teen-violence-hud-in-computer-assisted-reporting-project-debut","slug":"1992-january-morning-news-tackles-teen-violence-hud-in-computer-assisted-reporting-project-debut","url":"https://dwillis.github.io/uplink/article/1992-january-morning-news-tackles-teen-violence-hud-in-computer-assisted-reporting-project-debut.html","year":1992,"month":"January","monthNum":1,"headline":"Morning News tackles teen violence, HUD in computer-assisted reporting project debut","kicker":"Plugged In:","author_name":"Jon Schmid","author_title":"assistant projects editor","affiliation":"Dallas Morning News","summary":"The Dallas Morning News uses computer-assisted reporting to analyze death records and identify patterns of teen homicides in impoverished neighborhoods, while also investigating HUD scandal developers through multiple databases to reveal how little money was actually recovered in fines.","keywords":["violence","databases","investigation"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["9-track tape","GIS"],"full_text":"Dead men tell no tales. Butre- porters at the Dallas Morning News know that death records can reveal patterns reflecting grave social ills. In fact the News' first com- puter assisted project showed that an \"epidemic of violence\" is stalk- ing Dallas' teenagers The process that led to this discovery began by sifting teen deaths out of the city-owned death ecords, which the News has on on nine-track magnetic tape. Then teen homicides were pulled from that table, explained Allan Pusey, as- sistant projects editor at the News. \"Since there were no names, only dates and data, we worked backwards from police reports, developing a list of teen homi- cides from police reports,\" ex- plained Pusey. Using a computerized map- ping system, this data was plotted by neighborhood and the high- risk areas defined. It was discovered that \"a handful of impoverished, crime-rid- den neighborhoods accounted for the majority of teen homicides in 1989 and 1990,\" according to the story by reporter Bruce Tomaso. \"The result, I think, is a genu- ine balance of statistics versus hu- man experience,\" said Pusey. \"The statistics told us a lot of surprising information we wouldn't have known without the computer. For instance, in a race-conscious city like Dallas, the virtual non-existence of cross- racial slayings came as a complete surprise.\" Still, Pusey has not lost touch with the human element essential to every good story and recognizes computers as part of a good reporter's toolbox. \"I think the real triumph of the series was the human suffering Bruce brought to the story,\" Pusey said of Tomaso's story on teen vio- lence. \"I think the writing is extraor- dinary; powerful and empathetic without being maudlin. The com- puter could not and did not do that.\" HUD scandal The News' latest computer- assisted project helped produce a six-part update on the HUD scan- dal. The series looked at the 24 developers with the biggest HUD rehabilitation contracts. However, that information was not conveniently located on one database, which just goes to show, \"there's no way around the legwork,\" Pusey said. These developers and the locations of their projects were iden tified from a database obtain through a government agency, which Pusey was \"not at liberty to disclose.\" Then reporters Craig Flournoy and Bobbi Miller contacted the local housing authorities where the projects were located. Through the contracts at the local housing authorities they figured out exactly how much money and tax credits the developers received. Pusey also created a data- base on the size of the apartments in the developments. It turns out a lot of the units were single dwellings, which does little to help the families the HUD projects were supposed to tar- get. A further database, using information from the Inspector Gen- eral of HUD, recorded which devel opers were audited, how much t were fined and how much they actu- ally paid. \"It was dismally small,\" said Pusey of the amount remitted on the fines. And finally, Pusey cross-ref- erenced the list of top HUD develop- ers with FEC data available on nine- track tape. \"Suprise surprise, the biggest contributions were going to those congressmen overseeing housing,\" said Pusey. With this information, Flournoy and Miller set out across country to see what all this meant for the tenants who had to live in those buildings. The investigation \"found that these projects enriched a handful of politically connected private devel- opers while consigning thousand families to inadequate and overpriced apartments,\" reported Flournoy and Miller. The two reporters investigated 95 projects completed by the top 24 \"mod rehab\" developers. More than 250 tenants were interviewed. Another project done by the News' involved a computerization of the civil rights data kept by every school in Dallas and helped writer Jonathan Eig produce two stories. \"We utilized the reporting from each individual school to produce a database that encompassed all the Dallas-area school systems,\" Pusey explained. Computer examination of this data revealed that black students in the Dallas suburbs are assigned to special education far more frequently than their white classmates. Using the same database, a second story was produced which illustrated a \"wrinkle in the paddling pattern\" for Dallas schools. In other words, minority students were more likely to receive corporal punishmen than white students. \"Again, I think the project would have been all but impossible without the computerization of the data,\" said Pusey. After these initial successes Pusey said his \"worst fear and great- est joy is coming true. Everyone is thinking of projects and there is actu- ally a bottle neck.\" Pusey is putting together a proj- ect to join several different databases that, when combined, will provide reporters with a quick profile on a neighborhood. Using this system, reporters will be able to plot the address they're interested in on a computerized grid mapping system. This, in turn, will correspond to census tracks. With all these databases loader onto the News' mainframe, informa- tion like crime statistics, birth and death records and census data for a particular area will be available in one package.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-january-post-dispatch-reporters-find-hud-abuses-in-investigation","slug":"1992-january-post-dispatch-reporters-find-hud-abuses-in-investigation","url":"https://dwillis.github.io/uplink/article/1992-january-post-dispatch-reporters-find-hud-abuses-in-investigation.html","year":1992,"month":"January","monthNum":1,"headline":"Post-Dispatch reporters find HUD abuses in investigation","kicker":"","author_name":"Lisa Touye'","author_title":"MICAR","affiliation":"St. Louis Post-Dispatch","summary":"St. Louis Post-Dispatch reporters investigated how the city government distributed $20 million in HUD block grant funds, discovering that funding was not allocated equitably to neighborhoods in need and that the city's claims about housing spending were misleading.","keywords":["funding","neighborhoods","investigation"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["FoxPro","Atlas GIS","9-track tape"],"full_text":"In St. Louis, Mo., crumbling neighbor- hoods in need of city government funding don't necessarily get the money they need to survive. The St. Louis Post-Dispatch uncovered this and more. For more de- tails turn to p. 2 Suspicions arose last summer when the reporters found that some city wards had a much higher concentration of abandoned lots and buildings than oth- ers. In St. Louis, Mo., crumbling neighborhoods in need of city gov- ernment funding don't necessarily get the money they need to survive. That's what St. Louis Post Dispatch reporters Phil Linsalata, Tim Novak, and George Landau found in their investigation of city government's utilization of block grant money from the department of Housing and Urban Development. Suspicions arose last summer when the reporters found that some city wards had a much higher con- centration of abandoned lots and buildings than others. This led them to examine the city's use their annual $20 million in block grant money, which the city uses to help neighborhoods in trouble. \"We didn't know why certain areas had high concentrations of abandoned lots and buildings while others had low ones,\" Linsalata said. \"We suspected the city had under- taken an unspoken urban policy of triage - dividing neighborhoods into healthy areas, areas that were saveable but at risk, and terminal neighbor- hoods that were so distressed they were not worth putting money into.\" The reporters first step was to obtain the city's Grantee Performance Report - a report required by HUD. Th activity summary in the GPR to how much each program received but didn't tell what each program's unction was, or how much was spent, Linsalata said. To answer those questions, according to Linsalata, reporters requested work program contracts under the Missouri Open Records Act. After three weeks of negotia- tion, the city agreed to release the GPR and all the work program con- tracts - on paper. Stacks of the report and programs were sent to a local printer who copied them for the Post Dispatch. \"When we approached the city for the GPR and work program con- tracts, they intially wanted to charge for everything from copying fees to time needed to search for the rec- ords,\" Landau said. According to Landau, the GPR as not available on nine-track magnetic tape so Linsalata and Novak spent three weeks entering informa- tion from the paper reports into a database they created in FoxPro. Landau said that the city pre- pares the records and are required to provide copies for HUD. \"HUD is not required to keep the records for more than a few years, he said.\" Later they used Atlas GIS to map where block grant funds were spent in each ward. The end result showed that city administrators' claims were misleading. \"The city claimed that 45 per- cent of the block grant funding went to housing,\" Linsalata said. \"When we looked at it most of the money was spent fixing broken gutters and giving residents paint to repaint their omes. In 1990, 17 percent of the funding was actually spent on devel- oping homes for sale or rent; build- ing new homes or renovating old ones. It blew away the city adminis- trators myths.\" \"Neighborhoods are revived with substantial renovation and con- struction not quick fix-ups like new paint and gutters,\" Linsalata added. The reporters also found: That neighborhoods most in need of funding have received little block grant money for housing rede- velopment or renovation. Block grant money that was improperly used to supply car phones for an agency run by the mayor's niece, which coordinates city wide flower distribution, recycling pro- grams and city-wide cleanups. A corporation, run by a for- mer city official, received more than $80,000 in block grant money to teach homeowners to do minor re- pairs and train construction workers. The official hasn't held any home repair training classes in one year, and some construction worker train- ees have been in the program for eight years. City adminstrators countered that the Post Dispatch findings were seriously flawed because they exam- ined spending and not what was budgeted. Those adminstrators said that budgeting showed the city's intent to improve those neighborhoods. The reporters also discovered that renovated homes were sold in prosperous neighborhoods, while they were leased in the poorer St. Louis neighborhoods. \"Why should poorer neigh- borhoods not have the benefits of 'for sale' housing?\" Linsalata said.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-february-newsday-uncovers-nyc-tax-billing-mess","slug":"1992-february-newsday-uncovers-nyc-tax-billing-mess","url":"https://dwillis.github.io/uplink/article/1992-february-newsday-uncovers-nyc-tax-billing-mess.html","year":1992,"month":"February","monthNum":2,"headline":"Newsday uncovers NYC tax, billing mess","kicker":"","author_name":"Shay Wessol","author_title":"Reporter (MICAR)","affiliation":"MICAR","summary":"Penny Loeb's investigation revealed widespread errors in NYC's tax and billing system, affecting numerous residents with incorrect or missing bills.","keywords":["tax","billing","New York"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["9-track tape","Nine Track Express","XDB"],"full_text":"New York City resident Ellen Wolosh probably thought the city was joking when, in October 1990, they sent her a letter stating she had used about 1,377 gallons of water per day. Then she received the bill totaling $1,677 for the month. David Blake, who owns a small pharmacy in East Harlem, was forced to pay $7,635 in prop- y fees he didn't owe, so the city wouldn't take his building. New York Newsday investigative reporter Penny Loeb has plenty of Ellen Wolosh and David Blake stories to tell. She spent the better part of four months delving through 21 nine-track tapes on her personal computer. The tapes contained 17-million property records for Manhattan, Brooklyn, and Queens. When she was finished, the compiled information provided a clear picture of the city's system of taxing its residents. In a three part series that ran in December, 1991 Loeb found: At least half of the city's tax foreclosure filings contain mistakes resulting in incorrect billing, bills sent to the wrong address or, worst of all, no bills arriving at all until the city sends a whopping bill of thousands of dollars in supposedly unpaid bills and taxes. Thousands of homeowners lose their houses and property for unpaid bills and are charged rent by the city to stay. Even more are overcharged in their water and sewage bills. Loeb actually began investigating the city's property records in January, 1991. What she discovered and published then was that New York City had cloistered approximately $275 million in overpaid taxes from more than 200,000 property owners (See January, 1991 issue of Uplink). She didn't stop there. \"As soon as the other story came out in January, I started getting phone calls from people who had very strange billing problems, which showed me that there was a big prob- lem,\" Loeb said. \"I wanted to get the billing records.\" She immediately ran into problems obtaining the records from the city departments of Finance and Environmental Protection. \"I put in a request and nothing happened until late March, and at that point I asked the press person to intervene,\" Loeb said. \"In about six weeks, I got a tape, and then when I loaded it, it wasn't the right informa- tion. It was missing the properties that had been foreclosed on. So I went back and asked again.\" When she hit another road-block the Newsday attorneys jumped into the case, and in August Loeb got her data from the city Finance De- partment. \"I just kept at it and kept at it and went to a higher-up attorney,\" she said. Getting the water and sew- age billing records from the city Department of Environmental Pro- tection proved to be a two-month task. \"They turned me down imme- diately, saying that what I wanted was at finance,\" she said. \"It wasn't.\" With both sets of data firmly in hand, Loeb began the task of downloading 17 million records some of them variable length into her personal computer, using Nine- Track Express. \"It took forever,\" she said. \"I could never do the entire city.\" She then began her computer work by XDB to examine the records for Manhattan. One of the easiest ways to check the city's accuracy is to see how many bills were sent to vacant lots, she explained. The hunch worked. By checking the city's billing records against its property records, Loeb found that about $2 million was billed to vacant lots between June 1990 and August 1991. Indexing the data was mandatory, Loeb explained. She worked with several different data bases, the largest of which took up about 35 to 40 megabytes. The indexing took hours, but joining the two sets of data took even longer. Each combination required three \"loads\" and two \"joins\", she said. \"Each one took about 48 hours of computer time. What I would do is leave the computer and let it do the joins overnight.\" The more time she spent working with the records, the more she uncovered. \"I knew that there were billing problems, but I didn't expect things like vacant lots getting water bills,\" Loeb said. \"That's just one of those things you find going through the data.\" \"One the most bizarre occurrences Loeb uncovered was a Long Island lot carrying a $60,664 tab for unpaid water bills. Despite the fact that city agencies thought the property housed the Water's Edge restaurant, Loeb found the lot to be vacant and under the East River. \"That was partly luck. It's just one of those that took a lot of regular reporting,\" she said. \"I got the maps of the property and went out and physically walked the property. Finally the agencies admitted the restaurant wasn't on that lot. \"One of the keys to working with such a large number of records is finding good sources in the city's computer departments,\" Loeb said. \"I had sources that could explain the computer tape to me, when they were allowed to talk to me,\" she said. \"I (also) had property owners calling me, and I could look at their problem and look for others like it.\" To complement the database analysis, Loeb spent many more hours sifting through thousands of pages of documents, and conducted more than 100 interviews. About two weeks after the series ran and many New Yorker's eyebrows were raised, legislators announced they are seeking reforms in the city's billing strategies. Loeb, of course, is pleased with her work. \"It turned out to be a nice wrap-up to the year.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-february-herald-examines-mayors-warchest","slug":"1992-february-herald-examines-mayors-warchest","url":"https://dwillis.github.io/uplink/article/1992-february-herald-examines-mayors-warchest.html","year":1992,"month":"February","monthNum":2,"headline":"Herald examines mayor's 'warchest'","kicker":"Special interest contributions revealed","author_name":"Alan Levin","author_title":"Reporter","affiliation":"Boston Herald","summary":"An investigation into Boston Mayor Raymond Flynn's campaign finances reveals that nearly 90% of his identified donors from 1987-89 had direct financial ties to City Hall, including city employees, contractors, and developers, raising concerns about potential conflicts of interest as he negotiates with these same interests.","keywords":["corruption","campaign","investigation"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"On one day in July 1988, the top executives of Boston's largest employer, money market giant Fidelity Investments, gave Mayor Raymond L. Flynn a gift.\n\nAt least 15 of the firm's executives, many of whom had never given to the liberal Democratic mayor before, donated $7,300 to Flynn's campaign coffers at a fundraising breakfast.\n\nEvery Fidelity managing director gave to the mayor that year, many coughing up the legal maximum - at a time when the firm was negotiating with city officials to build a controversial new downtown Boston skyscraper.\n\nWithin weeks of the fundraiser, city officials joined Fidelity executives at a press conference to heap praise on the construction proposal.\n\nFor Flynn, whose name has been mentioned as a vice presidential candidate, accepting financial contributions from those with business before the city is common practice, the Boston Herald found in a computer examination of seven years of his campaign records.\n\nOut of the Flynn donors whose occupations were identified in the years 1987-89, nearly 90 percent were comprised of city employees, contractors and developers, downtown business interests, city no-bid contractors and others with direct financial ties to City Hall.\n\nHundreds of police officers, including a significant percentage of the department's top brass, gave to the mayor despite department regulations directly forbidding police employees from contributing to any candidate.\n\nThe Massachusetts Secretary of State is currently contemplating legal action against Mayor Flynn's administration.\n\nMore than 23,000 donations were logged by Flynn during his first two terms of office, but the state law governing campaign financing does not require listing each donors' occupation.\n\nBy the middle of last summer, Boston Herald City Hall Bureau Chief Joe Sciacca and I had roughed out a plan to probe Flynn's finances, obtained management support and contracted with a data entry firm to place the mayor's donor lists into a computer database.\n\nWe brainstormed for hours about how to get around the city's refusal to provide computer records to the public - despite the state's legal opinion that such records should be accessible.\n\nThough we had only the tiniest inkling of what we would find last summer, Flynn's $1.3 million cash warchest begged for close examination by a team of investigative reporters - especially after Flynn's name surfaced in rumors as a possible presidential candidate.\n\nBut uncovering the identities behind the anonymous names on Flynn's extensive donor lists proved infinitely more challenging.\n\nOnly names and addresses served as clues to the identity and possible motives - of the contributions to the candidate with some of the deepest pockets in Massachusetts.\n\nIn fact, city officials seem to delight in frustrating reporters' attempts to obtain electronic records. Just last fall, a city department head provided a Boston Herald reporter with a five-foot high stack of computer printouts to fulfill a request for a computer record. \"That's all you're going to get,\" said the official.\n\nFor our story, we settled on a dual strategy. We pursued computer records from independent agencies on the periphery of City Hall, where the public records law apparently means more than it does in the mayor's inner sanctum. And, reluctantly, we decided to pay several thousand dollars to enter a paper copy of the city payroll into a database.\n\nProgress was slow, but encouraging. A helpful source in the city Liquor Board gave us diskettes containing the names of the owners and managers of every bar, tavern and restaurant in Boston.\n\nSure enough, a number of persons associated with the more prominent establishments seemed to be loyal donors, especially the politically connected owner of the city's leading chain of nightclubs.\n\nBut soon, it became all too apparent that, while matching bar owners and other similar groups to Flynn's donor list gave us an interesting start - there was no similar source of data that would identify persons associated with real estate developers and key downtown business interests.\n\nOur sources had told us the owners and managers of these firms were Flynn's top backers, but without a means of identifying them by a rapid computer search, we would have to resort to combing telephone and business directories by hand.\n\nI was lamenting this fact one day with a veteran fundraiser. \"It's too bad it's not a federal campaign,\" he said. Federal law requires candidates to list the occupations of their financial backers.\n\nThen an idea hit. Why not match Flynn's campaign donors to federal records - already on computer - to obtain occupations for the people who gave to Flynn?\n\nMassachusetts Gov. Michael Dukakis' unsuccessful presidential bid in 1988 provided us an unusually large pool of Democratic contributors in the state who were likely to have also given to Boston's Democratic mayor.\n\nIt seemed simple on paper. I created a file of the approximately 21,000 records of persons living in Massachusetts who have given to various federal candidates in the years 1987 through 1989.\n\nA gruesome series of technical exercises followed.\n\nFirst, the federal record is structured so that there is only one field containing the last name, first name, middle initial and any title or salutation. This would not do.\n\nUsing the yWrite word processing software, which includes a powerful programming system, I was able to edit the records in such a way that the names were broken into four fields. Still, this process took the better part of a week because of the large size of the file and the seemingly endless series of steps it required.\n\nNext, comparing the federal records against the Flynn donors presented problems of accuracy. The computerized Federal Election Commission records do not include contributors' street addresses, making it more difficult to ensure that two matching names are indeed the same person.\n\nTo ensure the results were not polluted with erroneous matches, I only considered two names a legitimate match when first, last and middle names were identical, and when titles such as \"Jr.\" were the same.\n\nI followed this step with an intensive audit of the resulting file match to ensure that the results were accurate. For example, I manually examined every case in which the names matched, but the cities in which they lived were not the same. Hundreds of records were dropped from consideration to ensure accuracy.\n\nFinally, owing to the structures of both campaign files, the search produced a large number of duplicate records, which had to be eliminated to produce accurate results. Here, Elliot Jaspin proved helpful by writing a simple program for the XDB SQL database system that deleted the duplicates.\n\nAfter more than two weeks of sweating over the keyboard, we were rewarded with a gold mine.\n\nNearly 2,700 donations were now identified by occupations.\n\nIt was almost immediately apparent that real estate, development and construction employees dominated the list of Flynn donors that had been produced. One employer showed up numerous times: Fidelity.\n\nA Fidelity year-end report added the names of a few executives who had not given donations to federal candidates.\n\nOur paper's library revealed that Fidelity's fundraiser for the mayor coincided with discussions for its proposed skyscraper.\n\nThe clips also showed that the project had met with considerable opposition from historic preservationists because of its size. Despite the problems with the plan, city officials heaped praise on the project when it was first publicly announced.\n\nIn all the computer searches allowed us to identify the occupations of about half of Flynn's donors - or their immediate family members - in the years 1987-89.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-february-labarres-data-analysis-shows-prop-13-unfair-to-homeowners","slug":"1992-february-labarres-data-analysis-shows-prop-13-unfair-to-homeowners","url":"https://dwillis.github.io/uplink/article/1992-february-labarres-data-analysis-shows-prop-13-unfair-to-homeowners.html","year":1992,"month":"February","monthNum":2,"headline":"LaBarre's data analysis shows Prop. 13 unfair to homeowners","kicker":"Press-Enterprise exposes tax inequities","author_name":"Jon Schmid","author_title":"","affiliation":"MICAR","summary":"A Riverside Press Enterprise reporter used property tax databases to document how California's Proposition 13 creates vast disparities in property taxes between neighbors with comparable homes, with some homeowners paying 17 times more than their neighbors depending on when they purchased their properties.","keywords":["Proposition","taxes","property"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["9-track tape"],"full_text":"Under current California law, a homeowner in Santa Monica could end up paying 17 times more in property taxes than his or her immediate neighbor with comparable property.\n\nThat's due to the effects of the tax slashing initiative called Proposition 13 passed in 1978.\n\nWith the Supreme Court scheduled to test the constitutionality of the measure Feb. 23, Riverside Press Enterprise reporter Bob LaBarre decided to take a look at exactly how the alleged tax disparities stack up.\n\nIn a two part series LaBarre revealed that Prop. 13, originally designed to prevent modest homeowners from losing their houses, was producing huge differences in property taxes.\n\nOnly those homeowners who bought their houses before the end of 1975 fully benefit from the measure.\n\nAnd those who bought their homes since 1975 claim the inequities are a violation of the equal protection clause of the 14th Amendment of the Constitution.\n\nLaBarre expected the usual problems in obtaining a database for his story.\n\nBut when he went to the Riverside Assessor's office he found the staff was more than cooperative. They were so intrigued by the problem he presented they actually did the work for him.\n\n\"One of the things I learned was, 'ask the nerds,'\" said LaBarre. \"The systems people became so fascinated by the questions that they agreed to help me create the data base.\"\n\nJust 24 hours later LaBarre had the average tax paid by homeowners broken down by the last year its value was assessed.\n\nUnder Prop. 13 a home's value is not assessed unless it is sold. The flat rate for property taxes is one percent of the assessed value and it can be increased two percent per year.\n\n\"I was kind of disapointed, because it was so easy,\" said LaBarre about his first experience at computer sleuthing since finishing a MICAR seminar October.\n\nHowever, LaBarre realized that the average figure he was using might be inaccurate because older houses would naturally have a lower value, and thus a lower tax assesment.\n\n\"It was so easy that I thought that maybe the houses were not comparable, so I did the same thing for all the houses that sold in 1990,\" said LeBarre. \"What we did was we took from the assessors roll all the properties that had to be updated. It turned out there were 24,000 properties that had been sold.\"\n\nLeBarre took all the residential properties and broke them down by base year before they were sold. He then totalled up the assessment on the 1975 properties right before their sale and right after.\n\n\"That revealed that 1975 properties, when sold in 1990, were assessed at 21 percent of their market value,\" LeBarre explained. \"But 1990 properties were assessed at 85 percent of their market value.\"\n\nLeBarre then pulled out specific houses on individual streets in Palm Springs, Riverside and her show the effects of Prop. 13.\n\nIt would be nice to have Nine Track Express because every time we download a tape we have to rewrite the current program and that can take from a several hours to a day.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-march-clear-road-ahead-new-cable-improved-heads-may-keep-reporters-from-driveing-themselves-nuts","slug":"1992-march-clear-road-ahead-new-cable-improved-heads-may-keep-reporters-from-driveing-themselves-nuts","url":"https://dwillis.github.io/uplink/article/1992-march-clear-road-ahead-new-cable-improved-heads-may-keep-reporters-from-driveing-themselves-nuts.html","year":1992,"month":"March","monthNum":3,"headline":"Clear road ahead? New cable, improved heads may keep reporters from drive(ing) themselves nuts","kicker":"","author_name":"Jon Schmid","author_title":"","affiliation":"MICAR","summary":"Discussion of improvements in cable and hardware to resolve tape drive issues faced by reporters.","keywords":["tape drive","data","Qualstar"],"topics":["Technology","Investigative Methods","Journalism & Newsroom"],"technologies":["XDB","Qualstar 1260"],"full_text":"About this time last year Bob Paynter of the Akron Beacon Journal and Judy Miller of the San Francisco Chronicle were pulling their hair out in frustration over tape drive difficulties (See Uplink April, 1991). Paynter's Qualstar 1260 tape drive just never seemed to work. Sometimes it would do as he asked, and other times it would just rock back and forth interminably. Paynter, who was suffering from problems with XDB at about the same time, probably thought his PC and tape drive were possessed. \"I didn't know what was wrong with it,' he said. \"It was like asking a kid if his first car was operating properly. I just didn't know.'After a more than a few panic calls to Qualstar technicians, Paynter discovered his tape drive problem had a name - intermittent C-75 component (a capacitor in this case) on the read board. He also discovered that the \"intermittent\" problem was particularly aggravating because some- times the part would work and sometimes it wouldn't. Simply put, Paynter pack- aged up his Qualstar tape drive and sent it back to the company for re- pairs. Qualstar, as it so happens, is the company of choice for many newspapers because of the portable lightweight tape drives they manu- facture. The 1260 tape drive is an in- expensive, 6250 bpi-capable drive, and is usually purchased through CHI Corp. CHI manufactures the controller card and software that go inside the PC to make the tape drives and computers communicate. According to Qualstar techni- cian Dan Nguyen, Paynter's drive problems may be the exception and not the rule. There are other prob- lems causing similar symptoms that have simple solutions, he said. Qualstar customer support has a few \"quick fixes\" - such as cleaning the heads or re-seating the read and interface cables and the controller card - if your drive is acting demon-possessed. \"I've found that 50 percent of the time re-seating the interface card and the interface cable solves the problem,\" Nguyen said. And what about the other 50 percent? Well it's not that easy. The read cable, for example, has caused more than its share of headaches with no at-home solutions. Miller can more than relate to that particular problem. Now in possession of her fourth Qualstar tape drive, Miller was forced to send back two of her first three drives because they wouldn't function properly. \"Our first drive didn't work, so we sent it back,\" Miller ex- plained. \"CHI Corp. was really good to us, and they sent us another one that worked at first. Then cen- sus data arrived and it started to rock back and forth.\" All Miller could get was a 'C3' or head error. She immediately called Qualstar and talked to a tech- nician who walked her through head cleaning and reseating - making up 50 percent of the problem - and got the drive working again. The next day, Miller said, it just consistently started rocking. After sending the errant drive back to Qualstar Miller found that the replacement (the third by this time) they sent, wouldn't work at all. Ed Foldessy of the Wall Street Journal has a similar story to tell. \"We had plenty of problems right from the beginning,\" Foldessy explained. \"I would download one tape and everything would work fine. But when I'd download the next tape, all I would get is a read error.\" Foldessy said he'd then clean the heads - which became a ritual - and the same thing would happen, again and again and again. \"Qualstar finally sent me a new read cable, but after a few tapes the same problem would crop up again,\" he said. \"I finally gave up and sent the drive back to Qualstar.\" \"They told me their was nothing wrong with the drive, but they upgraded the hardware and shipped it back,\" Foldessy said. 'I haven't had any problems since.\" The culprit, according to John Thome of CHI Corp., is a grey rib- bon found in the older model drives connecting the green read-board to the head. When Uplink talked with him a year ago, Thome said that the green card-edge connector ribbon was the problem, and that CHI was sending customers new cables. According to Nguyen, Qualstar now uses a blue cable from a different vendor and reports much better performance. He also ex- plained that when you send your drive in for its yearly tune-up, Qualstar engineers replace pesky problem parts like the dreaded grey cable. Qualstar has also identified a read problem with the old Hamilton heads when reading certain tape brands and is now installing DRH heads on new machines. Nguyen recommends using only 3M 700 Blackwatch tapes. Brant Houston of the Hartford Courant, said that his current Qualstar drive is working fine. Houston, like Miller, went through four drives before getting the 1260 model upgrade. \"After a lot of misery we have finally got a drive that works fine,\" Houston said. \"The problem is that after going through that misery, there is a lot of mistrust, and my company would probably not pur- chase another one.\" Houston's drive problems, like those of Paynter, Miller, and Foldessy, had to do with the read cable. With that problem rememe- died, all are happy with the current performance of their drives. Other problems which Qualstar technicians have found within the past year include fried power boards (due to electrical surges), \"lost clusters,\" and faulty controller cards. Nguyen maintains that the most common problem plaguing the tape drive is the accu- mulation of dirt on the head. Qualstar recommends that you should clean the head and every- thing in the tape path every two hours of use. This includes rollers, guide and the tape cleaner below the head. Make sure to apply pressure when cleaning the head but not so hard as to scratch it. Clean the head horizontally with a swab or q-tip and isopropyl alcohol - 90 percent or higher, Nguyen said. \"I got to the point where I cleaned the heads every five min- utes, it was almost like a voodoo ritual,\" Paynter said. \"But to quite honest, I have been remiss since I haven't had problems.\" \"I've had no problems with the drive at all,\" Miller said. \"Eve- rything is working just fine.\" \"I guess you could say the fourth (drive) was a charm.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-march-loeb-selected-for-scripps-howard-award","slug":"1992-march-loeb-selected-for-scripps-howard-award","url":"https://dwillis.github.io/uplink/article/1992-march-loeb-selected-for-scripps-howard-award.html","year":1992,"month":"March","monthNum":3,"headline":"Loeb selected for Scripps-Howard award","kicker":"","author_name":"","author_title":"investigative reporter","affiliation":"New York Newsday","summary":"Investigative reporter Penny Loeb and data analyst Tom Braden have been named co-recipients of the Roy W. Howard Public Service Award for their Newsday series exposing problems in New York City's tax collection and billing, resulting in $275 million in refunds owed to property owners.","keywords":["investigation","journalism","award"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["PC"],"full_text":"Penny Loeb, investigative reporter for New York Newsday, and former MICAR research assis- tant Tom Braden, have been se- lected as co-recipients of the Roy W. Howard Public Service Award, pre- sented yearly by the Scripps-How- ard Institute. Loeb and Braden were se- lected for Loeb's Newsday series that exposed problems in New York City's collection of taxes and billing of services. Braden assisted Loeb by pro- viding some data analysis, and writ- ing a story for the series. by using a personal computer to analyze more than 17 million city finance records. The series was accomplished \"For once its a public service award that truly served the public,\" Loeb said. Because of her series the city owes its property owners $275 million due to overpayments. Braden said he was pleas- antly surprised to be selected as a co- recipient. \"I was surprised because I thought Penny would be the only recipient,\" Braden said. \"All I can say is that Penny is one helluva reporter.\" The award will be presented in April.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-march-mercury-news-finds-that-criminal-justice-isnt-color-blind","slug":"1992-march-mercury-news-finds-that-criminal-justice-isnt-color-blind","url":"https://dwillis.github.io/uplink/article/1992-march-mercury-news-finds-that-criminal-justice-isnt-color-blind.html","year":1992,"month":"March","monthNum":3,"headline":"Mercury News finds that criminal justice isn't 'color blind'","kicker":"Linking up:","author_name":"Christopher H. Schmitt","author_title":"","affiliation":"San Jose Mercury News","summary":"A San Jose Mercury News computer-assisted analysis of nearly 700,000 California criminal cases over nine years found that white defendants receive significantly better plea bargain deals than Hispanic or Black defendants accused of similar crimes. The study revealed racial disparities at virtually every stage of pretrial negotiation, with whites more successful at getting charges dropped or reduced.","keywords":["sentencing","discrimination","plea"],"topics":["Crime & Justice","Investigative Methods","Demographics & Census"],"technologies":["Paradox","9-track tape","PC"],"full_text":"In California, justice isn't color- blind. Constitutional guarantees of equality notwithstanding, that's what the San Jose Mercury News found recently in a computer-as- sisted study of nearly 700,000 crimi- nal cases over a nine-year period. As more and more cases are decided by plea bargain, whites as a group get significantly better deals than Hispanics or blacks who are accused of similar crimes and who have similar criminal backgrounds. Instead of judges or juries handing down verdicts, the fates of hundreds of thousands of defen- dants a year often are negotiated over the telephone, in hurried hall- way or parking lot conversations or in closed-door meetings. What happens? At virtually every stage of pretrial negotiation, whites are more successful than non-whites. They do better at getting charges dropped. They're better able to get charges reduced to lesser offenses. Our story began on the cam- paign trail in 1990. In get-tough-on- crime rhetoric, the two major candi- dates for governor including soon- to-be Republican Gov. Pete Wilson - were sounding a lot like outgoing Gov. George Deukamejian, who himself was elected mostly on a law-and-order pitch. So we decided to look at what nearly a decade of enacting new crime laws, appointing hundreds of \"common sense\" Deukamejian judges and building cavernous new prisons had done to cut crime and make neighborhoods safer. The answer was: Not much. One reason, we were told, was California's \"determinate sentencing\" law, which removes discretion from judges' hands by mandating specified prison terms for specified crimes. The idea behind this law was to remove racial disparity from sentencing. But, one of our sources suggested, what if dis- crimination had simply shifted elsewhere in the proc- ess? And thus our story was born. As we moved into it, it became clear the story was tailor-made for computer-assisted reporting. Simply doing some limited sampling of cases wouldn't work. Once you start looking at specific crimes by criminal background and by specific racial/ethnic group, the numbers can get pretty thin. And when that happens, it doesn't take much to distort results. Our main data sources were two state files: O. detailing adult felony arrest dispositions, the other pro- viding information on those sent to state prisons. We're pleased to say we handled the entire project on a PC, without having to resort to mainframe or systems people help at all. Our largest of several files was about 100 megabytes, and although we bumped into a limit on the number of fields allowed in a record, Paradox was a speedy database manager for combing through all the information. Getting the data was a best-of-times, worst-of- times affair. The felony arrest data came from a state agency still willing to make information available quickly and (in this case) at no charge. Getting the prison data, however, was an ordeal. In an experience very much like that recounted in Uplink recently by the Orange County Register, it took months of negotiation and haggling over price before we got what we wanted. Standing up for criminals' rights isn't politically popular these days, but we thought the story important enough to devote about seven months to it. But as time as that might seem, it still pales next to how lon, would have taken without our trusty nine-track tape reader and a 33-megahertz 386 clone.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-march-ohio-newspapers-battle-for-state-drivers-license-data","slug":"1992-march-ohio-newspapers-battle-for-state-drivers-license-data","url":"https://dwillis.github.io/uplink/article/1992-march-ohio-newspapers-battle-for-state-drivers-license-data.html","year":1992,"month":"March","monthNum":3,"headline":"Ohio newspapers battle for state drivers license data","kicker":"","author_name":"Bryan Edwards","author_title":"","affiliation":"MICAR","summary":"Two Ohio newspapers, the Dayton Daily News and Akron Beacon-Journal, simultaneously requested the same massive state drivers records database from the Ohio DMV, each facing significant technical challenges in analyzing the 60 million records using different computer systems and outdated equipment.","keywords":["databases","newspapers","analysis"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["9-track tape","mainframe","XDB","Paradox"],"full_text":"It was only a matter of time before two competing daily newspapers would request the same database at the same time.\n\nSuch was the case with two Ohio newspapers. The Dayton Daily News and the Akron Beacon-Journal both requested state drivers records from the Ohio Department of Motor Vehicles not knowing of the others request. Supplying the DMV with about 70 nine-track magnetic tapes, the Daily News received the database in November 1990 with intentions of finding repeat offenders of the D.U.I. law.\n\nBut the Daily News did not start analyzing the data until June of 1991, according to reporter John Erickson. \"This was such a huge database, we put it on the mainframe,\" he said.\n\nIn fact, the data included every person who was ever an Ohio driver. \"There were about 60 million records on the tapes, including drivers who were dead,\" Erickson said. The problem was much bigger than that. The records were divided into two separate databases one with the key field, names, addresses and so forth, and the other with all the conviction information. This meant that it could possibly take days to run a simple join.\n\nThen reporters found that the DMV didn't send them all the documentation for the tapes. They scrambled until they located a \"lower-echelon\" computer technician who helped unscramble the problem.\n\nTo add to the growing list of difficulties, they had to use a \"primitive\" software program called Quick Start, forcing technicians to write a different program for every task. \"They wrote a sorting program and gave us the results on paper,\" Erickson said. Not being able to use a personal computer was another disadvantage, according to Erickson. \"It's a lot easier to work on a PC,\" he said. \"You have much more control over the data. When you're on the mainframe you pass along what you want, but you can't play with it.\"\n\n\"It's like being at a press conference and not being able to ask questions,\" he added. While the Daily News was plodding ahead in their data analysis, the Beacon-Journal was hard at work analyzing the same database. The difference was that the Beacon Journal used a PC to analyze its data. But problems still arose.\n\n\"At the time we had a 1600 BPI tape drive and the tapes were 6250,\" said Bob Paynter of the Beacon-Journal. \"We wrote a program on the mainframe allowing the drive to read the data. Then we loaded it into the PC.\"\n\nPaynter explained that he used the mainframe to \"pare down\" the data pulling out only the records they needed. \"We ended up with about 5 million records,\" he said.\n\nMainframe technicians then copied the records on to a nine-track tape at 1600 bpi. Paynter then imported the records into his PC.\n\n\"If I would have had a 6250 bpi tape drive at the time, I would have done it all on a PC,\" Paynter said. \"Then the only time consuming problem would have been loading the data from 70 tapes.\"\n\nOnce past the drive problems, Paynter experienced others. \"Every time I used XDB I had trouble,\" he said. \"I had import problems, export problems. I literally spent weeks dealing with the XDB people on the phone.\"\n\nWhen Paynter began crunching numbers, XDB presented more problems. \"I would run it (query) through once and get one result, and get another result the next time,\" he explained. \"I just didn't trust it.\"\n\nWith the frustration level rising, Paynter decided to use another data analysis program called Paradox. The end result was rewarding, he said.\n\n\"It's worked great,\" Paynter said. \"But the XDB problems could have been something I was doing. I just don't know.\"\n\nBy September 1991 the Paynter had the results in hand and was able to publish a story about police officers and snowplow drivers with poor driving records.\n\nThe story surprised the Daily News.\n\n\"We thought we were the only ones who had it (the database) and I think we got a little too comfortable,\" said Steve Sidlo of the Daily News. \"All of a sudden the Beacon-Journal runs some pretty good stories, but it wasn't what we were working on. It became quite clear that the Beacon-Journal had the same database as the Daily News,\" Sidlo said. So the Daily News began pushing harder to finish the story on DUI repeat offenders. It yielded the story in November 1991.\n\nIn December, the Beacon-Journal ran its story on DUI repeat offenders. Too close for comfort for the Daily News, according to Sidlo. He said the Daily News might have waited another two weeks to run the story if it had not known about its cross-state rival. And both papers continue to use the drivers license database regularly. They have broken the data down to make a sub-database that includes only drivers in the circulation area of the respective paper. When a story breaks, they have quick access to information about drivers in their region.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-march-rape-series-points-out-serious-flaws-in-system","slug":"1992-march-rape-series-points-out-serious-flaws-in-system","url":"https://dwillis.github.io/uplink/article/1992-march-rape-series-points-out-serious-flaws-in-system.html","year":1992,"month":"March","monthNum":3,"headline":"Rape series points out serious flaws in system","kicker":"For discussion:","author_name":"Shey Wessol","author_title":"MICAR","affiliation":"Minneapolis Star-Tribune","summary":"A Minneapolis Star-Tribune investigation found that nearly one-third of rapists who underwent Minnesota's treatment programs were re-arrested within three years, compared to a one-quarter return rate for those who served jail sentences, raising questions about the state's sentencing and treatment procedures for sex offenders.","keywords":["recidivism","treatment","investigation"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["mainframe","9-track tape","floppy disk","Paradox","SAS"],"full_text":"The Minnesota Corrections Department believes in helping its convicted rapists and child molesters, but its therapy isn't work- ing. Instead, convicted rapists and child molesters walk away from jail sentences and attack again, the Minneapolis Star-Tribune found in a nine-month computer analysis of recidivism patters of sex criminals. Nearly one-third of all rapists who had undergone treatment pro- grams were re-arrested within three ars - compared to a one-quarter return rate of six years for criminals who served jail terms - the Star- Tribune reported. Star-Tribune reporters Allen Short and Donna Halvorsen, intern Dan Eggen and survey specialist Denise Brownfield began compil- ing the criminal histories of 932 first- degree sex criminals in February 1991. Their findings, published in November, shocked many Minne- sotans, and raised the ire of the treat- ment community by questioning the state's sentencing and treatment procedures. \"What we wanted to do was not only come up with a valid re- search project involving numbers, but we needed to show the human side as well,\" Short said. The two reporters conducted between 125 and 150 interviews with officials, victims and their fami- lies, convicts, therapists and re- searchers, Short explained. The investigative team used an IBM mainframe computer to track the criminal history of each individual for a 10-year period using information from three different state agencies, and the four major Minnesota treatment centers. The entire database, consisting of 10 data files, took up 20 megabytes of space, or 13,000 pages of single- spaced text on standard typewriter paper. The data was not available on nine-track magnetic tape, according to Rob Daves, assistant managing editor for research. \"We try to be as flexible as we can here and use whatever technology is available to support the reporters,\" he said. \"We don't let technology drive the re- porters.\" \"This project combined ele- ments of computer-assisted report- ing, social-science research and good old-fashioned shoe leather reporting,\" Daves said. From the Minnesota Sen- tencing Guidelines Commission the team acquired, on a diskette, the names and charges of 932 sex crimi- nals convicted on first-degree charges during the 1980s. Since Minnesota law requires actual penetration of the victim to warrant first-degree charges, the team knew they were getting criminals con- victed for the most serious attacks. That information was supple- mented with three megabytes of paper data containing the complete criminal histories of the 932 men from the Minnesota Bureau of Criminal Apprehension, which was keyed into the computer. \"We made a deal with them so that we could get the rap sheets on all of our guys,\" Daves said. All the information from the BCA was used for research pur- poses only, in the statistical analysis and the summary data. Finally, the Star-Tribune team went to the Minnesota Correc- tions Department and asked for in- formation on incarceration to calcu- late when the criminals were on the streets and when they weren't. That data, 400,000 bytes long, was also on paper and entered into the com- puter. \"We had some stumbling blocks along the way,\" Short said. \"In 1988, the Corrections Depart- ment said the tapes weren't there and that a newspaper couldn't do a valid study on sex crimes. We just went around the Corrections De- partment until the very end.\" All of the information was in- serted into Paradox databases, shaped into a flat file and uploaded into the paper's mainframe com- puter, Daves said. All the analysis was done using SAS, a statistical analysis program. Manipulating the data was a challenge, Brownfield said. \"I do mostly survey data here,\" she ex- plained. \"The biggest challenge, for me personally, was the different number of records for each person. It's the first time I've ever done anything like that.\" Short, the lead reporter on the project, began looking into sex crimes in 1988 after a number of rape-murders were reported. Some of the criminals had undergone treatment programs. \"As a reporter, I asked the Corrections Department if they had any information on the outcome of their treatment programs,\" he said. \"They didn't. At the same time, the state was spending millions of dol- lars on treatment.\" The Minnesota Corrections Department spends $6.3 million annually on treatment programs cure many of its first-degree sex offenders. But the agency has never conducted a study to find out if its program works, citing a lack of re- sources, the Star-Tribune reported. As a result of the series, both parties in the Minnesota legislature, as well as the attorney general, have proposed new legislative packages for sex offenders. \"In essence, the study engen- dered a rather heated debate in the treatment community, as it should have,\" Short said. But, he added, after spending 11 months reporting on rape, \"we don't want to write about rapists for a long time.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-april-courant-creates-profile-of-murders","slug":"1992-april-courant-creates-profile-of-murders","url":"https://dwillis.github.io/uplink/article/1992-april-courant-creates-profile-of-murders.html","year":1992,"month":"April","monthNum":4,"headline":"Courant creates profile of murders","kicker":"","author_name":"Brant Houston","author_title":"Investigative reporter","affiliation":"The Hartford Courant","summary":"The Hartford Courant developed a database to investigate unsolved murders of minority women, discovering patterns suggesting a possible serial killer and prompting police attention.","keywords":["murders","database","Hartford"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["XDB"],"full_text":"After doing computer-assisted reporting for a few years, I find myself and my colleagues increasingly attracted to those phrases such as \"a computer-assisted review of 1.2 million records found...\"\n\nIt's imposing, comprehensive and satisfying to see that number in print when weeks and months of your life prior to computers was spent sifting and stitching together a few hundred paper files for investigative projects.\n\nBut recently I had a sharp reminder of how some simple nuts and bolts computer-assisted reporting that uses only a few hundred records - can serve as a tipster, a backgrounder and an investigator, and end up having high impact.\n\nLast October, Courant police reporters noticed a streak of unsolved murders involving minority women in the Hartford area, and they mentioned it in a story in the back pages of the newspaper. However, the reporters had trouble going further or pinning down police investigators because they had no systematic files on the murders and our clip library is not electronic.\n\nI decided to create a computer file for the reporters that would include all of the unsolved murders of minority women in the state during the last few years. I created one file in an XDB database from our clips that included details of the murders and demographics of the victims. I knew it was incomplete, but it gave us about two dozen solid cases.\n\nWith a little sorting and comparing and working with the police reporters and Leah Segal, our Courant researcher, we culled the list to about six or seven cases. The reporters then began questioning police in the Hartford area, using the details from the database. The police thought we had a tipster and conceded that they had quietly been looking at links among five murders. That broke the first story that there might be a serial killer on the loose.\n\nThe police reporters, Gina Briscone and Andrew Julien, and I then collected more information for the database by visiting the sites where the victims were last seen and where their bodies were dumped and by interviewing victims' friends and families.\n\nThen I went to the state's medical examiner and asked for his computerized files. The medical examiner was worried about confidentiality but gave us several fields of information.\n\nUnfortunately, he didn't know how to download the information to diskette. I asked him to just give me a printout, and we borrowed a scanner from our advertising department and scanned about 200 records into the computer. Then I added some information from a death certificate tape we had recently acquired.\n\nAs the reporters were using the database to help do individual stories on the victims, I linked the various files together and started sorting and comparing while consulting experts on serial murders. It appeared there were similar murders throughout the state. On one interesting run, it appeared the murders generally occurred on a quarterly basis.\n\nWe also talked to a university professor who had data on 16 years of homicides in Connecticut. At our request, he did a quick run with our criteria and found the recent series of unsolved murders was a definite - if not statistically proven - blip on a chart of such murders.\n\nA few interviews later and we had two more stories that expanded the possible range of the murders and showed that police in different jurisdictions were investigating similar murders without talking to each other.\n\nBy now we had some investigators who were sure we had access to confidential files and wanted to know what we knew, and some of whom were full of culpas. One investigator told me point blank that we had done what police should have done months before.\n\nAs a result, the state's chief prosecutor convened a task force in January composed of police departments to look into at least 11 unsolved murders.\n\nA caveat: We never asserted there was a serial murderer at work. We simply established a pattern of murders that the police agreed should be seriously investigated. Even if two or three killers are not responsible for all the murders, would it be good news that there were 11 killers instead?\n\nWe also focused attention on the murders of minority women - some of whom were prostitutes - that had not been there before.\n\nSadly, many cities have a collection of unsolved homicides of black or Hispanic prostitutes that are dismissed as routine (murder is supposed to be an occupational hazard). In several other states, serial murders have plied their trade for years because society generally could care less about those kinds of victims.\n\nAnd our stories also allowed the newspaper to reaffirm its ties to our inner-city neighborhoods. We were thanked many times by victims' families and friends who thought these were the kinds of cases that the Courant wouldn't care about, much less investigate.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-april-mo-education-department-releases-test-scores-to-micar","slug":"1992-april-mo-education-department-releases-test-scores-to-micar","url":"https://dwillis.github.io/uplink/article/1992-april-mo-education-department-releases-test-scores-to-micar.html","year":1992,"month":"April","monthNum":4,"headline":"MO Education Department releases test scores to MICAR","kicker":"","author_name":"Russ Buettner","author_title":"research assistant","affiliation":"MICAR (Missouri Institute for Computer-Assisted Reporting)","summary":"A journalist uncovers how Missouri's Department of Education restricts access to standardized test data through bureaucratic obstacles, preventing even the state itself from analyzing results by demographic factors, though creative record-keeping tactics ultimately provide a workaround.","keywords":["data","education","access"],"topics":["Data & Databases","Investigative Methods","Education"],"technologies":["XDB","WordPerfect"],"full_text":"The Missouri Department of Education wrongfully imprisoned valuable data, at least until continued effort threatened to require movement. Since beginning an education project last September, the Missouri Institute for Computer-Assisted Reporting has acquired various computer files related to education. The Missouri Mastery and Achievement Test, this state's version of a standardized test, seemed a logical addition.\n\nThe department's usually helpful programmer could tell me nothing about the file. So I called John Jones, director of school data. In his lofty lobe, journalism serves no purpose beyond nuisance.\n\n\"Russ, I don't keep that information, and I don't want it,\" he said. \"If I had it, every newspaper in the state would be calling me for their little score sheets.\"\n\nYou can imagine the sickening image: a director of data providing data.\n\nI asked him who had the data. He said the department contracted with the Educational Assessment Center, another state agency, to administer, score and store the test results.\n\nSo I call Sharon Shattgin, assistant director of the center.\n\n\"See technically, we don't own the data,\" she told me. \"Technically, it's the property of each school district. We contract with each superintendent. The data is public information, but you have to get a release from the superintendent.\"\n\nBut I didn't want one district. I wanted all 453 districts.\n\n\"I guess you'd have to get a signed release from each district,\" she said.\n\nEven over the telephone, you can tell when someone is talking through a smirk. I asked her if there was anyone else I could speak to about this.\n\n\"Have you spoken to John Jones?\"\n\nAs ridiculous as it sounds in itself, this arrangement, clearly designed to block access to the data, had a much worse effect. As a result of not \"owning\" the data, the department of education has never analyzed test results in their entirety, nor by gender, race, rural versus urban, or in any other way. It only publishes an annual mean based on a 10 percent sample.\n\nIn other words, the state spends millions of dollars administering the tests, but does not use it to uncover weak spots.\n\nAlthough the leaders in Missouri education seemed to think they had cleverly locked up the data, even from themselves, they had unknowingly propped open a window for us.\n\nJones, incarcerator of data, had provided us earlier with a computer file containing the title, name and mailing address of every school administrator in the state. It took less than an hour to download the file, move it into an XDB table and export the table as a WordPerfect mail merge file. I then wrote a form letter to the superintendents explaining this ridiculous situation and informing them of their responsibility under the state's Sunshine Laws to respond within three working days.\n\nBut I never intended to actually print, let alone mail, those letters.\n\nInstead, I wrote a letter to Otis Baker, the director of elementary and secondary education, explaining our efforts and the apparent necessity of the mailing. I told him our form letters and return envelopes were addressed directly to him \"to insure their authenticity,\" and that each letter referenced his name and phone number \"should any of the 453 superintendents have any questions regarding the necessity of this procedure.\"\n\nI told him we had scheduled everyone on our staff for an envelope-stuffing party on March 3, three working days after he received the letter. If I didn't hear from him by then, out the letters went.\n\nWhen I arrived at the MICAR office on March 3, I found that Mr. Baker had called.\n\n\"He said he needs to talk to you TODAY,\" said MICAR Administrative Assistant Michele Ward. \"He said he just got your letter today. He'll be in meetings all day, but his secretary will interrupt when you call. He definitely needs to talk to you today.\"\n\nOtherwise deaf bureaucrats seem to perk up to the threatening sound of 400-plus letters and phone calls thundering in over a few days.\n\n\"I just got your letter today,\" he said when I called. \"Don't send out those letters. You've gone to too much trouble already. I'll send out a release letter today.\"\n\nIn an eventual story, we might tell Missouri's citizens more than they know about their education system. We might also tell Missouri's education leaders more than they apparently want to know about the jobs they do. Yet I doubt the Department of Education will begin graciously providing files.\n\nNonetheless, maybe computers can help us free more wrongfully imprisoned data by presenting the jail keeper's greatest menace - the threat of imminent labor.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-april-will-early-version-woes-hurt-xdb","slug":"1992-april-will-early-version-woes-hurt-xdb","url":"https://dwillis.github.io/uplink/article/1992-april-will-early-version-woes-hurt-xdb.html","year":1992,"month":"April","monthNum":4,"headline":"Will early version woes hurt XDB?","kicker":"","author_name":"David Raziq","author_title":"","affiliation":"MICAR","summary":"Journalists using the XDB database software report significant technical problems including import failures, indexing errors, and inconsistent query results, leading some to switch to alternative programs, while others continue to praise XDB's speed and flexibility.","keywords":["database","software","journalism"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["XDB","dBase","Paradox","Datacase"],"full_text":"For two weeks in July 1989, Bob Paynter was supposed to be on vacation. But during his stay at a North Carolina beach, he frequently found himself on the phone. The problem: his software. Paynter specializes in computer-assisted reporting for the Akron Beacon-Journal, and he says his database program of choice, XDB, was giving him major problems.\n\n\"I was very concerned,\" says Paynter. \"I was involved in a project involving campaign finances that required me to move a large database back and forth between XDB and Dbase.\"\n\nPaynter says his troubles began when he tried to import the information back into XDB. The program wouldn't accept it. So he placed the first of many calls to the software company.\n\n\"There were people joking in our department that suggested I demand a development fee from XDB,\" Paynter says.\n\nIn the last few months the staff of MICAR has heard a number of complaints from XDB users. Bob Paynter's story is representative.\n\nFor example, Paynter himself finally ended up shipping his entire database to XDB. During his break, XDB's technical support eventually solved his difficulty with a new import engine.\n\nIt wasn't long before he experienced another set of problems - indexing a large file.\n\n\"It would take a split second to index a file of almost two million records so I knew something was screwed up,\" he says.\n\nWhen Paynter queried about the values within the file, his computer came up blank.\n\nSo after another month trouble-shooting with XDB's technical support staff, Paynter got the file to index. But he found a new bug: Given the exact same query, the program would return different answers.\n\nAnd that, he explained, was the last straw.\n\n\"I said 'I just can't trust this,'\" he says. \"And I talked to other database managers, and nobody had ever heard of XDB.\" So after consulting friends, Paynter switched to the Paradox database system.\n\nEd Foldessy of the Wall Street Journal has also abandoned XDB for his record analysis. \"XDB is not graceful with numbers,\" he says. \"I encountered every conceivable problem working with it.\"\n\nAccording to Foldessy, one of those problems included sorting fields. \"It wasn't giving a good sort,\" he said. \"You might have .01, .02, but then you'd have a number that was out of order.\"\n\nLike Paynter, Foldessy says he \"will never use XDB again,\" and instead relies on Datacase, his corporation's recommended program.\n\nHowever, many reporters stand by the XDB product. Elliot Jaspin, the director of the Missouri Institute for Computer-Assisted Reporting, is one of the pioneers in this reporting technique. He has had a long standing relationship with XDB.\n\n\"When I was teaching at Columbia University, I began searching for a database program,\" says Jaspin. \"So I went down to PC Magazine's software library and tried them all. And XDB was the winner, hands down.\"\n\nJaspin lists speed, flexibility and ease of use as some of the product's strong points. Because he also uses XDB in his computer-assisted reporting seminar, he says the program has some other advantages. \"It's a good teaching tool,\" he says. \"It has a simple interface, and it has good implementation of structured query language.\"\n\nBut Jaspin also readily admits that the program has had an \"unacceptably high number\" of problems. Specifically, Jaspin mentioned problems indexing large files, and he has noticed an \"unevenness\" in XDB's support staff.\n\nAccording to Mike Waters, XDB's Director of Sales, most of the problems were found on an earlier version of the software.\n\nFor example, Waters explained that Paynter's indexing problem was due to the index pointer being defined as a small integer. \"So once you got up to a little over 31,000 unique indexes, you ran out of numbers,\" he says.\n\nWaters said his company corrected those types of problems with the 2.4 version of XDB and currently markets a 2.41 version of the software.\n\nWaters also explained that the once-limited indexing function of the 2.3 software could have caused Paynter to get different answers for the same query. \"When you do the query, the optimizer inside is actually building indexes on a slide,\" he said. \"So if he was dealing with a lot of records with the 2.3 version, he could have gotten inconsistent results if he got more than 31,000 on an index scheme.\"\n\nHowever, Waters said he was unaware of any complaints about sorting fields of numbers, as was the problem Ed Foldessy experienced.\n\n\"As a matter of fact,\" he says, \"I offer free products to any of these guys that made a comment to you. I'd love for them to try them again.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-april-bytes-bits-and-nibbles","slug":"1992-april-bytes-bits-and-nibbles","url":"https://dwillis.github.io/uplink/article/1992-april-bytes-bits-and-nibbles.html","year":1992,"month":"April","monthNum":4,"headline":"Bytes, bits and nibbles..","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Winners of this year's IRE reporting competition exposed workplace safety failures and racial profiling in law enforcement through investigative journalism using computer-assisted analysis.","keywords":["investigation","discrimination","safety"],"topics":["Investigative Methods","Crime & Justice","Public Safety"],"technologies":[],"full_text":"Racist cops and lethal working conditions were a few of the problems exposed by the winners in this year's IRE reporting competition.\n\nMike Casey and Russell Carollo of the Dayton Daily News won for their series \"Worker Safety in America.\" The team used computer analysis of records from the Occupational Safety and Health Administration to document the failure of that agency to protect American workers.\n\nCarolyn Tuft of the Belleville (Ill.) News-Democrat won for a four-month computer-assisted investigation of city traffic tickets that revealed the city's police department was targeting black motorists in attempt to deter them from entering the city.\n\nWinners will receive their awards in June at the IRE National Conference in Portland, Ore.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-may-dear-readers","slug":"1992-may-dear-readers","url":"https://dwillis.github.io/uplink/article/1992-may-dear-readers.html","year":1992,"month":"May","monthNum":5,"headline":"Dear Readers","kicker":"","author_name":"David Washburn","author_title":"Editor","affiliation":"Uplink","summary":"David Washburn introduces himself as the new editor of Uplink, replacing Bob Jackson, and announces a redesign effort by new designer Joyesha Bhattacharya to make the newsletter more user-friendly with enhanced graphics and reader engagement.","keywords":["editor","redesign","newsletter"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Dear Readers: I would like to take this opportunity to introduce myself as the new editor of Uplink. I am taking over for Bob Jackson, who is still with MICAR. and can be reached at his old number. However, beginning with this issue. I will field any questions or comments concerning Uplink.\n\nWe also have added a new designer to our staff. Joyesha Bhattacharya created Uplink's new cover design, and in the coming months will be working on a complete makeover of the newsletter.\n\nThe purpose of the redesign is to build on Bob's foundation and make Uplink a more \"user friendly\" newsletter. Upcoming issues will feature more graphics and photos, plus greater opportunities for reader input. We want Uplink to be as useful a tool to you as your computer.\n\nWe appreciate continued correspondence from our readers: not only your successes, but your problems and pitfalls as you boldly go where few reporters have gone before. Please call us at (314) 882-0684 with any questions or comments. - David Washburn Editor","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-may-star-reporter-dodges-usda-obstacles","slug":"1992-may-star-reporter-dodges-usda-obstacles","url":"https://dwillis.github.io/uplink/article/1992-may-star-reporter-dodges-usda-obstacles.html","year":1992,"month":"May","monthNum":5,"headline":"Star reporter dodges USDA obstacles","kicker":"","author_name":"David Raziq","author_title":"","affiliation":"MICAR","summary":"Greg Reeves of the Kansas City Star navigates complex challenges in accessing USDA farm subsidy data, which contributed to the Pulitzer Prize-winning series 'Failing the Grade.' His investigation uncovered manipulation in farm subsidy claims involving multiple fake ownerships to bypass payment limits.","keywords":["USDA","subsidy","investigation"],"topics":["Investigative Methods","Data & Databases","Agriculture & Land"],"technologies":["XDB","Qualstar 1260","Vax","Nine Track Express","Datatrieve"],"full_text":"Star reporter dodges USDA obstacles By David Raziq MICAR \"From the standpoint of computer-assisted reporting. this was a techno-nightmare.` said Kansas City Star reporter Greg Reeves. \"It took three gigabites of space. the tapes blew up my Qualstar 1260 taped- rive and the number of records overwhelmed XDB.\" But Reeves isn't complaining. because the \"nightmare he's describing is \"Failing the Grade a comprehe- sive and critical look at the United States Department of Agriculture that this April won his paper a Pulitzer Prize for national reporting. The effort of reporters Mike McGraw. Jeff Taylor. Reeves and editor Mark Ziemann, the seven-par series also won an Investiga tive Reporters and Editors Award earlier this year. The Star's investigation began nearly two years ago as the result of previous stories by reporters McGraw and Taylor. One such story included the problem of dirty grain farmers and grain producers adding dirt to grain export to increase the weight and therefore the price of shipments. That and other chronic problems. savs Reeves. convinced the Star that the USDA's troubles \"went much deeper\" than any- one realized and SO the project began. Eleven months later. Reeves joined the investigation. \"For my part of it. he says. \"I asked the USDA for the last five years in farm subsidy pay- ments. The U'SDA pays out over ten billion dollars in subsidies and Reeves suspected he might find some ir- regularities to those payments. But Reeves effort soon hit a snag when the government denied his request. We started throwing lawyers at each other. Reeves said. but \"unlike other agencies that ultimately agree to give you the records. the USDA said go ahead and sue us. I think they denied us access simply because they had never given them to anybody before and because bureaucrats tend to believe its their infromation and not the public's. Greg Reeves spent months trying to access USDA farm subsidy tapes. But getting the tapes was the easy part. Three months later a compromise was finally reached: The USDA would provide 8.2 million com- puter records of only 1990's payments. Those tapes included the check's serial number the date, county, and state the check was issued for, and even an identi- fying number for the intended farmer but excluded the reiver's name and address. However that was just the beginning of Reeve's problems, because with the tapes in hand. he now experienced difficulties in down-loading the information into his computer. \"Our tapedrive could not read these tapes. The tape quality was marginal and the Qualstar 1260 is a piece of trash when it comes to reading marginal tapes.\" Reeves says that although a recent upgrade has solved most of the Qualstar's reading problems. he felt discouraged at the time. Starting in July 1991. Reeves says he \"worked nearly around the clock seven days a week to load these tapes to my PC... We failed.\" So Reeves decided to install the nine tapes on the Star's Vax Mainframe processer using Nine Track Express for the download and then Datatrieve, the mainframe's database software. for the analysis. \"I wrote hundreds of programs to extract this data,\" says Reeves. and in some cases ran each of them dozens of times\" for a two-month period. Finally he created an analysis based on those farmers who got , most payments along with their county and state. the investigation was still missing one thing: the names of the farmers or producers receiving the money. So using the check's serial number, the date of issue, and purpose, Reeves began calling various county USDA offices to match the checks with their receivers. As a result, one of the investigation's findings included the unchecked growth of what Reeves calls `Mississippi Christmas Trees.\" \"We've found that in ten states.\" he says. \"there are more farmers getting farm subsidy pay- ments than there are farms. What's happened is the farmer's have split up their farms under SO many different ownerships in order to skirt the USDA's fifty-thousand dollar per farmer limitation.\" Asa result. the Star's team found wives. children, cousins, uncles, and even family doctors and lawyers all listed as producers with the USDA and thus receiving subsidy checks. Although Reeves depended on a mainframe for \"Failing the Grade,\" he says he still primarily believes in PC-oriented research methods and has some recommendations for those organizations just starting their own computer-assisted programs. Go for a PC-based set up that takes care of your needs at least two years out: namely a 486 33 megahertz with a minimum two giga-byte harddrive,\" he says. In addition, Reeves recommends Foxpro. Paradox, or SAS database software for the analysis, Nine Track Express for downloading tapes, and a Pertech. Cipher. or Storagtech tape drive. \"The whole set-up\" says Reeves. would be around twenty thousand dollars.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-may-letters","slug":"1992-may-letters","url":"https://dwillis.github.io/uplink/article/1992-may-letters.html","year":1992,"month":"May","monthNum":5,"headline":"Letters","kicker":"","author_name":"Lex Alexander","author_title":"Investigative/Database Reporter","affiliation":"Hartford Courant","summary":"A journalist shares their experience with XDB database software versions 2.30 and 2.41, describing problems with sorting accuracy, computational errors with large numbers, and RAM memory management issues, and noting improvements in version 2.41.","keywords":["software","database","computation"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["XDB","Toxic Release Inventory"],"full_text":"Dear Uplink:\n\nI got my April 1992 Uplink today and was intrigued by the reports of problems with XDB software. I have had some of the same problems, and I'll be waiting to hear what solutions arise.\n\nI got XDB 2.30 in May 1991. We have not yet attempted any projects as complicated as those described by Robert Paynter, so I hadn't run into the problems with indexing, but I had noticed two other problems mentioned in the article: the software would give some sorts out of order (the decimal point example cited by Ed Foldessy of the Wall Street Journal is a perfect parallel), and its computations were not always trustworthy.\n\nAs an example of the former, I've run a fair number of queries on census data, and I frequently found that when I calculated percentages-say, ranking census tracts by percentages of minority residents-two percentages that were a point or two apart would occasionally appear out of order. I eliminated most of those problems by simply adding another float decimal place (F10 utility menu, then F5 user profile, then F6 format options, then F8 float decimal places), but that didn't get rid of them all, and even with version 2.41 I've had to keep a close on any calculations and orderings in which decimal places were significant. Perhaps I should've been less tolerant.\n\nI ran into the other computational problems in working with a cut-down version of the 1989 Toxic Release Inventory data for North Carolina. I found that XDB 2.30 could rank them accurately statewide-it seemed to ignore any record with a toxics total exceeding seven digits, of which we had at least one. (This wasn't apparent from the data itself; fortunately, we had other TRI reports with which to compare some of our data and could tell right off that there was a problem.) However, version 2.41 totals the larger numbers accurately.\n\nOne other problem that 2.41 solved for us was using the PC's random access memory. We bought four megs of RAM with the machine, and in my relative ignorance of computers I'd assumed that if it was in the machine, it was being used. Meanwhile, when I tried to import census data into 400-field tables and kept getting the X10 error message-insufficient RAM I just figured that four megs wasn't enough and built smaller tables. I didn't learn about basic versus extended memory until stumbling over Jack Warner's computer column from Cox News Service on the wire a couple of months ago- explained what the problem was, although it didn't suggest anything to solve it that worked for us. Dave Stroble, our audiotext product manager and a PC fiend, futzed around with my AUTOEXEC.BAT and CONFIG.SYS files but couldn't solve the problem. Then we got XDB 2.41 and its extended memory manager. I don't know exactly how it works, but since I installed 2.41, I've had no more problems with totaling numbers or importing files with many fields. (I haven't had occasion to try 400, but I have gone as high as 288 with no problem. Not only that, 2.41 appears to import about 20 percent faster than 2.30.)\n\nIf XDB has figured out how to fix the ordering bug, I'd like to hear about it. Meanwhile, please keep us posted on other problems, even if it doesn't make for as compelling reading as does the Hartford Courant's serial killer!","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-may-bytes-bits-and-nibbles","slug":"1992-may-bytes-bits-and-nibbles","url":"https://dwillis.github.io/uplink/article/1992-may-bytes-bits-and-nibbles.html","year":1992,"month":"May","monthNum":5,"headline":"Bytes, bits and nibbles","kicker":"","author_name":"Lex Alexander","author_title":"Investigative/Database Reporter","affiliation":"Hartford Courant","summary":"Reporters across the United States are increasingly adopting computer-assisted reporting techniques to conduct investigations and access government databases, with newsrooms like the Raleigh News and Observer demonstrating the effectiveness of this approach despite obstacles from state and local governments attempting to restrict access to digital records.","keywords":["computers","reporting","databases"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["9-track tape","mainframe"],"full_text":"Reporters and editors around the United States are warming up to the idea of integrating computers into journalism. And if the packed seminar on computer-assisted reporting at the Investigative Reporters and Editors conference in Atlanta is any indication, the coming years will see the concept catch fire in our nations newsrooms.\n\n\"This is not a luxury. This is not the future. This is a way to get information now,\" said Bill Dedman, I.R.E. Board Member and freelancer. Dedman, who is currently writing a book on computer-assisted reporting along with MICAR founder Elliot Jaspin, said the use of computers in investigative journalism is changing the way government and the media do business together.\n\nThe Freedom of Information Act and access laws are slowly beginning to facilitate the process for those who use this new technology.\n\n\"The general premise is that if you can get it on paper, you can get it on computer.\" But, Dedman said state and local governments are routinely putting roadblocks in the way of reporters who attempt to gain access to computer records.\n\nFor example:\n\nThe state of North Carolina's attempt to charge the Raleigh News and Observer more than $12 million for the state's driver license records, which are on nine-track magnetic tape. The state and the newspaper are currently in litigation to decide on a fair price for the records.\n\n\"The federal government has spent billions of dollars inputting information on more than 32,000 mainframe computers,\" Dedman said. \"And it spends no money telling you which databases it has.\"\n\nDespite the government's obstacles, reporters are turning out award-winning stories that would have been impossible to complete without computers.\n\nThe News & Observer has arguably one of the best computer-assisted programs in the country. With about half of their newsroom trained in analyzing computer databases, the newspaper churns out computer-based stories on a daily basis. Here's a taste of what the News & Observer has been working on the past few months:\n\n*Computer analysis of North Carolina Marriage records for the past 20 years yielded an interesting story for reporter Mary Miller. Her analysis found that although Friday is the most popular day to get married at the Wake County Courthouse, the 13th is always a slow day. It seems that people are superstitious about the 13th, no matter what day of the week it falls on.\n\n*News & Observer reporter Carol Blaney performed a computer analysis of the state's escheat and unclaimed property program and was able to detail how uncashed checks, stocks, forgotten accounts, jewelry and cash left at hospitals, banks and other businesses can be found. Her story, \"Wake residents owed $305,579: Escheats office holding millions in unclaimed funds,\" explains how the state runs the program and how the interest is used to fund the program and provide college scholarships.\n\n*In February, reporters Steve Riley and C.E. Yandle performed a computer analysis of records from the labor department and the state office of the chief medical examiner. They found that because of Labor Department oversights, many on-the-job deaths remain uninvestigated. Their story, which ran last month, detailed seven cases of workers dying on the job. The deaths were never reported to all the correct authorities and never fully investigated. The News & Observer story stated that the Labor Department misses one worker death in four and the department relies on employers and the news media to notify them of worker deaths instead of checking in with medical examiners and taking a more proactive stance.\n\nAlso busy turning out computer-assisted stories are Doug McInnis and Steve Stephens of The Columbus Dispatch. The reporters recently completed a computer analysis of drug-crime records and conducted scores of interviews. The end result was a series entitled \"Drug killings boom in just 16 months.\"\n\nThe series depicts the drive-by killings, crack killings and other drug killings that have shattered Columbus' image as a big city without big-city crime. The series includes graphics of the rising pace of drug-related killings and statistics revealing that they have become the leading type of homicide in Columbus.\n\nComputers are being used more and more on breaking stories. For example, when a plane crashed in Dayton, a reporter using a computer discovered that the plane had previously had 32 serious maintenance problems.\n\nHal Straus from the Atlanta-Journal Constitution pointed out the trend toward using the computer to enhance solid beat reporting. Journal-Constitution reporters used a database to trace campaign contributions to Atlanta's governor and lt. governor. They found that the sponsor of a bill, allowing beverage companies to label just about anything short of sewage as spring water, had contributed heavily to the two campaigns.\n\n\"It took five minutes to search the database and get a really good story,\" Straus said.\n\nIRE seminar\n\nThe widespread interest in computer assisted reporting has led IRE to establish its first all-day seminar on the subject at its national conference in Portland, Oregon. The conference will take place from June 11 to June 14, 1992. For more information telephone IRE at (314) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-may-easing-the-fear-of-a-financial-collapse","slug":"1992-may-easing-the-fear-of-a-financial-collapse","url":"https://dwillis.github.io/uplink/article/1992-may-easing-the-fear-of-a-financial-collapse.html","year":1992,"month":"May","monthNum":5,"headline":"Easing the fear of a financial collapse","kicker":"","author_name":"Trebor Banstetter","author_title":"","affiliation":"MICAR","summary":"A New Jersey journalist created a bank safety ranking system using federal deposit insurance data to ease consumer fears following the 1991 Bank of New England collapse, helping residents evaluate the security of their local financial institutions.","keywords":["banks","safety","ranking"],"topics":["Business & Economy","Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"Easing the fear of a financial collapse by Trebor Banstetter MICAR The bank of New England failed in early January of 1991. It was the biggest bank failure in the history of the United States, and the financial catastrophe caused great concern among New Jersey residents who kept their money in local banks and thrifts. \"The people of New Jersey were very nervous about the safety of their banks,\" said Dan Woods, business writer for the Record of Hackensack, N.J. The fear was SO pervasive, Woods said, that a thrift in Patterson, N.J., had a \"run de- positors withdrawing all of their savings because of a takeover by United Savings Bank, \"There was poor communication, and people thought their money was going to disappear, he said. \"We wanted to give them some information on how safe their bank was. Woods wanted to create a ranking of banks and thrifts that would assist consumers in choos- ing what financial institution was safest. He decided to use a pub- licly available database provided by Ferguson and Co. The people of New Jersey were very nervous about the safety of their banks. We wanted to give the information on how safe their bank was. -Dan Woods The data was based on Federal Deposit Insur- ance Corp. call reports, which are detailed financial reports banks have to file each quarter. and are avail- able on a CD-ROM disk. Obtaining the information from the disk and downloading it into it database program is very similar to using nine-track magnetic tapes. The best way to determine a rating for a bank or thrift. Woods decided. was to use a formula compar- ing an institution's capital and its problem loans. \"In il failed bank, the losses from the problem loans keep piling up and eating away the capital. he said. Each bank's capital and problem loans was divided by the bank's assets to produce a percentage. The capital percentage gave an indicator of how much the bank had saved over the years to act as w-a cushion of last resort. The problem loans indicated how good the bank was at making loans. The capital percentage and the problem loan percentage were given equal weight in the ranking. and were converted into the standard deviation to equalize the numbers. Then, each bank had a ranking between one and five. \"After the story ran. consumers were wildly enthusiastic, Woods said. \"This gave them a way to judge the basic safety of their bank. One of the biggest concerns Woods had was making sure the story presented the data accurately and fairly. \"We were very con- scious of the fact that people would use these ratings to move their money, SO we checked the data and methodology to the extreme. he said. He also pointed out banks and thrifts that had low rankings but good recovery pro- grams. Woods made sure to include in- formation on deposit insurance. SO readers understood their savings were insured by the federal government if under $100.000. \"We didn't want to cause hysteria.\" he said. In 1992 Woods did a second ranking. this time including several significant out-of-state banks. \"Again. the response was overwhelming. The stories have won several awards, including a third prize in the enterprise category from the New Jersey Press Association. They also won a C.T. Rutgers award for business reporting. Woods encourages others to try similar rank ings of financial institutions in their areas. but warns that extreme caution and care must be used. - If you print the wrong data. it can be devastating. he said. But. he added. \"It's definitely worth the effort. I have received calls from senior citizens who told me they would sleep a little easier.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-june-foxpro-paradox-or-xdb","slug":"1992-june-foxpro-paradox-or-xdb","url":"https://dwillis.github.io/uplink/article/1992-june-foxpro-paradox-or-xdb.html","year":1992,"month":"June","monthNum":6,"headline":"FoxPro, Paradox or XDB","kicker":"","author_name":"Bob Jackson","author_title":"Director","affiliation":"MICAR","summary":"A comparison of three popular database management programs—FoxPro, Paradox, and XDB—used in computer-assisted reporting, examining their strengths and weaknesses based on journalists' real-world experiences.","keywords":["database","software","reporting"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["FoxPro","Paradox","XDB","SAS","DataEase","SQL"],"full_text":"FoxPro, Paradox or XDB\n\nby Bob Jackson MICAR\n\nWhich should you choose?\n\nWhen you're in the middle of indexing more than 250,000 records you need to be using a database software that can accomplish the task in a quick, efficient and easy manner.\n\nMore importantly, if you're sorting the same 250,000 records, you need software you can trust.\n\nIn the world of computer-assisted reporting, there are plenty of database management programs to choose from. Foxpro, XDB and Paradox are the most widely used by CAR programs. Other programs like PC SAS and DataEase are also popular in some newsrooms.\n\nWhich is the best?\n\nThat depends on who you talk to.\n\nIf you were to ask Brant Houston, who manages the computer-assisted reporting program for the Hartford (Conn.) Courant, the answer would be simple:\n\n\"You make the software you have work for you.\"\n\n\"A lot of it has to do with personal taste, and what works best for you,\" Houston said. \"And part of it has to do with the task you are trying to accomplish.\"\n\nHouston primarily utilizes XDB, but also depends on Paradox and PC SAS to analyze data. He is also in the process of taking a look at Foxpro's capabilities, and may make the switch from XDB in the future.\n\nCurrently, he explained, XDB's performance is getting the job done.\n\n\"We're using the new version of XDB (version 2.41) and although we are still encountering bugs, the software is very much improved over earlier versions.\" Houston said that XDB is easy for reporters to learn, and is also great for smaller databases.\n\n\"But the larger the database the slower XDB runs,\" he explained.\n\nMany reporters started their programs with XDB, but have since made the switch to other products because of different software problems (see April, 1992 Uplink). Reporters have cited problems indexing and sorting, but according to XDB management, those problems have been ironed out.\n\nIndexing with the earlier versions of XDB, for instance, became a problem when the software encountered more than 31,000 unique indexes.\n\nAlthough XDB has taken a bad rap because of past software problems, Elliot Jaspin, director of the Missouri Institute for Computer-Assisted Reporting, still stands by the software.\n\nJaspin lists speed, flexibility and ease of use as some of the product's stronger points. He also says that XDB has a simple interface and has good implementation of structured query language. Jaspin uses XDB in MICAR's quarterly seminars, which attract reporters from across the country.\n\nGeorge Landau, of the St. Louis Post-Dispatch, was one of the computer-assisted reporting project managers to dump XDB.\n\n\"I started with XDB for a year and found that my tables were getting too large for the software to handle quickly, and I began having problems,\" Landau said. \"Other bugs in the software made me lose confidence in the product and I was forced to switch to something I could trust.\"\n\nLandau now uses Foxpro version 2.0 and is extremely pleased with the product, calling it \"fast, full featured and bug free.\"\n\n\"Foxpro is very fast and I've found that it does a better job of chewing through large volumes of data,\" he said. \"It has the analytical power of SQL but runs much faster.\"\n\nLandau is also impressed with its report writing function. \"It's wonderful to work with. It provides you with easy to read reports quickly.\"\n\n\"I've found that Foxpro can analyze greater amounts of data quicker and it has greater analytical power than XDB,\" Landau added. \"It is the nicest, easiest software I have found.\"\n\nBob Paynter, of the Akron Beacon-Journal, switched from XDB to Paradox because of major problems encountered with the software. He made the switch, he explained, after consulting some colleagues.\n\n\"I said 'I just can't trust this (XDB),'\" Paynter said. \"And I talked to other database managers, and nobody had ever heard of XDB.\"\n\nParadox is a menu driven program that doesn't require the user to know procedural language (such as SQL), making it easier to use. The drawback to using Paradox is that, like XDB, the program has a hard time managing large volumes of data.\n\nPat Stith, who manages the CAR program at the Raleigh (N.C.) News and Observer, is one of a growing number of Foxpro users. \"We stumbled into Foxpro and are extremely happy with it,\" he said.\n\n\"Foxpro is excellent because it is much faster than XDB,\" Stith said. \"We can go through 1 million records in a very short period of time. You just don't sit around with Foxpro.\"\n\nStith explained that Foxpro is simple to use, which makes it easy to teach to reporters. For Stith that's important because he is responsible for conducting the News and Observer's two-week training seminars for reporters. The seminars, which are run periodically, have trained about 30 of the newspaper's reporters.\n\n\"I like Foxpro for three reasons,\" Stith said. \"It's easy to understand; I believe it is the fastest database handler on the market; and I have had zero problems with accuracy.\"\n\nThe Boston Herald's Dave Armstrong agrees. \"Queries that would take overnight with XDB are taking a matter of minutes with FoxPro,\" Armstrong said. \"We will never go back to XDB.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-june-tech-tips","slug":"1992-june-tech-tips","url":"https://dwillis.github.io/uplink/article/1992-june-tech-tips.html","year":1992,"month":"June","monthNum":6,"headline":"Tech Tips","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A tutorial on using SQL to categorize continuous data into ranges, demonstrating how creating a range field and grouping by it is more efficient than repeatedly running queries with different WHERE clauses.","keywords":["SQL","database","ranges"],"topics":["Data & Databases","Investigative Methods","Health & Environment"],"technologies":["SQL"],"full_text":"SUBJECT: building ranges in SQL\n\nPROBLEM: CBS Evening News obtained a database of approximately 15,000 children who took a test to measure the lead levels in their bodies. While the database had individual scores ranging from 0 to 45, CBS wanted a report showing the number of children with lead levels of:\n\n10 to 15\n16 to 20\n21 to 25\nover 25\n\nThere was also the possibility that the producer would want further analysis of the children within one or several of the ranges.\n\nSOLUTION: One approach would be to do a count based on the values in the test field:\n\nselect count (*) from leadtest where test >= 10 and test < 16\n\nThis approach requires running the same query, with the test ranges modified, again and again for any further analysis. Also, there would be several pages of reports to sift through. Not a major problem, but not a very elegant solution either.\n\nA better approach is to create a new field called \"range.\" This is a character field 8 bytes long. The next step is to run the following update command:\n\nupdate lead test set range = \"10 to 15\" where test >= 10 and test < 16\n\nAfter this command is run four times for each range, the following query will produce a one page report with all the necessary information.\n\nselect range, count (range) from leadtest group by 1\n\nAny subsequent analysis would use the range field to group the children presumably saving a substantial amount of time. If the table were larger, space could be saved by making the range field a one byte code and using a second table called \"Ranges\" with a code field and the text such as \"10 to 15.\" A join of the two tables would then produce the same results.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-june-what-does-it-take-to-run-a-successful-computer-assisted-reporting-program","slug":"1992-june-what-does-it-take-to-run-a-successful-computer-assisted-reporting-program","url":"https://dwillis.github.io/uplink/article/1992-june-what-does-it-take-to-run-a-successful-computer-assisted-reporting-program.html","year":1992,"month":"June","monthNum":6,"headline":"What does it take to run a successful computer-assisted reporting program?","kicker":"Still on the ground","author_name":"Trebor Banstetter and Lou Grieco","author_title":"","affiliation":"MICAR","summary":"Step-by-step guide on how to build better ranges in SQL for effective data reporting and analysis.","keywords":["SQL","ranges","data"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["XDB","Nine Track Express","UNISYS","PC"],"full_text":"The answers vary widely. But as more and more newsrooms across the nation begin to use computers and data analysis on a daily basis, reporters and editors are looking for the key ingredients that separate the successes from the failures. For some newspapers, providing reporters with the ability to read data off government computer tapes is an investment that gives a quick return. For others, however, the pay-off is not so swift. Starting a computer-assisted program can be a long, complicated, expensive process that offers limited results. Uplink interviewed principal reporters in several computer-assisted programs to find out why some are flying high and others are having trouble getting off the ground.\n\nJournalists who have experimented with computer-assisted reporting with few results give a number of reasons why they have had trouble starting a successful program: unenthusiastic reporters and editors; rarely used equipment that makes it hard to find help when problems occur; and the biggest issue, time. It is often difficult to balance other reporting duties with training and working on computer tapes.\n\n\"It's working out, but very, very slowly,\" says Dave Davis, who runs the program at the Cleveland Plain Dealer. Most editors, he says \"go crazy if you suggest they give you someone for two days.\"\n\n\"There're so many stories that have to be done for the next day's paper,\" says Rae Davidson, who runs the program at the Toledo Blade. Davidson, who received training two years ago, says the Blade's program is \"definitely in its infancy.\" Only four stories have come out of the program. All of the reporters whom she originally trained have since left the newspaper, though she is currently working with another reporter.\n\nSteven Bennish, a reporter for the Columbia (Mo.) Daily Tribune, trained at the MICAR two years ago. He says that even with the full support of the managing editor, he has produced no stories with what he learned.\n\n\"I don't think there's really been the time to do it,\" Bennish says. The Tribune employs eight full-time reporters.\n\nThe amount of money invested varies at the different newspapers. For the Tribune, the investment was a few hundred dollars for Bennish's training, because the paper hoped to use MICAR's equipment at the University of Missouri-Columbia. Davidson says the Blade paid $10,000 for equipment, and about $500 for XDB.\n\nThe Roanoake (Va.) Times and World News paid $10,000 for its equipment. However, leadership became an issue when the person in charge of the program resigned only weeks after attending a MICAR conference.\n\nCopy editor Jim Ellison took over, mainly because he had a PC at home and knew a little about computers. He considers the program \"halfway there,\" though it hasn't produced any stories. Ellison says he has \"had to work in between writing headlines.\"\n\n\"My biggest problem is I forget this stuff as I go along,\" Ellison says, adding that the only way to become proficient with computer equipment is to keep working on it.\n\nEllison hasn't been trained in NineTrack Express, so sorting out problems with tapes takes him longer as he tries to reinvent the wheel. This problem also affects the program at the Plain Dealer. Davis says the newspaper invested $80,000 in equipment and has done some stories with it. But the Plain Dealer bought the UNISYS system, which Davis considers a mistake, because few programs use the system. \"It's a problem if you're out of the loop,\" he says. \"Then you're always on new ground,\" when you need help with your system. \"You spend a lot of time and energy doing things people have already done.\" This adds to the problem of limited work time that reporters have at the Plain Dealer. Davis says that about a dozen of them can load tapes and do some basic queries. However, the newspaper is far from management's goal to have the program \"touch everybody's beat and everybody's job.\"\n\nSome CAR programs and their reporters have moved into the new information age with relative ease. What have been the keys to these successful transitions?\n\n\"Persistence,\" said St. Louis Post Dispatch reporter George Landau. \"You run into so many problems working with government data. You need patience and determination to solve them.\"\n\nLandau said the Post's program began when \"they'd had some stories that could have benefitted from data analysis.\" A senior editor, Richard Weil, initiated the program in 1989. \"I'd been working GA for a couple of years, and was pretty well versed in computers, so I was tapped for it.\"\n\nGreg Reeves, reporter for the Kansas City Star who did the data analysis for the Pulitzer winning \"Failing The Grade\" series, said that helping other reporters with ongoing stories has been one of the successful qualities of the Star's program. \"I've started some stories, but the peripheral stories are the most important.\"\n\nBrant Houston, of the Hartford Courant, agrees that helping other reporters is essential. \"You've got to have a willingness to spend a lot of time with people answering questions. If you think you're the only one who should be doing this, you're going to choke your program to death.\"\n\nHouston also attributes some of his success to a willingness to stick with it, despite the initial hours. \"You have to realize that 'full time' means anything from eight to 16 hours a day. It's as though for 10 hours you're a journalist, for four hours you're a technician, and for two hours you're a database administrator.\n\n\"There's no question that success is a two-edged sword.\"\n\nPat Stith, of the Raleigh News and Observer, credits \"the newspaper's patience\" with determining the success of their program.\n\nHe added that an open mind is helpful. \"I learn something new every day. You never learn it all, you just get a little better as you go along.\n\n\"I've loaded 40 databases, more or less, and each one presents a different problem.\"\n\nImplementing a computer-assisted reporting program is not cheap, and financial support from the publication is essential.\n\n\"We were able to wrangle about $12,000 for equipment when we started,\" said Landau. \"The biggest investment was my salary, letting me do this full time.\"\n\nLandau said that they were able to sell the idea to the Post's management as a \"secret weapon\" to gain a competitive edge over the now-defunct St. Louis Sun.\n\nMost agree that to ensure success, a program should include more than one reporter. In-house training of reporters is essential to computer-assisted reporting. \"This should be a routine tool in the newsroom used by a wide range of reporters,\" Stith said. \"This is exactly where we want to go.\"\n\nStith has gone a long way to train his colleagues at the News and Observer and familiarize them with the technology, holding classes for reporters, editors and news researchers. \"The class was successful in destroying the mystery of the computer. That's progress. You can't teach people computer-assisted reporting in 20 hours. It requires a lot of practice.\"\n\nStith said that about 15 reporters have produced one or more computer-assisted stories for the News and Observer in the last year. However, he said, \"We don't have a situation where those reporters can do all their own work independent of anyone else.\"\n\nIt is important that computers be used for a broad spectrum of stories, Stith said. \"We've used databases in reporting business stories and features,\" he said. \"If we can get a database, load it in and crank out the data, why not use it for features?\"\n\nThe Courant's Houston said being able to \"crank\" out the data is vital to a program. \"The challenge now is to continue to integrate it into the newsroom-without creating a lot of bottlenecks,\" Houston said. \"We have to figure out how to make it more accessible to people on the fly.\"\n\nAlthough Landau said he has been doing most of the Post's data analysis by himself, \"I would like every reporter to be able to do their own analysis when they need to. They would need training and periodic refreshers.\"\n\nHouston said it is vital to have more than one person possessing the skills. \"You need a couple of people who are up on things,\" he said. \"Otherwise, if you're sick or go on vacation, everything comes to a grinding halt.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-june-flying-high","slug":"1992-june-flying-high","url":"https://dwillis.github.io/uplink/article/1992-june-flying-high.html","year":1992,"month":"June","monthNum":6,"headline":"Flying High","kicker":"","author_name":"Trebor Banstetter and Lou Grieco","author_title":"","affiliation":"MICAR","summary":"Computer-assisted reporting (CAR) programs at major newspapers have succeeded through reporter persistence, collaborative support, adequate funding, and newsroom-wide training that demystifies data analysis technology. Key pioneers from outlets like the St. Louis Post-Dispatch, Kansas City Star, and Raleigh News and Observer emphasize the importance of patience, multi-reporter involvement, and organizational commitment to make CAR a standard newsroom tool.","keywords":["persistence","training","collaboration"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Some CAR programs and their reporters have moved into the new information age with relative ease. What have been the keys to these successful transitions?\n\n\"Persistence,\" said St. Louis Post Dispatch reporter George Landau. \"You run into so many problems working with government data. You need patience and determination to solve them.\"\n\nLandau said the Post's program began when \"they'd had some stories that could have benefitted from data analysis.\" A senior editor, Richard Weil, initiated the program in 1989. \"I'd been working GA for a couple of years, and was pretty well versed in computers, so I was tapped for it.\"\n\nGreg Reeves, reporter for the Kansas City Star who did the data analysis for the Pulitzer winning \"Failing The Grade\" series, said that helping other reporters with ongoing stories has been one of the successful qualities of the Star's program. \"I've started some stories, but the peripheral stories are the most important.\"\n\nBrant Houston, of the Hartford Courant, agrees that helping other reporters is essential. \"You've got to have a willingness to spend a lot of time with people answering questions. If you think you're the only one who should be doing this, you're going to choke your program to death.\"\n\nHouston also attributes some of his success to a willingness to stick with it, despite the initial hours. \"You have to realize that 'full time' means anything from eight to 16 hours a day. It's as though for 10 hours you're a journalist, for four hours you're a technician, and for two hours you're a database administrator.\n\n\"There's no question that success is a two-edged sword.\"\n\nPat Stith, of the Raleigh News and Observer, credits \"the newspaper's patience\" with determining the success of their program.\n\nHe added that an open mind is helpful. \"I learn something new every day. You never learn it all, you just get a little better as you go along.\n\n\"I've loaded 40 databases, more or less, and each one presents a different problem.\"\n\nImplementing a computer-assisted reporting program is not cheap, and financial support from the publication is essential.\n\n\"We were able to wrangle about $12,000 for equipment when we started,\" said Landau. \"The biggest investment was my salary, letting me do this full time.\"\n\nLandau said that they were able to sell the idea to the Post's management as a \"secret weapon\" to gain a competitive edge over the now-defunct St. Louis Sun. Most agree that to ensure success, a program should include more than one reporter. In-house training of reporters is essential to computer-assisted reporting. \"This should be a routine tool in the newsroom used by a wide range of reporters,\" Stith said. \"This is exactly where we want to go.\"\n\nStith has gone a long way to train his colleagues at the News and Observer and familiarize them with the technology, holding classes for reporters, editors and news researchers. \"The class was successful in destroying the mystery of the computer. That's progress. You can't teach people computer-assisted reporting in 20 hours. It requires a lot of practice.\" Stith said that about 15 reporters have produced one or more computer-assisted stories for the News and Observer in the last year. However, he said, \"We don't have a situation where those reporters can do all their own work independent of anyone else.\" It is important that computers be used for a broad spectrum of stories, Stith said. \"We've used databases in reporting business stories and features,\" he said. \"If we can get a database, load it in and crank out the data, why not use it for features?\" The Courant's Houston said being able to \"crank\" out the data is vital to a program. \"The challenge now is to continue to integrate it into the newsroom—without creating a lot of bottlenecks,\" Houston said. \"We have to figure out how to make it more accessible to people on the fly.\" Although Landau said he has been doing most of the Post's data analysis by himself, \"I would like every reporter to be able to do their own analysis when they need to. They would need training and periodic refreshers.\" Houston said it is vital to have more than one person possessing the skills. \"You need a couple of people who are up on things,\" he said. \"Otherwise, if you're sick or go on vacation, everything comes to a grinding halt.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-june-correction","slug":"1992-june-correction","url":"https://dwillis.github.io/uplink/article/1992-june-correction.html","year":1992,"month":"June","monthNum":6,"headline":"CORRECTION","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Uplink issues a correction regarding its previous coverage of the Kansas City Star's Pulitzer Prize-winning USDA investigation, clarifying errors about IRE award wins, technical details, and reporter attribution.","keywords":["correction","journalism","investigation"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["9-track tape","mainframe"],"full_text":"Last month, Uplink reported on the Kansas City Star's investigation of the USDA which won the Pulitzer. However we incorrectly reported that the same series won this year's IRE award. We were also in error in stating that reporter Greg Reeves used the software Ninetrack Express to download the USDA's records into his mainframe precessor. Finally, the dirty grain' series which helped spark the Star's main investigation was written by then reporter Mark Zieman and not \"Failing the Grade\" reporters Mike McGraw and Jeff Taylor. Uplink regrets the errors.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-june-putting-a-price-tag-on-data","slug":"1992-june-putting-a-price-tag-on-data","url":"https://dwillis.github.io/uplink/article/1992-june-putting-a-price-tag-on-data.html","year":1992,"month":"June","monthNum":6,"headline":"Putting a price tag on data","kicker":"","author_name":"Diana Borsi","author_title":"","affiliation":"MICAR","summary":"The media's battle with the Missouri Department of Revenue over charging for access to public information and its implications.","keywords":["information","media","Missouri"],"topics":["Government & Politics","Journalism & Newsroom"],"technologies":[],"full_text":"When you're up to your ears in alligators, it's hard to remember that the original idea was to drain the swamp.\n\nThat's probably what the head of Missouri's Department of Revenue thought when a proposed bill intended to bring in money instead drew the wrath of the press and questions on access to information.\n\nMissouri Senate Bill No. 597 started years ago as a way to raise money by charging for driver's records. The most recent draft of the proposed bill divided public information users into five separate categories with \"amounts charged by the department of revenue for providing copies of driver records based on categories of customers.\"\n\nBill 597 ended, killed by the lack of passage by the Missouri Legislature, but not before it spawned a dialogue between the Missouri Press Association and the Department of Revenue on access to information. The debate also raised the issue of the wisdom of allowing states to use access to information as a revenue source. It also should send up a warning flag to the press in other states.\n\n\"It was never intended to narrow the scope of information,\" said Ray Wagner, director of Missouri's Department of Revenue.\n\n\"Senate Bill 597 was proposed years ago to maintain funding for the Department of Revenue. We didn't think it would have any impact on the media for two reasons,\" he said. First, in the final version of the bill, the media's rates were no higher than the current fees. Second, the department can waive the fees under certain circumstances and often does in the case of media requests, Wagner said.\n\nThe Missouri Press Association took another view. \"We don't think the state should be selling public information for a profit,\" said Doug Crews, executive director of the MPA.\n\nAn ad hoc MPA public documents group met with Wagner and hammered out a four-point letter of recommendation. It included the MPA's position against the state selling information and recommended that the Department of Revenue charge all public information requestors the same fees which should comply with Missouri's open access statute. It also should \"not exceed the actual cost of document search and duplication\"; publish a list of fees for document search and duplication; and develop as soon as possible of on-line access to public records to \"provide for citizens the easiest, widest and least expensive access to the Department's data.\"\n\nInstead of a fight, MPA got support.\n\n\"I fully understand their concerns and I don't disagree with their concerns,\" Wagner said. According to him, the bill won't be back. Instead the Department will seek funds from other sources, make adjustments in personnel and reallocate resources, Wagner said.\n\nBut the issue of money and information remains.\n\n\"This is an example of a nationwide phenomenon,\" said Dr. Sandra Davidson Scott, an assistant professor of communication law at the University of Missouri School of Journalism. \"It's a fact that the state government never has enough money to do what they want to do and so people start casting about for ways to make money.\"\n\nAnd sometimes that means state agencies trying to cash in on the often lucrative information brokering business, a move Scott opposes. \"I don't want to see records accessibility as a money making activity,\" she said, which would create a potential for information restriction. \"You start getting into a lot of problems,\" Scott said, \"especially if you start dividing citizens into categories.\"\n\nWhile the beast of information for money is dead in Missouri, the national economic crunch could bring it to life in other states.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-july-politics-power-and-pcs","slug":"1992-july-politics-power-and-pcs","url":"https://dwillis.github.io/uplink/article/1992-july-politics-power-and-pcs.html","year":1992,"month":"July","monthNum":7,"headline":"Politics, Power and PCs","kicker":"","author_name":"Ronald Campbell","author_title":"reporter","affiliation":"Orange County Register","summary":"This report challenges the assumption that money rules politics by analyzing campaign donations in Orange County. It reveals that powerful local Democrats contributed significantly to campaigns and that fundraising in 1991 was unexpectedly low.","keywords":["politics","donations","analysis"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Orange County is the home of Super-rich moderate Republicans who like to make political contributions and middle-class conservatives who can't afford to. Guess which group dominates Republican primaries in California's GOP stronghold?\n\nIf you bet on the monied moderates, you bet wrong. But so did several wealthy developers who have thrown their chips behind a succession of moderate Republicans, only to see them buried at the polls by hard-right conservatives.\n\nThe Orange County Register documented the split between the county's money and its votes through a computer analysis of nearly 150,000 campaign donations.\n\nThe database was the starting point for several pre-election analyses challenging widely held beliefs, such as the omnipotence of money in politics.\n\nFor example, we showed that local Democrats, usually portrayed as inept sideliners, in fact were powerhouse givers. Orange County raised $5 million for Democratic candidates for the legislature and Congress between 1987 and 1991. But most of this money went to candidates outside the county.\n\nWe also established that 1991 was a lousy year for fundraising, much worse than would be expected in a normal off-year. The falloff couldn't be blamed on a lack of competition. Using a simple test for a candidate's seriousness-could the candidate raise $10,000 an off year?-we documented that political competition was measurably hotter in '91 than in '89.\n\nAnd when one Republican congressman threatened to challenge another GOP incumbent for a newly created seat, we quickly showed the race would be a mismatch, at least in fundraising. (Despite our analysis, the stronger candidate pulled out, unwilling to get into a fight with a rival who publicly said he had nothing to lose.)\n\nComputers were made for stories like these. They are the perfect tool for a political junkie. But there are a few obstacles to watch for before diving in:\n\n*First obstacle: The records are old. Candidates file their end-of-year reports on Jan. 31. Transcribing the reports from paper to computer tape easily can take an additional one to two months. There are only two ways around this obstacle-hire your own people to enter just the records you want or pray that the Federal Elections Commission begins requiring candidates to file reports on computer.\n\nThe Register did not get its state and federal records for 1991 installed until late April. You might be able to shorten the time by subscribing to a database service, but even a little searching on-line will quickly outweigh the cost of maintaining your own database in the newsroom.\n\nThe built-in delay means you can pretty well give up the idea of tracing, say, all donations from trash haulers to legislative candidates until months after the fact. But for spotting long-term trends, nothing beats a computer.\n\n*Second obstacle: The records are filled with errors. Not just the deliberate obscuring of a donor's name or employment. That sort of error I can understand; in a weird sort of way, I can respect it.\n\nBut why would anyone report a donation on \"2/29/90,\" much less several dozen donations on this non-existent date? My database program refused to load 10,000-record tables when there was as much as one misdated contribution. It took me hours to figure out what was wrong.\n\nA second type of error is guaranteed to foul up any database analysis. I refer to inconsistent names. \"Lyon, William M\" and \"Lyon, Gen Wm\" and \"Lyon, William\" all refer to one of the nation's largest homebuilders, a member of the Forbes 400 whose hobbies are collecting antique cars and bankrolling Republicans. But to my computer there are three Lyons, not one.\n\nA few steps will solve this problem. First cast a wide net for donors. If you're looking for everyone who gave more than $10,000, look beyond that set to everyone who gave more than say, $2,500. If you want to get the 10 largest donors, get the 20 largest. This will probably turn up a few donors with inconsistent names. Next run a wildcard search for every last name on your list of top donors; beware of \"Mr.,\" \"Mrs.\" and middle initials. Finally, select a standard name for each donor and use the \"update\" command to change non-conforming records.\n\n*Third and worst obstacle: Lack of imagination. The day you get campaign contributions loaded on the newsroom PC, the political reporters at your paper will clamor for the names of top donors and for total donations by city or by date. After that, ideas come few and far between.\n\nBreaking out of the obvious-question mold is hard. One possibility is to collect conventional wisdom from political writers, then figure out how to prove or disprove it. The conventional wisdom was that Orange County, a stronghold of conservative voters, spent its money the same way; the computer showed otherwise.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-july-bytes-bits-and-nibbles","slug":"1992-july-bytes-bits-and-nibbles","url":"https://dwillis.github.io/uplink/article/1992-july-bytes-bits-and-nibbles.html","year":1992,"month":"July","monthNum":7,"headline":"Bytes, Bits and Nibbles","kicker":"","author_name":"David Washburn","author_title":"","affiliation":"","summary":"Two journalism projects showcase computer-assisted reporting: a Greensboro News & Record survey on sin and morality won an award, and a St. Louis Post-Dispatch investigation revealed political favoritism in workers' compensation settlements worth $12 million over 3.5 years.","keywords":["reporting","investigation","award"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["XDB"],"full_text":"Buyer Beware\n\n'Sin' study wins award\n\nGreensboro, N.C. News & Record staff writer Andrew Barron received a Wilber Award in April from the Religious Public Relations Council for \"Sin: The Triad has a lot to say about it,\" the News & Record's first computer-assisted reporting project. (Published in the November, 1991 Uplink)\n\nThe package, published Oct. 13, 1991, examined the views of News & Record readers on what acts constitute sin, the relative importance of the Ten Commandments and the seven deadly sins, and other topics.\n\nMore than 1,100 readers returned the published survey, database reporter Lex Alexander used the newsroom computer and XDB to tabulate the results, cross-referencing them by major religious denominations in the Greensboro area. He also compared respondents' demographics with Census Summary Tape File 1A data for the coverage area to see how they differed from the population at large.\n\nAlthough the tabulated responses were interesting, Barron's thoughtful interviews with clergy and lay people alike (including an entire Sunday school class) made the package worth reading. Way to go guys!\n\nMaking claims in Missouri\n\nGeorge Landau of the St. Louis Post-Dispatch combined computerized campaign contributions and computer records of the Second Injury Fund, an obscure workers compensation fund, to show that appointees and political supporters of Missouri's Attorney General (and candidate for governor) William L. Webster win up to three times as much in settlements and fees from the fund than attorneys without those connections. The total value of excess settlements was above average at $12 million over 3.5 years.\n\nThe Post-Dispatch separated lawyers who file claims against the state's Second Injury Fund into three groups, each with its own degree of success in winning money from the fund. In almost all cases, the lawyers kept 25 percent of each settlement as their fee.\n\nAverage settlements paid from the fund:\n**Lawyers with no political connections- $3,548\n**Lawyers who gave to Webster's campaign- $5,307\n**Lawyers Webster has hired to defend the fund $11,808.\n\nAs journalism enters a new information age, reporters who use computers to assist them in reporting must not only negotiate the price of a reporters' notebook with a Wal-Mart clerk, but also learn to negotiate the price of a tape drive with a computer hardware salesperson. The negotiation process, not to mention the price, is quite different.\n\nAdam Berliant, co-owner of Electronic Public Information Consultants, a company geared towards obtaining, analyzing and reporting on electronically stored government records, will be the first to say that government bureaucrats are not the only obstacles to a database reporter.\n\nIn October of 1991 E.P.I.C. founders Berliant, Tom Braden and David Hinchman began operations in Seattle. A top priority was to buy a tape drive. The fledgling company wanted a Cipher drive, known for its high quality and reliability. However, a new Cipher drive can cost up to $14,000. E.P.I.C. could not afford a new Cipher, so they decided to buy one used for $6,000 from Contech Computer Corporation of Wilmington, N.C..\n\nBerliant negotiated the financing, which included a $2,000 down payment. He says he was told by Contech's president Robert Conti that E.P.I.C. would be spinning tapes within a week after ordering the drive. But after a month of waiting they were still without a drive. Berliant was tired of waiting and decided to cancel the order.\n\nA few weeks later he received just over $600, less than a third of his down payment. He was told by Conti that all of Contech's written purchase agreements include a restocking charge if an order is canceled. The charge, which covers the reconditioning of a drive when the customer sends it back to the warehouse, amounts to 20% of the purchase price.\n\nBerliant contends that E.P.I.C. should not have to pay a restocking charge because they never received the drive and had only paid 30% of the $6,000 purchase price. He has contacted the FBI and the Better Business Bureau but has not received his money back.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-july-sin-study-wins-award","slug":"1992-july-sin-study-wins-award","url":"https://dwillis.github.io/uplink/article/1992-july-sin-study-wins-award.html","year":1992,"month":"July","monthNum":7,"headline":"'Sin' study wins award","kicker":"Bytes, Bits and Nibbles","author_name":"","author_title":"staff writer","affiliation":"Greensboro News & Record","summary":"Andrew Barron of the Greensboro News & Record won a Wilber Award for a computer-assisted reporting project examining readers' views on sin, combining survey data analysis with interviews from local religious leaders and community members.","keywords":["journalism","survey","religion"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["XDB"],"full_text":"Greensboro, N.C. News & Record staff writer Andrew Barron received a Wilber Award in April from the Religious Public Relations Council for \"Sin: The Triad has a lot to say about it,\" the News & Record's first computer-assisted reporting project. (Published in the November, 1991 Uplink)\n\nThe package, published Oct. 13, 1991, examined the views of News & Record readers on what acts constitute sin, the relative importance of the Ten Commandments and the seven deadly sins, and other topics.\n\nMore than 1,100 readers returned the published survey, database reporter, Lex Alexander used the newsroom computer and XDB to tabulate the results, cross-referencing them by major religious denominations in the Greensboro area. He also and compared respondents' demographics with Census Summary Tape File 1A data for the coverage area to see how they differed from the population at large.\n\nAlthough the tabulated responses were interesting, Barron's thoughtful interviews with clergy and lay people alike (including an entire Sunday school class) made the package worth reading. Way to go guys!","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-july-making-claims-in-missouri","slug":"1992-july-making-claims-in-missouri","url":"https://dwillis.github.io/uplink/article/1992-july-making-claims-in-missouri.html","year":1992,"month":"July","monthNum":7,"headline":"Making claims in Missouri","kicker":"Bytes, Bits and Nibbles","author_name":"","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"A St. Louis Post-Dispatch investigation reveals that lawyers connected to Missouri Attorney General William L. Webster receive significantly higher settlements from the state's Second Injury Fund, with connected attorneys winning up to three times more than unconnected lawyers.","keywords":["corruption","investigation","settlements"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"George Landau of the St. Louis Post-Dispatch combined computerized campaign contributions and computer records of the Second Injury Fund, an obscure workers compensation fund, to show that appointees and political supporters of Missouri's Attorney General (and candidate for governor) William L. Webster win up to three times as much in settlements and fees from the fund than attorneys without those connections. The total value of excess settlements was above average at $12 million over 3.5 years.\n\nThe Post-Dispatch separated lawyers who file claims against the state's Second Injury Fund into three groups, each with its own degree of success in winning money from the fund. In almost all cases, the lawyers kept 25 percent of each settlement as their fee.\n\nAverage settlements paid from the fund: **Lawyers with no political connections- $3,548 **Lawyers who gave to Webster's campaign- $5,307 **Lawyers Webster has hired to defend the fund $11,808.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-july-buyer-beware","slug":"1992-july-buyer-beware","url":"https://dwillis.github.io/uplink/article/1992-july-buyer-beware.html","year":1992,"month":"July","monthNum":7,"headline":"Buyer Beware","kicker":"Bytes, Bits and Nibbles","author_name":"","author_title":"","affiliation":"","summary":"A startup journalism company struggles to recover a $2,000 down payment after canceling an order for a used computer tape drive from a vendor who applied unexpected restocking charges despite never delivering the equipment.","keywords":["journalism","negotiation","payment"],"topics":["Technology","Business & Economy"],"technologies":["magnetic tape"],"full_text":"As journalism enters a new information age, reporters who use computers to assist them in reporting must not only negotiate the price of a reporters' notebook with a Wal-Mart clerk, but also learn to negotiate the price of a tape drive with a computer hardware salesperson. The negotiation process, not to mention the price, is quite different.\n\nAdam Berliant, co-owner of Electronic Public Information Consultants, a company geared towards obtaining, analyzing and reporting on electronically stored government records, will be the first to say that government bureaucrats are not the only obstacles to a database reporter.\n\nIn October of 1991 E.P.I.C. founders Berliant, Tom Braden and David Hinchman began operations in Seattle. A top priority was to buy a tape drive. The fledgling company wanted a Cipher drive, known for its high quality and reliability. However, a new Cipher drive can cost up to $14,000. E.P.I.C. could not afford a new Cipher, so they decided to buy one used for $6,000 from Contech Computer Corporation of Wilmington, N.C..\n\nBerliant negotiated the financing, which included a $2,000 down payment. He says he was told by Contech's president Robert Conti that E.P.I.C. would be spinning tapes within a week after ordering the drive. But after a month of waiting they were still without a drive. Berliant was tired of waiting and decided to cancel the order.\n\nA few weeks later he received just over $600, less than a third of his down payment. He was told by Conti that all of Contech's written purchase agreements include a restocking charge if an order is canceled. The charge, which covers the reconditioning of a drive when the customer sends it back to the warehouse, amounts to 20% of the purchase price.\n\nBerliant contends that E.P.I.C. should not have to pay a restocking charge because they never received the drive and had only paid 30% of the $6,000 purchase price. He has contacted the FBI and the Better Business Bureau but has not received his money back.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-july-the-greensboro-news-and-record-has-north-carolina-dairies-crying-over-spilt-milk","slug":"1992-july-the-greensboro-news-and-record-has-north-carolina-dairies-crying-over-spilt-milk","url":"https://dwillis.github.io/uplink/article/1992-july-the-greensboro-news-and-record-has-north-carolina-dairies-crying-over-spilt-milk.html","year":1992,"month":"July","monthNum":7,"headline":"The Greensboro News & Record has North Carolina dairies crying over spilt milk.","kicker":"","author_name":"Lex Alexander","author_title":"","affiliation":"Greensboro News & Record","summary":"The Greensboro News & Record uncovered a major bid-rigging scheme where North Carolina dairies defrauded school systems of approximately $8 million between 1982 and 1989 by manipulating milk contract bids, requiring the newspaper to manually collect and compare data from 133 separate school districts.","keywords":["dairies","investigation","fraud"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["VuText","DataTimes"],"full_text":"The Greensboro News & Record has North Carolina dairies crying over spilt milk.\n\nFederal and state investigators believe that between 1982 and 1989, North Carolina dairies rigged bids for school milk contracts and got as much as $8 million of illicit revenue, 2 cents at a time, from the state's school children.\n\nThe News & Record first considered looking into the bidding in July 1991, just after we got our newsroom computer and about two months after National Public Radio reported that a North Carolina-based dairy had been implicated in school milk bid-rigging in Georgia.\n\nAt first, we had very little to go on. We assumed that the U.S. Department of Agriculture or the state Department of Public Instruction would have bidding information for school milk contracts in a computer file in one central location. They didn't. The records were kept on paper at 133 separate school system offices.\n\nBut in August, after another dairy admitted having rigged bids in a school system in an adjacent county, we jumped in anyway. We requested (via form letter) information from all 133 systems, back to 1981, on which dairies had bid on a contract, what kind of contract it was (whether it kept a price firm for the entire school year or allowed the price to vary with changes in the cost of raw milk, which was set by the state dairy commission), and how much per half-pint of milk they were charging.\n\nWhat we got back created a stack of paper 2 feet high. The only help our tape drive offered was to give me a place on which to set the stack.\n\nBecause schools were required to keep such records only for three years and three months, many systems had records only as far back as 1988. And, with school about to start, many child nutrition directors felt like they had more important things to do than help us.\n\nBut other systems fell all over themselves to give us everything they had. And some of them had a lot: About 40 systems had usable records all the way back to 1981.\n\nWe also got an unexpected break. Early on, one school system's child nutrition director told us that federal marshals had told her that her system definitely had not been touched by the bid-rigging. We were able to use bids in that system as a benchmark for comparing bids in other school districts; in almost every case, the differences were dramatic.\n\nFrom mid-August until Christmas, I divided my time between typing in the information we had received and calling school systems to beg, plead, cajole, and just generally make enough of a pest of myself to get them to give us what we'd asked for, or fill in the blanks where information we got was incomplete.\n\nComparing bid prices statewide with bid prices in the system where no bids had been rigged lent credence to investigators' claims that the dairies had received about $8 million in illicit revenue from milk sales between 1982 and 1989.\n\nIt wouldn't have been cost-effective for us to sue the holdouts, and ultimately we got at least some usable information from 112 systems.\n\nMeanwhile, I ran up a huge bill in our library, using VuText, DataTimes and other databases to check publications from New Jersey to Texas for valuable information on school milk bid-rigging in other states.\n\nI talked to dairy industry experts who told us what kind of bidding picture we could expect if bidding had been honest-and what we could expect if it hadn't. Then, during Christmas week, I sat down to crunch the numbers.\n\nThe analyses and other information showed that:\n\nNorth Carolina dairies frequently enjoyed exclusive, long-term relationships with school systems in the towns or counties where they were located.\n\nIn any given year, prices bid by a particular dairy generally varied by 15 percent or more. One dairy's bid prices varied 60 percent. Our experts considered those swings significant in an industry in which, because of perishability, profit margins seldom exceed 2 percent.\n\nThose swings also were consistent with investigators' suspicions that dairies had bid intentionally high prices in systems where they had no intention of competing for the contract. Such bidding created the appearance of competitive bidding where there really was none.\n\nI also traced the movement of milk prices against the dairy commission's changes in the cost of raw milk that the processors had to pay. (The commission had been disbanded in 1990 and its records destroyed, but a source in state government was able to fax me copies of commission records dating to 1981.) I found that the prices bid by dairies tended to go up more quickly, and some came down more slowly, than the cost of raw milk, the dairies' biggest expense. That meant, if all other expenses were equal, the dairies were making more money. In fact, between 1982 and 1986, their second biggest expense, the price of gasoline, was falling, meaning even more profitability.\n\nComparing bid prices statewide with bid prices in the system where no bids had been rigged lent credence to investigators' claims that the dairies had received about $8 million in illicit revenue from milk sales between 1982 and 1989. This analysis suggested that the dairies got that $8 million the hard way-cents at a time.\n\nThe analyses did not conclusively prove bid-rigging. In fact, despite subpoenas for dairy company travel, expense and telephone records, the feds couldn't prove bid-rigging here until two dairy executives secretly began cooperating with them in the spring of 1991. But in records from 1982 to 1989, when word of the federal investigation in other states spread to North Carolina, the analyses did show a strong similarity between the bidding patterns of the three dairies that had admitted rigging bids in North Carolina and the four other large dairies that had not. (Three of those have been implicated in other states.)\n\nThis was a highly technical subject. A spreadsheet program would have helped tremendously during the analyses and in generating graphics that would have made the stories clearer to the reader, but we don't yet have one.\n\nThere weren't a lot of real people in the package with anything meaningful to say. The only helpful source I could find in the state's dairy industry stayed strictly off the record. And it's not at all clear yet what impact the package has had. Since publication on March 1, seven months after our investigation began, no new indictments have been handed up or guilty pleas announced. The investigation continues at this writing. The project gave me some practice in setting up relational databases working from paper records, and it helped demonstrate to other staffers the value of computer-assisted reporting.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-july-tech-tips","slug":"1992-july-tech-tips","url":"https://dwillis.github.io/uplink/article/1992-july-tech-tips.html","year":1992,"month":"July","monthNum":7,"headline":"Tech Tips","kicker":"","author_name":"Elliot Jaspin","author_title":"","affiliation":"","summary":"A technical guide explaining how to extract specific state records from the Toxic Release Inventory tape while preserving the document control number that links related pollution data, using NineTrack Express to filter records by state code without wasting disk space.","keywords":["extraction","database","filtering"],"topics":["Data & Databases","Health & Environment","Investigative Methods"],"technologies":["9-track tape","Nine Track Express"],"full_text":"Subject: Extracting information from a tape.\nProblem: The Toxic Release Inventory uses what is called either a \"submission number\" in its 1987 documentation or a \"document control number\" in 1989. The number, which is 15 bytes long, occurs at position 5 in each of the different record types that make up the TRI file. The information in this field is crucial for a number of reasons.\n\nThe document control number is used to identify a specific source of pollution such as a plant, and is included in the various documents that are filed by the polluter. Using this number you can tie together all related bits of information on one site that are scattered throughout the TRI file. The number also contains the state where the plant is located. This information occurs as the last two bytes of the number.\n\nMost people who use TRI are interested in information on polluters in one state or at most a small group of states. How then do you extract only those records for a specific state while keeping the all important document number intact during the transfer.\n\nSolution: It is possible using a NineTrack Express to define a field in two different ways. Thus, you can first define a field called \"DocNo\" that begins at position 5 for a length of 15. When you transfer information from the TRI file, you would make sure to import this field.\n\nAt the same time, you can also define the last two bytes of the document control number as the field. This field begins at position 18 for a length of 2.\n\nAssuming that you only want those records where state equals \"KY\", there is no need to import the field. A state field that only contains \"KY\" would only take up valuable space on the hard disk.\n\nHowever, you can still filter on this field even though you will not be importing. As each record is read from the tape, NineTrack Express will check to see if the state field contains \"KY\" and import the rest of the fields including the 15 byte document control number.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-july-journalists-from-around-the-country-log-on-at-the-portland-ire-conference","slug":"1992-july-journalists-from-around-the-country-log-on-at-the-portland-ire-conference","url":"https://dwillis.github.io/uplink/article/1992-july-journalists-from-around-the-country-log-on-at-the-portland-ire-conference.html","year":1992,"month":"July","monthNum":7,"headline":"Journalists from around the country \"log on\" at the Portland IRE conference.","kicker":"","author_name":"Jon Schmid Sotomayor","author_title":"","affiliation":"MICAR","summary":"Over 400 reporters attended a standing-room-only conference on computer-assisted reporting at an Investigative Reporters and Editors event in Portland, Oregon, featuring seminars, hands-on practice with database programs, and demonstrations from experienced journalists on how to use computers for investigative work.","keywords":["reporting","computer","journalism"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["9-track tape","XDB","Paradox","FoxPro"],"full_text":"Question: Why on earth would 400 reporters get up at 8 a.m.?\n\nAnswer: A whole day of lectures and demonstrations at the largest conference ever on computer-assisted reporting.\n\nInvestigative Reporters and Editors sponsored the program on computer-assisted reporting as part of its national conference held in Portland, Oregon June 11-14.\n\nBut judging from the turnout, most conference attendees considered the day essential, rather than optional.\n\n\"I have been going to these conferences for 10 years and I have never seen an 8 a.m. conference with standing room only,\" said IRE president Mark Middlebrook.\n\nIn addition to the seminars and panels, a computer practice room staffed by volunteers gave everybody the opportunity to practice downloading data from 9-track tape. Participants tested a variety of database management programs, including XDB, Paradox and Foxpro. Experienced journalists gave demonstrations of different mapping programs and anybody could try tapping into research libraries through on-line services.\n\nThe National Library on Money and Politics gave away more than 200 free diskettes with FEC information listed by state. And IRE sold, for $4, about 100 shareware diskettes each containing a sampler of 10 utilities programs.\n\nThe nine seminars and panels Thursday ranged from recognized experts discussing the direction of computer-assisted reporting to nuts-and-bolts demonstrations on how it is actually done.\n\nStatistics sage Phil Meyer talked about the past, present and future of the craft and called for more sophisticated analysis instead of the usual \"quick-hit\" approach. However, Elliot Jaspin assured the audience that basic counting and sorting can still lead to great stories.\n\nOther seminars covered more practical problems for journalists eager to start computer sleuthing, such as \"Getting Started: Computer-Assisted Journalism on a Budget in Your Newsroom.\"\n\nAccomplished reporters shared war stories and tips on building databases, prying tapes from unwilling government bureaucrats and mining huge databases, such as the census.\n\nIn \"Deciphering the Gibberish: A Short Course in Government Databases\" the audience watched the computer screen, projected overhead, as reporters George Landau, of the St. Louis Post-Dispatch, Penny Loeb of New York Newsday, and Bob Port of the St. Petersburg Times typed through the methodology used for some of their best stories.\n\n\"Computer-assisted journalism has always been pushed by reporters,\" said Bill Dedman, IRE boardmember and Pulitzer Prize winning journalist. \"This (conference) means the seeds have been planted at several hundred more newspapers.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-august-beating-them-at-their-own-game","slug":"1992-august-beating-them-at-their-own-game","url":"https://dwillis.github.io/uplink/article/1992-august-beating-them-at-their-own-game.html","year":1992,"month":"August","monthNum":8,"headline":"Beating them at their own Game","kicker":"","author_name":"Lou Grieco","author_title":"","affiliation":"MICAR","summary":"Reporters face challenges in accessing electronic records due to bureaucratic tactics and legal ambiguities. Journalists are often confronted with delays, high costs, and record redaction issues. Strategies to combat these include legal action and efficient request filing.","keywords":["bureaucrats","electronic records","access laws"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["9-track tape"],"full_text":"No longer able to baffle the press with talk of bytes and bits, bureaucrats are now using access laws to play the old shell game with reporters requesting electronic records.\n\nHow far will they go?\n\nWhen Philadelphia Inquirer systems editor Al Hasbrouck wanted four years of criminal court dockets from the city, he had to fight for them. Then he had to fight for a copy of the codes for the more than 17,000 lawyers in the Philadelphia area, which he got in the form of a six-inch thick printout. Then he had to continue fighting for the codes on ninetrack magnetic computer tape, the way he had originally requested the records.\n\nBoston Herald staff writer Dave Armstrong was familiar with the old-fashioned method of record redaction: the black pen marks that eliminated the parts of a paper record a journalist was not allowed to see. But Armstrong was not prepared for the new redaction \"logic\" that would prevent him from seeing any part of a computer tape, even the fields that hadn't been redacted.\n\nJournalists are not alone in recognizing the potential of computer-assisted reporting. Government bureaucrats with whom reporters must deal to get tapes also see what journalists can do with personal computers and are learning new ways to deny or delay access to electronic records.\n\nSometimes the resistance comes in an unsophisticated bluff, in which the reporter is told the records are not public, in the hope he or she will simply go away. But new strategies, some depending on various state access laws, can keep reporters busy just trying to get tapes.\n\n\"They've gotten real creative,\" Hasbrouck says. \"Public servants understand what we can do to embarrass them through computer records.\"\n\nPennsylvania law is unclear on how public records are to be provided, and Hasbrouck says bureaucrats have stalled him in several ways. One time, an agency gave him a record layout in COBOL code, which Hasbrouck is unfamiliar with, so he needed to get a translator.\n\nIn the case of the court dockets, Hasbrouck had to go through endless negotiations. After he \"finally wrung it from the hands of the good civil servants,\" he spent several weeks to get the codes he needed to understand the dockets.\n\nEven in states where the requestee cannot determine what form public records are released in, reporters can still get the runaround. In Massachusetts, where the law says bureaucrats must give records in the forms requested if possible, interpretation makes the difference.\n\n\"There's one trick in particular we have a problem with,\" Armstrong says describing the redaction ploy. Another part of the Massachusetts law says that bureaucrats do not have to create records for reporters, just turn over what they already have. Certain Massachusetts agencies have decided that redaction in computer records creates new records, because of the programing required to provide records with certain fields deleted. So, the agencies say, the reporter cannot see the original records, because of the confidential fields. However, the reporter also has no right to see the records without the confidential fields because the \"new\" records aren't covered under state law.\n\n\"It's interesting, and at the same time it's bothersome,\" Armstrong says. The Herald does have a court case pending that will address this interpretation. Armstrong says he's optimistic about winning, though it may be the start of a long legal battle.\n\nJerry Uhrhammer, a reporter with the Tacoma (Wash.) Morning News Tribune, is skeptical of a new state public access law. Under the law, which went into effect earlier this year, Washington agencies have five days to respond to requests. The previous law mandated a non-specific \"promptly.\"\n\nBefore, when he could insist on a prompt response, Uhrhammer says he often got answers within 24 hours of his requests. Now, with the new law in effect, Uhrhammer says the five-day rule may actually slow his requests down. He is finding that bureaucrats often wait until the last possible day to start working on a request.\n\n\"[The] most common stall is the delay, the foot dragging,\" Uhrhammer says. For him, the new law may give more excuses for foot-dragging officials.\n\nTexas access laws were good, until electronic records became predominant. The state can determine the form the records will be provided in. Allen Pusey, assistant projects editor for the Dallas Morning News, says that many times, the paper receives huge printouts of records.\n\n\"I think that's been done specifically to make it harder,\" Pusey says. The Morning News uses data entry to create a database if it receives paper records. \"Until we fix the law, it's the only thing we can do.\"\n\nCost is another factor that agencies can use to keep reporters at bay. Hasbrouck, at Bucks County, Pa., found that agencies charged six cents per record, which is good for lawyers who want specific records but costly for reporters interested in spotting trends. Hasbrouck notes that it doesn't cost any more to store six or 6,000 records.\n\nIn Washington, Uhrhammer has also found cost to be a problem. One agency offered to provide tapes, but wanted to charge $40 per hour of programming, and said the duplication will take 40 hours. Uhrhammer says this is \"outrageous and it's not legal.\" Under Washington law, agencies can only charge the cost of reproduction, and Uhrhammer says both the price and time estimates are way too high. The matter has not been resolved yet.\n\nBut regardless of new strategies, the favorite bureaucratic tactic remains the bluff. These battles are often easily won in court, but can be costly and time-consuming. Letters are exchanged by various attorneys, and if the requestor consulted the access laws before making the request, the bureaucrats will give in.\n\n\"A lot of times it's like taking a sledgehammer and banging them over the head,\" says Armstrong.\n\nPusey has his own strategy for waking up the civil servants. The Morning News has a well-known policy of pursuing legal fees as well as access. \"That I think gets their attention.\" While Pusey concedes that some bureaucrats are getting more sophisticated about computer-assisted reporting, many will rely on the bluff because they don't know the access laws and don't care.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-august-a-few-tips-on-making-computer-requests","slug":"1992-august-a-few-tips-on-making-computer-requests","url":"https://dwillis.github.io/uplink/article/1992-august-a-few-tips-on-making-computer-requests.html","year":1992,"month":"August","monthNum":8,"headline":"A Few Tips on Making Computer Requests","kicker":"","author_name":"Sandra Davidson Scott","author_title":"","affiliation":"","summary":"A tipsheet on how to phrase public records requests for computer data, addressed to the correct custodian and specific about format.","keywords":["public records","records requests","tips"],"topics":["Investigative Methods","Government & Politics"],"technologies":[],"full_text":"1. Make requests as specific as possible and direct them to the designated custodian of that particular type of record. Remember that the custodian may need computer experts to handle your request, so requesting a meeting with both the custodian and his or her computer experts at the outset may speed up the process. At the meeting you can discuss such issues as how much of a computer expert's time it would take to retrieve your request.\n\n2. Take time to know your state's computer laws relating to your request before the meeting, including:\n\nExemption laws. These laws which exempt certain records from public disclosure are sometimes scattered throughout the statutes instead of being contained within the chapter or title on access to public records. Be prepared to argue why your request is not exempt.\n\nRedaction laws. If a record you request contains some information that is exempt and some that is open to disclosure, does your law say that the custodian must delete the exempt information and give you the rest? If not, you may have to argue for redaction. (Note that the United States Supreme Court did not rule in favor of redaction of F.B.I. records in Department of Justice v. Reporters Committee for Freedom of the Press, 489 U.S. 749 (1989).)\n\nTailoring laws. Do your statutes say that custodians must provide records in computer form or better yet in the format you request if the computer system is capable of doing so? If your statutes don't address tailoring you may have to argue that what you are requesting does not constitute creating a new record. \"We don't have to do programming\" may be an argument you'll face. Think in terms of the ease with which your request can be fulfilled; you may have to argue that a few computer strokes does not constitute programming.\n\nCosts. If your statute reads in terms of \"reasonable\" or \"actual\" cost, be prepared to argue precisely what that imprecise language means. You want to limit the costs to costs of search and copying at a maximum, and you want to avoid paying for overhead.\n\n3. Think twice before filing a lawsuit. If you have \"fuzzy\" statutes, you may be better off living with uncertainty and arguing with custodians than living with a bad court decision. For instance, it may be better to have no court decision if you have statutes that do not address the question of redaction than to have a court decision that says custodians of records do not have to redact records.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-august-connections-that-pay-off","slug":"1992-august-connections-that-pay-off","url":"https://dwillis.github.io/uplink/article/1992-august-connections-that-pay-off.html","year":1992,"month":"August","monthNum":8,"headline":"Connections that pay off","kicker":"","author_name":"David Raziq","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"St. Louis Post-Dispatch reporters used computer databases to investigate alleged corruption in Missouri's Worker's Compensation Second Injury Fund, uncovering improper relationships between state-hired defense lawyers and Attorney General William Webster involving campaign contributions and settlement negotiations.","keywords":["corruption","investigation","database"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["FoxPro","9-track tape","microfilm","ASCII"],"full_text":"Connections that pay off\n\nSt. Louis Post reporters use computers to uncover scandal involving state worker's comp lawyers and Missouri Attorney General.\n\nFor years, few Missourians had heard of the state's Worker's Compensation Second Injury Fund. Created by a tax on insurance premiums, it gave workers the opportunity to sue the fund for extra disability when an on-the-job injury aggravated a previous injury or medical condition. The fund was created to ensure that workers got a square deal from worker's comp. However, by early 1992 some Missouri lawyers had voiced complaints about they way in which the fund was being administered.\n\n\"A couple of attorneys who had worked on the fund had been calling us for months,\" says reporter George Landau of the St. Louis Post-Dispatch, \"They were disgusted with the way it worked.\"\n\nAnd how it worked, claimed these attorneys, bordered on the illegal. Fifteen private lawyers had been hired by state Attorney General William Webster to defend the second Injury fund. However, Webster also allowed these lawyers to bring suit against the same fund on behalf of their own clients. Hence, these \"Special Attorneys General\" would negotiate settlements among themselves, one lawyer suing the fund one day while another lawyer would defend it. Later, they would switch roles.\n\nIn addition, the Post's sources claimed that these lawyers or their firms had made contributions to Webster's campaign for Governor. Hence, claimant's lawyers who had also contributed allegedly received better settlements for their clients.\n\nAfter receiving the complaints, the Post launched an investigation and planned to use a computer to create a database that combined election contribution data with records from the second injury fund to support the allegations.\n\nBut, like many states, Missouri's state election contribution data is not computerized, but kept in a hardcopy format. State law also requires agencies to charge 50 cents per photocopy which can become prohibitively expensive.\n\nHowever, foresight and a little serendipity nipped these problems in the bud. \"Post reporter Terry Ganey and the capital bureau had campaigned for a few years for a database of campaign contributions,\" Landau said. So when the story came along, an office manager had already been typing the contribution reports into a Foxpro data entry program. Plus, the Post bought the reports in a microfilm format for five dollars per reel, with each reel containing two thousand pages.\n\n\"We were just lucky,\" says Landau, \"because she had just finished the Webster campaign contributions from 1988 to 1991 when I got the second injury fund database from the Division of Worker's Comp.\"\n\nAcquiring the database, which listed the names of attorneys filing claims, attorneys defending the fund and approving settlements, and the settlement amounts, was a difficult task. The Post team had to negotiate with the Attorney General's office for the tapes. At first the Attorney General's office gave the Post the runaround saying they didn't have the information the newspaper had requested.\n\nGaney, Landau, and the Post lawyers decided that it was time to give the bureaucrats an attitude adjustment. \"This involved a lot of hasseling and discussions between our attorney and the Attorney General's office.\" And five weeks later: \"They finally agreed we were entitled to the electronic data.\"\n\nOn the nine-track tape Landau received was a \"print-image\" of a print-out the Post had previously acquired, with column headings and page numbers. \"The data had the same position in each line of the file,\" says Landau, so he just edited out those lines containing the page-numbers and headings. The result: a fixed position ASCII document that he could now convert to Foxpro. It was time to get to work.\n\n\"The hard part of a project like this is matching names because they are spelled differently in each database,\" says Landau, \"so I developed some little tricks.\" For example, in the second Injury database had dozens of attorneys with more than 20 cases, each with their names spelled differently from case to case. \"So, what I would do,\" says Landau, \"is sort the names alphabetically into a new table and the like-names would usually cluster.\n\nOnce I was sure this cluster was the same person, I put a 'plus' in front of the first name-occurence and a 'minus' in front of the last name-occurence. Then I would write a program that replaces a new field in the original database with a consistent name, usually the first name in the cluster. So now everything was spelled consistently.\"\n\nLandau says he applied this technique throughout the data, saving him \"hours and hours of time.\" Then he wrote another Foxpro program that allowed him to link the record numbers in the Webster contributers database with the record numbers of the now consistently-named attorneys in the second Injury database. Another program then went through both databases, looked at each record number, and aggregated all the contribution data for each lawyer into the 2nd Injury table. \"That may be a complicated way to do it\" says Landau, \"but I wanted to end up with a single table that told the whole story.\"\n\nHowever, Landau says it took more than a computer table to tell the story and praised his partner's reporting skills. \"I did all the computer stuff but Terry Ganey did some really dynamite reporting by working her sources. And if it hadn't been for those sources saying how the fund worked, we wouldn't have had as clear an idea of what we were looking for.\"\n\nLandau continues: \"It can be hard to analyze a database and if you don't know what you're looking for the story won't jump out at you. Thats obvious, but people can forget it when you do computer stories.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-august-tech-tips-tech-tips-tech-tips-tech-tips","slug":"1992-august-tech-tips-tech-tips-tech-tips-tech-tips","url":"https://dwillis.github.io/uplink/article/1992-august-tech-tips-tech-tips-tech-tips-tech-tips.html","year":1992,"month":"August","monthNum":8,"headline":"Tech Tips Tech Tips Tech Tips Tech Tips","kicker":"","author_name":"Elliot Jaspin","author_title":"","affiliation":"","summary":"A database management technique uses the XDB commands Update and Xleft to create a standardized name field by extracting the first portion of company names, allowing grouping and sorting despite spelling variations without manual correction of each entry.","keywords":["database","standardization","deduplication"],"topics":["Data & Databases","Technology","Investigative Methods"],"technologies":["XDB"],"full_text":"Problem: A database that lists people and companies is nearly impossible to work with because of variations in spelling. A company such as Amalgamated Zool Suits might be listed as:\n\nAmalgamated Zoot Suits\nAmalgamated ZootSuits\nAmalgamated Zoot-Suits\nAmalgamated Zoot Suits Inc.\nAmalgamated Zoot-Suit Inc.\nAmalgamato Loot Soots\n\nBecause the database is large, Correcting the spelling of every entry is not an option.\n\nSolution: If we look at the variations in the spelling of Amalgamated Zoot Suits, notice that changes occur somewhere in the last half of the name: We can make use of that fact to create a new name field that uses only the first half of the name field. This is done in XDB by using two commands: Update and Xleft.\n\nThe first step of course is to create a new field in our hypothetical database called Badspell that will hold the new name we are creating. Let us assume we are going to use the first 11 letters of the name field. Thus the \"NewName\" field will be a \"char\" field 11 characters wide.\n\nWe then run the following command:\n\nUpdate Badspell set NewName = xleft(name, 11)\n\nWe can now use the \"NewName\" field to group, sort and order our database. Or can we?\n\nWhile the \"NewName\" field is an improvement, it is not a perfect solution. Notice that the last entry for Amalgamated Zoot suits contains a misspelling with in the first 11 letters. If we try to group on the \"NewName\" field, \"Amalgamatd\" will be seen as a different company. There are two possible solutions: grab fewer letters to get around the misspelling er correct the origional entry and rerum the update command\n\nReducing the size of \"NewName\" might work except that there is also a person in the database named Amalgam, Arnold.\" If we take too few letters, \"Amalgam, Arnold,\" and \"Amalgamated Zoot Suits\" might wind up being lumped together.\n\nIf might be simpler to run the following command:\n\nUpdate Badspell\n\nset NewName - \"Amalgamated\" where xdeff(name,11 Amalgamated\n\nHowever, there also might be two companies who start with the word \"Amalgamated.\" To account for that we could make \"NewName\" held larger and grab more letters from name or alternatively, keep NewName the same size and use the Update command to change\n\n\"Amalgamated\" to \"Amalgam Zoo\" wherever the first 13 letters in \"name\" field are \"Amalgamated Z\" The second approach would leave us with two different \"NewName\" entries:\n\n\"Amalgamaled\" for \"Amalgamated Federated Industries\"\nAmalgam Zoo\" for \"Amalgamated Zoot Suits\"\n\nWhile we may face several more runs to account for such problems, this approach accomplishes two things: It \"Solves\" the large majority of misspellings on the first pass of the database and it allows us to \"Correct\" any other misspellings in subsequent without actually having to go into the database and make the individual changes.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-august-what-they-didnt-teach-you-in-school","slug":"1992-august-what-they-didnt-teach-you-in-school","url":"https://dwillis.github.io/uplink/article/1992-august-what-they-didnt-teach-you-in-school.html","year":1992,"month":"August","monthNum":8,"headline":"What they didn't teach you in school","kicker":"","author_name":"William Syken","author_title":"","affiliation":"MICAR","summary":"Journalists trained in computer-assisted reporting discover significant gaps between classroom instruction and real-world newsroom application, facing challenges with equipment selection, database accessibility, and staffing limitations.","keywords":["reporting","training","technology"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Vax","PC","Qualstar 1600 BPI","Qualstar 6250","9-track tape"],"full_text":"What they didn't teach you in school\n\nSo you want to start a computer-assisted reporting program. Maybe you've read some books, or taken a seminar on how to use the equipment. Now you're ready to give it a whirl in the newsroom.\n\nNo problem, right?\n\nWell\n\nMost reporters said there was a huge difference between the real world and the controlled atmosphere of the seminar classroom.\n\nJerry Uhrhammer of the Morning News Tribune in Tacoma, Washington said the learning curve was steeper than what he expected after completing a training seminar at the Missouri Institute for Computer-Assisted Reporting in Columbia, Mo.,\n\n\"Learning how to do this is like learning a foreign language, and you don't learn it in two weeks,\" Uhrhammer said. \"You need to immerse yourself and speak it constantly. I'm still in the learning process.\"\n\nUhrhammer also said the Morning News Tribune's computer-assisted reporting program got off to a slow start because, as a money saving maneuver, they tried to get their program running with a Vax business computer.\n\n\"When we found that was not working we went out and bought the hard drive and the 486,\" he said.\n\nAt the Akron Beacon Journal, Bob Paynter's also found that his paper's initial equipment purchase designed to save money soon proved inadequate. The Journal started out with the Qualstar (1600 BPI). Now he has the Qualstar 6250, but still finds the going slow. Paynter said his biggest problem has been setting up a user-friendly way to get reporters into databases.\n\nPaynter doesn't have programming background, and has struggled to make his datasets accessible.\n\n\"I've aquired a lot of databases that would be valuable to reporters on a daily basis,\" he said. \"How do you spread it around without going through a Jaspin-like seminar?\"\n\nSome papers complain they simply haven't had the opportunity to use their equipment.\n\nRae Davidson of the Blade in Toledo, Ohio, said tight staffing at her paper had prevented people from committing themselves to in-depth computer-assisted projects.\n\n\"The reporters we have are very interested,\" she said. \"But this is the nineties. We may be in the age of technology, but we're also in the age of doing more work with fewer reporters.\"\n\nJudy Nichols of the Arizona Republic who took the MICAR seminar in January, said she has had an almost trouble-free experience with her computer-assisted program and credits Russ Beutner, a former MICAR employee and Republic intern for her success. \"But even with Russ here, it has been a learning experience for me and the editors,\" Nichols said. \"When you start to do analysis, it can get more in depth than you expected rather quickly.\"\n\nUhrhammer also said he hadn't used the equipment as much as he would like, but cited a different reason. He said after getting the drive he has been involved with investigations that either didn't involve computers or had been based from information on diskette rather than 9-track tape. He has only used the tape drive to download census tapes.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-september-the-seductive-technology","slug":"1992-september-the-seductive-technology","url":"https://dwillis.github.io/uplink/article/1992-september-the-seductive-technology.html","year":1992,"month":"September","monthNum":9,"headline":"the seductive technology","kicker":"","author_name":"Ronald Campbell","author_title":"","affiliation":"The Orange County Register","summary":"A journalist reflects on how technological seduction led him to pursue a lengthy investigation into California bus safety that ultimately yielded no significant story, illustrating how access to large databases can mislead reporters into searching for problems that don't exist.","keywords":["investigation","database","failure"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["MISTER"],"full_text":"at 8:20 p.m. on May 9, a sleeping 3-year-old boy was whipped from his mother's lap by an exploding tire tread and dragged to his death beneath the wheels of a bus.\n\nThe bizarre death of Ramon Prado provided an occasion - an excuse, really - for The Orange County Register to publish results of its seven-month probe of bus safety in California.\n\nThe story showed that the California Highway Patrol routinely fails to meet its legal mandate to inspect buses. But as the story also noted, California's buses have racked up an enviable safety record. And even if CHP inspectors had found and inspected the bus on which Ramon Prado later died, they could have done nothing about a floor so thin that a tire tread could breach it; the floor met federal standards.\n\nSo much for seven months work. The story, and the endless research that went into it, illustrate what I call seduction by technology. Entranced by the siren song of high tech, I pursued a story long after my instincts told me it wasn't there. Most articles in Uplink tell of triumphs. This is a story of failure and the four journalistic myths that made it possible.\n\nIt began in July 1991, with a bus crash near Palm Springs that took the lives of seven people. The Register immediately began casting about for a way to investigate bus safety. We discovered the CHP's Management Information System of Terminal Evaluation Records, aka MISTER.\n\nIt sounded perfect: computerized records on 90,000 bus and truck companies, including every traffic ticket, every accident and every CHP inspection of bus or truck terminals. CHP's reaction made the database seem even better: They demanded $2,800. Over the next three months we argued the price down to $81.18.\n\nEnter Myth No. 1: If a bureaucrat hinders or refuses access to public documents, those records must be explosive. That might have been true, sometimes, in the days when governments kept its records in manila folders. Today, when agencies routinely keep gigabyte-sized databases, it's absurd to presume that custodians know what they're hiding.\n\nUnder the spell of Myth No. 1, I was easy prey for Myth No. 2: Gather enough facts and you're bound to find a great story. All reporters play hunches, looking under rocks for something odd. With computers comes the temptation to look under every grain of sand on the beach.\n\nThat is what I did with MISTER. Had I faced a similar quantity of paper records, I instantly would have turned away for a reality check. I would have called some experts, read some reports and chosen one or two angles to explore. Instead I plunged into the database with all the direction of a near-sighted man on a foggy night.\n\n\"How many people die on buses?\" I asked the database. Very few. \"How often do buses crash?\" Not often. \"What does 'often' mean?\" Good question, since even CHP doubts the mileage numbers in MISTER.\n\nOne day I extracted a list of multiple-injury bus accidents. The heaviest toll by far, more than 50 injuries, was for an accident a year earlier within a few miles of the Register. I could not find a word about that accident in the paper. A local CHP flack looked it up for me: The bus was ferrying prisoners between court and jail; most of the \"injured\" were inmates complaining of whiplash.\n\nMyths No. 3 and 4 made an already bad situation nightmarish. I suspect both myths are common among computer-assisted reporters.\n\nMyth No. 3: Databases are the perfect reflection of the real world. Think about that one for a second. Intellectually, of course, no one believes it. But emotionally we tend to invest databases with a God-like aura of omniscience. They seem so complete.\n\nIn fact, however, databases merely record what their keepers meant them to record, to the degree their keepers can get the information.\n\nIt's easy to lose sight of those limitations. MISTER listed hundreds of companies that no longer existed. It omitted bus and truck accidents on local roads, unless police went to the trouble of informing the CHP. MISTER's record of hazardous spills abruptly stopped in 1990 when CHP \"temporarily\" stopped entering the data.\n\nBefore I ever used a mainframe, I was a devotee of Myth No. 4: Mainframes are big, fast and unstoppable - the Incredible Hulk of the Information Age. The myth is correct on one point. Mainframes are indeed big. So was Tyrannosaurus Rex. I thought I needed a mainframe to handle MISTER; at 160 megabytes, it would have stretched the capacity of my PC and overwhelmed XDB, the database program I use. It would have taken weeks to get a few simple answers, I thought.\n\nSo I used a mainframe instead. And it did not take weeks to get simple answers; it took months.\n\nWe lumbered along, the Register's information services department and I, for three months before I framed my first query to the MISTER database. It took time to install the hardware linking my PC with the mainframe, time to teach me how to use the link, time to load MISTER and then to load it again when the first layout didn't work.\n\nThen the real problems began. Contrary to its legend, I found the mainframe balky. I would ask what I thought was a simple question and then wait and wait - until an irritated computer tech would call, asking why I was consuming 90 percent of the central processing unit's capacity.\n\nI would have to ask four or five questions on the mainframe where one would have sufficed on the PC. And after other users began to complain in droves about that crazy new user monopolizing the computer, the high priests of the mainframe restricted my access. As the weeks dragged into months, I began to dread afternoons digging into the database. Few findings excited me, and never for long; if I found an intriguing pattern in the river of printouts, I usually found a flaw an hour or a day later.\n\nRamon Prado rescued me from all this. Months before his death I had noticed that the CHP was failing its mandate to inspect every bus terminal. Absent a healthy dose of mayhem, this was just one more tiny instance of decaying state service. Prado's death, which no inspector could have prevented, make bus inspections newsworthy. In retrospect, my failure with MISTER offers a couple of easy lessons: First, never use a computer you don't control; the control issue alone makes a newsroom PC superior to a mainframe. Second, before you buy a database, before you spend months probing its secrets, find out its weaknesses.\n\nBut the major lesson is one that I am not yet sure I have mastered: A database is not a story; it just might lead to a story, but it is not a story. Lose sight of that lesson, and you too can be seduced by technology.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-september-problem-solving-at-us-news-and-world-report","slug":"1992-september-problem-solving-at-us-news-and-world-report","url":"https://dwillis.github.io/uplink/article/1992-september-problem-solving-at-us-news-and-world-report.html","year":1992,"month":"September","monthNum":9,"headline":"PROBLEM SOLVING AT U.S. NEWS AND WORLD REPORT","kicker":"","author_name":"John Bare","author_title":"Intern","affiliation":"U.S. News and World Report","summary":"U.S. News and World Report used computer-assisted reporting to investigate medical devices through FDA data, discovering significant obstacles including duplicate records and unstructured text fields that prevented proper data analysis.","keywords":["database","FDA","implants"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["ATEX","Oracle","SQL Forms","SPSS","WordPerfect","Lotus 1-2-3","VAX","mainframe","PC"],"full_text":"U.S. News and World Report's summer plunge into computer-assisted reporting illustrated a multi-method approach to problems encountered in working with the Food and Drug Administration's Medical Device Reporting system. As an intern hired to help develop computer-assisted reporting projects at U.S. News, I worked with section editors, senior writers, associate editors and administrative staff. The magazine's library and its corporate data services division devoted substantial resources to computer-assisted reporting projects. In addition to the magazine's standard word-processing system, ATEX, I used Oracle version 6.0 with SQL Forms, SPSS/PC+ version 4.01, WordPerfect version 5.1 and Lotus 1-2-3 release 3. An AST 386SX/20 connected to a VAX mainframe by an ethernet network was set aside for computer-assisted projects. The result was \"Danger: Implants\" (Aug. 4, 1992), a six-page story in the News You Can Use section. Using the MDR computer tapes, U.S. News examined five medical devices set to undergo regulatory scrutiny next year. U.S. News devoted the most attention to penile implants, combining traditional reporting techniques with more scientific methods. The magazine's corporate data services division loaded the MDR tapes onto the VAX and created a database using Oracle. From my personal computer, I was able to log on to the VAX in just a few seconds. Using Oracle, I could retrieve selected records, sort by fields and produce reports that could be imported into other software packages for additional analysis. Soon after obtaining the FDA tapes, however, we discovered a major obstacle. When records were sorted by accession number (the MDR equivalent of identification numbers) and product code, the report revealed several problems, including thousands of duplicate records. An FDA programming error had rendered the tapes virtually useless. More meetings with the FDA were scheduled to resolve the problem. The one thing we learned from the tapes is that key information contained in a text field cannot be sorted. The MDR's massive \"event description* field consists of a text paragraph composed by data entry personnel that describes how and why the medical problem occurred. With penile implant reports, for instance, the patient's age (if listed at all) is in the event description field, as is the date of the implant operation and the details of the medical device problem. Because this information is buried in a text field, we could not instruct a database program to sort the penile implant records by patient age, implant date or problem type. Even worse, the form in which the MDR information is archived is inconsistent. The event description field may say that the penile implant patient is 42 years old, or it may say that he was born 8/12/50. One entry might describe the problem as a \"leak.\" Another might say \"lost fluid.\" Still another might say \"leakage from reservoir.\" An implant might protrude, extrude, erode or break the skin. This haphazard style of reporting device problems made it difficult to devise a surefire method of dumping the event description paragraphs into a text analysis or word processing system for more detailed analysis. So just as social scientists use multiple methods to tackle tough problems, we added another prong to the computer-assisted reporting task. While waiting for a new set of accurate MDR tapes, we completed an old fashioned, hand-coded content analysis of penile implant problem reports, using a sample of 1,196 implant records drawn from microfiche copies of the MDR tapes. We recorded product names, manufacturer names, the report description (death, serious injury or malfunction), report date, date of implant and patient age. In addition, we established nine categories of problems and coded the reports accordingly. Data from the code sheets were keypunched into Lotus and then imported into an SPSS system file for statistical analysis. With the push of a button, we could find out things such as 8 percent of the penile implants involved infections and the average age of patients with penile implant problems is 56. By subtracting the implant date from the MDR report date (SPSS \"compute\" command), we created a variable of the number of years between the implant operation and the problem report. The distribution ranged from zero years (implant and problem in same year) to 16 years but was skewed heavily, with about 62 percent of the problems occurring within three years of the implant date. By this time, new MDR tapes arrived, and integrity checks indicated that there were no problems. The tapes contained approximately 176,000 medical device problem reports from 1984 through mid-June 1992. Of these, 8,064 records were singled out as penile implant problem reports. In our continuing effort to cull from the event description field information about why the implant problems occurred, we created separate Oracle reports for each of the 13 penile implant models cited most often in the MDR tapes. The reports contained the full text of the event descriptions. A macro WordPerfect program was written to search for terms and count frequencies. We counted the number of times such terms as \"leak,\" \"lost fluid\" and \"infection\" appeared in the event descriptions. As explained earlier, the FDA's inconsistent reporting style makes this sort of measure imperfect. Here, however, the method served as a backup check to the hand-coded content analysis sample. Another check came from the FDA, whose analysts are able to sort medical device problems by categories. U.S. News obtained FDA analyses of problem types for the 13 penile implants most often. Because the FDA's system did not always fit our needs, it was not desirable to rely solely on FDA data. but again it was a valuable check. This multi-method approach produced three sources of information regarding specific types of penile implant problems: the content analysis of 1,196 records, the frequency counts from WordPerfect and the FDA analysis. From this we produced a box that ran at the close of the story listing the most common problems associated with various models of penile implants. For media such as U.S. News that are just starting to utilize computer-assisted reporting techniques, the road to completing projects is filled with an endless string of potholes. As crucial as it is for reporters to narrow their research question and devise an effective methodology, it is just as important to be able to react positively when glitches occur. Whether or not such projects ultimately succeed depends in large part on the willingness of the news organization to assemble a team of players flexible and creative enough to solve the problems they could not have foreseen. For U.S. News and other media that have moved past discussing computer-assisted reporting ideas and actually produced publishable work, they now face new challenges. They must educate editors and reporters about the advantages of computer-assisted methods, provide year-round in-house training and make computer-assisted reporting techniques part of their daily routine.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-september-real-help-for-data-junkies","slug":"1992-september-real-help-for-data-junkies","url":"https://dwillis.github.io/uplink/article/1992-september-real-help-for-data-junkies.html","year":1992,"month":"September","monthNum":9,"headline":"Real help for data junkies","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Association of Public Data Users provides journalists and data professionals with a newsletter and conferences to stay informed about developments in public data, including policy updates, new data releases, and technical information.","keywords":["data","journalism","newsletter"],"topics":["Data & Databases","Journalism & Newsroom","Government & Politics"],"technologies":["CD-ROM"],"full_text":"Keeping up with the wide range of developments in the field of public data can be pretty tough for reporters on the beat.\n\nBut the Association of Public Data Users can help with that.\n\nA sort of data-junkie support group, the association provides a forum for interaction between data users, sources and producers through its newsletter and conferences.\n\nupdates readers on the latest round in the user fee debate: U.S. Courts might be given authority to fund facility costs through fees on the use of the judicial system.\n\nThe eight- to ten-page newsletter is packed full of information that would be pretty dry to anyone except a data addict.\n\nSharing the front page was an article about a proposal by the Office of Management and Budget to amend federal policies relating to information dissemination, records management and cooperation between state and local governments.\n\nThe front page stories of the May 1992 issue of the APDU Newsletter were esoteric but informative.\n\n\"The Mother of All User Fees\"\n\nInside pages are filled with short descriptions of new government and private data releases available through a variety of mediums. Some of this stuff is really far out, yet fascinating.\n\nThe Calendar section lists instructional seminars around the country.\n\nUseful tidbits of information, including legislative updates and technical information, are sprinkled throughout the newsletter. For example, the Census Bureau has announced an error in STF3A data, and it looks like CD-ROMs are falling far short of their expected 20-year lifespan.\n\nIt costs $300 to join the Association of Public Data Users. Besides a newsletter, members get a telephone contact list for government data agencies and can share the cost of a database with other members. For information write to APDU, Princeton University Computing Center, 87 Prospect Avenue, Princeton, N.J. 08544.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-september-tech-tips-eliminating-happy-faces-and-other-garbage-characters","slug":"1992-september-tech-tips-eliminating-happy-faces-and-other-garbage-characters","url":"https://dwillis.github.io/uplink/article/1992-september-tech-tips-eliminating-happy-faces-and-other-garbage-characters.html","year":1992,"month":"September","monthNum":9,"headline":"Tech tips: Eliminating happy faces and other garbage characters","kicker":"","author_name":"Brant Houston","author_title":"","affiliation":"The Hartford Courant","summary":"A guide for journalists new to computer-assisted reporting on cleaning corrupted business data from diskettes using XYWrite III Plus and XDB database software to prepare files for matching and merging.","keywords":["data","database","journalism"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["XDB","XyWrite"],"full_text":"Subject: Cleaning up data from a file on diskette so you can run a match in XDB.\n\nProblem: You find out that a small state agency has collected information on thousands of businesses and keeps that information on a diskette. You want to merge that information with your own database of a couple of hundred business names.\n\nBefore you can worry about spelling differences in the names, you discover that the agency's file is, not surprisingly, fraught with problems, including garbage characters. And your problems aren't over after you import it into database software.\n\nSolution: If you are a journalist who is just getting into computer-assisted reporting, you probably don't have programming skills. But that's not a problem if you have a quick text editor and use a little imagination with XDB.\n\nOne commercial text editor is XYWrite III Plus. Copy the agency file onto your hard disk as BIZINFO and look at it by calling it up through XYWrite.\n\nThere is the file in all its ugliness with ASCII characters (such as the ubiquitous and ironic happy face) popping up here and there. The nice thing about XYWrite III Plus is that it has an ASCII menu from which you can pluck the offending character and put in the header area.\n\nBy typing CH /{ }// in the header area and hitting enter, XYWrite will race through the file and eliminate the happy faces. You can do whatever search-and-destroy missions you need to on other garbage characters. You can also eliminate unwanted spaces with CH / II, and add a character, if needed, with the same function.\n\nAnother nice feature of XYWrite is that it stores a backup of the original file in case you get excited by all the destruction and blast away an innocent character.\n\nWhen you import BIZINFO as a comma delimited file, the quote marks around the company names also get imported. (This actually happened to me.) That means there are company names with quotes around them in BIZINFO, and there are no quotes around company names in your own file (call it BIZNAME).\n\nRather than taking awhile to figure out what is going on (which might mean fathoming bits and bytes, hidden characters and tabs, etc.), there is a quick solution.\n\nExport BIZNAME from XDB as a comma delimited file with quotes around characters. Then re-import it as a fixed file, BIZNAME2. That brings those names back into XDB with quotes. Your company names now have a chance of matching because they have quotes around them in both files. At this point you might want to use XDB's command Xleft to run a join on the first nine characters in name field in each file in order to pick up most of your matches. For example: Select A.*, B.Name From BIZINFO A, BIZNAME2 B Where Xleft (A.Name,9) = Xleft(B.Name,9)\n\nMake sure you index both name fields before running it. You may have to clean up the data further by using XDB's Update command, but you are well on your way at this point. I'm sure there are other effective text editors and that a little programming knowledge would go a long way toward solving these problems in a better manner. The bizarre importing of names is just an example to show how to clean and shape data by using the import and export functions in XDB. Nonetheless, these tricks do come in handy when importing data from an optical scanner or an online source such as the Federal Election Commission.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-september-correction","slug":"1992-september-correction","url":"https://dwillis.github.io/uplink/article/1992-september-correction.html","year":1992,"month":"September","monthNum":9,"headline":"CORRECTION","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A correction to last month's Tech Tips article regarding a database command for handling company name spellings in the BADSPELL database.","keywords":["database","correction","command"],"topics":["Data & Databases","Journalism & Newsroom","Technology"],"technologies":["SQL"],"full_text":"In last month's Tech Tips, Elliot Jaspin made some suggestions for dealing with various spellings of a company, such as \"Amalgamated Zoot Suits,\" within a database called BADSPELL. But Uplink printed one command incorrectly. The correct command is: Update BADSPELL Set NewName = \"Amalagated\" Where Xleft(name, 11) = \"Amalgamated\" Our apologies.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-september-bits-bytes-and-nibbles","slug":"1992-september-bits-bytes-and-nibbles","url":"https://dwillis.github.io/uplink/article/1992-september-bits-bytes-and-nibbles.html","year":1992,"month":"September","monthNum":9,"headline":"Bits, bytes and nibbles","kicker":"","author_name":"","author_title":"","affiliation":"MICAR","summary":"MICAR founder Elliot Jaspin has left the University of Missouri to join Cox Newspapers in Washington, D.C., though he will continue advising the organization. A reporter at the Anchorage Daily News discovered data quality issues in the Home Mortgage Disclosure Act database where Alaska and Arkansas shared the same state abbreviation code.","keywords":["journalism","database","lending"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["HMDA"],"full_text":"Elliot Jaspin, MICAR's founder, has left the University of Missouri School of Journalism to take a position with Cox Newspapers in Washington, D.C. We'll miss his smiling mug around the office, but he's promised to continue advising us on technical matters and help out when he can with MICAR's programs. In fact, his first return engagement was last month when he came to help teach a week-long TRI-DART seminar.\n\nMICAR and its programs will continue at the School of Journalism with some help from Investigative Reporters & Editors, a non-profit organization also based at the University of Missouri.\n\nBruce Melzer at the Anchorage Daily News discovered the feds don't know their state abbreviations. When Melzer tried to call up Alaskan lending institutions in the Home Mortgage Disclosure Act data, he got Arkansas too. Apparently whoever entered the data used \"AK\" as the abbreviation for both states at least part of the time. HMDA data is divided into two files. \"LARS Public\" contains information about the loan and loan applicant. This first file lists lending institutions by a numeric code composed of two fields: \"respondent id\" and \"agency code.\" This code must be cross-referenced with the second file, \"TS Public,\" which contains the name and address of the lending institutions, including two-letter state abbreviations. Inaccurate state abbreviations make filtering the TS file by state difficult. But it's not likely anybody would need to do this. Instead, use the numeric state code in LARS to filter for applicants in the desired state. Then search the TS file using the institution code to get names and addresses of lending institutions. Uplink needs your input! Send bits to MICAR, 120 Neff Hall, University of Missouri, Columbia, Mo. 65211.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-october-quick-hits-writing-a-computer-assisted-story-in-less-than-a-week","slug":"1992-october-quick-hits-writing-a-computer-assisted-story-in-less-than-a-week","url":"https://dwillis.github.io/uplink/article/1992-october-quick-hits-writing-a-computer-assisted-story-in-less-than-a-week.html","year":1992,"month":"October","monthNum":10,"headline":"Quick hits! Writing a computer-assisted story in less than a week","kicker":"","author_name":"Alison Boggs","author_title":"Graduate student","affiliation":"University of Missouri School of Journalism","summary":"Van Denton used a database to quickly produce a more detailed story on lobbyist spending, illustrating the potential of computer-assisted reporting.","keywords":["reporting","database","lobbyists"],"topics":["Investigative Methods","Government & Politics","Journalism & Newsroom"],"technologies":["FoxPro"],"full_text":"Van Denton, a beat reporter for the Raleigh (N.C.) News & Observer, was doing a routine assignment. He didn't need to look for a faster way. But he did, and he ended up with much more than a routine story. All it took was a newly formed database and a floppy disk.\n\nEvery year, Denton's paper does a story on the amount of money lobbyists spend wining and dining legislators. The information was only available on hard copy, which required searching through voluminous documentation, making calculations and coming out with a story in a couple of weeks.\n\nDenton started early, looked around a little and found person in the secretary of state's office who was building a database with all the needed information. A couple of weeks later, when all the data was in, he took a floppy disk down to the secretary and had the file copied, at no cost to the newspaper.\n\nHis story was ready two days later and included not only how much lobbyists spent, in total and per legislator, but also was broken down by interest group, such as medicine or insurance. He got a better, more complex story, in less time.\n\nThere are hundreds of examples of reporters who have done in-depth investigations over months, or even years, using computer-assisted reporting. However, reporters all over the country, from North Carolina to Washington state, are turning over stories in a week or less with the help of a database and some creative thinking. These reporters are putting out faster, harder-hitting stories long before their competition.\n\nHere's a simple rule to follow: If you're going through volumes of documents, ask if there's a tape somewhere. Then find someone who knows what to do with it.\n\nPat Stith has been The News & Observer's computer expert for the past two years. He said the paper put out 31 articles in the past year using computer-assisted reporting. Also, a 20-hour training program was set up to remove the mystique for beat reporters.\n\nDenton took the course, which he said made him aware of possibilities of this technique. On the lobby story, he was faced with 866 reports to be analyzed over the course of a couple of weeks. He looked for another way. Then he found out about the database at the secretary of state's office.\n\nOn a Wednesday, Denton got a copy of the database and brought it back to Stith in the newsroom. Stith said the \"little bitty database,\" a flat file, took \"five seconds\" to load into Foxpro. Reports were ready for Denton to take home by the end of the day.\n\nDenton had requested reports on many possible combinations of the information. When he went through the reports that Wednesday night, he tagged all the special interest groups. The next morning, Stith created a new field for groups, such as oil, insurance and medicine, and printed more reports calculating amount spent by each group.\n\nDenton did follow-up interviews and had the story ready on Friday. But it ran on Sunday's front page, because, after all, they were weeks ahead of their competition.\n\n\"I'm more interested in something I can use every day,\" said Denton. \"It's just a tool to get a story like any other tool.\"\n\nPenny Loeb of New York Newsday made it through 500 pages of a 1,000-page document before she thought, \"This has got to be on a tape somewhere.\" Sure enough, it was.\n\nShe and Kevin Flynn were investigating allegations that attorneys were making contributions to Attorney General Robert Abrams' campaign for the U.S. Senate while they had business pending with his office. The reporters discovered that Abrams had received more than $120,000 for his campaign from attorneys who do business with his office, including $33,000 from lawyers who had matters pending before him at the time of the contributions.\n\nAfter six hours of searching through documents, Loeb looked for a tape. She got a tape from the attorney general's office, but a record layout, telling her how to load it, was not included. She counted some fields and made her own record layout. She loaded the tape in about two hours.\n\nWith the tape, Loeb could immediately go into the database and see names of attorneys who had pending business.\n\nLoeb printed two sets of reports, one from the attorney general's list of pending business and one from the Federal Election Committee's list of campaign contributions. Since neither list identified law firms, she had to do a little creative thinking.\n\nLoeb asked herself what distinguishing characteristics are in law firms' names. Of course - ampersands! She printed reports of all occupations with ampersands in the company name. Then she went through both lists by hand, matching up the names.\n\nLoeb loaded the tape on Friday, Aug. 28. By the following day, she had identified 12 instances of lawyers making contributions while they had matters pending. However, she wasn't sure she had the right firms.\n\nOn Tuesday, Sept. 1, she and Flynn got on the phones to the law firms to interview them and confirm the data. Loeb said this took three days and was the most time-consuming part of the story.\n\nBy Friday, Sept. 4, the writing was done. The story was scheduled for Sunday the 6th, but ran on Tuesday the 8th. All together, Loeb and Flynn churned out the hard-hitting story in eight days.\n\nAnother story that took about a week came from Brant Houston at The Hartford (Conn.) Courant. A breaking story on a possible serial murderer had been unfolding over the course of a couple weeks. Houston built his own database in XDB, using clips on unsolved murders. He included dates, descriptions of the female victims, time and place the body was found, demographic information and race. This took a few hours.\n\nNext, Houston went to the medical examiner's office and made a separate database, to which he added significantly more information. After a couple of days, he had a clean printout with a couple of hundred records.\n\nAfter two to three days of interviewing and updating information, the first story came out. Houston had established a possible pattern and enough connections that the state created an investigative task force. Additional stories were published and Houston's stories acted as a \"tipster\" for law enforcement officials.\n\n\"Once you have a database in-house, you seldom use it just once,\" Houston said.\n\nBob Wodnik of The Everett (Wash.) Herald agreed. His XDB database on building permit records had been used for numerous stories. Wodnik, Scott North and Jim Muhls did a story in three days on increased development which led to more stormwater runoff into an already flood-prone area.\n\nWith this story, there was no problem of building, finding or copying a database. It was there, and the reporters had become used to it, which made the quick story even easier.\n\nGeorge Landau, at the St. Louis Post-Dispatch, said his paper had done quick stories with in-house databases, too. Three examples came from computerized Missouri death certificates:\n\nDeaths from breast cancer were occurring at a higher rate (in certain age ranges) in the city of St. Louis than in the county or in the state as a whole.\nNo decline was shown in the number of drowning deaths among children for the last 12 years in the St. Louis area or the state as a whole.\nThe rate of asthma deaths increased in Missouri over the last 12 years.\n\nStith said it's a realistic goal for the News & Observer to run a computer-assisted story every week.\n\n\"You could sit there in one database and do ten stories,\" he said. But, he added, the objective is to find one good story and get it in the paper.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-october-when-tri-is-not-enough-try-state-hazardous-waste-manifests","slug":"1992-october-when-tri-is-not-enough-try-state-hazardous-waste-manifests","url":"https://dwillis.github.io/uplink/article/1992-october-when-tri-is-not-enough-try-state-hazardous-waste-manifests.html","year":1992,"month":"October","monthNum":10,"headline":"When TRI is not enough, try state hazardous waste manifests","kicker":"","author_name":"Adam Berliant","author_title":"Founder","affiliation":"EPIC (Electronic Public Information Consultants)","summary":"Hazardous waste manifests provide detailed tracking of toxic materials that can supplement federal TRI data, with about half of U.S. states offering computerized access to these records for investigative journalism purposes. Reporters can use manifest data to identify illegal dumping, verify environmental compliance, and enhance local accountability in environmental stories.","keywords":["manifests","hazardous","tracking"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"When it comes to environmental issues, particularly the treatment and disposal of hazardous material, Uplink often touts the EPA's Toxic Release Inventory. TRI is a database with untold uses, but it contains generalized data and provides little detail with regard to the fate of hazardous materials.\n\nTRI records scarcely go beyond \"Off-site Transfer\" when describing the fate of the millions of tons of hazardous materials that were stored, treated and/or shipped offsite by the thousands of large-quantity waste generators in the United States.\n\nTRI serves as excellent summary material and a good initial source for information concerning manufacturers and their toxic chemical discharges. The challenge for computer-assisted reporters is to find new databases on local or state levels and use them to develop more detailed applications in the environmental arena to supplement the federal perspective. At Electronic Public Information Consultants (EPIC), we tap into data that helps make this possible: state hazardous waste manifests.\n\nHazardous waste manifests are one of the many consequences of the 1976 Resource Conservation and Recovery Act. In theory, RCRA is supposed to help environmental enforcement agencies track every ounce of hazardous material \"from cradle to grave.\" In short, the act compels manufacturers that create and ship hazardous materials to complete Uniform Hazardous Waste Manifest forms every time waste leaves their facilities.\n\nDespite the name, there is no such thing as a standardized UHWM form. Some manufacturers actually generate their own forms. However, RCRA does specify reporting requirements. The manifest must not only describe hazardous material in superb detail, but also must specify who is transporting the material, where it is going (even if out of state), when the material is shipped, and just about anything else you would want to know about the material's transport, delivery and disposal.\n\nUsing hazardous waste manifests, reporters can track the transport of hazardous materials through their state. In all likelihood, they can also identify the source and destination of the most menacing material, cross-reference the data with TRI and state enforcement data to determine if discharge figures are consistent, and even identify illegal transport activities. Needless to say, reporters can thereby add a tremendous amount of detail to TRI's \"Off-site Transfer\" records.\n\nThe good news for computer-assisted reporters is that about half the states in the United States are using computers to compile UHWMs.\n\nThey are: Alaska, Arkansas, California, Connecticut, Delaware, Illinois, Indiana, Louisiana, Maryland, Massachusetts, Michigan, Minnesota, Missouri, Nevada, New Hampshire, New York, New Jersey, Oregon, Rhode Island, Texas, Vermont, Washington and Wisconsin. As of last year, Alabama, West Virginia, and Maine were planning to automate.\n\nThe other states, as you might imagine, are complaining about volume problems and their relative inability to handle the information. Ohio, for example, supplements its current system (or lack thereof) by purchasing manifest databases from bordering states - Pennsylvania, Michigan, and Indiana all of which have automated systems. You can take a similar approach if your state is not using computers. At this point, it is difficult to be specific about fees for data. EPIC paid $86 for Texas manifests, Illinois manifests are free of charge, and New Jersey charges $4.50 per second of CPU time with a $500 maximum fee.\n\nRequesting data should not be difficult. A data processor at the Department of Natural Resources in New Jersey boasted that he fills hundreds of requests for manifest databases each month (don't worry, he couldn't recall doing much work for journalists). Whether he was exaggerating or not, he did make it clear that obtaining data was no problem.\n\nState agencies store manifests anywhere from three years to indefinitely. Unfortunately, every state will have a different agency responsible for compiling the manifest data, and each will have different policies concerning format and maintenance.\n\nOne way to get started would be to call the agency in your state responsible for enforcing environmental permits (e.g. Department of Natural Resources, state Environmental Protection Agency, or Department of Ecology). If that fails, call your state Department of Transportation. EPIC has started to compile as many of these hazardous waste databases as possible. Feel free to call us.\n\nFor more information regarding hazardous waste manifests, there is an excellent, free resource manual called \"A State Guide to the Uniform Hazardous Waste Manifest\" published by the National Governors Association (444 North Capitol Street, Washington, D.C., 20001-1572). In addition to describing the manifest process in detail, the guide identifies appropriate agencies and contacts in every state.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-october-the-national-library-on-money-and-politics-searches-databases-for-you","slug":"1992-october-the-national-library-on-money-and-politics-searches-databases-for-you","url":"https://dwillis.github.io/uplink/article/1992-october-the-national-library-on-money-and-politics-searches-databases-for-you.html","year":1992,"month":"October","monthNum":10,"headline":"The National Library on Money & Politics searches databases for you","kicker":"","author_name":"Richard Mullins","author_title":"Researcher and systems administrator","affiliation":"National Library on Money & Politics","summary":"The National Library on Money & Politics provides journalists with processed Federal Election Commission data dating back to 1979, enabling easier access to campaign finance information for reporting on political contributions and funding sources.","keywords":["campaign","finance","contributions"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["CSV","FoxPro","dBase","Paradox","CompuServe"],"full_text":"Think about what the civics and history books tell young citizens about democracy and voting, how the members of Congress represent The People in their district, about the Power of One Person in the Voting Booth.\n\nAnd now, think of this, from the Real World. Every member of Congress has two constituencies. The first has the most people - the citizens of that district: the few who voted for the winner, the few who voted against and the majority who didn't vote at all.\n\nThe other constituency, the one not in the schoolbooks, has the most power. The congressperson knows every member. They gave the money for the expensive campaign. And the high cost of modern campaigning means that the congressperson spends a great deal of time with this limited group of people who have the money and motivation - business, political, ideological - to give to campaigns.\n\nThere are lots of stories in tracing what money does to politics. Most of the information is on record. It affects just about every aspect of the lives of the people who read your paper or listen to your broadcasts. Since you're a reporter, you should be figuring out what the data means and then telling the stories to your readers and viewers.\n\nSo what does it take to get started? You can go to the Federal Election Commission with your FOIA request, pay the fees, load the tapes, convert the data to a form your software can import and start figuring out how to write the queries.\n\nBut the National Library on Money & Politics has already done this. You can get more information, in less time, with less trouble. And in many cases it will cost your news organization less than the mileage to the state high school football championship.\n\nThe Library already bought all the tapes, with all the data, all the way back to 1979. It can give you information that covers just the states, districts, officials and years you're interested in. You can get the data for just one congressperson now, and after those stories are done, decide to order all the data about the two senators in your state.\n\nThe FEC's job is to maintain the records and make them available to the public. The Library's job is help reporters figure out what data they need and get it to them quickly. If reporters need help deciding how to approach the data, or finding the stories in the data, the Library staff can help. They know campaign finance and they've been reporters, too.\n\nThe Library gives you information that's not in the FEC records - a coding system that identifies the political/ideological/business interest of the Political Action Committees. The FEC has the PAC's full official name and the sponsor, but many PAC names give you no clue about what their interest is.\n\nSome common requests for data: Send the tables of all the PAC and individual contributions to a congressperson over a certain period. How much money did Senator Z get from energy interests? From the transportation industry? Some questions can be answered with just a fax or a return phone call: How much money have cable industry PACs given since 1980?\n\nYou may want to start out with everything for your state since 1979. That would include all the individual contributors in your state who gave to PACs, political parties and presidential candidates; and all the contributions from PACs and individuals to Congressional candidates in your state.\n\nThe data is put on floppies in the format you choose: CSV format, a DBF file for Foxpro and dBase, or a Paradox file. We can send it to you Federal Express or, even better and faster, to your Compuserve E-mail.\n\nThe National Library on Money & Politics is affiliated with the Center for Responsive Politics. The Library is a non-profit, non-partisan public education service. The offices are at 1320 19th St. NW, Washington, DC 20036. The Library's phone number is (202) 857-0318.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-october-foxpro-adds-a-new-dimension-to-sql","slug":"1992-october-foxpro-adds-a-new-dimension-to-sql","url":"https://dwillis.github.io/uplink/article/1992-october-foxpro-adds-a-new-dimension-to-sql.html","year":1992,"month":"October","monthNum":10,"headline":"FoxPro adds a new dimension to SQL","kicker":"Tech Tips:","author_name":"Dan Woods","author_title":"Reporter","affiliation":"The Raleigh News & Observer","summary":"FoxPro's SQL point-and-click interface makes database querying accessible to journalists of all skill levels, from beginners to experienced users, enabling more reporters to engage in computer-assisted reporting. The tool simplifies SQL use and automates coding tasks, democratizing technical skills across newsrooms.","keywords":["FoxPro","SQL","journalism"],"topics":["Technology","Investigative Methods","Journalism & Newsroom"],"technologies":["FoxPro","SQL"],"full_text":"FoxPro's Point and Shoot SQL Interface can smooth the way for beginners.\n\nThe great thing about Structured Query Language is that it is simple, powerful and for the most part, easy. Beginners can get started quickly, and pros can go as fast as they need to. It's no surprise that SQL has been adopted by reporters enthusiastically.\n\nFor the first stage of a program of computer-assisted reporting, SQL releases the power of the highly-motivated individuals who are driving the effort. But after a small cadre of reporters has hit a few home runs, knowledge must spread throughout the newsroom in order to avoid a bottleneck. That's why training eventually becomes one of the top, if not the top priority to ensure the success of a program of computer-assisted reporting.\n\nFor those news organizations using FoxPro Version 2.0, the Relational Query By Example interface provides a tremendous tool for teaching SQL to beginners. No matter how easy SQL may seem to the initiated, beginners need, and should have, as much help as possible.\n\nWhat the Relational Query By Example interface does is provide a point and shoot menu interface for SQL. For each portion of the SQL statement, the user chooses from a menu instead of typing the command. For the seasoned SQL programmer, this may seem awkward at first. I know it did for me, but now I write almost all of my SQL statements using RQBE.\n\nThe benefit for beginners is that they don't have to sit there facing a SELECT statement, wondering what to type next. With the RQBE interface, they can play around and use menus to build the statement they want as follows.\n\nTo enter the RQBE interface, a FoxPro user creates a new query file by choosing New from the file menu and specifying the file type as Query. This displays the interface screen.\n\nWhen FoxPro creates the query, it assumes that whichever database is open will be the subject of the query. If this isn't the correct database, then it can be cleared from the screen, and another database can be added from a menu of all available databases. If a database named \"DB1\" were open, then that database would be used for the query and its fields would be displayed in the Select Fields window.\n\nThe user can change which fields appear in the query, or to choose various functions, such as SUM, AVG, COUNT, etc.\n\nWhen a user adds a second database to the query, say \"DB2\", FoxPro pops up a menu-driven interface to the WHERE clause. Instead of typing in WHERE DB2.IDNUM = DB1.IDNUM, the WHERE interface allows a user to point and shoot. First a menu of all the fields in the DB2 database is presented so the user can select the appropriate field to join the databases together. Then the user selects the comparison operator, (LIKE or =, EXACTLY LIKE, etc.), and finally a menu of fields from the DB1 database appears.\n\nAfter the WHERE menu has been filled out, the where clause will appear in the large box at the bottom left of the screen. Also, all of the fields in the DB2 database will appear when the Select Fields menu is chosen. The same style interface is available for the ORDER BY, GROUP BY and HAVING clauses.\n\nThe menus are nice for queries that use several databases, because the job of placing the alias tags that identify which field belongs to which database is handled automatically.\n\nWhen I first started to use this interface, I balked because I thought that it was a replacement for SQL, not a way to learn it. As soon as I clicked on the See SQL box, I realized I was wrong. Clicking on this box displays the SQL query built by the interface. A beginner can point and shoot for a while, then look at the SQL that he or she has created.\n\nOnce the query has been completed, then it can be executed. The output can be directed to a temporary database for browsing, a permanent database for later use, or a report for printing.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-october-an-electronic-forum-on-computers-in-journalism-debuts","slug":"1992-october-an-electronic-forum-on-computers-in-journalism-debuts","url":"https://dwillis.github.io/uplink/article/1992-october-an-electronic-forum-on-computers-in-journalism-debuts.html","year":1992,"month":"October","monthNum":10,"headline":"An electronic forum on computers in journalism debuts","kicker":"Bits, bytes and nibbles","author_name":"Elliot Parker","author_title":"curator of CARR-L","affiliation":"University of Louisville / Central Michigan University","summary":"The University of Louisville has created CARR-L, an online forum for journalists and journalism educators to discuss computer use in reporting, covering topics from database searching to investigative reporting techniques.","keywords":["forum","journalists","technology"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet"],"full_text":"The University of Louisville in Kentucky has set up an online forum for working journalists and journalism educators, news librarians and researchers.\n\nComputer-Assisted Research and Reporting List, or CARR-L, as this corner of cyberspace is called, focuses on the use of computers in journalism.\n\nTopics range from text processing and graphics to online database searching, computer communications and investigative reporting, according to Elliot Parker, one of CARR-L's curators.\n\nThe nature of the exchange depends on the users, but here are a few topics which have passed through CARR-L recently: Where do we get state-level information in electronic form and how do we know what is available?; is there a net source for backgrounding information on hurricanes?; software reviews, job announcements.\n\nFor more information on CARR-L, contact Elliot Parker, Department of Journalism, Central Michigan University, Mount Pleasant, Mich., 48859, (517) 774-3196.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-october-more-errors-surface-in-hmda-data","slug":"1992-october-more-errors-surface-in-hmda-data","url":"https://dwillis.github.io/uplink/article/1992-october-more-errors-surface-in-hmda-data.html","year":1992,"month":"October","monthNum":10,"headline":"More errors surface in HMDA data","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"reporter","affiliation":"Anchorage Daily News","summary":"The 1990 Home Mortgage Disclosure Act data contains significant errors in state coding, with Alaska's actual error rate reaching 35 percent due to misclassified Arizona loan applications, far exceeding the Federal Reserve's accepted 3 percent threshold.","keywords":["HMDA","data","errors"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"More problems have surfaced in the 1990 Home Mortgage Disclosure Act (HMDA) data.\n\nLast month Anchorage Daily News reporter Bruce Melzer told Uplink that the Feds used the state abbreviation \"AK\" for Arkansas as well as Alaska in the bank address file.\n\nThis didn't seem like a problem at the time since the loan applications, listed in a separate file, used a numeric code to identify states.\n\nHowever, Melzer found that \"02\", the code for Alaska, was also used for Arizona part of the time. Of course, this means Alaska records could be hidden under some other state code.\n\nWhen asked about this, Cornelius Driggins at the Federal Reserve said the problem falls within their accepted 3 percent error rate.\n\nUnfortunately for Melzer, the error rate in Alaska runs closer to 35 percent, with 2,077 Arizona loan applications recorded under \"02\".","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-november-pursuing-the-paper-trail","slug":"1992-november-pursuing-the-paper-trail","url":"https://dwillis.github.io/uplink/article/1992-november-pursuing-the-paper-trail.html","year":1992,"month":"November","monthNum":11,"headline":"Pursuing the paper trail","kicker":"","author_name":"Jerry Uhrhammer","author_title":"Reporter","affiliation":"The Morning News Tribune","summary":"Washington state reporters used computer data to uncover government negligence, with taxpayers paying over $76 million in settlements due to employee misconduct.","keywords":["government","negligence","data"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["PC"],"full_text":"While covering the case of a Washington state ferry boat captain accused of sexual harassment, News Tribune courts reporter Cynthia Flash began hearing stories of other lawsuits pending against the state. She wondered how many.\n\nWith the help of a computer, she found out. There were thousands, many thousands, not just for sexual harassment but also for illegal rectal probes of prison inmates and for people killed and maimed because of wrongful conduct and neglect by state employees.\n\nAn analysis of computerized state records led to a three-day series that revealed for the first time that state taxpayers were paying a heavy price for the negligence and misconduct of state employees - more than $76 million in settlements in less than a decade. The annual average was $9 million. Although ranking only 18th in population, the state was paying more per capita in liability damage awards than any other state in the contiguous United States. The series was the first major computer-assisted project at The (Tacoma, Wash.) Morning News Tribune and, as this piece is written, its findings are reverberating in the campaign to elect a new attorney general. Aside from that impact, its success underscored the fact that time-tested techniques of investigative reporting - specifically, following the paper trail - should be an integral part of computer-assisted reporting. Following the paper trail and using your noggin to figure out where documentation can be found are vital. The big difference is that, in these times, the indexes to the records, if not the records themselves, are kept on computer. In this project, the computer data led the way to the paper trail.\n\nWhile there is a tendency to measure computer-assisted projects by the number of megabytes in the database you're crunching, don't be afraid to think small. In Washington state, many obscure public agencies have their own little fiefdoms. Instead of storing data on a state megacomputer, they tend to keep their records on PCs. The whole database you're after may fit on a few diskettes.\n\nSomething I learned in two decades of following paper trails: Read the applicable state laws before you get too far into your investigation. Become knowledgeable about how the system is supposed to operate.\n\nThat's what I did when I was asked to do the computer work for Flash's investigation. The risk management statutes yielded a quick dividend; I found that the risk management agency was required, by law, to maintain a computerized tracking system of all claims filed against the state.\n\nThe trouble was this computerized tracking system was, by law, confidential. However, the same law held that claims filed against the state were public records, available for inspection. Our strategy: Press for release of the data that was public, even though it was on a \"confidential\" database.\n\nWhen Flash made her first visit to the agency, she was offered a voluminous print-out containing the key information of the public claims data kept on the computer. Altogether, there were some 25,000 cases.\n\nThe printed data was so diffused, so overwhelming, that it was virtually unusable. We needed the computer data in order to make sense of anything.\n\nI made a public records request for the computer data, specifically the portion from the hard copy claims the state law said was public record.\n\nNothing like that had happened before at the risk management agency. The director sent the request to the attorney general's office for a legal opinion. It took six weeks to get an answer.\n\nFinally we got word that our request had been approved; we would be given the computer data for all closed cases, some 20,000 of them, together with the type of case - civil rights, wrongful death, personal injury, etc. (Later, we got an additional 5,000 records of open cases.)\n\nUsing XDB, I began breaking down the claims data for each department, year by year.\n\nThe information was used in an information graphic with the number of cases per department and total dollars paid in settlements. I also tracked, year by year, the number of cases of each particular type. Civil rights complaints (including sexual harassment) turned out to be the fastest-growing type of claim.\n\nThis data was inserted into a QuattroPro spreadsheet that was invaluable for figuring percentages and making other calculations.\n\nBy manipulating the data, I was able to pinpoint which specific departments had the worst records.\n\nBut the most important use of the computer data was linking the worst cases, those with the biggest settlements, to claim numbers. This enabled Flash to go back to the agency and inspect the hard copy claims.\n\nIt was a case of using the computer trail to help us explore the paper trail, and its importance cannot be overstated. The hard copy claims were a rich source of detail, including names, addresses and telephone numbers, helping us to contact and interview the victims and survivors.\n\nThe series found that while total claims against the state were declining slightly, civil rights violations were skyrocketing - and so were settlements. The state ferry system, with the worst record, finally instituted a training program for employees.\n\nBut the series also showed that some state agencies simply kept doing things the same as ever, despite the fact that their mistakes were costing the taxpayer millions of dollars.\n\nFlash found a 1978 case in which an auto plunged off an icy bridge, killing the driver. A Department of Transportation engineer recommended installing a $22,000 guard rail.\n\nNothing was done.\n\nFive years later, another driver plunged off the bridge during a rainstorm, falling 40 feet to the gully below. The victim suffered a head injury that caused paralysis, memory loss and emotional problems.\n\nLess than a year later, a third car slammed into a bridge girder during a heavy rainstorm. The driver suffered a broken back.\n\nThe last two victims sued the state and won nearly $1.8 million. And the bridge? It still isn't fixed. The Department of Corrections, which operates the state's penal institutions, turned out to be a major trouble spot. One series of cases involved a convicted rapist who was released from the sexual psychopath program at a state mental hospital. Over the next three years before he was caught, the rapist brutalized 26 Tacoma-area women.\n\nSo far the state has paid some $640,000 for its mistake in releasing the rapist, settling lawsuits brought by 18 of the victims. And the total is climbing as more victims sue.\n\nFlash also unearthed the existence of \"Dr. Bigfinger,\" the practice at state prisons of performing body cavity searches on all inmates admitted to the maximum security units.\n\nPrison officials maintained the body cavity search policy was necessary to keep the prisons free from drugs, knives and rapes. The inmates, however, charged that the rectal searches amounted to rape.\n\nIn 1987, a federal judge ruled the policy unconstitutional, finding that prison officials used the policy for punitive, rather than safety reasons.\n\nThe state stopped the \"Dr. Bigfinger\" practice when the first lawsuits were filed. But the issue of whether damages should be awarded to the inmates is being appealed by the state.\n\nSo far, the illegal searches have cost the state $194,528. The state has paid 65 inmates $200 to $500 for each search. Some cases are still pending.\n\nFlash emerged from the project as a believer in computer-assisted reporting. \"The computer pointed the way,\" she said.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-november-six-successes-in-computer-assisted-election-coverage","slug":"1992-november-six-successes-in-computer-assisted-election-coverage","url":"https://dwillis.github.io/uplink/article/1992-november-six-successes-in-computer-assisted-election-coverage.html","year":1992,"month":"November","monthNum":11,"headline":"Six successes in computer-assisted election coverage","kicker":"","author_name":"Marie-Susanne Langille","author_title":"","affiliation":"MICAR","summary":"Computer-assisted journalism investigations across the country analyzed FEC campaign finance records to reveal patterns in political contributions, including Hispanic donor trends, Senate race funding sources, and corporate soft-money donations to political committees.","keywords":["contributions","campaigns","money"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["dBase","Lotus 1-2-3"],"full_text":"This is a roundup of computer-assisted election stories from around the country on everything from campaign contributions to voting patterns.\n\n\"Looking into the deepest pockets\"\n\nBy Rick Mendosa and Sarah Acosta Hispanic Business (October 1992) This computer-assisted story shows how Hispanic contributions to congressional and presidential candidates, national party funds and political action committees more than tripled over the past three presidential elections. Interestingly, a disproportionate amount of the $3.16 million Hispanics contributed in this election cycle as of March 31 is coming from Florida and is going to Republicans, according to the report. Mendosa and Acosta also isolated the largest Hispanic political giver. The National Library on Money & Politics, a Washington-based non-partisan, non-profit research organization, searched FEC records from 1979 to 1992 for 3,000 Hispanic surnames supplied by Hispanic Business and isolated 30,000 contributions. Mendoza and Acosta took these records, in comma delimited form, and analyzed them in dBASE by year, party, state type of contribution.\n\n\"Senate race ante up to $1 million\"\n\nBy Dan Popkey The Idaho Statesman (Oct. 11, 1992) Reporter Dan Popkey used FEC data to see just how seriously Republican and U.S. Senate hopeful Dirk Kempthorne took his own campaign call to \"take our government back from the politics as usual crowd.\" Popkey's analysis showed Kempthorne raised twice as much money from political action committees as his opponent. Using the same data, Popkey showed Kempthorne's financial support came from timber and mining interests while his opponent's came from labor unions and agriculture interests. The Statesman also documented how much of each candidate's support came from out-of-state. The reporting took three to five days, Popkey said. The Statesman got the FEC data from the National Library on Money & Politics and used Lotus spreadsheet to sort contributors by address, occupation and name.\n\n\"Bush holds big lead in corporate donations\"\n\nBy Lance Williams San Francisco Examiner (Oct. 13, 1992) President Bush fell behind Democratic candidate Bill Clinton in California popularity polls in this year's presidential campaign, but the GOP was ahead of the Democrats almost three to one in state soft money donations, according to the Examiner.\n\nComputer-assisted reporter Lance Williams looked at FEC records on state corporate contributions to four Democratic and four Republican committees from the beginning of the election cycle through June 1st to show how the GOP had raised $5.5 million in Californian \"soft-money\"-- beefy company donations exempt from election finance reform laws that are meant to lessen the influence of special interest money in presidential campaigns. Williams' analysis showed the GOP has received particularly large donations from energy firms worried about laws restricting oil drilling, agribusiness interests worried about losing federal subsidies and real estate interests that would profit from a Bush plan to free up wetlands for development. The Examiner also listed the three top donors in each of six sectors, from energy to entertainment. During this one-month, one-person project, Williams looked at more than 7,000 contributors and used dBASE to sort data according to interest group.\n\n\"Paying attention to political influences\"\n\nBy Thomas Huang and Alex Marshall, with computer analysis by Tom Boyer The Virginian Pilot and the Ledger-Star (Oct. 18, 1992) A Virginian Pilot/Ledger Star survey of congressional records and a computer analysis of campaign contributions illustrated a political maxim that is easy to complain about but difficult to document: money buys influence. Computer-assisted reporter Tom Boyer used an FEC database supplied by the National Library on Money & Politics to show an alignment between PAC campaign contributions and the political positions of three congressional incumbents. Boyer also isolated each candidate's top five donors and the amounts, as well as the top 10 donors overall. Boyer recommends dealing with the Library instead of the FEC. \"FEC tapes are expensive and cumbersome to work with, and to make sense of PAC contributions it's a huge help to have the detailed industry codes the National Library folks have added.\" The best way to keep up to speed with contribution data, Boyer says, is have National Library data plus a link to the FEC's on-line system. The best way to use PAC-donation data, Boyer suggests, is in routine congress coverage. \"We've plowed campaign finance data into our day-to-day coverage of the Virginia General Assembly,\" he says. \"Legislators hate it, and lobbyists hate it even more. Some lawmakers become timid about carrying a PAC's water if they know their vote will end up beside their contributions in the next day's paper.\"\n\n\"Bush slipped in suburbs\"\n\nBy Ted Mellnik and Allen Norwood The Charlotte Observer (Nov. 11, 1992)\n\nA post-election story by the Observer matched precinct results with 1990 census data to show how the swing away from George Bush in North Carolina was more pronounced in cities and suburbs.\n\nBush support in state urban and suburban areas dropped since 1988 by more than 15 percentage points, compared with a 13.6 point drop across rural areas, according to the study.\n\nThe Observer also used census data to analyze election returns for four groups of precincts, each group dominated by people with similar education, occupations and household income.\n\nThe study showed Bush lost support among upper-income voters, mostly executives and professionals, and that most of those votes went to Perot. The votes Bush lost among middle-class voters were more likely to go to Clinton.\n\nFor this precinct analysis story, the Observer started with block-group level figures on income and education, then used Atlas GIS to make precinct-level estimates and SPSSPC to search for data patterns and to create precinct groups. The paper used dBASE to relate demographic class with precinct vote results.\n\n\"The politics of reform\"\n\nBy Patti Epler, with computer analysis by Jerry Uhrhammer The (Tacoma, Wash.) Morning News Tribune (Oct. 1992)\n\nThe News Tribune used computers to analyze the impact of a Washington state campaign financing initiative before the votes were cast.\n\nCritics of Initiative-134, aimed at limiting the money going into political campaigns, said it was a political maneuver to boost the influence of business at the expense of labor interests. The criticism focused on a clause prohibiting the use of some union dues for political purposes.\n\nThe News Tribune's analysis of 1990 campaign records from the state Public Disclosure Commission bore out the critics' claims by showing that I-134 would have cost Democratic candidates $1.2 million dollars in labor backing. Overall, I-134 would have cost the Democrats $400,000 more than the Republicans in campaign contributions for that election cycle.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-november-looking-into-the-deepest-pockets","slug":"1992-november-looking-into-the-deepest-pockets","url":"https://dwillis.github.io/uplink/article/1992-november-looking-into-the-deepest-pockets.html","year":1992,"month":"November","monthNum":11,"headline":"Looking into the deepest pockets","kicker":"","author_name":"Rick Mendosa and Sarah Acosta","author_title":"","affiliation":"Hispanic Business","summary":"Hispanic political contributions to congressional, presidential, and party candidates have more than tripled over three presidential elections, with a disproportionate amount from Florida going to Republicans in the current cycle.","keywords":["Hispanic","contributions","Republicans"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["dBase"],"full_text":"This computer-assisted story shows how Hispanic contributions to congressional and presidential candidates, national party funds and political action committees more than tripled over the past three presidential elec- tions. Interestingly, a disproportionate amount of the $3.16 million Hispanics contributed in this election cycle as of March 31 is coming from Florida and is going to Republicans, according to the report. Mendosa and Acosta also isolated the largest Hispanic political giver. The National Library on Money & Politics, a Washington-based non-partisan, non-profit research organization, searched FEC records from 1979 to 1992 for 3,000 Hispanic surnames supplied by Hispanic Business and isolated 30,000 contributions. Mendoza and Acosta took these records, in comma delimited form, and zed them in dBASE by year, party, state type of contribution.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-november-senate-race-ante-up-to-dollar1-million","slug":"1992-november-senate-race-ante-up-to-dollar1-million","url":"https://dwillis.github.io/uplink/article/1992-november-senate-race-ante-up-to-dollar1-million.html","year":1992,"month":"November","monthNum":11,"headline":"Senate race ante up to $1 million","kicker":"","author_name":"Dan Popkey","author_title":"reporter","affiliation":"The Idaho Statesman","summary":"Reporter Dan Popkey analyzed FEC data to reveal that Republican Senate candidate Dirk Kempthorne raised twice as much money from PACs as his opponent, contradicting his campaign message against politics as usual, with funding primarily from timber and mining interests versus his opponent's labor and agriculture support.","keywords":["campaign","fundraising","analysis"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Lotus 1-2-3"],"full_text":"Reporter Dan Popkey used FEC data to see just how seriously Republican and U.S. Senate hopeful Dirk Kempthorne took his own campaign call to \"take our government back from the politics as usual crowd.\" Popkey's analysis showed Kempthorne raised twice as much money from political action committees as his opponent. Using the same data, Popkey showed Kempthorne's financial support came from timber and mining interests while his opponent's came from labor unions and agriculture interests. The Statesman also docu- mented how much of each candidate's support came from out-of-state. The reporting took three to five days, Popkey said. The Statesman got the FEC data from the National Library on Money & Politics and used Lotus spreadsheet to sort contributors by address, occupation and name.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-november-bush-holds-big-lead-in-corporate-donations","slug":"1992-november-bush-holds-big-lead-in-corporate-donations","url":"https://dwillis.github.io/uplink/article/1992-november-bush-holds-big-lead-in-corporate-donations.html","year":1992,"month":"November","monthNum":11,"headline":"Bush holds big lead in corporate donations","kicker":"","author_name":"Lance Williams","author_title":"Computer-assisted reporter","affiliation":"Francisco Examiner","summary":"A computer-assisted analysis of FEC records reveals that Republicans raised nearly three times more soft money than Democrats in California through June, with particularly large donations from energy, agribusiness, and real estate interests supporting policies favorable to their industries.","keywords":["fundraising","donations","analysis"],"topics":["Government & Politics","Business & Economy","Data & Databases"],"technologies":["dBase"],"full_text":"President Bush fell behind Democratic candidate Bill Clinton in California popularity polls in this year's presidential campaign, but the GOP was ahead of the Democrats almost three to one in state soft money donations, according to the Examiner. Computer-assisted reporter Lance Williams looked at FEC records on state corporate contribu- tions to four Democratic and four Republican committees from the beginning of the election cycle through June 1st to show how the GOP had raised $5.5 million in Californian \"soft-money\"-- beefy company donations exempt from election finance reform laws that are meant to lessen the influence of special interest money in presidential campaigns. Williams' analysis showed the GOP has received particularly large donations from energy firms worried about laws restricting oil drilling, agribusiness interests worried about losing federal subsidies and real estate interests that would profit from a Bush plan to free up wetlands for develop- ment. The Examiner also listed the three top donors in each of six sectors, from energy to entertainment. During this one-month, one-person project, Williams looked at more than 7,000 contributors and used dBASE to sort data according to interest group.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-november-paying-attention-to-political-influences","slug":"1992-november-paying-attention-to-political-influences","url":"https://dwillis.github.io/uplink/article/1992-november-paying-attention-to-political-influences.html","year":1992,"month":"November","monthNum":11,"headline":"Paying attention to political influences","kicker":"","author_name":"Thomas Huang and Alex Marshall","author_title":"","affiliation":"The Virginian Pilot and the Ledger-Star","summary":"A Virginia Pilot/Ledger Star investigation used computer analysis of FEC campaign contribution data to demonstrate the correlation between PAC donations and congressional voting patterns. Reporter Tom Boyer recommends integrating campaign finance data into routine political coverage as a way to increase transparency and accountability.","keywords":["money","contributions","influence"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"A Virginian Pilot/Ledger Star survey of congres- sional records and a computer analysis of campaign contributions illustrated a political maxim that is easy to complain about but difficult to document: money buys influence. Computer-assisted reporter Tom Boyer used an FEC database supplied by the National Library on Money & Politics to show an alignment between PAC campaign contributions and the political positions of three congressional incumbents. Boyer also isolated each candidate's top five donors and the amounts, as well as the top 10 donors overall. Boyer recommends dealing with the Library instead of the FEC. \"FEC tapes are expensive and cumbersome to work with, and to make sense of PAC contributions it's a huge help to have the detailed industry codes the National Library folks have added.\" The best way to keep up to speed with contribu- tion data, Boyer says, is have National Library data plus a link to the FEC's on-line system. The best way to use PAC-donation data, Boyer suggests, is in routine congress coverage. \"We've plowed campaign finance data into our day-to-day coverage of the Virginia General Assem- bly,\" he says. \"Legislators hate it, and lobbyists hate it even more Some lawmakers become timid about carrying a PAC's water if they know their vote will end up beside their contributions in the next day's paper.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-november-bush-slipped-in-suburbs","slug":"1992-november-bush-slipped-in-suburbs","url":"https://dwillis.github.io/uplink/article/1992-november-bush-slipped-in-suburbs.html","year":1992,"month":"November","monthNum":11,"headline":"Bush slipped in suburbs","kicker":"","author_name":"Ted Mellnik and Allen Norwood","author_title":"reporters","affiliation":"The Charlotte Observer","summary":"The Observer analyzed North Carolina's 1990 census data alongside precinct-level election results to demonstrate that George Bush's support declined more sharply in urban and suburban areas than rural ones, with upper-income voters shifting to Perot and middle-class voters to Clinton.","keywords":["election","demographics","analysis"],"topics":["Government & Politics","Demographics & Census","Investigative Methods"],"technologies":["Atlas GIS","SPSS","dBase"],"full_text":"A post-election story by the Observer matched precinct results with 1990 census data to show how the swing away from George Bush in North Carolina was more pronounced in cities and suburbs. Bush support in state urban and suburban areas dropped since 1988 by more than 15 percentage points, compared with a 13.6 point drop across rural areas, according to the study. The Observer also used census data to analyze election returns for four groups of precincts, each group dominated by people with similar education, occupations and household income. The study showed Bush lost support among upper-income voters, mostly executives and professionals, and that most of those votes went to Perot. The votes Bush lost among middle-class voters were more likely to go to Clinton. For this precinct analysis story, the Observer started with block-group level figures on income and education, then used Atlas GIS to make precinct-level estimates and SPSSPC to search for data patterns and to create precinct groups. The paper used dBASE to relate demographic class with precinct vote results.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-november-the-politics-of-reform","slug":"1992-november-the-politics-of-reform","url":"https://dwillis.github.io/uplink/article/1992-november-the-politics-of-reform.html","year":1992,"month":"November","monthNum":11,"headline":"The politics of reform","kicker":"","author_name":"Patti Epler","author_title":"The (Tacoma, Wash.) Morning News Tribune","affiliation":"The Tacoma Morning News Tribune","summary":"The News Tribune used computer analysis of campaign finance data to demonstrate that Washington's Initiative-134 would disproportionately harm Democratic candidates by reducing labor union contributions by $1.2 million compared to Republican losses of $800,000.","keywords":["initiative","campaign","financing"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"The News Tribune used computers to analyze the impact of a Washington state campaign financing initiative before the votes were cast. Critics of Initiative-134, aimed at limiting the money going into political campaigns, said it was a political maneuver to boost the influence of busi- ness at the expense of labor interests. The criticism focused on a clause prohibiting the use of some union dues for political purposes. The News Tribune's analysis of 1990 campaign records from the state Public Disclosure Commis- sion bore out the critics' claims by showing that I- 134 would have cost Democratic candidates $1.2 million dollars in labor backing. Overall, I-134 would have cost the Democrats $400,000 more than the Republicans in campaign contributions for that election cycle.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-november-new-bill-promises-access-to-federal-databases","slug":"1992-november-new-bill-promises-access-to-federal-databases","url":"https://dwillis.github.io/uplink/article/1992-november-new-bill-promises-access-to-federal-databases.html","year":1992,"month":"November","monthNum":11,"headline":"New bill promises access to federal databases","kicker":"","author_name":"Suzanne Langlois","author_title":"Reporter","affiliation":"MICAR Missouri School of Journalism","summary":"Pending legislation to make government data more accessible online through the Government Printing Office has been approved by the House Administration Committee, though commercial data vendors have lobbied to narrow its scope by reducing funding and limiting the range of information available.","keywords":["legislation","government","online"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Internet","mainframe","Federal Register","Congressional Record"],"full_text":"P ending legislation could make electronically stored government data more accessible to the public. During the last session of Congress, the House Administration Committee approved the Government Printing Office (GPO) Electronic Information Access Enhancement Act of 1992. The bill would require the GPO to provide on-line public access to the Federal Register, the Congressional Record and a directory of federal information. The bill would also provide for on-line access to some publications under the control of the superintendent of documents and other departments. The act 5983) awaits a floor vote in the House and a vote in the Senate when Congress reconvenes. The legislation is a boon for report- ers. On-line access would make it easier and less expensive to retrieve and use government information. \"The bill would particularly help small news organizations by making available some of the things that can only be accessed through tapes now,\" said Penny Loeb, an investigative reporter for New York Newsday and a member of the board of directors of Investigative Reporters and Editors. \"It would make it possible, for example, to go on-line cheaply and get background on a specific issue, like hazardous waste. It opens up a whole avenue of opportu- nity,\" she said. Loeb testified on the bill at the hearings. The bill is the brainchild of the Taxpayer Assets Project and the American Library Association, and the result of the efforts of numerous organizations including the American Society of Journalists and Authors, Project Censored and the Coalition of Networked Information. James Love, director of the Taxpayer Assets Project, anticipates congressional approval, but not of the legislation he originally sought. \"It's more a matter of how good a bill we'll get, not if we'll get a bill,\" Love said. Rep. Charlie Rose (D-N.C.) and Vice President-elec Albert Gore sponsored the original legislation, commonly called the GPO Gateway/WINDOWS bills. But somewhere between their initial proposals and the revised House- approved version, lobbyists for com- mercial data vendors convinced Republican members of the House Administration Committee to narrow the scope of the legislation. The most significant exclusion in the revised bill is funding, Love said. The original bills suggested appropriating $3 million this year and $10 million next year for development of on-line databases. The revised bill doesn't offer any appropriations. The new bill also limits the range of on-line information. The Gateway/ WINDOWS bills sought an array of line databases, including the federal LEGIS and JURIS systems, the SEC's EDGAR system, MEDLINE and the FDA Bulletin Board. The revised bill only mandates that the Federal Regis- ter, the Congressional Record and the federal information directory go on- line. The original bills provided for on- line access through mainframe net- works like Internet and through federal depository libraries. The revised bill abandoned mandatory network access in favor of a study of Internet access. The new bill still allows for free access for 1,400 federal depository libraries and will charge all other users cost of dissemination only. Fees should be significantly less expensive than those charged by private data vendors. Love said that Rep. Rose and other members of Congress have promised to give the bill priority next session. Despite the limitations of the revised bill, the legislation would provide the public with government information for which taxpayers continually pay but can rarely access. \"We should all push to make it better,\" Loeb said. \"Even though it's small step, it's something.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-november-unraveling-those-mysterious-string-functions","slug":"1992-november-unraveling-those-mysterious-string-functions","url":"https://dwillis.github.io/uplink/article/1992-november-unraveling-those-mysterious-string-functions.html","year":1992,"month":"November","monthNum":11,"headline":"Unraveling those mysterious string functions","kicker":"Tech tips","author_name":"George Landau","author_title":"Reporter","affiliation":"St. Louis Post-Dispatch","summary":"This article explains how string functions in database managers and spreadsheets can standardize and match data across different sources by allowing journalists to split, extract, and manipulate text fields.","keywords":["database","functions","journalism"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["FoxPro","dBase","Paradox","XDB"],"full_text":"S tring functions are the E. Coli of computer- assisted reporting. Digesting a database can be impossible without them, but who likes to talk about it? I, for one, don't like even to admit knowing such stuff. But this being a forum for geekspeak, I swallow my pride and cough up the following. All about string functions: String functions are the features of a database manager or spreadsheet that allow you to alter text in a field through truncation or substitution. Say you want to match two databases in which names and addresses are the only identifiers. One of the databases is a list of private lawyers who occasionally do work for the state; the other is a list of contributors to the campaign of the state's attorney general, who hires the private lawyers without competitive bidding. The database of lawyers has records like this: LAWYER NAME ADDRESS Dew E. Cheatham 2424-A Greenback Way Hope U. Cansue II One Mercantlie Center And the contributors database looks like this: AME ADDRESS Cheatham, Dew E.Jr. 2424 Greenback Cansue, Hope U. H #1Mercantile Ctr. To minimize the amount of manual compari- son you'll have to do between the databases, you'd want to standardize the formats of names and addresses. It would help to split each name field into four new fields: LASTNAME, FIRSTNAME, MIDDLEINIT and JRSR. You could split the address fields into NUMBER, STREETNAME and STREETSUFFIX. I don't have the space to go through each step, but I'll explain how you could use string functions to extract each piece of information for the new fields. (Note: I'll be referring to string functions available in FoxPro 2.0; I believe that similar functions are available in dBASE and Paradox, although XDB users might find their selections of string functions to be more limited.) FoxPro has a handy function called AT( ) which returns the location of a character in a string. For example, in the NAME field with the value \"Cheatham, Dew E. Jr.,\" the function AT(\",\",name) returns the number 9 - the position of the comma from the start of the NAME field (the capital \"C\" is in position 1). If there were two commas in the NAME field and we wanted erything to the left of the second one, the AT( ) action could help us: AT(\",\",name,2) will give the position of the second comma. AT( ) can have two or three items between its parentheses: the first item is the string to search for (it can be more than one character), the second is the field to search, and the third, optional item denotes which occurrence of the string to search for. To extract the last name from the field contain- ing \"Cheatham, Dew E. Jr.,\" we want the first eight characters starting from the left. The function LEFT(name,8) would yield the string \"Cheatham\". To automatically extract all characters to the left of a comma, not matter where the comma appears, you can combine the LEFT() function with the AT() function: LEFT(name,at(\", \"name)-1) will do the trick. Another function, SUBSTR( ), can pull a string of any length from any position in a text field. SUSTR(name,3,6) would return the characters \"eatham\": the function went to the third character in the NAME field and extracted the next six characters. You could write a short program that scans through each record of the file, combining SUBSTR() with the AT() function to pull out text between the first and second spaces that occur in a name field with the format, \"Firstname I. Lastname.\" Then, using the LEN() function, you could copy that text to the MIDDLEINIT field if the text is only one or two characters long, or copy it to the FIRSTNAME field if it is longer. Another string function that can prove mighty useful is STRTRAN(). This one automatically searches a field for any string you specify and replaces it with whatever else you specify -- or deletes it altogether. STRTRAN(name,\"eat\",\"chow\") would replace \"Cheatham, Dew E. Jr.\" With \"Chchowham, Dew E. .Jr.\" You can use this function to remove all punctuation - STRTRAN(name,\".\") would strip the periods from the name field, for example - or to standardize the spellings of small numbers in addresses or street names (STRTRAN(address,\"One or STRTRAN(address,\"Tenth\",\"10th\"). One last tip: A string function called SOUNDEX() can be useful in matching phoneti- cally similar words with minor spelling variations. The function returns a four-character code describing the sound of a word; \"Diane,\" \"Dyann\" and \"Deanne\" all produce the same SOUNDEX code of D500. This function is useful in identifying possible matches that deserve closer inspection - matches that the computer would otherwise miss. If string functions seem a little complicated, don't worry. These things start to make sense only after you've experimented with them for a while. But they are worth learning, because they'll save you lots of time in the long run. If you have any questions, don't hesitate to call me at the Post- Dispatch: (314) 340-8296.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-november-osha-data-analyzes-information-for-you","slug":"1992-november-osha-data-analyzes-information-for-you","url":"https://dwillis.github.io/uplink/article/1992-november-osha-data-analyzes-information-for-you.html","year":1992,"month":"November","monthNum":11,"headline":"OSHA DATA analyzes information for you","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"","summary":"OSHA DATA is a New Jersey-based service that analyzes Occupational Safety and Health Administration records for insurance companies, offering background checks starting at $195 with discounted rates for frequent use, while also providing specialized database searches and interpretation of OSHA violations data.","keywords":["safety","insurance","database"],"topics":["Data & Databases","Public Safety"],"technologies":[],"full_text":"OSHA DATA, in Basking Ridge, N.J., offers analyses of Occupational Safety and Health Administration records. Set up to do claims adjusting for insurance companies, OSHA DATA will do a one-time background check of a single company for $195.00. Because this company performs many requests in any given month for insurance companies, the rates go down with multiple uses. If a company uses. their services 30 times a month, for example, the cost per use is $105. The OSHA violations database, which includes violations for the whole country from July 1972 to the present, is available from the Department of Labor. OSHA's phone number is (202) 219-7008. OSHA DATA can do specialized searches such as which standard OSHA is emphasizing in a specific area, why some companies are never inspected or which safety and health officers are stricter when evaluating violations. OSHA DATA can help its customers decide how to go about searching the database and explain the findings. For information about OSHA DATA, write 28 Bradford Lane, Basking Ridge, N.J. 07920-1548 or call (908) 766-4551.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-november-new-me-furnishes-a-pc-friendly-newsroom-in-chicago","slug":"1992-november-new-me-furnishes-a-pc-friendly-newsroom-in-chicago","url":"https://dwillis.github.io/uplink/article/1992-november-new-me-furnishes-a-pc-friendly-newsroom-in-chicago.html","year":1992,"month":"November","monthNum":11,"headline":"New M.E. furnishes a PC-friendly newsroom in Chicago","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"","summary":"New Chicago Sun-Times managing editor Julia Wallace, who pioneered computer-assisted reporting at USA Today, aims to bring data analysis capabilities into the newsroom through a planned PC-based network conversion. Wallace views advanced reporting technology as now essential for competitive newspapers rather than a luxury.","keywords":["technology","reporting","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["PC"],"full_text":"Resourceful reporters at the Chicago Sun- Times, accustomed to doing data analysis on PC's at home, got a morale booster with the appointment of their new managing editor. Julia Wallace left USA Today in September to take the helm at the Sun-Times. - Wallace helped establish USA Today's computer-assisted reporting program and was managing editor of special projects when she left. \"USA Today had a very structured program,\" explained Wallace. \"We don't have the same resources here, but some people have already figured out how to use computers for the appropriate stories.\" The Sun-Times' upcoming conversion to a PC-based network should help get data analysis out of reporters' living rooms and into the newsroom. When USA Today started its computer- assisted reporting program, it was a luxury that gave the paper an edge over competitors, said Wallace. But she now considers such technol- ogy a necessity. \"Times have changed so much that it's just another thing newspapers have to be able to do,\" said Wallace.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-december-giving-credit-where-credit-is-due","slug":"1992-december-giving-credit-where-credit-is-due","url":"https://dwillis.github.io/uplink/article/1992-december-giving-credit-where-credit-is-due.html","year":1992,"month":"December","monthNum":12,"headline":"Giving credit where credit is due","kicker":"","author_name":"Dan Woods","author_title":"Reporter","affiliation":"The Raleigh News & Observer","summary":"The article discusses the challenges of crediting computer-assisted reporting work in newsrooms and suggests potential solutions for fair attribution.","keywords":["credit","newsroom","reporting"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["spreadsheet"],"full_text":"Now that the first part of the invasion is complete and computer-assisted reporting has a toehold in newsrooms around the country, a new question is facing editors and reporters: How do we apportion credit for computer-assisted stories? The tapeloading, datacleaning, reportwriting and spreadsheet analysis done to create computer-assisted stories straddles the technical and journalistic worlds. Some of the work may not be reporting, but most of it clearly is. As of yet, no clear trend for attribution has emerged. That the problem has arisen at all is a sign of the coming of age of computer-assisted reporting programs at newspapers across the country. In most newsrooms, reporters and editors first worry about getting a few projects into the paper, and any issues of credit are secondary. But at a growing number of newspapers, including The Dayton Daily News, The Hartford Courant, Newsday, The News & Observer in Raleigh, and the St. Louis Post Dispatch, computer-assisted reporting is enough of an everyday activity to call for a policy on how to assign credit and tell readers who did the work on a story. Possible solutions range from no credit lines at all to a special byline for data analysis. In between these two poles are tag lines at the end of stories ('Marie Smith contributed to this story'), separate boxes with attribution for the data analysis and co-bylines. The issue begs the question of why bylines are put on stories in the first place. By and large, bylines exist to show who wrote the story, who reported the story, who is responsible for its accuracy and whom readers should contact with questions and tips. If your newspaper includes bylines only to indicate the author, then no credit would be required for a data analyst, unless he or she also wrote the story. But if bylines are motivated by any of the other reasons, then some sort of credit should appear. An informal survey of newspapers revealed several approaches. At USA Today, no credit is given for routine computer analysis. On big projects or analysis so complex that the method of analysis is an important part of the story, attribution appears as part of a credit box or in the box that explains how the analysis was performed. USA Today uses this approach for its annual thrift rating story, which first appeared in 1988. Shawn McIntosh, special projects editor at the paper, said the credit boxes serve the purpose intended for them: directing readers to the right source. 'When we first did the bank analysis four years ago, the credit boxes explained who made the critical decisions,' McIntosh said. The flurry of calls from bankers who wanted to argue about the formulas used in the story came to the people who did the data analysis, rather than to the reporters, who wrote and reported the story but weren't able to explain the details of how the analysis was designed and performed. 'We're weird here because the data analysts are editors, so typically there is no credit for the editor,' said McIntosh. 'We get all the credit that editors usually get, which is not much, which is fine by me.' She added that the only time any friction over credit for a computer-assisted story had arisen was with respect to awards won for the stories. Because the data analysts weren't mentioned in the bylines, they weren't always recognized when awards were granted, which raised some hackles. This approach has two problems. The first is that data analysis is reporting. The analyst, in effect, interviews the data using report writers, database managers and spreadsheets and comes up with information that in many cases makes up the bulk of the story. Generally, the analyst works with reporters to formulate questions and then manipulates the data to answer them. 'It's the same as going into documents. It's just the same,' said Brant Houston, a reporter at The Hartford Courant who does most of that newspaper's computer analysis. Even if an editor is performing an analysis, the task is still reporting and credit should be awarded. The second problem is that although readers are the most important constituency for any newspaper, other reporters, editors and publishers read and take note of what appears underneath each reporter's byline. The 'Data Analysis' byline seems like a good solution because it describes exactly who did what. 'It's not a matter of vanity; it's not a matter of modesty. The crucial issue is that people are doing work in a new field and if credit isn't given then it will give the impression that they aren't getting journalistic work done,' said Houston. 'Here I am in a pilot program really getting things moving, and you can run into editors who aren't giving you any credit. Then upper management says 'What are they doing? We are giving them money and equipment and taking this risk. Where's the product?' Houston insists on a byline when he does a substantial amount of work on a story, and that policy has been informally adopted in The Courant's newsroom. Houston said that when he started the program of computer-assisted reporting three years ago, some editors were reluctant to grant credit because they didn't realize what was involved in the analysis 'Certainly, there are editors who think that you hit a magic button and the stuff appears with no effort at all,' said Houston. 'Editors are more sensitive to it now. It's a challenge to make people understand how much work you are really doing.' While granting a co-byline to the data analyst conveys that that person has contributed to the story, it doesn't make clear who did what. The St. Louis Post-Dispatch has addressed this issue by granting a 'Data Analysis' byline. George Landau, the investigative reporter who does most of the computer analysis at the Post-Dispatch, said that the data analysis byline was aimed at other reporters, as well as readers. 'The 'Data Analysis' byline allows readers with comments to contact the right person (and keeps me from having to pass along dozens of questionable 'tips' to my colleagues),' said Landau. 'It also announces to staffers that the story relies at least partly on computer analysis. I mean, reading such stories is one of the best ways for reporters to start thinking about what a computer can do for them.' When working on a computer-assisted story where he isn't involved in the reporting, Landau rarely takes a hand in writing the first draft. He does however insist on reading the story at least once to make sure that the information is being interpreted correctly. Here is my recommendation: At The News & Observer, we have decided to adopt a policy of granting a co-byline to those who perform the data analysis for a story. For the first two years of our program, few bylines were granted. Attribution for the data analyst, if it came at all, usually appeared in a tag line at the end of the story. The main problem with this approach is that for most computer-assisted stories the data analysis creates the story. It's not fair or accurate to acknowledge that work in a tag line, which is generally present to indicate a minor contribution to a story. The 'Data Analysis' byline seems like a good solution because it describes exactly who did what. The reporter gets the byline for the traditional work of reporting and the analyst gets credit for the analysis. I am uncomfortable with this solution for the long term, because eventually using computers to help report on stories will be routine and not worthy of special mention. The data analysis should be recognized because it is reporting, not because it involves computers. Furthermore, as newsrooms attract more and more specialists to cover abstruse fields, it will be common to have highly trained economists or literary critics on staff. Should they get 'Economic Analysis' or 'Deconstruction' bylines? I think it's better to call us all reporters, give us bylines, and be done with it.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-december-two-weapons-fight-unfair-database-pricing-old-fashioned-haggling-and-knowing-the-law","slug":"1992-december-two-weapons-fight-unfair-database-pricing-old-fashioned-haggling-and-knowing-the-law","url":"https://dwillis.github.io/uplink/article/1992-december-two-weapons-fight-unfair-database-pricing-old-fashioned-haggling-and-knowing-the-law.html","year":1992,"month":"December","monthNum":12,"headline":"Two weapons fight unfair database pricing: old-fashioned haggling and knowing the law","kicker":"","author_name":"Sandra Davidson Scott","author_title":"Attorney","affiliation":"St. Louis Post-Dispatch","summary":"Missouri's public records law caps fees for computer data access to only duplication and staff costs, resulting in much lower prices than states like Texas that allow overhead charges; agencies often quote inflated prices hoping requesters won't negotiate, but those who push back can achieve significant reductions.","keywords":["records","fees","Missouri"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["mainframe"],"full_text":"Recently I got a call from a man in Texas who wanted computer tapes of Missouri's drivers license records. He had called the Department of Motor Vehicles. The price quoted to him? $9,000. Three years ago, MICAR and the St. Louis Post-Dispatch got copies of DMV tapes. When first asked the cost, the DMV said $7,000. Fortunately for us requesters, Missouri has one of the nation's best statutes for keeping costs low. Missouri Revised Statutes § 610.026(2) limits fees to duplication costs and staff time only: 'Fees for providing access to public records maintained on computer facilities shall include only the cost of copies and staff time required for making copies.' That statute gave MICAR and the Post-Dispatch ammunition to take aim at the high price. Quickly the DMV dropped its price to $1000. After a meeting attended by then-Director of MICAR Elliot Jaspin and myself, the attorney for the Post-Dispatch, and a host of persons from the DMV (including the Director of Revenue, who supervises the DMV, and computer experts), the asking price dropped to $550. Negotiations on costs in Missouri are fairly straightforward because only two questions on costs arise- duplication costs and 2) staff time required for the requester's project. Period. Jaspin, ever a prankster, later popped into my office to tell me, 'The DMV isn't going to give us the tapes for $550.' He waited until he saw steam and then said, 'Nope. $440.' The process had taken less staff time than anticipated. For nearly 4 million records, $440 wasn't bad. So why the $9,000 asking price to the requester from Texas? I asked that question of the current Director of Revenue. 'Maybe he talked to the wrong person?' came the reply. Maybe. But the $7,000 asking price first quoted to MICAR a couple of years ago came to mind. But even $7,000 was cheap compared to the $30,000 price quoted to the Texas requester for some Texas computer records. Texas puts requesters on notice that its costs may be Texas-sized. Under Texas Revised Civil Statutes article 6252-17a, § 8(b), 'The costs of providing the record shall be in an amount that reasonably includes all costs related to providing the record, including costs of materials, labor, and overhead.' 'Overhead' could include helping to pay for the building and utilities! Why would state agencies ask such high prices, though, even in states with user-friendly cost statutes? My observation is this: Agencies sometimes ask a higher price than statutes warrant, and if the requester pays it, that's the end of it. But, if the requester balks, then negotiations may occur, and prices may drop to the maximum amount allowed by statute. The reason that agencies sometimes quote such high prices is that some requesters pay without even batting an eyelash. To, say, a large insurance company or telemarketer, $7,000 or even $30,000 may be no big deal-a simple cost of doing business and not a call to arms. The information business is highly lucrative. The Information Industry Association (IIA), based in Washington, D.C., represents more than 650 companies. Companies such as Knight-Ridder and Dow Jones are buying government data on tapes, loading it into mainframes and granting subscribers on-line access. That government might want to partake more in this multibillion-dollar industry is not surprising. Should government have a straightforward policy of charging commercial users more for records than other users? The federal government and some states do. Of course, this entails screening requests on the basis of 'purpose,' something which some states do not permit. On the federal side, the Office of Management and Budget published its Freedom of Information Reform Act of 1986; Uniform Freedom of Information Act Fee Schedule and Guidelines on Jan. 18, 1987. The report said that Congress clearly intended to distinguish between commercial and noncommercial users and to shift 'some of the burden' of costs onto the commercial users. The Kentucky statute continues: 'It shall be unlawful for a person to obtain a copy of all or any part of a database. for a: a) commercial purpose without stating the commercial purpose; or b) specified commercial purpose, and to use or to knowingly allow the use of the database. for a different commercial purpose; or c) noncommercial purpose, and to use or knowingly allow the use of the database. .for a commercial purpose. A newspaper, periodical, or radio or television station shall not be held to have used or knowingly allowed the use of the database for a commercial purpose as a result of its publication or broadcast unless it has given its express permission for such commercial use.' (Emphasis added.) Clearly, in Kentucky the news media are not considered 'commercial.' (How noncommercial some news outlets truly are, of course, is a regrettable fact.) Arizona § 39-121.03 is more restrictive than Kentucky's law and so broadly worded that it is not clear whether newspapers would be 'commercial users': '[C]ommercial purpose' means the use of a public record. for any purpose in which the purchaser can reasonably anticipate the receipt of monetary gain from the direct or indirect use of such public record.' (Emphasis added.) Arizona also provides for a trebling of costs of records, as well as attorneys' fees, if a requester 'obtains public records for a commercial purpose without indicating the commercial purpose or who obtains a public record for a noncommercial purpose and uses or knowingly allows the use of such public record for a commercial purpose New Mexico § 15-1-9(C) restricts all use of government's computer databases. A requester of a database has to agree, among other things, 'not to make unauthorized copies of the database'; 'not to use the database for any political or commercial purpose unless the purpose and use is approved in writing'; and 'not to allow access to the database by any other person unless the use is approved in writing.' The requester also must 'pay a royalty or other consideration to the state.' New Mexico law adds: '[A]ny person who reveals to any unauthorized person information contained in a computer database or who uses or permits the unauthorized use or access of any computer database is guilty of a misdemeanor and upon conviction the court shall sentence such person for a definite term not to exceed one year or to payment of a fine not to exceed $5,000 or both....'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-december-la-times-tackles-department-of-transportation-data","slug":"1992-december-la-times-tackles-department-of-transportation-data","url":"https://dwillis.github.io/uplink/article/1992-december-la-times-tackles-department-of-transportation-data.html","year":1992,"month":"December","monthNum":12,"headline":"L.A. Times tackles Department of Transportation data","kicker":"","author_name":"Hsiao-Yin Hsueh","author_title":"","affiliation":"MICAR Missouri School of Journalism","summary":"Los Angeles Times journalist Michael Parrish and computer analyst Richard O'Reilly analyzed 10 years of hazardous material transportation data from the Department of Transportation, discovering that hazmat incidents rose 37 percent between 1982-1991, with truck-related injuries soaring 374 percent.","keywords":["hazmat","analysis","transportation"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["mainframe","SAS"],"full_text":"One morning in February 1991, Sacramento, Calif., residents woke to a thunderous blast when a fully loaded gasoline-tanker truck rolled and skidded 100 feet and landed on a parked car at the edge of a tranquil suburban neighborhood. On July 14, 1991, a Southern Pacific freight train filled with toxic chemicals derailed at a bridge near the small town of Dunsmuir, Calif., falling 40 feet into the Sacramento River below. To many, these might seem to be two incidents with no connection. One damaged property for blocks and burned two homes to the ground. The other spilled tons of weed killer into the Sacra- mento River, killing virtually every organism along 40 miles of river. Michael Parrish of the Los Angeles Times sensed something might be wrong with the hazardous material transportation in this country when he saw the two disasters. Sparked by the thought, he began researching the transportation of hazardous materials, or 'hazmat,' by both rail and truck, to put together a picture of the national scene. Parrish's speculation proved to be on target when he obtained computer tapes covering the past ten years from the U.S. Department of Transportation's Hazardous Material Incident Reporting System. A total of 67,657 spills had been reported to the Department of Transportation from January 1982, through December 1991. 'There is a story here,' Parrish thought. Parrish needed someone to transform the data on those computer tapes into comprehensible information. He enlisted the help of Richard O'Reilly, the Times' director of computer analysis. For the past three years, O'Reilly has devoted his time to doing data projects for the reporting staff. O'Reilly took the tapes from Parrish and used an IBM mainframe computer and the data analysis software SAS to conduct the research. Unlike database programs which use structured query language to sort data, SAS needs a tailor- made program to analyze the data. Each time O'Reilly analyzed a new set of data, he needed to write a new program. But SAS helped him do hundreds of cross-tabs and frequency counts on which areas of the country had the biggest problems with hazardous material and whether particular kinds of chemicals had problems with particular kinds of containers. A number of significant findings emerged. He found that from 1982 to 1991, hazmat incidents rose 37 percent. Incidents involving trucks, which carry most of the hazmat, went up 34 percent. Injuries to people as a direct result of the truck spills soared 374 percent. Incidents on the nation's railroads were up 36 percent. And almost all the deaths, 106 out of 108, involved tanker trucks. O'Reilly had talked about the structure of the database with people at the Transportation Department earlier to avoid pitfalls. Still, he encountered problems. There were three sets of data within the database: the incident report, the containers report and the remarks report. In a train accident, each car of the train would be counted as separate spill, but there was actually only one train wreck. In the container database, for example, if there was a glass jar inside a cardboard box and the box was one of four within a larger carton, there would be a separate report on the same spill for each of the containers. Each layer the material leaked through was counted as a different spill. To solve this problem, O'Reilly spent hundreds of hours producing a summary report of all the different findings and sent it to the Department of Transportation for spot-checks to verify his methodology and findings. The people there got the same answers as O'Reilly. With the data in hand, reporters Parrish and Tom Furlong interviewed police officers, government officials, highway accident investigators, critics, hazmat specialists, engineers, rail inspectors and package delivery workers. The result was 'Danger in Transit,' a three-day series in the Times. Appreciation letters and phone calls started coming in soon after the series appeared Sept. 20-22, 1992. Readers told the Times that they didn't realize the danger of the hazmat transportation until reading those articles. 'That (computer-assisted reporting) was terrific research,' said Furlong, whose article in this series was seen on Sept. 21. 'Using that information, we were able to go out and document the accidents and explore what had happened That may be the best journalism you can do.'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1992-december-getting-rid-of-those-pesky-duplicate-records","slug":"1992-december-getting-rid-of-those-pesky-duplicate-records","url":"https://dwillis.github.io/uplink/article/1992-december-getting-rid-of-those-pesky-duplicate-records.html","year":1992,"month":"December","monthNum":12,"headline":"Getting rid of those pesky duplicate records","kicker":"Tech tips","author_name":"Elliot Jaspin","author_title":"","affiliation":"Cox Newspapers","summary":"This article explains how to eliminate duplicate records in a database using programming languages, comparing the process to manually sorting through papers and removing duplicates by matching key fields and deleting matching second records.","keywords":["database","duplicates","programming"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["XDB","Paradox","dBase"],"full_text":"One of life's more vexing problems - along with faulty plumbing and telephone solicitations - is getting rid of duplicate records. You can, of course, use the DISTINCT command, but it has its limitations. If you want to eliminate the second record with a duplicate field in a list, DISTINCT has no way of distinguishing between which record came first and which second.\n\nThe only alternative, short of manually deleting each record, is to use the programming language that is part of any database software. While the following example is done using the procedural language in XDB, the same approach can be modified to work with PAL in Paradox or any of the xbase products.\n\nBefore writing any code to accomplish this task, imagine how you might do this if you were working with a stack of papers instead of electronic records. You would first sort the records to make sure that the first record you wanted of any duplicates would always appear first. Next you would take the first record off the stack, and compare a key with the second record. If the two fields matched you would throw away the second record. But if they didn't match, you would save the second record and use it to match against the record. You would then repeat the process until you had gone through all the records.\n\nWhile the instructions to the computer will appear different from the way it is described above, the method is the same.\n\nLet us assume our list of records looks like the following table, called Doubles:\n\nKey\tCompany\tLast\tFirst\n57\tAjax\tSmith\tJohn\n44\tWidget\tClinton\tBill\n44\tWidget\tRoebuck\tBubba\n32\tWidget\tWheedle\tSkeeter\n88\tRebar\tLybdenum\tMolly\n88\tRebar\tStang\tArnold\n\nThe first step is to select a stack of records and sort them according to our criteria. In XDB this means running a select statement and assigning the result to a label.\n\nQ1: SELECT FROM DOUBLES ORDER BY KEY, COMPANY, LAST\n\nJust as with our hypothetical stack, you then select the first record and load the comparison value into a variable.\n\nFIRST Q1 < This selects the first record, and COMPVAL = Q1.KEY < the value in the key field is loaded into compval.\n\nOur next step is to select the second record from the list before we begin repeating the process.\n\nNEXT Q1\n\nTo loop through the entire list we can use a WHILE statement. In this case the WHILE statement is constantly checking to see if the next record grabbed from the list is the last record in the list. In XDB, determining the last record is done by checking the \"stat\" value. If STATUS is \"1\" then there are still more records. But if the status value changes to \"0\" we know we have reached the end of the list. What we are saying in the next statement is to keep taking records from the list until there are no more records - until the value of status is \"0.\"\n\nWHILE (STATUS[Q1])>0\n\nWe're now ready to make the actual comparison, using an \"if-then-else\" statement.\n\nIf the two values match then delete the second record, grab another record, make it the comparison value and repeat the process. ELSE makes the second record the comparison value.\n\nIF COMPVAL = Q1.KEY If they match, DELETE Q1 < delete the second record, NEXT Q1 < get another record COMPVAL = Q1 KEY < make it the comparison value. ELSE COMPVAL = Q1 KEY < Otherwise make the ENDIF second record the comparison value.\n\nWhether we decide to delete a record, the last thing we need to do is get another record.\n\nNEXT Q1\n\nTo repeat the whole process we end this short routine with an ENDWHILE. That forces our program to loop back to the WHILE statement and repeat the process until, of course, we run out of records.\n\nNote that this method only deletes the first duplicate record. If more than one duplicate exists, another approach must be used.\n\nNow if we could just write a program to prevent salesmen from hawking life insurance by phone.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-december-compuserve-hosts-a-january-forum-on-computer-assisted-reporting","slug":"1992-december-compuserve-hosts-a-january-forum-on-computer-assisted-reporting","url":"https://dwillis.github.io/uplink/article/1992-december-compuserve-hosts-a-january-forum-on-computer-assisted-reporting.html","year":1992,"month":"December","monthNum":12,"headline":"CompuServe hosts a January forum on computer-assisted reporting","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"","summary":"CompuServe's Journalism Forum will host a two-week online lecture in January on computer-assisted reporting, featuring Chris Feola discussing program setup and Larry Krumenaker covering online research techniques.","keywords":["reporting","journalism","online"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["CompuServe"],"full_text":"The CompuServe Journalism Forum will host a two-week-long online lecture about computer-assisted reporting sometime in January. The exact date for the lecture has not been set. Chris Feola, of the Waterbury Republican-American in Connecticut, will lecture about setting up your own computer-assisted journalism program. Larry Krumenaker, a freelancer who's written about online research for Quill and PC Today, will give several days of lectures about online research. The lecturers will post their lessons on the computer bulletin board during the first part of the week and then devote the end of the week to answering questions. For those who don't already dial in to Compuserve, call 800-848-8990 for information.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1992-december-the-database-files-tracks-and-reviews-valuable-but-obscure-databases","slug":"1992-december-the-database-files-tracks-and-reviews-valuable-but-obscure-databases","url":"https://dwillis.github.io/uplink/article/1992-december-the-database-files-tracks-and-reviews-valuable-but-obscure-databases.html","year":1992,"month":"December","monthNum":12,"headline":"The Database Files tracks and reviews valuable but obscure databases","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"","summary":"The Database Files is a new newsletter that reviews online databases and software useful for journalists, covering lesser-known resources beyond major vendors and featuring in-depth analyses of database features, costs, and access instructions.","keywords":["databases","journalism","newsletter"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":["CD-ROM","Dialog","Westlaw"],"full_text":"It's virtually impossible to keep track of the growing number of databases out there, much less sample each one for its usefulness. That's where The Database Files can help. This new newsletter includes CD-ROM and software reviews but its focus is online databases. There are currently more than 5,000 online databases in existence, said John Ullmann, publisher of The Database Files. The newsletter will inform readers about databases which are valuable to journalists but usually unknown because they aren't offered by major vendors, such as Dialog. The Database Files features an in-depth review of a database every issue, including costs, pitfalls and instructions on how to link up. A regular column on bulletin boards profiles \"one or two boards that have content or clientele that would be useful for newsrooms to know about\" The November issue had a meaty article on Westlaw. This isn't the most obscure database, but the authors answered just about every question imaginable. We look forward to more such thoroughly researched reviews with more esoteric, equally valuable databases. The Database Files is published six times each year. Subscription costs $100, or $95 prepaid, and includes one or two special supplements on selected topics in addition to the regular newsletter. Ullmann said he will provide a complimentary copy for review. For information write to The Database Files, 5622 Wood Lane, St. Louis Park, MN 55436, or call (612) 925-6210.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-january-read-this-before-scanning-text","slug":"1993-january-read-this-before-scanning-text","url":"https://dwillis.github.io/uplink/article/1993-january-read-this-before-scanning-text.html","year":1993,"month":"January","monthNum":1,"headline":"Read this before scanning text","kicker":"","author_name":"Barb Pearson","author_title":"","affiliation":"USA Today","summary":"USA Today's attempt to scan 7,500 pages of Congressional spending records proved far more difficult than expected, with scanning technology struggling with small text, formatting issues, and ink marks that created numerous character recognition errors requiring months of manual correction.","keywords":["scanning","Congress","errors"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Paradox","Xerox Imaging Systems K5200","Kurzweil","ASCII"],"full_text":"A look at how Congress spends its money was USA Today's first experience with scanning. Perhaps it was the size of the project (7,500 pages) or the number of records (325,000) or the complexity of the subject - whatever, it turned out to be a nightmare.\n\nUSA Today and Gannett News Service wanted to look at what it costs to run Congress and to identify, among other things, the top spenders. We started by asking Congressional leaders for a computer tape of all 1991 expenses by House members' offices, committees and support departments. They refused, saying all the information was publicly available in four books kept by the Clerk of the House. At least one person acknowledged that providing the information on tape, rather than paper, would make analysis easier.\n\nIn order to proceed with the project, we chose to scan the four books into the computer. The plan was to have the tabular data scanned and converted into fixed-position ASCII files, which we could then read into Paradox.\n\nWe quickly dismissed buying a desktop scanner and doing the work ourselves because of the volume, and the slowness and inferiority of portable models. Instead, we hired a company that estimated scanning on its professional machine would take four weeks. Try four months.\n\nThe company used the Xerox Imaging Systems K5200, a Kurzweil hardware and software system priced at $14,500. Stacks of pages are automatically fed into the scanner, much like a high-volume copying machine.\n\nThis is a very time-consuming process. The book's agate type stymied the splitter. The splitter software is designed to separate touching characters, making it possible to \"read\" them. If it splits improperly, all sorts of strange things can happen. For example, the letter O could be read as a C. The splitter has problems with small and bold text and with characters that are angled or italic. Well, the House books are all of that.\n\nAny stray or unwanted marks on a page, such as ink smudges and page numbers, introduced errors.\n\nAmong our problems:\n\nOn many pages, the Government Printing Office had not printed the type square on the page. The software has a tolerance of about 5 percent skew. anything more than that requires an operator to manually place the page in the machine.\n\nThe result: Lines of garbled or wrong characters: 7's appeared as I's, 8's as 3's. (Some scanning errors were comical - shamed employee, instead of shared; resistant executive, instead of assistant.) Data from two lines often converged into one. Plus, the tabular structure of the original data was often lost, creating errors later when we imported the fixed file into Paradox.\n\nIt took months of extensive work to clean up the ASCII files. It was crucial that every number be double-checked with the original. And, in order to do any collapsing of data and analysis, we needed to correct any errors in staff names, job titles and government contractors so they were consistent.\n\nAlthough it's hard to estimate the overall error rate caused by scanning, on some pages more than a fourth of the characters or numbers were wrong. Before you proceed with a scanning project carefully consider the quality of your originals. Is the type clear, preferably not agate, smudge-free, and printed straight on pages? Testing is crucial. You should have several dozen, representative pages scanned and then doublecheck the electronic data with the original. How many errors were introduced and how difficult will correction be? Is scanning worth it if the cleanup is extensive?\n\nIn researching this article, our contractor told me that scanning technology has improved dramatically, even since we began our project a year ago. We hope so. The quality of the conversion we got was so poor that, had we known at the time, we wouldn't have gone the scanning route. Instead we'd have cut back on the data we wanted from the House books and had it double-keyed.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-january-working-around-a-software-bug","slug":"1993-january-working-around-a-software-bug","url":"https://dwillis.github.io/uplink/article/1993-january-working-around-a-software-bug.html","year":1993,"month":"January","monthNum":1,"headline":"Working around a software bug","kicker":"Tech tips","author_name":"Elliot Jaspin","author_title":"","affiliation":"Cox Newspapers","summary":"The article explains a software bug in XDB versions prior to 2.41 that incorrectly parses delimited files containing commas within quoted fields, and provides a practical workaround using alternative delimiter characters.","keywords":["bug","software","workaround"],"topics":["Technology","Investigative Methods"],"technologies":["XDB","Nine Track Express","Access"],"full_text":"Working around a software bug\n\nSolving some problems with delimited files in XDB\n\nBy ELLIOT JASPIN Cox Newspapers\n\nAnyone who has worked with software for any length of time will soon encounter a bug. It may be large or small, but once you hit the enter key and watch a program sail off into cyberspace, it can be awfully irritating.\n\nMy personal favorite involved software for aircraft navigation. The software worked perfectly until the plane flew across the equator. Then the onboard computer commanded the plane to fly upside down. Hmmmmm.\n\nWhile every programmer will try to produce software with as few bugs as possible, the chances of producing a perfect piece of code seems remote. The trick for the user is to find an acceptable \"workaround\" when a problem is encountered.\n\nA particularly irritating bug in some versions of XDB involves importing delimited files. Each field in a delimited file is separated by a character, usually a comma, and text fields are enclosed within quotation marks. Thus, a record might appear as:\n\n\"Doe, John\", \"123 Maple St.\", \"Bristow, VT\"\n\nAlthough there are four commas in our example, software should be able to see this as a record with only three fields, because two commas are within quotation marks. When the software encounters the first set of quotation marks, it should know that anything that follows is data. If it encounters a comma it should not treat it as a delimiter. Once it reaches the terminating set of quotation marks, the software should consider the comma that follows as the end of one field and the beginning of another.\n\nAlas, versions of XDB prior to release 2.41 see the quotation marks but keep on looking for the comma as a delimiter. The results are usually commastrophic (ouch).\n\nWhile the problem is potentially fatal, the workaround is simple. XDB allows the user to specify what the field delimiter is when importing a delimited file. By the same token NineTrack Express allows the user to pick any character that can be entered at a keyboard as a delimiter. Instead of using a comma, transfer a file in NineTrack Express using a \"^\" or a \"|\", both characters rarely or ever are found in a file. Once you have transferred the file from tape, specify the character in XDB, and the file should import perfectly.\n\nA few closing notes, this particular bug in XDB was corrected in version 2.41. But a similar bug has popped up in the database program from Microsoft: Access. The software gets confused when it encounters a record such as:\n\n\"Bartowski, \"Buddy\"\", \"123 Maple St.\", \"Bristow, VT\"\n\nAs I said at the start, software bugs are everywhere, like","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-january-series-leaves-politicians-with-no-one-blame-for-recession-but-themselves","slug":"1993-january-series-leaves-politicians-with-no-one-blame-for-recession-but-themselves","url":"https://dwillis.github.io/uplink/article/1993-january-series-leaves-politicians-with-no-one-blame-for-recession-but-themselves.html","year":1993,"month":"January","monthNum":1,"headline":"Series leaves politicians with no one blame for recession but themselves","kicker":"","author_name":"Chris Feola","author_title":"","affiliation":"The Republican-American","summary":"A major investigative journalism project used spreadsheet analysis of government data to reveal that Connecticut families paid over 50% of their income in taxes, exposing wasteful federal spending and exposing that the Social Security Trust Fund relies on IOUs rather than actual reserves.","keywords":["taxes","spending","investigation"],"topics":["Data & Databases","Government & Politics","Business & Economy"],"technologies":["Quattro Pro"],"full_text":"We acquired a filing cabinet full of data available only on paper, and 25 megabytes directly from government computers via modem. Since this was primarily a numbers project, we moved everything into a spreadsheet Borland's Quattro Pro for Windows. We used Quattro to build 3-D spreadsheets with each topic - the breakout on federal spending, for example - in a separate sheet that cranked its results into the master sheet. This kept the master sheet simple enough to be read by person other than Werner Von Braun. \"The Tax on Living\" ran every day for more than two weeks, ending the day before the election. It showed that the average Connecticut family was paying a little over 50 percent of their income in taxes and how those taxes made it financially attractive for businesses such as UPS to pay to move their operations to other states. It showed that 42 percent of the cost of a bottle of liquor and more than 50 cents on every gallon of gasoline went for taxes. It showed taxes where people expected - on gas - and where no one would have ever guessed - one on gas pump nozzles and another for pump gauges. It also showed the money was being spent less than wisely. Not only did Congress increase its budget 17 percent last year, the money provided for things such as the House and Senate photo studios and radio and television studios. And then there's overhead. We obtained government figures that showed more than $300 billion went for overhead - rents, telephone and utility bills and airfare, among other things. (That's the total for the entire federal budget, not a separate appropriation.) That doesn't include a dime for programs or salaries. Perhaps the biggest fuss occurred after we obtained an internal Congressional document that revealed the ugly truth about the Social Security Trust Fund: There isn't one. All Social Security taxes go straight into the treasury, where they are spent just like the rest of the taxes. The money is replaced with \"IOUs,\" the document went on to say. Sen. Joseph Lieberman, D-Connecticut, called the system a sham, a fraud, and said it amounted to nothing more than flim-flam. You always wonder if a project like this has any effect. We know it was read. We were deluged with calls and letters, and many readers took the time to fill out and mail us a do-it-yourself total tax form we printed. The key to this series was putting families and faces on the numbers. It is one thing to explain that X amount of dollars are going out in taxes. It is another to show photos of the family paying thousands in taxes and have them explain they can't afford allergy shots for their child.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-january-projects-at-the-san-francisco-examiner-and-the-lawrence-eagle-tribune-uncover-what-else-political-waste-and-greed","slug":"1993-january-projects-at-the-san-francisco-examiner-and-the-lawrence-eagle-tribune-uncover-what-else-political-waste-and-greed","url":"https://dwillis.github.io/uplink/article/1993-january-projects-at-the-san-francisco-examiner-and-the-lawrence-eagle-tribune-uncover-what-else-political-waste-and-greed.html","year":1993,"month":"January","monthNum":1,"headline":"Projects at the San Francisco Examiner and the Lawrence Eagle-Tribune uncover (what else?) political waste and greed","kicker":"","author_name":"Dick Rogers, Brad Goldstein and Ed Achorn","author_title":"Reporter","affiliation":"San Francisco Examiner and Lawrence Eagle-Tribune","summary":"Journalists describe their computer-assisted reporting projects, including a major investigation into San Francisco's AIDS programs using database analysis and a congressional campaign finance study tracking political contributors.","keywords":["reporting","database","investigation"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["FoxPro","PC","spreadsheet"],"full_text":"Some of our far-flung friends recently wrote us with details about their computer-assisted reporting projects. From Dick Rogers: It began with a telephone call in February to Lisa M. Krieger, the Examiner's medical writer. The caller, a health professional in town, described what he believed was waste and incompetence in the use of city, state and federal funds. The task of evaluating the 100-plus AIDS programs carried out in San Francisco was beyond our resources, but it turned out that the city was about to conduct individual evaluations of every publicly funded grant program. The process took months. It wasn't until October that we were able to go through each file folder. There were 124 - each one containing scores on at least 11 performance categories, as well as five or more other entries and written comments. None of it was computerized. I had been doing some computer-assisted reporting over the last two years, most of it on my home PC. In September, the paper acquired a Compaq 486 with a 200-megabyte hard drive, FoxPro database software and a spreadsheet. So the AIDS story was my first official opportunity to demonstrate the number-crunching virtues of the computer. We entered the key information into the database, adding a few fields for certain calculations we had in mind. It took about five hours. Instead of sitting down with a calculator and two big notebooks worth of numbers to look for patterns, averages and totals, we let the computer do the work. The Compaq is a screamer - the typical sorting process was completed by the time I looked up from the keyboard to the screen. If we sorted incorrectly or failed to include an important category, it didn't matter. We just did it again. Compared to the drudgery of doing all the work by hand, the five hours spent keying the numbers was time well-spent. The result was a two-day series built around our analysis of the city evaluations. The punch line: San Francisco has created a far-flung system of AIDS programs that often fall short of their goals and duplicate efforts, wasting money that could be used to improve services. The stories created quite a stir among AIDS groups and provided us with a follow-up story - a vow by AIDS officials to straighten out the system. If it weren't for the computer, we'd still be fiddling with the calculator and the two big notebooks of numbers. From Brad Goldstein and Ed Achorn: For two years, reporters at The Eagle-Tribune have been using computers to explore the link between contributors and political candidates. Last fall, the computer put us on the trail of perhaps the biggest story of the campaign in a local congressional race. A local man, a former congressman, was running for the 5th district again, touting his record as a successful businessman. When we downloaded his federal election records and sorted on business, we quickly discovered some patterns: members of New York investment firms and their wives were plowing tens of thousands of dollars into his Massachusetts campaign. One of the contributors was Robert E. Brennan, who was accused of bilking small investors of millions of dollars during the 1980s. Further research disclosed that the individual investors and their firms had links to Brennan. Our check with state securities officials revealed that many of them were under investigation in several states on securities fraud charges. A check of reports filed by the candidate's business ties with the Securities and Exchange Commission uncovered other interesting side-lights. Brennan's First Jersey Securities had sold penny stock in the candidate's company during the '80s; a second firm whose members contributed heavily to his campaign was selling his company's stock now. The stock sales helped the company provide a $700,000 retirement package for the candidate, although the company had had three years of losses totaling $2.8 million. We were able to give readers a broader perspective on the candidate's career as a self-made businessman. The Eagle-Tribune broke the story on the front page of its Sunday paper. The Boston Herald followed the next day with information by the candidates' opponent.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-january-bits-bytes-and-nibbles","slug":"1993-january-bits-bytes-and-nibbles","url":"https://dwillis.github.io/uplink/article/1993-january-bits-bytes-and-nibbles.html","year":1993,"month":"January","monthNum":1,"headline":"Bits, bytes and nibbles","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article lists several upcoming seminars and workshops on computer-assisted reporting for journalists, including programs from the Education Writers Association, University of North Carolina, and Indiana University, featuring prominent journalists and instructors.","keywords":["journalism","training","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["VU/TEXT","Nexis","DataTimes"],"full_text":"Bits, bytes and nibbles\n\nMICAR's March seminar is already full. But MICAR isn't the only wheel in town when it comes to training journalists in computer-assisted reporting. Here's a list of some upcoming seminars. If we've missed any, please send the information to Uplink.\n\nThe Education Writers Association will present \"Computer Power in Education Reporting\" Feb. 26 in Arlington, Va.\n\nThe one-day workshop co-sponsored by The Freedom Forum is advertised as a \"day packed with story ideas, tips on getting started and ways to solve problems in your newsroom.\"\n\nGuest speakers will include Pulitzer Prize winner Elliot Jaspin of Cox Newspapers, Aleta Watson from the San Jose Mercury News, Joseph Garcia from the Dallas Morning News and Pat Ordovensky from USA Today.\n\nThe conference is free, but space is limited to 40 participants. For information call Lisa Walker at (202) 429-9680.\n\nThe University of North Carolina at Chapel Hill will present a \"Workshop in Precision Journalism: Computer-Assisted Reporting for the Next Century,\" July 12-16 in Chapel Hill, North Carolina. Topics include the use of statistics for journalists, an overview of statistical software packages, the graphic representation of data and an introduction to the data holdings of major social science archives. The class will also present online search strategies for full-text databases such as VU/TEXT, NEXIS and DataTimes. Phil Meyer, author of Precision Journalism, will be one of the instructors. Other teachers include sociologists and librarians from the university. The seminar costs $750 but early registration, before May 15, is $650. For more information, call Dr. Beverly Wiggins at (919) 966-2350 or write to the Workshop in Precision Journalism, Institute for Research in Social Science, University of North Carolina, Chapel Hill, NC 27599-3355.\n\nIndiana University's National Institute for Advanced Reporting presents its fourth annual conference, \"Computers Equal Power Journalism,\" March 12-14 in Indianapolis. Session leaders will show participants how to build computer-assisted projects from the ground floor up. Some of the session topics include: Methodology of Working with Statistics; Dealing with Dirty Data: Ethical Problems; and Management of Computer Resources: In-house Training. Speakers include Pulitzer Prize winning investigative reporters Don Barlett and Jim Steele; Dwight Morris, editor of special projects at the Los Angeles Times Washington, D.C., bureau; Don Fry, writing coach of The Poynter Institute; and Pat Stith, reporter at the Raleigh News & Observer, Raleigh, N.C. The seminar costs $135. For information call Deb Perkins at (317) 274-2776 or write to the Conference on Computer-Assisted Journalism, I.U. School of Journalism at Indianapolis, 902 W. New York Street, ES 4104, Indianapolis, IN 46202-5154.\n\nThe Transactional Records Access Clearinghouse will hold the TRAC Workshop on Nuclear Regulatory Commission Data May 20-23 at Syracuse University in New York. The workshop will teach individuals how to use NRC data as the basis for investigations. Attendance to the workshop will be limited to 15 persons. The cost is $200 for registration and $180 for lodging and meals. For information contact Randi Maroney, TRAC, 478 Newhouse II, Syracuse University, Syracuse, NY 13244, (315) 443-3563.\n\nAmerican University will offer \"Journalists, Computers and the Federal Government: A Mid-Career Seminar for Reporters and Editors\" July 18-23 in Washington, D.C. The seminar promises hands-on experience working with databases maintained by various government agencies as well as commercial online services. Ethics, access and statistical concepts will also be covered at the American University seminar. While prior computer experience is not required, the seminar will be limited to 10 to 12 journalists who have at least two years Washington reporting or editing experience. The cost is $650 for early registration before March 1 and $750 after March 1. For more information call Wendell Cochran at (202) 885-2002 or write to the American University School of Communication, 4400 Massachusetts Ave. N.W., Washington, DC 20016.\n\nInvestigative Reporters and Editors and The News & Observer will hold \"Computing: The News Frontier\" October 21-24 in Raleigh, North Carolina. The seminar promises to put \"bytes into your bark\" through hands-on training in computer-assisted reporting. Topics include: finding and negotiating for government databases; exploring local, state and federal on-line databases; and using digitized maps. Participants will return with a budget and plan for starting a computer-assisted reporting program at their own news organization. For information write to Dan Woods, The News & Observer. P.O. Box 191. Raleigh. NC 27602.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-february-deadly-cure","slug":"1993-february-deadly-cure","url":"https://dwillis.github.io/uplink/article/1993-february-deadly-cure.html","year":1993,"month":"February","monthNum":2,"headline":"deadly cure","kicker":"","author_name":"Dave Davis and Ted Wendling","author_title":"","affiliation":"The Plain Dealer","summary":"A Plain Dealer investigation using Freedom of Information Act requests and the NRC's NUDOCS database uncovered over 40 deaths from medical radiation overdoses that the NRC claimed not to know about, including 9-year-old Dwight Golstein who died from double cobalt-60 doses, prompting a criminal investigation into hospital coverups and systemic failures in radiation safety oversight.","keywords":["radiation","investigation","oversight"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["NUDOCS"],"full_text":"As a daily assignment, the spill of radioactive phosphorus-32 inside a research laboratory at the Cleveland Clinic wasn't much - a 12-inch story inside. But clinic officials refused to identify a nearby music school that had been contaminated in the incident. They issued a challenge: file a Freedom of Information Act request and try to get the name from the Nuclear Regulatory Commission.\n\nWe did. And more than 1000 FOIA requests later, we wrote \"Lethal Doses Radiation That Kills,\" a five-day series that was published by The Plain Dealer in December.\n\nThe series could not have been done without access to the NRC's online database NUDOCS. We spent hundreds of hours searching and analyzing the 2 million records on NUDOCS, looking for leads on radiation accidents and victims.\n\nThe resulting stories documented the failure of the NRC to protect the public from radiation overexposures and accidents in our nation's hospitals. When we asked NRC officials how many people had died from medical overdoses of radiation, they knew of none. We found more than 40, including two that we identified on the agency's NUDOCS system.\n\nAmong the dead was Dwight Golstein, a 9-year-old who died of radiation-induced respiratory failure in 1988 after receiving double-doses of cobalt-60 radiation at Alta Bates Medical Center in Berkeley, Calif. Golstein was undergoing treatments for a tumor in his sinus cavity.\n\nThe radiation was a slow killer. After a few months, Dwight's dark skin turned jet black and began to peel, spinal fluid drained from his right ear and his swollen tongue forced its way out of his mouth.\n\nIt was as if Dwight had been in an atomic bomb blast. His mother, Barbara, watched him die.\n\n\"Dwight's face was disfigured, and his tongue was so fat he looked like a little monster. He really looked grotesque and he knew it, but I wanted him to know that we loved him.\"\n\n\"I made sure to hug and kiss him,\" Golstein told us.\n\nDwight's final days are etched forever into his mother's memory. \"I told Dwight before he died that it was OK to leave us,\" Golstein told us. \"He was fighting hard. I told him that he was going to go to heaven and live with Jesus and that it was better for him. Three days later, he died.\"\n\nWhen we asked the California Bureau of Radiological Health if anyone there knew of Dwight's death, officials said no. They then went to the hospital, where they were told that no overdose has occurred. After reading the series, California officials visited the hospital again. This time Alta Bates officials admitted that they had not been truthful with the agency. A criminal investigation is now underway into what state officials openly call a coverup by the hospital.\n\nAnd Alta Bates is not unique. We found many hospitals where radiation safety has been under-funded, neglected and, in some places, openly scorned. We found doctors and nuclear medicine technicians who had overdosed patients, sometimes deliberately, and then falsified records and lied to regulators to cover up. And we probed the NRC's shameful investigation of the nation's most serious medical radiation catastrophe, the overexposure of 413 cancer patients who died, and the NRC didn't bother to find out what happened to them.\n\nMany of the patients suffered from terminal cancers. For them, the radiation overdoses of up to 41 percent left wounds that would not heal and hastened their demise. But many victims, such as 25-year-old Edna Gail Valentine, an elementary school teacher, were expected to survive their cancers.\n\nEdna was 1½ months pregnant with her first child when she began radiation treatments that were meant to kill any lingering cells left after surgery for Hodgkins Disease. Four months later, both she and the unborn baby were dead.\n\nHer sister, Barbara Hoover, and scores of other families of victims say they were told nothing by hospital and NRC officials.\n\n\"They burned her lungs,\" said Hoover, after reading an NRC report that had not been made available to the family. \"They lied to us. They knew what happened all along, and they didn't tell us.\"\n\nHoover said she would never forget the last time she spoke to her sister. She had received a call at the school where they both taught. By the time she reached the hospital, they were wheeling Edna into emergency surgery.\n\n\"She told me she was afraid,\" Hoover said. \"I said, 'Yeah, I would be too. But everything will be fine. We'll see you when you get out.' Edna never awoke from the surgery.\n\nAfter reading about Riverside, NRC Chairman Ivan Selin said he didn't realize what the victims and their families went through. \"We haven't helped them and we really should,\" Selin said. Later, in commenting on their series, he said: \"If two reporters can work for a few months and come up with what you got, that's a pretty good indication that we're not doing our jobs.\" Selin promised major reforms in the agency's medical programs.\n\nFor a reprint of the series, call Dave Davis or Ted Wendling at (216) 344-4808.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-february-acquiring-information-from-the-nuclear-regulatory-commission","slug":"1993-february-acquiring-information-from-the-nuclear-regulatory-commission","url":"https://dwillis.github.io/uplink/article/1993-february-acquiring-information-from-the-nuclear-regulatory-commission.html","year":1993,"month":"February","monthNum":2,"headline":"Acquiring information from the Nuclear Regulatory Commission","kicker":"","author_name":"Dave Davis","author_title":"","affiliation":"The Cleveland Plain Dealer","summary":"The Nuclear Regulatory Commission's NUDOCS database is an invaluable free resource containing over 2 million records since 1978 that journalists can access to research radiation incidents, nuclear facilities, and regulatory actions. Reporters can request a password to search the database at no cost and retrieve full documents through the NRC's Public Documents Room for five cents per page.","keywords":["database","nuclear","journalists"],"topics":["Data & Databases","Investigative Methods","Health & Environment"],"technologies":["NUDOCS"],"full_text":"Our series on radiation mishaps in hospitals found numerous flaws with the Nuclear Regulatory Commission, but an unwillingness to provide information was not among them.\n\nThe best proof of this I can offer is the agency's online database NUDOCS, an invaluable source of information for reporters who cover the environment, or who occasionally write about hospitals, universities, nuclear power plants, radioactive waste or other related issues.\n\nNUDOCS, agency lingo for Nuclear Document System, holds more than 2 million records gathered by the NRC since 1978.\n\nWithin about a week of issuing inspection reports, technical studies, fines and special investigations, NRC officials have the information summarized and entered on the NUDOCS system. NUDOCS also is an excellent source for correspondence, reports and other records generated by other federal and state agencies, as well as outsiders such as environmental groups, concerned citizens and reporters.\n\nFor example, a search for records that mention \"Ohio\" brings up 2,010 hits. Plug \"death\" or \"killed\" into the NUDOCS search menu and 426 records appear. NUDOCS has 10,141 records generated by the U.S. Environmental Protection Agency, 46 by the Federal Bureau of Investigation and 19 by Vice President Al Gore.\n\nThere are 21 records, mostly Freedom of Information Act requests, for which I am listed as the author.\n\nNUDOCS also holds congressional correspondence and transcripts of hearings and important meetings.\n\nBecause NUDOCS offers only summaries of documents - ranging from a few sentences to dozens of pages in length - it must be used in conjunction with the NRC's Public Documents Room, which holds the actual records. With the document accession number retrieved from NUDOCS, you can call the Public Document Room's reference desk at (202) 634-3273 and get the records by fax or by mail. The cost is five cents a page.\n\nThere is no cost for searching NUDOCS. The NRC doesn't charge for time on the system and the call that connects you is to a toll-free 800 number.\n\nAnd while NUDOCS was created for use by NRC officials and the libraries that serve as NRC repositories, you can get access. To request a password, simply write to: Gerald F. Cranford Director, Office of Information and Resource Management, U.S. Nuclear Regulatory Commission, MNBB6219, Washington, D.C. 20555.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-february-tracing-more-complete-nrc-data","slug":"1993-february-tracing-more-complete-nrc-data","url":"https://dwillis.github.io/uplink/article/1993-february-tracing-more-complete-nrc-data.html","year":1993,"month":"February","monthNum":2,"headline":"TRAC'ing more complete NRC data","kicker":"","author_name":"David Burnham and Susan Long","author_title":"","affiliation":"TRAC","summary":"The Transactional Records Access Clearinghouse (TRAC) obtained and transformed the Nuclear Regulatory Commission's massive computerized database into an organized knowledge base to reveal the agency's actual enforcement activities, including inspection and violation records for commercial nuclear power plants from 1975 to 1989. The process required extensive data cleaning and verification to address inconsistencies and missing information in the NRC's original files.","keywords":["database","nuclear","transparency"],"topics":["Data & Databases","Investigative Methods","Health & Environment"],"technologies":["9-track tape","floppy disk","PC","Macintosh"],"full_text":"In recent years, federal agencies such as the Environmental Protection Agency and the Census Bureau have been making computerized databases public. While these \"hand out\" databases can provide reporters important, useful information, most of the data files available through federal agencies focus on external situations, and not on the internal workings of agencies. The central mission of the Transactional Records Access Clearinghouse is to obtain computerized records of federal enforcement agencies and transform them into databases that reveal what these agencies actually are doing. The TRAC Nuclear Regulatory Commission database, for example, covers the regulation of nuclear power reactors from 1975 to August of 1989. (TRAC currently is updating this database.) The database is available in various formats: a six-volume report series, diskettes suitable for use on either PCs or MacIntosh computers and nine-track computer tape. TRAC's first step in gathering the NRC data was to obtain a list of the dozens of record systems then used by the NRC. We decided initially to focus on what the NRC called its \"766\" file. TRAC requested this file under the Freedom of Information Act. The massive file, with millions of different entries, was reproduced on a tape that we provided for the agency. Obtaining the raw 766 file was only the beginning of the long process of developing what we call a \"knowledge base,\" an organized collection of information that allows the user to examine the workings of an agency. Although the 766 file contains details of NRC inspections undertaken at both commercial power plants and other licensed facilities such as universities and hospitals, TRAC decided to focus its attention on the 130 commercial plants which came under NRC regulation from its formation in 1975 through August of 1989. There were, however, a lot of problems. The NRC only provided the sketchiest information on the format and file contents of the tape, and most of the specific codes were not defined. In cases where codes were defined, they didn't always match the information in the file. The file contained detailed information such as where and when each inspection was conducted, how long it required to complete, who conducted it and whether the inspection resulted in the operator being cited for any violations. From 1975-89 for power plants only, the data indicated that the NRC conducted close to 50,000 inspections detected about 27,000 violations. The NRC's records are surprisingly inconsistent, sometimes showing that a single reactor became operational on several different dates. An accurate list of the name of each commercial nuclear power plant, the utility that owned it and the exact date it was granted its pre-construction license, its construction license, when it became operational and when it was permanently closed down had to be compiled by TRAC to make the 766 file useful. The inconsistencies in the data required double checking the information with the NRC and, in some cases, with the utilities themselves. The purpose of this intense effort was to provide more than just NRC information about individual reactors. The TRAC data also presents the information in terms of the periods that each reactor goes through during its total life. With TRAC data, it is possible to determine whether an individual reactor had more or fewer violations during the construction period than it did during its operational period. The basic accuracy of the NRC data, however, was not the only problem. NRC had changed the way it categorized the seriousness of violations several times - starting with a three-category system, moving to a six-category system in the early eighties, and then back to a five-category system. Spotting these changes, not documented anywhere in the raw 766 file, was critical. We also sought to locate information about what the NRC did after the inspector detected a violation, the place where the 766 file left off. When were penalties assessed, how big were they, against whom were they applied? No one we talked to at the NRC knew of any computerized information on penalties, so TRAC reluctantly began building its own computerized files drawn from internal paper records. Very late in this process, TRAC discovered that the NRC did in fact have such a file - the Enforcement Action Tracking System (EATS) which was not widely known within the NRC. We obtained EATS under the FOIA. But EATS used a different identification system than the 766 file. The file that TRAC developed from the paper records supplied cross-linking \"IDs\" so that violations and penalties could be matched. Comparison of these two independent sources of information on penalties allowed further checks of NRC data accuracy. Knowing the precise number of violations at a particular plant can be interesting. But developing information so that a user can compare the performance of one utility against that of another or determine whether NRC inspectors around the country are enforcing regulations in a uniform way transforms a normal \"database\" into a \"knowledge base,\" a tool that can provide genuine insights into both the workings of an important federal agency and the industry it regulates.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-february-taking-bytes-out-of-the-education-apple","slug":"1993-february-taking-bytes-out-of-the-education-apple","url":"https://dwillis.github.io/uplink/article/1993-february-taking-bytes-out-of-the-education-apple.html","year":1993,"month":"February","monthNum":2,"headline":"Taking bytes out of the education apple","kicker":"","author_name":"DIANNA BORS MICAR","author_title":"","affiliation":"MICAR","summary":"Investigative reporters used computer database matching to uncover serious problems in school systems, including hundreds of convicted felons working as teachers and bus drivers, and the discovery that criminal background checks were inadequate or ignored by school districts.","keywords":["felons","databases","investigation"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["9-track tape","Paradox","XDB","Excel","Macintosh","floppy disk"],"full_text":"Child molesters for teachers. Felons driving the school bus. Racism manifesting itself with more minority students in remedial classes and fewer in gifted programs.\n\nEducation stories like these can be done without years of training or rooms full of computer equipment.\n\nJoel J. Smith, an investigative reporter at the Detroit News, used nine-track tape to compare Michigan's Department of Corrections database of 550,000 convicted criminals with a Michigan Department of Education database of 97,000 school teachers and 15,000 school bus drivers.\n\nHe found roughly 200 convicted felons in the classrooms with another 200 driving school buses. Charges ranged from murder to child molestation. Weighing the right to privacy against the responsibility to protect students, Smith pointed to one case in particular: One teacher convicted of molesting a 15-year-old girl served his time, went and taught in another state, returned and was rehired - once again dealing with young girls. \"Shouldn't the school district know about this? They didn't,\" Smith said.\n\nThe Department of Education claimed it checked for criminal records. Later the Department admitted it only checked when applicants for a teaching certificate admitted they'd been convicted of a felony. But when Smith looked at applications, he found that \"in every single case, they lied.\"\n\nAfter the initial computer match, many names were culled by gender or by first and last name checks. Smith used a stand-alone IBM for the initial matches which turned up the 400 names. With the information copied onto a diskette, he put it on an IBM PC and used Paradox software.\n\nIn one case, a custodian was fired when he stopped showing up for work. Garcia discovered the missing janitor was in jail for murder.\n\nDallas Morning News reporter Joseph Garcia found 200 school employees with felony convictions that no one knew about. Using tapes, Garcia compared the Dallas Independent School District's payroll records with Dallas County Prosecutor data, which contains demographics, the criminal charge and its disposition.\n\nAlong with dozens of drug offenses, the search turned up three sexual offenders. Garcia discovered in two cases that judges had ruled that employers - the school district - didn't have to be told.\n\nAgain, the applications had asked about convictions, but \"by and large, the people just out-and-out lied,\" Garcia said.\n\nNot every successful computer-assisted education story starts with a tape.\n\nUsing XDB on a home computer and information provided on diskette, Reagan Walker of the (Jackson, Miss.) Clarion-Ledger found that test scores more closely correlated with the basic poverty level of a school than expenditures per pupil.\n\nWalker looked for correlations using information furnished by the State Department of Education on diskette for a mere $250. The data included test scores by district, school expenditures by pupil, the worth of a mill per district and the number of courses offered by each high school.\n\nShe found a strong correlation between the poverty level and low test scores, high dropout rates and low ACT scores. But Walker wanted to know why.\n\nAfter analyzing the data, she concluded that schools that had less money offered fewer high school courses and less resources to their students. This in turn meant students scored lower on ACT tests and had fewer opportunities for higher education.\n\nAleta Watson, with nine years of experience on the education beat, uses Microsoft EXCEL spreadsheets on a Macintosh to bring data together and make comparisons.\n\nHer paper, the San Jose Mercury News, covers an area that includes nearly 50 school districts. But there is no central repository of education data for the area. Spreadsheets help her draw together information from the various districts and make comparisons.\n\nWatson also looked at administrative costs of each district as a percentage of its budget. \"It was pretty shocking. Administration wasn't that big chunk [of the budget],\" Watson said. Despite solid numeric evidence, \"People still called and said there was too much administration,\" she said.\n\nIn 1987 she looked at asbestos in the schools. Loading information from each district into the computer, she discovered how many schools had removed the substance, where asbestos remained, whether it is in classrooms or other areas and what proportion of the schools have asbestos.\n\nA spreadsheet can also reveal hidden segregation. In 1991 Watson used a spreadsheet to look at a school five years after a desegregation court order and found that Hispanics were still more likely to be in remedial classes and less likely to be in advanced placement classes. In building this database, Watson also discovered that half of all Hispanic students have had a D or an F on their report card.\n\nSmith also checked his computer matches using old-fashioned reporting. \"I never ran a name that I didn't manually look up and verify if it was the same person. I talked to virtually every person I used,\" Smith said. In some cases, he simply sent them a letter, and they contacted him.\n\nThe final results? Michigan enacted a safeguard law and criminal checks are now run on all schoolbus drivers and teachers.\n\nJonathan Eig of the Dallas Morning News used spreadsheets to prove racial lines still exist in school. But in this case, Eig used information that was gathered by the federal government but analyzed and released.\n\n\"We scooped the agency,\" Eig said. Using the Justice Department's Civil Rights Division surveys, he found racial differences among the districts in his area in remedial and gifted classes, corporal punishment and graduation rates.\n\nBut while the Department requires all schools to gather the information, it collects information from only a few. To obtain the data for this award-winning 1991 series, Eig had to write Freedom of Information letters to the Department and various schools in his district.\n\nThen, using a spreadsheet, he proved a great gulf still divides students.\n\nFor example, he found that at schools with predominantly black students, teachers had less experience and few had master's degrees. He also found that the numbers of minorities in remedial classes were up while the numbers of minorities in gifted and talented classes were down - and that minorities got corporal punishment more often. This held true for blacks and Hispanics but not Asians. Looking at the information by district, he discovered that the oft-criticized Dallas Independent School District was more even-handed than suburban schools in many ways. In suburban schools, minorities were more likely to end up in special education classes.\n\nThe articles sparked a parents movement and started people talking about the issue, Eig said.\n\nReporters don't have to be computer experts or full-time investigative reporters to turn out hard-hitting computer-assisted education stories. Even beginning reporters can deal with data that would stump a reporter without a computer.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-february-secrets-of-the-superprogrammers","slug":"1993-february-secrets-of-the-superprogrammers","url":"https://dwillis.github.io/uplink/article/1993-february-secrets-of-the-superprogrammers.html","year":1993,"month":"February","monthNum":2,"headline":"Secrets of the superprogrammers","kicker":"Techtips","author_name":"Dan Woods","author_title":"Reporter","affiliation":"The News & Observer","summary":"The article explores the legacy of 'superprogrammers' from the mainframe era who used ingenious but complex data compression techniques, and how modern database systems have largely obsoleted their methods while making data more accessible to everyday users.","keywords":["programming","database","efficiency"],"topics":["Technology","Data & Databases"],"technologies":["mainframe","SQL","FoxPro","Paradox","XDB"],"full_text":"Secrets of the superprogrammers\n\nBy DAN WOODS The News & Observer Raleigh, N.C.\n\nDecades ago, in the datazoic era of the 1960s and 1970s, huge mainframes ruled the earth. Behemoths they were, cared for by a priesthood of superprogrammers, who loved to compress every field of every record into the smallest number of bits. After the personal computer enlightenment in the early 1980s, memory became cheap and processors exploded with power. In this era, ease of use became more important than efficient storage.\n\nThe relational database revolution sprang from the premise that the method for asking questions should be independent of the way the data is stored. Structured Query Language was born to help people ask questions of data in an organized way. Instead of the compressed and Byzantine ways superprogrammers stored data, relational databases are stored in flat files, most of it in character form. With a relational database, asking a new question means writing a new SQL statement. In a zoic database, asking a new question meant asking the superprogrammer to write a new program. Superprogrammers loved this. For the most part, the superprogrammers have fossilized, and no longer are able to make life difficult for the rest of us. They still exist, though, and frequently surface in state government bureaucracies, which have resisted for years competitive pressures that drove businesses to modernize. In this environment, data tends to be crammed together in wonderfully creative ways. Superprogrammers must be given their due. Their methods stored the most data in the least space. One of the favorite techniques used by superprogrammers is using bit flags to store Boolean or logical variables. This is elegant because logical variables are those that can be either true or false, bits can be 0 or 1 - a perfect match. It is efficient because one byte can store 8 bit flags. In the era of P.C. enlightenment, one logical variable takes up one byte, which is no problem because storage abounds. Superprogrammers also would store numbers such as dates in portions of integers. I recently encountered a two-byte date field that had the following format: Bits 0 through 7 were the year, Bits 7 through 10 were the month, Bits 11 through 15 were the day. The bits are numbered from 0 to 15 starting at the right.\n\nThis is a super-compact way to store dates. Each date is stored in a two-byte integer, rather than in a six-byte string of characters, like most dates (i.e., YYMMDD). Despite their efficiency, these fields are an unholy nuisance! Unless you write Assembly Language code, the lingua franca of the superprogrammer, (or its modern equivalent, C) it seems that bit flags or two-byte dates will be impossible to use with a language like FoxPro, Paradox or XDB. Fear not, however. Such fields are accessible to those unskilled in the ways of the superprogrammers. For even though superprogrammers can squish data together in ingenious ways, they cannot stop numbers from being numbers, and therein lies the solution.\n\nA byte that contains 8 bit flags, for example, is a number from 0 to 255. If we look at the first 16 values in binary, hexadecimal and decimal notation, we can see a way out of our dilemma.\n\nBinary / Hex. / Dec.\n0000 0000 / 00 / 0\n0000 0001 / 01 / 1\n0000 0010 / 02 / 2\n0000 0011 / 03 / 3\n0000 0100 / 04 / 4\n0000 0101 / 05 / 5\n0000 0110 / 06 / 6\n0000 0111 / 07 / 7\n0000 1000 / 08 / 8\n0000 1001 / 09 / 9\n0000 1010 / OA / 10\n0000 1011 / OB / 11\n0000 1100 / OC / 12\n0000 1101 / OD / 13\n0000 1110 / OE / 14\n0000 1111 / OF / 15\n\n(Remember the bits are numbered from 0 to 7, starting at the left. Also, in hexadecimal A = 10, B = 11 etc. up to F = 15.)\n\nHow do we determine if an individual bit of this character is 0 or 1? First, treat the byte as a number. In FoxPro that means reading the byte in as a one-character field, then using the ASC function to translate it to a number. The number field will have to be 3 bytes long in FoxPro, because 8 bits can store a number from 0 to 255. If we look at the byte as a number we can see some interesting things. Bit 7, the rightmost bit, is on - meaning 1 - when the number is odd, and off - meaning 0 - when the number is even. Bit 6, the second from the right, is on when the remainder after dividing the number by 4 is greater than 1. Bit 5, the third from the right, is on when the remainder after dividing by 8 is greater than 3. Bit 4 is on when the remainder after dividing by 16 is greater than 7. Another way of talking about the remainder of a number is to use the term modulus. Number modulus y is the remainder of x after dividing the number by y. If x is less than y then x is the same as its modulus with respect to y. In FoxPro this function is written MOD(x,y). MOD(7,4)=3 because 7 has a remainder of 3 after being divided by 4. MOD(8,16)=8 because 8 has a remainder of 8 when divided by 16. MOD(4,2)=0 because 2 divides evenly into 4.\n\nWe can then write the following FoxPro IF THEN statements to decode the bit flags in a single byte field called BITFLAG:\n\nIF MOD(BITFLAG,2)>1 THEN Bit 7 is on\nIF MOD(BITFLAG,4)>1 THEN Bit 6 is on\nIF MOD(BITFLAG,8)>3 THEN Bit 5 is on\nIF MOD(BITFLAG,16)>7 THEN Bit 4 is on\nIF MOD(BITFLAG,32)>15 THEN Bit 3 is on\nIF MOD(BITFLAG,64)>31 THEN Bit 2 is on\nIF MOD(BITFLAG,128)>63 THEN Bit 1 is on\nIF MOD(BITFLAG,256)>127 THEN Bit 0 is on\n\nThis code will leave any number of bit flags a smoking ruin. We use the same technique to decode the compact dates, which are stored as follows in a two-byte, unsigned binary integer we'll call TWOBDATE. When stored as a binary number, TWOBDATE has the following structure:\n\nyyyyyyymmmmddddd\n\nwhere y are bits indicating the year, m indicate the month, and d the day.\n\nTWOBDATE is converted from a two-byte integer to a FoxPro decimal number using Nine Track Express or some other tape conversion utility. The FoxPro number would have to be five bytes wide because two bytes, or 16 bits, can store a number from 0 to 65535.\n\nThe year field is stored in bits 0 through 7. That leaves 9 bits of the 16-bit number remaining. Nine bits can store a number from 0 to 511. The year bits, in essence, are storing the number of 512's in the 16-bit number. (That's equivalent to the three leftmost digits of a 9-digit decimal number storing the number of millions.) To find out the number of 512's in the number we take the number, subtract its remainder with respect to 512, and then divide by 512. Thinking of this in millions rather than 512's may make it clearer. The number of millions in a number is the number minus the first 6 digits, divided by a million. This formula extracts the year from the first 7 bits:\n\nyear=(TWOBDATE-MOD(TWOBDATE,512))/512\n\nWe handle the next two fields in a similar fashion. For the month, bits number 8 to 11, we note that there are 5 bits to the right, which can store from 0 to 31. That means the 8th to 11th bits are counting the number of 32's in the number. We eliminate bits 0 to 7 by taking the number modulus 512. The formula to extract the month is therefore:\n\nmonth=(MOD(TWOBDATE,512)-MOD(TWOBDATE,32))/32\n\nThe day is easy. We simply take the number modulus 32.\n\nday = MOD(TWOBDATE,32)\n\nPutting this all together with other FoxPro functions to convert the entire two-byte date into a FoxPro date, we come up with the following command:\n\nREPLACE ALL FOXDATE WITH CTOD(STR((MOD(TWOBDATE,512)-MOD(TWOBDATE,32))/32)+\"/\"+STR(MOD(TWOBDATE,32))+\"/\"+STR((TWOBDATE-MOD(TWOBDATE,512))/512,2,0))\n\nTake that, superprogrammers!","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-february-where-to-look-for-electronic-information","slug":"1993-february-where-to-look-for-electronic-information","url":"https://dwillis.github.io/uplink/article/1993-february-where-to-look-for-electronic-information.html","year":1993,"month":"February","monthNum":2,"headline":"Where to look for electronic information","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"MICAR","summary":"Journalists struggle to locate government databases, but a University of Minnesota doctoral candidate is creating a comprehensive directory of Minnesota's 2,000+ state government databases to serve as a reference tool for reporters. The article suggests other states could benefit from similar initiatives.","keywords":["databases","journalists","government"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Journalists frequently call MICAR to ask: \"How do I know what databases are out there?\" While the government spends a lot of money maintaining its databases, it makes little effort to tell the public what's available. Two places to start looking are the Federal Database Finder and the State Database Finder by Mathew Lesko. These sources scratch the surface. However, once you understand the philosophy behind CAR - that every government agency, from the federal to the local level, which maintains computer records has, in essence, an obtainable electronic database - you get an appreciation for how difficult it is to come up with a comprehensive listing. But it looks like reporters in Minnesota might get lucky. Andris Straumanis, a doctoral candidate at the University of Minnesota's school of journalism, has been cataloguing some of the more than 2,000 databases located in state government agencies throughout Minnesota. The Minnesota Database Directory, tentatively set for publication in the spring of 1993, is designed specifically to be a reference tool for journalists. The idea for the directory arose out of two conferences on the use of electronic public records hosted by the Minnesota Journalism Center. Maybe journalism professionals and students in the other 49 states could get together and do the same.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-february-greensboro-news-and-record-wins-a-court-battle","slug":"1993-february-greensboro-news-and-record-wins-a-court-battle","url":"https://dwillis.github.io/uplink/article/1993-february-greensboro-news-and-record-wins-a-court-battle.html","year":1993,"month":"February","monthNum":2,"headline":"Greensboro News & Record wins a court battle","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"investigative/database reporter","affiliation":"The News & Record","summary":"The News & Record in Greensboro, North Carolina won a court order to access computerized property tax records from Guilford County after challenging the county's claimed costs and arguments that the records were not public. The case highlights ambiguities in North Carolina's public records law regarding electronic records and what constitutes reasonable fees.","keywords":["access","records","court"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["9-track tape"],"full_text":"The News & Record in Greensboro, N.C., recently won a victory in the ongoing battle for access to computerized information. This month a judge finally ordered Guilford, N.C., county officials to turn over to reporters property tax listings stored on computer tape. The News & Record first sought access to the records last March but balked when county officials said the 164,000 records would cost $8,200. \"They were asking a nickel a record, which was the same cost they charge for individual paper records,\" said Lex Alexander, investigative/database reporter for The News & Record. When The News & Record challenged the cost, county officials changed their tune and argued the records weren't public - at any price. On Feb. 5, a judge rejected the county's argument that making the records public would violate state law prohibiting public officials from using information about individual income. The judge did not, however, settle the issue of cost.\n\n\"The agreement is a little weird,\" said Alexander. \"They will give us a copy of the tape now, and if the court rules the cost is some price we don't want to pay, we return the file.\" North Carolina Public Records Law includes electronic records but does not clearly define what constitutes \"reasonable cost,\" said Alexander.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-february-more-on-car-seminars","slug":"1993-february-more-on-car-seminars","url":"https://dwillis.github.io/uplink/article/1993-february-more-on-car-seminars.html","year":1993,"month":"February","monthNum":2,"headline":"More on CAR seminars","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"","summary":"The Freedom Forum and Arizona State University's Cronkite School are hosting a four-day computer-assisted journalism seminar featuring renowned investigative journalists and hands-on training with databases and digital tools. The program is primarily open to newspaper staffs in the Southwest region.","keywords":["journalism","seminar","database"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["CD-ROM","9-track tape"],"full_text":"The Freedom Forum and Arizona State University's Walter Cronkite School of Journalism are sponsoring the fourth annual seminar for reporters and editors in computer-assisted journalism March 4-6. Morning sessions will feature nationally known journalists, including Donald Barlett, two-time Pulitzer Prize-winning investigative reporter and coauthor of the Philadelphia Inquirer series \"America: What Went Wrong;\" Larry Sanders, database editor of USA Today, John Ullmann, former projects editor of the Minneapolis Star-Tribune. Afternoons will be devoted to hands-on work with online databases, CD-ROM and 9-track tapes. The seminar is open to newspaper staffs in Arizona, New Mexico, Colorado, Utah and Nevada, but other applicants will be considered if space is available. For more information contact Ed Sylvester at (602) 965-4210.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-february-micar-boot-camp-seminar","slug":"1993-february-micar-boot-camp-seminar","url":"https://dwillis.github.io/uplink/article/1993-february-micar-boot-camp-seminar.html","year":1993,"month":"February","monthNum":2,"headline":"MICAR boot camp seminar","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"MICAR","summary":"MICAR is offering a week-long intensive boot camp seminar from May 23-28 teaching journalists how to download and analyze data from magnetic tape using hands-on training in database management and SQL. The course covers magnetic tape storage, relational databases, data validation, and electronic records law, with class size limited to 12 students.","keywords":["seminar","journalism","database"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["9-track tape","SQL","DOS"],"full_text":"MICAR will hold a week-long seminar in computer-assisted reporting May 23-28. This is the famous MICAR \"boot camp\" in which students learn the process of downloading and analyzing data stored on nine-track magnetic tape through hands-on training. Seminar topics include: the facts behind magnetic tape storage, relational database theory, Structured Query Language, testing for data validity and the law regarding electronic records. Students return to their newsrooms ready to apply what they've learned and begin producing stories. Extensive computer knowledge isn't required. The first day of class includes computer basics and getting around in DOS. But because of the grueling pace, only the highly motivated are encouraged to attend. Class size is limited to 12 to ensure individualized attention. Cost varies according to circulation size or market share. For more information, contact MICAR, 120 Neff Hall, Columbia, Mo., 65211, (314) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-march-getting-data-into-the-right-hands-the-readers","slug":"1993-march-getting-data-into-the-right-hands-the-readers","url":"https://dwillis.github.io/uplink/article/1993-march-getting-data-into-the-right-hands-the-readers.html","year":1993,"month":"March","monthNum":3,"headline":"Getting data into the right hands: the readers'","kicker":"","author_name":"Chris Feola","author_title":"Computer-Assisted Reporting","affiliation":"Waterbury Republican-American","summary":"The article discusses how the Waterbury Republican-American tackled the challenge of making data accessible to all reporters by equipping them with notebooks and creating user-friendly data access programs.","keywords":["data accessibility","notebooks","reporting"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["XDB","Paradox","WordPerfect","PC"],"full_text":"If your database ran queries in a forest, but no one was there to access it, would your readers hear about it? The Zen cliche about the tree falling in the forest is an examination of existence. My hackneyed refrain is a question of practicality.\n\nIn the end, a newspaper is no better than the editions it publishes. A paper can have racks of the latest equipment, access to every piece of data known to man, the deepest thinkers thinking the deepest thoughts - and all of that matters not if the paper that rolls off the presses each night is a piece of junk.\n\nHere's the same question, then, in a more serious form: How do we get our data out of our databases and into the hands of our reporters? We all know how to do it on the big projects. We all know how to run XDB on 42 bazillion records. But what about the stories that aren't brainsurgery? What if a reporter doesn't need to do an asymmetrical outer join on a series of campaign contribution tables? What if they just want to know if Mr. Big gave the mayor money - Yes or No? And what if they need to know on deadline? What if they need to know at the council meeting, so they can ask the mayor tonight why he's voting on Mr. Big's project?\n\nWe faced these questions at the Waterbury Republican-American about a year ago when we realized we were running incomplete stories because no one outside the computer-assisted reporting team had a clue when it came to the resources we had available. We decided to attack the problem on two fronts: We started equipping all our reporters with i386 notebooks to give them their own computer access; and we began building information appliances to allow them to get the data without having to learn Paradox Application Language or anything similar.\n\nThe notebook setup is fairly straightforward. Each reporter is given a notebook - we are currently using AST i386SX 20 megahertz machines with 100 megabyte hard drives, six megabytes of RAM and 2400 baud FAX modems.\n\nAll that comes in a six-pound package that runs on batteries and fits in a briefcase. We also hand out monochrome VGA monitors and full-size keyboards that plug into the notebooks. In effect, each reporter has a notebook for the field and a desktop for the office.\n\nApplication appliances are programs designed to give users point-and-shoot access to data - the idea is that they are no more difficult to use than a microwave.\n\nHere's an example. We have a Paradox database listing all those who contributed to our mayor's election war chest. Rather than having all our reporters learn Paradox - we teach everyone who is interested, but we don't force anyone - we gave them access to the data through a menu choice in their word processor.\n\nHere's how we did it. First, we stripped our Paradox tables down to raw ASCII, then imported them into WordPerfect for Windows 5.1. Then we built a WordPerfect macro and hung it on a word processor menu.\n\nThe macro pulls up the file and posts a dialog box that asks \"What is the name of the person?\" The reporter types the name in and clicks \"OK.\" The macro then runs a search based on the name. If there is a hit, the information is pasted into the story the reporter is working on. If the search comes up empty, the macro tells the reporter \"That person is not on the list\" and sends them back to their story.\n\nIn either case, the reporter has an answer as fast as they can ask the question.\n\nWe have used this technique on more than just this list. Reporters can also search municipal employee lists for their towns with just a click of a mouse button. And we're doing the same thing with census data for their towns.\n\nThis is probably not the most elegant piece of programming you've ever seen. But here's the bottom line: Our reporters can get their hands on the data they need in seconds. And so they use it.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-march-orange-county-register-settles-the-score-on-soft-money-contributions","slug":"1993-march-orange-county-register-settles-the-score-on-soft-money-contributions","url":"https://dwillis.github.io/uplink/article/1993-march-orange-county-register-settles-the-score-on-soft-money-contributions.html","year":1993,"month":"March","monthNum":3,"headline":"Orange County Register settles the score on soft-money contributions","kicker":"","author_name":"Kristin Baird","author_title":"","affiliation":"MICAR","summary":"Orange County Register reporter Ronald Campbell analyzed campaign contribution data to identify California's largest soft-money donors during the 1991-92 elections, discovering $13.4 million in contributions and revealing industry patterns in political giving.","keywords":["donations","campaigns","analysis"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["FoxPro","CD-ROM"],"full_text":"The stats read like the salary list of a major league baseball team: ARCO - $804,842; Michael Kojima - $613,850; Sony Corp. $332,650; Walt Disney Co. - $196,207.\n\nBut this game was politics, played by corporations and private individuals from California who donated large sums of soft money during the 1991-92 elections. The final score: $13.4 million in soft money contributions from California alone.\n\nRonald Campbell of the Orange County Register analyzed campaign-contribution computer tapes from the National Library on Money & Politics to tally up the scorecard on California's soft-money contributors.\n\nSoft money donations go directly to the national political party instead of a particular candidate. This provides an exception to the federal law that limits campaign contributions to $1,000 a person and prohibits businesses and labor unions from contributing.\n\nUsing FoxPro on a 386, Campbell quickly generated a long list of wealthy Californians who took advantage of the soft money loophole. But who were these people and corporations? That proved to be the hardest and most interesting part of Campbell's research.\n\nMany contributors were household names, such as Sony, Chevron, Sunkist and Peter Norton.\n\nBut what about Ben D. Kelts, or C.S. Ishiyama & Co.? To unmask mystery donors, Campbell turned to a new \"toy\" in the Register library - a national phone directory on compact disc. Campbell struck gold.\n\n\"I went to the machine, typed in 'Kelts, Ben,' and there was one of them,\" he said. \"Of all the dumb luck, to find a guy who gives $100,000 listed in the phone book.\"\n\nCampbell also got help from his fellow reporters in identifying other contributors. After identifying most of the donors, Campbell then broke the contributions down by industry. The help he needed was right there on computer tape. The National Library on Money & Politics had designated fields on the tape with interest codes.\n\n\"That is very useful,\" Campbell said. \"If you want to know how much money came from the entertainment industry, you have the computer go through, take the interest code and sum the amount.\"\n\nParty loyalties became evident through the industry breakdown. Hollywood and labor unions went Democrat; energy moguls and agriculture went Republican.\n\nBut something was missing - the developers. Campbell noticed only a small number of them appeared among the big campaign contributors. An explanation was needed; Campbell decided to compare contributions to those made in 1988, a peak year in the Orange County economy.\n\nCampbell looked once again to the National Library of Money & Politics. Because the Federal Election Commission did not require parties to report soft money until 1991, the library warned Campbell that the 1988 tapes would not be complete. Campbell was willing to take the risk.\n\n\"It is constructive to compare the partial picture of the late '80s with the complete picture of the '90s, because the contrast was quite glaring,\" he said.\n\nGlaring, indeed. Real-estate developers who had dominated the county's soft-money heavy-hitters in 1988 were replaced in 1991-92 by a restaurant group and a health-care conglomerate.\n\nThe incomplete data from 1988 did cause a few problems. Several corporations listed as contributors in 1991-92 were known to be members of the Team 100, a designation for those that donated $100,000 or more to the Bush/Quayle campaign in 1988. When the 1988 computer tape failed to show several Team 100 members and the Republican party declined to release information on donations above $100,000, Campbell could only guess that the corporations may have donated larger amounts in 1988 as well.\n\nCampbell is still following up on the story. Since the article's publication on Feb. 1, Campbell has discovered the identities of two previously unknown donors and is working on several others. But one question still weighs heavily on his mind.\n\n\"What the blazes are they getting for the money?\" Campbell said. \"The side with the most money wins. But what they do because of that money, I don't know.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-march-the-data-entry-ordeal","slug":"1993-march-the-data-entry-ordeal","url":"https://dwillis.github.io/uplink/article/1993-march-the-data-entry-ordeal.html","year":1993,"month":"March","monthNum":3,"headline":"The data entry ordeal","kicker":"","author_name":"Deirdre Shesgreen","author_title":"","affiliation":"MICAR","summary":"Investigative journalists face challenges with data entry for their stories, as hiring professional services often results in inaccurate work, leaving reporters to spend months entering data themselves or turning to cheaper but less reliable offshore options.","keywords":["data","journalism","accuracy"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"The data entry ordeal\n\nWhat to do when you don't have unfailingly accurate slave labor\n\nBy Deirdre Shesgreen\nMICAR\n\nIs the task of putting a few hundred thousand records on computer stopping you from cracking that great campaign finance story?\n\nData entry is not most investigative journalists' idea of a good time - but often it comes with the territory.\n\nHiring a data entry service is a solution, but they come with problems of their own.\n\nThe Kansas City Star learned the hard way while investigating the abuse of Missouri's \"Second Injury\" worker's compensation fund.\n\n\"We tried a so-called professional typist to do this, and it was disaster,\" said Rich Hood, the Star's political correspondent. \"Among other things, she left out the contribution amounts. She went like a bat out of hell, but it was unusable.\"\n\nthe Star spent hundreds of hours entering the data itself. \"We checked and rechecked the accuracy of the thing so we could stand behind the story,\" Hood said.\n\nIf the government wants to create a database, they can go to the nearest prison and enlist inmates to do the tedious task.\n\n\"Two ways that government inputs info into computers is through civil servants and prison inmates,\" said John Roure, who runs Optic Solution, a company that builds databases for law-enforcement agencies, unions, PACs and other associations.\n\nInmates at the Texas Department of Criminal Justice have done data entry for workers compensation commission reports, records from the Department of Transportation and other agencies, said Phillip Anderson, industrial supervisor of the Huntsville prison.\n\nBut the government mainly uses prisoner labor for non-sensitive data entry, Roure said.\n\nUnfortunately, newspapers don't have access to the same human or financial resources. So enthusiastic and dedicated reporters end up spending months in front of a computer screen, coming away with a great story but glazed eyes.\n\nOne source of cheap labor newspapers do have access to is off-shore data entry companies. Though using a plant in Mexico that pays its employees $4 a day may raise some ethical eyebrows, it is cheaper than the U.S. firms.\n\nMartin Horan, who runs Horan Data Services Inc., in Cincinnati, just lost a bid to an off-shore data entry company.\n\n\"We really can't compete with their prices,\" Horan said.\n\nA local firm wanting to keep its business in the United States asked Horan to give them his best price. Unfortunately, Horan said, \"the price they got from the off-shore place was half the best price I could offer.\"\n\nBut the offshore data services' accuracy is a problem. Horan said that the local company is frustrated with the quality of the off-shore service.\n\n\"They don't have the luxury of handling questions about the data on a timely basis, so they usually don't handle them at all.\"\n\nWhen the St. Petersburg Times investigated abuse of a Florida law that allowed people charged with a crime to erase all records of their arrest, they had to go through hundreds of thousands of pages of information, said Times reporter David Barstow.\n\nWhere information is easily understandable, they could have sent the data to a data entry service, said Barstow. But with some types of documents, the chance of error, or of missing a crucial clue, was too great.\n\nAnd as you sort through the piles of paper, other things can come up. \"You can't send it to someone who doesn't know what you're thinking,\" he said.\n\n\"We did consider trying to get journalism students to do it because it is pure drudgery - mindless work - but the only way to be absolutely confident about the data is to enter it yourself,\" Barstow said.\n\nBarstow said the St. Petersburg Times did consider giving campaign contribution data to an off-shore data entry service.\n\n\"We explored the possibility of farming out to Jamaica, but it was still enormously expensive and it was not a possibility in terms of hardcore reporting. Too many things can go wrong.\"\n\nIn the United States, data entry is an expensive endeavor. Getting 100,000 typed records with 16 fields put into a database could cost $27,000 - $54,000 if you want the information double-checked, said Gayle Myers of International Data Corp. in St. Louis. And it could take on average four weeks, depending on other factors.\n\nInternational Data estimates costs on a per-record basis. The rate is higher for smaller jobs because they still charge a set-up fee. For 1,500 typed records with 16 fields, the bill would be between $250 and $425.\n\n\"They (data entry services) don't handle small jobs. They're used to doing things for IBM. If they do a few thousand records they charge exorbitant rates,\" Barstow said.\n\nBut for the Cleveland Plain Dealer, the $10,000 tag on the database was worth keeping their reporters reporting. The Plain Dealer paid 18 cents per record.\n\n\"There's a trade off with concerns about accuracy and judgment calls. But if you do data entry you're not doing reporting,\" said Dave Davis, investigative reporter for the Plain Dealer. Davis said that if it doesn't take reporters away from reporting for very long, they would prefer to do data entry in house.\n\n\"But it's not worth it if it's something that's going to take lot of time,\" Davis said. Time pressures added incentive to hire outside help.\n\nSo the Plain Dealer went to a company they knew and had worked with before.\n\n\"We created the record layout and hired people to go through the records and input the data. We were looking over their shoulder the whole time.\"\n\n\"I told them not to guess and gave them my home number. They called a lot.\"\n\nAnother possible solution is illustrated by a cooperative effort among state legislative reporters and a professor at Pennsylvania State University.\n\nAn organization of full-time Pennsylvania state government reporters funded a project to computerize all the campaign contributions to the candidates for governor and state legislature in 1989 and 1990.\n\n\"The reason I got interested in the project was because I spent about a year tracking contributions from lobbyists - just doctors, lawyers and insurers. But there were 253 legislators,\" said David Morris, then-president of the Pennsylvania Legislative Correspondence Association and a state reporter for The Associated Press.\n\n\"I built my own database with index cards. Then I decided it was time to move into the 20th century,\" said Morris, who is now head of the AP bureau in Sacramento.\n\nHe teamed up with James Eisenstein, a Penn State professor of political science who has studied campaign finance since 1976. With help of work-study students, Eisenstein computerized almost 35,000 contributions in all, plus another 2,000 records of summary information.\n\n\"We computerized all contributions except individual contributions between $50 to $250,\" said Eisenstein.\n\nThe project cost about $20,000. \"What we are trying to do should be done by the state,\" Eisenstein said.\n\nEisenstein has tried to keep the project going by forming a coalition of all people who would be interested in the information.\n\nThe Pennsylvania Newspaper Publishers' Association was the largest financier, giving about $5,500.\n\nIn return, three versions of the information were made available to all members of PNPA.\n\nThe first - the blue book - was an index of all contributions by name of candidate, type of contribution and alphabetical listing of contributor in each category.\n\nThe second - the red book - gave all PAC contributions sorted by the name of the PAC and then the name of the recipient.\n\nThe third - a database application - allowed users to make specific inquiries by candidate name, by contributor name and SIC code. The searches could be specified by chamber, party, election outcome, and date.\n\n\"So, for instance, you could find out all contributions that went to Republican senate losers in the primary between March 2 and April 2 in 1990. It is a very powerful program,\" said Eisenstein.\n\nBy including the SIC codes, \"it does one thing that no other data base does,\" according to Eisenstein. It automatically classifies PACs by industry.\n\nThe date functions lets the reporter see if there is a relationship between the timing of a contribution and the legislative agenda.\n\nBut the project is petering out. The coalition only came up with $8,250 to computerize 1992 campaign contributions, not enough to get past the primary election reports.\n\nBut Eisenstein said he hasn't given up on the idea yet.\n\n\"About 20 organizations got at least one version of the reports,\" Eisenstein said. \"We had a good, but not massive, response.\"\n\nUsing the database \"became pretty routine,\" said Morris. Now that he's moved to Sacramento, Morris said he misses the luxury of using the database.\n\n\"It's a pain in the butt because you have to go through all this paper by hand. You have to go to the secretary of state, find the paper and take notes or make copies.\"\n\nMorris is trying to get some news organizations interested in starting up a California database, but \"it's a long, cumbersome process,\" he said.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-march-adventures-in-sampling-using-brute-force-to-compensate-for-your-ignorance-about-statistics","slug":"1993-march-adventures-in-sampling-using-brute-force-to-compensate-for-your-ignorance-about-statistics","url":"https://dwillis.github.io/uplink/article/1993-march-adventures-in-sampling-using-brute-force-to-compensate-for-your-ignorance-about-statistics.html","year":1993,"month":"March","monthNum":3,"headline":"Adventures in sampling: Using brute force to compensate for your ignorance about statistics","kicker":"Techtips","author_name":"George Landau","author_title":"Reporter","affiliation":"St. Louis Post-Dispatch","summary":"A journalist describes using computer sampling and random simulations to determine whether suspicious settlement disparities among 15 corrupt lawyers in a Missouri workers compensation fund were statistically significant or due to chance.","keywords":["sampling","statistics","investigation"],"topics":["Data & Databases","Investigative Methods"],"technologies":["FoxPro","Paradox"],"full_text":"Using your computer's ability to generate random numbers and to repeat tasks thousands of times in a few minutes, you can use common sense - rather than a statistical formula - to figure out how likely it is that the results of your investigation are a chance occurrence.\n\nThe technique is called sampling. It requires only that you write a program that tells your computer to dip repeatedly into the data while keeping track of the results.\n\nI decided to use this technique when I was working with the data from a corrupt workers compensation fund in Missouri. Using a database of about 12,000 claims, we had identified 15 lawyers who were working both sides of the fund - alternately filing claims on behalf of injured workers and then, with other cases, working for the state defending the fund against excessive claims.\n\nOur analysis showed that this group of 15 lawyers somehow managed to win settlements that were about three-and-a-half times higher than everybody else's. One question we were asking ourselves (and that we knew our critics might raise) was this:\n\nWhat are the odds that this disparity between the 15 lawyers and the rest (who numbered about 1,000) is due to chance? In other words, if you randomly picked groups of 15 lawyers from the database, how often would their combined average settlement be 3.5 times higher than everybody else's?\n\nAs you can probably gather at this point, the answer lay in the question itself. We created a table that listed each attorney whose name appeared in the database, along with the total number of cases he filed against the fund and the grand total of his settlements.\n\nThen we wrote a program (in FoxPro, although you could do the same in Paradox) that instructed the computer to do the following:\n\n1. Do three things 15 times:\n   1. Pick a number between 1 and the total number of records.\n   2. Goto that record (i.e., if the random number is 553, goto the 553rd record).\n   3. Mark the record, either by tagging it for deletion or setting a value in a marker field.\n2. Count the total value of settlements and the total number of claims for these 15 lawyers (summing the marked records only).\n3. Calculate their average settlement (total dollars/total cases) and place this number in a field called \"avg_15\" in another database table called \"results.\"\n4. Count the total value of settlements and the total number of claims for everyone but those 15 lawyers (summing the unmarked records only).\n5. Again, calculate their average settlement (total dollars/total cases) and put this number in the \"results\" table, into a field called \"avg_rest\" in the same record you just wrote to.\n6. Unmark all records in the table of lawyers.\n7. Do it all again. And again. And again.\n\nWe used a FOR/NEXT loop that counted from 1 to 10,000, doing the above routine 10,000 times. It took maybe 20 minutes to run. If the database had been much larger or the computer slower, we simply would have run it overnight. We probably didn't need to repeat the sampling loop so many times, but hey - we wanted to be sure.\n\nWith the inhumane chore of data sampling done, the answer to the question - how often would this happen by chance - was quickly in hand. We just counted the number of records in our \"results\" table where the 15 lawyers' average settlement was at least 3.5 times larger than everybody else's. There were about 60 such records among those 10,000. So the likelihood that our story was the result of chance was 6 out of 1,000, or 0.6 percent.\n\nKnowing that pollsters and scientists are happy when the uncertainty falls below 5 percent, we were downright giddy.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-march-two-computer-assisted-stories-win-ire-awards","slug":"1993-march-two-computer-assisted-stories-win-ire-awards","url":"https://dwillis.github.io/uplink/article/1993-march-two-computer-assisted-stories-win-ire-awards.html","year":1993,"month":"March","monthNum":3,"headline":"Two computer-assisted stories win IRE awards","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"","summary":"Reporters from The Cleveland Plain Dealer and The National Law Journal won 1992 IRE awards for investigative reporting using computer-assisted research to expose environmental inequality and radiation safety failures.","keywords":["investigation","database","toxins"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":[],"full_text":"Congratulations to reporters at the The Cleveland Plain Dealer and The National Law Journal whose computer-assisted stories won 1992 IRE awards for investigative reporting.\n\n\"Unequal Protection: the Racial Divide in Environmental Law,\" by Marianne Lavelle, Marcia Coyle and Claudia MaClachlan of The National Law Journal won in the category for newspapers with circulation less than 75,000.\n\nThe National Law Journal reporters used computerized records to prove that a community's color influences the way the federal government cleans up toxic waste sites and punishes polluters. They used pre-collected data and also built two of their own databases for the story.\n\n\"Lethal Doses - Radiation That Kills,\" by Dave Davis and Ted Wendling of the Plain Dealer won in the category for newspapers with circulation over 75,000. Davis and Wendling used several Nuclear Regulatory Commission data files to show how the agency has failed to protect the public from radiation overexposures and accidents in the nation's hospitals. The February issue of Uplink featured a story by Davis and Wendling on how they did the series.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-march-namebase-offers-67000-hard-to-find-names","slug":"1993-march-namebase-offers-67000-hard-to-find-names","url":"https://dwillis.github.io/uplink/article/1993-march-namebase-offers-67000-hard-to-find-names.html","year":1993,"month":"March","monthNum":3,"headline":"NameBase offers 67,000 hard-to-find names","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"","summary":"NameBase is a searchable database of over 67,000 names compiled from investigative books and periodicals, specializing in intelligence, foreign policy, and political elites with detailed citations available on diskette for $79.","keywords":["database","intelligence","citations"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["floppy disk"],"full_text":"Public Information Research offers NameBase - a database of citations for over 67,000 names of groups and individuals.\n\nNameBase, compiled from over 400 investigative books and thousands of pages from periodicals, is said to specialize in sources that are ignored by expensive on-line search services. Citations for each name include author, title, date, and page number. Almost all titles contain a screen of annotations.\n\nAreas emphasized include the international intelligence community, U.S. foreign policy, political elites from the right and the left, assassination theory, Latin America and big business. Leading letter and phonetic searches can be used to locate difficult or transliterated names.\n\nSome sources which have been added within the last year include: 2,800 names from the 1991 Membership Directory for the Association of Former Intelligence Officers, 395 names from the 1992 Federal Staff Directory; 996 names from the State Department Biographic Register.\n\nNameBase fits on diskette and costs $79. Updates are available for $39. For information, contact Public Information Research, P.O. Box 5199, Arlington VA 22205, (703) 241-5437.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-aprilmay-suspicious-figures","slug":"1993-aprilmay-suspicious-figures","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-suspicious-figures.html","year":1993,"month":"April/May","monthNum":4,"headline":"Suspicious figures","kicker":"","author_name":"David Armstrong","author_title":"","affiliation":"Boston Herald","summary":"David Armstrong discusses the challenges and resistance faced when requesting public records in electronic form, highlighting instances where government cost estimates were inflated.","keywords":["records","government","cost"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["magnetic tape"],"full_text":"Two recent requests for public records in electronic form by the Boston Herald highlight many of the pitfalls and irritating techniques used by bureaucrats to keep information from reporters and their computers. In the first case, which began with a request more than two years ago, the Herald sought a copy of the Massachusetts motor vehicle registration database. We planned to use the database more as a people finder and research tool for reporters than for a particular project. However, the database's potential for projects was obvious. The response to the request for the registration file was unfortunately predictable. The information we requested was available, but would cost $168,000. However, the Massachusetts Registry of Motor Vehicles said it would offer the Herald a reduced fee of $3,134 because the Herald was a media outlet and was not using the information for commercial purposes. While $3,134 was better than $168,000, it was still unacceptable. The state said it had recently granted the rival Boston Globe the same reduced fee and the Globe purchased the database. We wanted to see the rationale for the Globe fee and formally requested a copy of any documents reflecting price estimates for the Globe. This was important because, under Massachusetts law, a state agency can only charge the 'actual cost' of producing the information requested by a public record seeker. Sure enough, the Registry of Motor Vehicles bilked the Globe. The agency charged the newspaper $600 a minute for CPU time. Yes, $600 a minute. In early 1991, the Herald questioned the CPU cost charged to the Globe, which was the basis for the fee the Registry offered to charge the Herald. We informed the Registry that if the CPU time was calculated for a full year, based on 50 percent usage, the total cost would be $155 million. The registry's computer, purchased in 1989, cost only $2.7 million. The Registry, however, refused to cave in and said the Herald was getting a great deal at $3,134. The Herald disagreed and appealed to the state's public records office to rule on the dispute. Finally, on Dec. 29, 1992, the public records office determined the Registry price was inflated and ordered the agency to provide the database to the Herald at a price reflecting the actual cost. On Jan. 11, 1993, the Registry wrote to the Herald and informed the newspaper it was revising the price for the driver registration database. The final price? $77. We bought it. In a second case, the Herald sought a computer tape copy of the Boston area mass transit authority's payroll. Initially, the transit authority simply ignored our request. After some prodding, from the Herald and the state's public records office, the authority gave us an estimate of $570 to provide the requested information. The estimate was bogus. The authority said two employees - a programmer and payroll manager - would have to work six hours to fulfill the request. This wasn't a problem since we realized some programming was necessary. There was a problem, however, with the hourly rate the authority intended to charge the Herald for the work of these two employees. When the hourly rate was calculated for a full year's salary, these two mid-level employees appeared to earn in excess of $90,000. I called the public records office and said if this was the actual salary of these two employees, the Herald would purchase the tape immediately. Imagine what the high-level officials were earning. However, if this wasn't their actual salary, I informed the record's officials the newspaper wanted a new estimate. Only days later a new estimate arrived. This estimate was $203, less than half of the previous total. The authority said it had mistakenly included all of the employee fringe benefits and overhead costs in the hourly rate. The moral of this story? Never trust an estimate from a government agency. Double check all the figures and question anything that appears to be suspect. I've found many agencies back down after they have been caught trying to jack up the price and keep public records private.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-aprilmay-cd-rom-may-be-the-future-but-dont-give-up-your-tape-drive-yet","slug":"1993-aprilmay-cd-rom-may-be-the-future-but-dont-give-up-your-tape-drive-yet","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-cd-rom-may-be-the-future-but-dont-give-up-your-tape-drive-yet.html","year":1993,"month":"April/May","monthNum":4,"headline":"CD-ROM may be the future, but don't give up your tape drive yet","kicker":"","author_name":"Deirdre Fleming","author_title":"","affiliation":"Missouri School of Journalism MICAR","summary":"Government agencies are transitioning from nine-track tapes to new storage technologies, with CD-ROMs and automatic tape library systems emerging as the primary alternatives by the early 2000s to improve efficiency and reduce labor costs.","keywords":["storage","technology","government"],"topics":["Technology","Data & Databases","Government & Politics"],"technologies":["CD-ROM","9-track tape","erasable laser storage discs"],"full_text":"Thomas Temin, chief editor of Government Computer News, has predicted that by the year 2000 government agencies will replace nine-track tapes with CD-ROMs. What the metamorphosis means for journalists is that CD-ROM drives will become a necessity to access federal information. And the change could take place swiftly as the cost of CD-ROMs comes down. Since 1986, Jerry McFaul, a computer scientist with the U.S. Geological Survey (USGS), has helped a number of government agencies convert to CD-ROM information storage and retrieval with the group he formed, SIGCAT (Federal Special Interest Group for CD-ROM Application and Technology). The lifespan of CD-ROMs has been questioned and SIGCAT, along with CD-ROM manufacturers, have begun testing the longevity of the media. Mike Martin, a science data systems technologist at the California Institute of Technology's Jet Propulsion Laboratory, reported in the March issue of CD-ROM Professional that the idea of CD-ROMs lasting for hundreds of years, error-free, may yet only be a dream. But Martin concluded in his report that CD-ROM is probably the most durable and long-lived media today. Still, Robert Deller, a consultant for the government on computer equipment and the director of market research for Selbre Associates Inc. in Bethesda, Md., said that in five years, nine-track tapes will be replaced by erasable laser storage discs, which have been around for the last four years commercially, but are still too expensive for widespread use in government agencies. Deller added that recently the move in government agencies has been to closed cartridges used in automatic tape libraries. He said many government agencies have turned to automatic tape libraries because the computer-run tape libraries are more efficient than open reel, nine-track tapes. Ed St. Jean, director of input process systems for the Internal Revenue Service, said that two years ago the IRS launched a pilot project using automatic tape libraries that use cartridge tapes to store data. St. Jean said that today the IRS uses mostly nine-track tapes because tapes are inexpensive. But, he said after using the automatic tape libraries, the agency expects to implement the new technology by the year 2008 in order to process information more efficiently. St. Jean said the greatest benefits of the automatic tape library have been the savings in human resources and the decreased risk of damaging the tapes. Because nine-track tapes can be creased or bent, St. Jean said, cartridges are a more reliable method of processing information without altering the data. 'In our environment the task of loading the large number of tapes is too labor intensive,' St. Jean said. 'We have to mount 2,000 tapes e week to load into the master file. If we had on line, we wouldn't need an operator to mount the tapes.' Other government agencies that use the libraries include the Department of Treasury, the Department of Agriculture and the Department of Health and Human Services. Deller argues that as automatic tape libraries give way to laser discs in the next five years, CD-ROMs will continue to be used by government agencies. A problem that hastens the move to CD-ROMs is that nine-track tapes are sensitive to dust particles and can be creased and damaged. While nine-track tapes deteriorate eventually, there is doubt that nine-track tapes will ever be extinct. Temin said that when CD-ROMs or cartridges are commonly used to store public information, the information previously stored on nine-track tapes will remain on the tapes for years before the data is transferred to another means of storage. 'Just like punch cards still exist out there somewhere, nine-track tapes will be around for a long time, probably 20 years,' Temin said. Deller said nine-track tapes will still be used in governmental agencies to a small degree to move data because they are inexpensive, portable and they can hold a large amount of data. St. Jean also agreed that nine-track tapes not be replaced entirely, although they will used less by government agencies. He said ti IRS will continue to use nine-track tapes to share information with agencies that still only use nine-track tapes.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-aprilmay-bringing-computer-assisted-reporting-to-every-reporters-repertoire","slug":"1993-aprilmay-bringing-computer-assisted-reporting-to-every-reporters-repertoire","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-bringing-computer-assisted-reporting-to-every-reporters-repertoire.html","year":1993,"month":"April/May","monthNum":4,"headline":"Bringing computer-assisted reporting to every reporter's repertoire","kicker":"","author_name":"Tom Foster","author_title":"The Post-Standard Syracuse, N.Y.","affiliation":"The Post-Standard Syracuse","summary":"The Post-Standard in Syracuse has democratized computer-assisted reporting by making data access easy and intuitive for all reporters, enabling daily use of databases for source-finding, fact-checking, and story details rather than limiting such tools to specialized projects. By prioritizing user-friendly interfaces through customized programs and familiar tools like electronic phone directories, the newspaper has integrated database resources into routine newsroom operations.","keywords":["databases","reporting","newsroom"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["9-track tape","CD-ROM","Paradox"],"full_text":"Every successful computer-assisted reporting effort chronicled in Uplink, there probably is an unfinished story languishing in newsrooms around the country. Reporters using computers for the first time jump in with enthusiasm. Then reality sets in. They often find themselves struggling to tame the mountain of data they obtained on 9-track tape or (Ugh!) key-punched themselves. These once-energetic souls now stare blankly at their screens. They are frustrated, puzzled and intent on avoiding eye-contact with editors. Two weeks later, there's still no story in the paper. Computer-assisted reporting often becomes the domain of a few reporters and editors who work on projects. That's unfortunate. Ideally, newspapers should be able to use computers to assist in investigations AND improve stories reporters write on a daily basis. Chris Feola outlined the Waterbury (Conn.) Republican-American's approach to democratizing the process in the March issue of Uplink. Feola's bottom line: Our reporters can get their hands on data they need in seconds. And so they use it. We've had a similar experience at The Post-Standard in Syracuse. By concentrating on making data access as easy as possible, we've been able to make use of our databases for stories in nearly every day's paper. For the most part, this involves helping reporters on deadline locate sources, check facts and add detail. A reporter scrambling to cover a late-night fatal fire, for example, can count on being able to quickly find a list of neighbors to call, identify the building's owner and its assessed value, and obtain a demographic profile of the neighborhood's housing values, rents and income levels from census data. The most popular databases we have acquired are telephone directories on CD-ROM. Hardly an hour goes by without a reporter or editor using one of these databases. We use a NYNEX product which is updated monthly and a national directory called ProPhone, produced by ProCD Inc. The latest version of ProPhone allows searching by address, providing instant access to a national crisscross directory. It's an inexpensive gem. Because the telephone book is such a familiar tool to reporters, the transition to the electronic directory is painless for most. From there, it's a small step to use other databases. Other heavily in-house databases include voter registration city and county assessment rolls, government rolls, campaign contribution records for federal, state and local candidates and 1990 census data. All of the databases, with the exception of those on CD-ROM, are accessed using programs written in Paradox's application language, or PAL. All of the directory changes, query structures, report parameters and other necessities usually required to tap database resources are invisible to users. All a reporter has to do is keep making choices from menus, then type in the name, address or value he or she is searching for. Writing these customized programs isn't as complicated as it might sound. All of the applications at The Post-Standard were produced by reporters and editors with no prior programming experience. The databases enhance our newspaper every day. The surprise is that this approach may have helped us produce better investigative projects. If we had set out to devote all our resources to long-term computer-assisted stories, we would have drawn on the talents of only a few reporters and editors. By having the entire staff familiar with our databases, we benefit from the story ideas they generate when they use the system. When a reporter gets an idea, he or she learns how to use interactive Paradox to do the analysis. Since the reporter already is familiar with the data, the learning curve is not as steep. Consequently, we have a long and growing list of computer-assisted stories that we have been able to turn around fairly efficiently. Examples include: Detailing faults in prison health care by tracking the treatment of more than 1,100 inmates who had died in the state's care in the past decade; Analyzing salary differences between men and women in county government; Dissecting a public agency's telephone records to help show that its top executive was doing his job from California; Uncovering an obscure tax exemption the city was granting developers by designating new parking garages as bomb shelters. Writing a feature on people with leap year birthdays. Not every computer story has to be an investigative project. This staple of Feb. 29 editions everywhere was painless because the voter registration roll provided the reporter with a list of names of people born on leap day, complete with phone numbers. All of those stories involved different reporters, and several other members of the staff are being drawn by increments into computer-assisted, long-term stories. We've had our share of glitches and failures. But so far every reporter who has been involved with a computer-assisted story has survived. A few are even brazen enough to look their editors in the eye.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-aprilmay-the-national-law-journal-uncovers-injustice-in-environmental-cleanup","slug":"1993-aprilmay-the-national-law-journal-uncovers-injustice-in-environmental-cleanup","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-the-national-law-journal-uncovers-injustice-in-environmental-cleanup.html","year":1993,"month":"April/May","monthNum":4,"headline":"The National Law Journal uncovers injustice in environmental cleanup","kicker":"","author_name":"Kelly Devine","author_title":"","affiliation":"Missouri School of Journalism MICAR","summary":"The National Law Journal's investigative project revealed that the EPA has enforced environmental protection laws unequally, cleaning up toxic waste sites faster in white communities while allowing pollution to persist in minority neighborhoods. The reporters overcame significant government data access barriers to conduct computer-assisted analysis demonstrating this racial divide in environmental law enforcement.","keywords":["pollution","discrimination","environment"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["floppy disk","PC","XyWrite","WordPerfect"],"full_text":"Fortunately, one regional EPA officer was very cooperative and agreed to produce a national Superfund diskette for the NLJ reporter at no charge. It's no surprise that wealth and race have long separated neighborhoods and communities nestled in the American landscape. It is surprising, however, that the U.S. government has allowed waste and pollution to fester in minority communities, becoming a deadly element of that injustice. Children suffer birth defects. Mothers and fathers battle cancer. Brothers, sisters, aunts, uncles complain of pink eye and asthma, ringworm and respiratory problems. This is life in one of the many minority neighborhoods that are not receiving equal protection under environmental law. The Environmental Protection Agency has promoted a racial divide in the United States by acting faster to clean up toxic waste sites in white communities and punishing polluters in white neighborhoods more severely. These are some of the startling discoveries The National Law Journal made when reporters Marianne Lavelle, Marcia Coyle and Claudia MaClachlan embarked on an eight-month project combining computer-assisted analysis of thousands of incidents with in-depth portraits of families who live each day in these minority communities that have become hazardous pollution dumping grounds. The article 'Unequal Protection: The Racial Divide in Environmental Law' not only unveiled the bias of environmental law in the U.S. government, but also earned the NLJ an IRE Award for its accomplishments. It was not easy. The road to obtaining documentation on diskette from the government was paved with denials, rejections, and non-cooperation. 'We knew that there was so much data that we would need to do the project by computer,' Lavelle said. The reporters' first difficulty was obtaining computer data on Superfund locations. Superfund is a complex federal program for identifying and cleaning up the worst of the nation's hazardous waste sites. The original diskette the NLJ received from the EPA for $200 was inadequate and incomplete. The data did not include when cleanup occurred and what happened after the identification of a Superfund site. At first, the EPA denied that such data even existed. But eventually officials conceded that the information could be obtained through each EPA regional office. In all, 10 FOIA requests were filed with the 10 regional offices. 'They responded very slowly, and each field office had a different response, from outright refusal to wanting to charge us $500 for the information,' Lavelle said. Fortunately, one regional EPA officer was very cooperative and agreed to produce a national Superfund diskette for the NLJ reporter at no charge. 'It wasn't through FOIA requests but through reporting skills and a reasonable EPA officer that we were able to obtain the data we needed,' Lavelle explained. Reporting skills helped to open still another door when the NLJ reporters attempted to obtain data from the Census Bureau. Because the bureau sells its data to individual contractors, retrieving the data can be very expensive, Lavelle explained. A contractor wanted to charge them $10,000 for the information. 'Luckily, through some reporting we were able to locate a non-profit organization, the Population Reference Bureau, whose purpose is to make census data public,' Lavelle said. 'They merged our EPA data with their census data and gave it to us on a computer disk we could use on our personal computers for $2,000.' After obtaining the various data, the reporters built two databases titled 'Enforgraph' and 'Supergraph,' linking demographic data with EPA's enforcement results and with EPA's Superfund progress, respectively. They collected and entered data on diskette using their Magnavox 386 SX-16 personal computer and the word-processing programs XYWrite and WordPerfect. The reporters also corrected nearly 200 ZIP codes from EPA's records by calling corporate headquarters, post offices, city halls and EPA regional offices, as well as interviewing agency engineers to fill in the various gaps in the EPA's Superfund data. They continued to manipulate and massage the data on their personal computers. Then they conducted nearly 500 interviews for their report, including many minority families living in these highly polluted communities. Overall, the project took eight months. Five months were spent gathering and analyzing the computer data and researching environmental injustice, two months were spent reporting in communities and one month was reserved for writing and editing the article. 'My advice to reporters wanting to get involved in computer-assisted reporting is to jump into a project,' Lavelle, who had never worked on a computer-assisted project before. 'In my opinion, you need to use computers to learn how to do it. Only by using your hands can you really learn.'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-aprilmay-turning-print-image-files-into-databases","slug":"1993-aprilmay-turning-print-image-files-into-databases","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-turning-print-image-files-into-databases.html","year":1993,"month":"April/May","monthNum":4,"headline":"Turning print-image files into databases","kicker":"Tech tips","author_name":"Brant Houston","author_title":"","affiliation":"The Courant","summary":"Journalists can convert messy print-image files from government agencies into usable databases by using word processing programs or specialized software to automatically remove headers and formatting rather than manually deleting them.","keywords":["data","files","automation"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["XyWrite","Monarch"],"full_text":"Turning print-image files into databases\n\nIn the land of data garbage one of the most painful files to get from a government agency is a print file.\n\nThis file looks just like the information you get on a printout and contains header information that looks like \"page one, judicial files of america, section 203AB, the office of Eddie Haskell, 10/20/92\" and so forth for each \"page\" of information.\n\nIf you can't convince the agency to give you a nice, flat file then it's laundry time.\n\nYou can clean these files, if they are small, by throwing them into a word processing program like XYWrite III Plus and manually defining the offending headers and deleting them.\n\nThis won't work very effectively for a 10 megabyte file, however, and that is where, whether you are a programmer or not, you should head for a program.\n\nAt The Courant, I discovered that a good person in our publishing systems had a handy XYWrite III Plus program that with a little modification would go in and destroy headers until it reached the bottom of the file.\n\nAll you have to do is identify a mark or string of characters that establishes the beginning of the header - \"page\" for example - and the end of the header - \"92\" perhaps and run it through its loops. Sometimes you might have to make other pass through the data, but the job gets done. I know the same kind of program can be written more gracefully in other word-processing programs or database programs, but the message of this tip is not to waste time on manual deletion when a little programming can help. Also, I have been told that a database program - Monarch - takes care of this problem during importing and look forward to reading about it while trying to find money to buy it.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-aprilmay-monarch","slug":"1993-aprilmay-monarch","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-monarch.html","year":1993,"month":"April/May","monthNum":4,"headline":"Monarch","kicker":"Tech tips","author_name":"Tom Boyer","author_title":"","affiliation":"The Virginian-Pilot and Ledger-Star","summary":"Monarch is a software tool that converts fixed-format printed reports into usable databases and spreadsheets, allowing journalists to extract data from agencies that cannot provide digital files without requiring programming expertise.","keywords":["software","database","reporting"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Monarch","FoxPro","9-track tape","Excel","dBase","Nine Track Express","XyWrite","mainframe"],"full_text":"Somebody in our data-processing department recently tipped me off to a piece of software that can digest a report print-image file and turn it into a live database. It's called Monarch, and I think its a gem.\n\nMonarch is like a database report writer in reverse. It takes fixed-format reports - even very complex slices them into data fields. You can export to a bunch of formats, including .wks spreadsheet, dbf database, or delimited ASCII.\n\nFor reporters, it helps with those agencies who aren't set up to output fixed or delimited files to tape, but who send their key data out in long printed reports. If you have a good FOIA, you can pay them to produce a tape file for you, or you can FOIA their print file if it's what you need. It's usually a minor programming change for them to filch a t file - such as a payroll list - from the mainframe's ant queue and output it to 9-track tape or floppy disk. Monarch requires no programming, and you can figure it out in a morning. The new version handles report files up to 30,000 pages. It is very fast on a 486. (No, I'm not getting a commission.)\n\nThere are three catches: 1) It must be a fixed-format report with fields of information appearing at the same spot within a line. 2) If it's outputted to tape, you'll need a utility to move the entire file en masse to your PC hard drive. 3) When you get the file in your PC, you may have to clean out weird print characters, extra line feed commands, etc. (More on that below.)\n\nMonarch works by recognizing the various lines of a report by their contents. Say you have a print-image of a report on criminal incidents (this is a report we get weekly). Each page contains headers and footer, and within the headers are three \"detail lines\" of information describing each crime.\n\nCRIME DATE TIME ADDRESS PROPERTY TAKEN OWNER BUILDING ENTRY METHOD SUSPECT SUSPECT OCCUPATION\n\nThe information, in a fixed format, looks like this:\n\nBURGLARY 042193 1330 421 APPLE LANE STEREO HOUSTON BRANT CONDO REAR DOOR KICKED IN SCHMID GRADUATE STUDENT\n\nWorking with this file in Monarch, you first teach it to recognize, or \"trap\" the first detail line by the pattern of characters, spaces and numbers. Once it knows the first detail line, it can pick the two lines immediately below it. Then you \"paint\" each bit of information with the cursor and give it a name. Monarch then chops up the report into fields according to your instructions. It can even include header and footer information as fields. Monarch has some analysis features of its own, but I generally have it export to .dbf files and do the analysis in FoxPro.\n\nNow for the hairy part. Sometimes the print-image files contain mainframe junk: weird IBM printer instructions, extra line feeds and such. If Monarch chokes on these you'll know because the report won't look right on your screen - you need to remove the offending characters.\n\nI won't go into detail, but you can use the hex dump feature of Nine Track Express to diagnose the characters that are giving you problems and write a routine to remove them or replace them with spaces. A word processor like XYWrite can clean up a print file with its search-and-replace capability, but its very slow with big files (editor's note: Brant Houston says the newest version of XYWrite is fast even with files up to 20 meg). I use the text editor that comes with FoxPro 2.0, which can search and replace the entire ASCII character set and is blazing fast with big files.\n\nAll this sounds complicated, but once you have it figured out, it need not take long. You can run a complete Monarch translation from a DOS batch file, and it can spit out thousands of records in a couple of minutes. In one case I've gotten the whole routine - from tape, through Monarch, to FoxPro's report writer, to an ASCII report that runs in the paper down to 10 minutes.\n\nMonarch is put out by Personics of Wilmington, Mass. Call them at (800) 445-3311 and they send you a demo disk free. The price is currently $395, and you may be able to get it cheaper from discount software houses.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-aprilmay-bits-bytes-and-nibbles","slug":"1993-aprilmay-bits-bytes-and-nibbles","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-bits-bytes-and-nibbles.html","year":1993,"month":"April/May","monthNum":4,"headline":"Bits, bytes and nibbles","kicker":"","author_name":"","author_title":"","affiliation":"Uplink","summary":"Uplink newsletter switches to bimonthly publication schedule from April through summer due to reduced student availability, while maintaining $20 annual subscription rates with more content per issue.","keywords":["publication","newsletter","summer"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"You might have thought your April issue of Uplink got lost in the mail, but from now until the end of the summer, we'll be publishing on a bimonthly basis. That's because our student help drops off over the summer months.\n\nOne-year subscriptions to Uplink still cost $20. We'll try to pack more stories and tips into each issue.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-aprilmay-uplink-becomes-bimonthly","slug":"1993-aprilmay-uplink-becomes-bimonthly","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-uplink-becomes-bimonthly.html","year":1993,"month":"April/May","monthNum":4,"headline":"Uplink becomes bimonthly","kicker":"","author_name":"","author_title":"","affiliation":"Uplink, University of Missouri","summary":"Uplink announces that current subscriptions will be extended to include 12 full issues and calls for readers to submit success stories, problems, and insights about computer-assisted reporting.","keywords":["subscriptions","reporting","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Current subscriptions will be extended to include 12 full issues.\n\nPlease remember, in order for Uplink to be the \"forum for computer-assisted reporting,\" we need your success stories, your problems, your ideas and insights into computer-assisted reporting. Write to: Uplink 120 Neff Hall University of Missouri Columbia, MO 65211","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-aprilmay-an-invitation-for-paradox-users","slug":"1993-aprilmay-an-invitation-for-paradox-users","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-an-invitation-for-paradox-users.html","year":1993,"month":"April/May","monthNum":4,"headline":"An invitation for Paradox users","kicker":"","author_name":"","author_title":"projects editor","affiliation":"Syracuse Post-Standard","summary":"Tom Foster at the Syracuse Post-Standard has developed Paradox applications to make data available to newsroom reporters and seeks to connect with other journalists who have worked with Paradox database systems.","keywords":["Paradox","database","newsroom"],"topics":["Journalism & Newsroom","Technology","Data & Databases"],"technologies":["Paradox"],"full_text":"Tom Foster, projects editor at the Syracuse Post-Standard, wants to share ideas with other reporters or editors who have developed Paradox applications. The article on page three of this issue of Uplink describes how Tom has successfully made data available to reporters in the newsroom, in part through programs written in Paradox's application language. If you share Tom's penchant for Paradox, contact him at (315) 470-3071 or on Compuserve 70401,1732.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-aprilmay-jon-schmid-moves-south-for-a-new-job","slug":"1993-aprilmay-jon-schmid-moves-south-for-a-new-job","url":"https://dwillis.github.io/uplink/article/1993-aprilmay-jon-schmid-moves-south-for-a-new-job.html","year":1993,"month":"April/May","monthNum":4,"headline":"Jon Schmid moves south for a new job","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Jon Schmid is departing MICAR to join the Raleigh News & Observer as an assistant database editor, with Matt Reavy, a doctoral student, taking over his editorial duties at Uplink and professional seminar responsibilities.","keywords":["MICAR","database","journalism"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"Jon Schmid will be leaving MICAR next month to work as an assistant database editor for the Raleigh, N.C., News & Observer. Jon has been handling much of MICAR's role with the professional community while pursuing a master's degree in journalism since Elliot Jaspin left the Institute last June. Matt Reavy, a doctoral student in journalism, will take over Jon's role as editor of Uplink and will help Andy Scott, executive director for Investigative Reporters & Editors, teach the professional seminars. The next TRI-DART seminar is scheduled for May 23-28. Both Andy and Matt will handle requests for data analysis and data sales until a new MICAR director is appointed.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-junejuly-work-is-killing-me-lax-labor-laws-imperil-texas-teens","slug":"1993-junejuly-work-is-killing-me-lax-labor-laws-imperil-texas-teens","url":"https://dwillis.github.io/uplink/article/1993-junejuly-work-is-killing-me-lax-labor-laws-imperil-texas-teens.html","year":1993,"month":"June/July","monthNum":6,"headline":"Work is killing me! Lax labor laws imperil Texas teens","kicker":"","author_name":"Sarah Baxter","author_title":"Special projects reporter","affiliation":"Houston Chronicle","summary":"Nancy Stancill uncovered numerous instances of Texas teenagers suffering injuries and long work hours under unenforced child labor laws, prompting legislative action.","keywords":["child labor","Texas","injuries"],"topics":["Investigative Methods","Government & Politics","Health & Environment"],"technologies":["9-track tape","EX IQ"],"full_text":"Four teenage girls are murdered in an Austin yogurt shop, where two of the girls routinely closed the shop alone at night.\n\nDuring a single year, more than 13,000 Texas teenagers suffer burns, sprains and other injuries, often ignored by their employers.\n\nCountless other teenagers fall asleep in school or fail their classes after staying up half the night at jobs where they work more than 40 hours a week.\n\nNancy Stancill, special projects reporter at the Houston Chronicle, analyzed nine-track tapes to uncover these and countless examples of how Texas teenagers pay the price for living in a state with one of the weakest child labor laws in the country.\n\nThe tapes, from the Texas Worker's Compensation Commission, revealed 13,328 injuries to Texas teenage workers from January 1989 to October 1992. The injuries included 15 deaths.\n\nStancill became interested in the story after learning that Texas' child labor law, on the books since 1981, had never been enforced. The law required inspectors to monitor children's work, but the state had never actually hired any such inspectors. The Texas Employment Commission was given responsibility for the law in 1990, but the legislature never allotted the money to enforce it.\n\nThe law sets 14 as the minimum working age, but Stancill found some teenagers who said they started working when they were as young as 10 years old. The law also allows 16- and 17-year-olds to work unlimited hours, causing many to fall behind in school, and placing some in danger when left alone to close shops at night.\n\nEven if an employer is caught violating the Texas child labor law, Stancill discovered, the maximum penalty is a $500 fine.\n\nStancill's first request for the records was ignored. \"They said the information was confidential,\" she explained. \"Some of it was, but most of it wasn't.\"\n\nAfter proving she was entitled to the records under the Open Records Act, Stancill convinced the state to comply with her request. In fact, officials didn't even charge her for the records. But the commission refused to reveal the names of the injured children or their employers, citing concerns about privacy.\n\nThe commission did provide detailed information describing injuries to teenagers and the types of businesses involved. But in a measure to protect the identity of specific employers, the commission refused to identify even the county of the employer, and instead identified employers by region.\n\nStancill was given names of some businesses that had been cited for violation of child labor laws by the U.S. Department of Labor. But the department only revealed businesses inspected by special strike forces.\n\nSince the department normally checks for child labor violations when inspecting a business for some other reason, Stancill felt revealing only those businesses cited for violation by strike forces would be unfair.\n\n\"They claimed there was no way to extract all the company names because they weren't filed the same way,\" she said. \"So it wasn't the full picture.\"\n\nAfter analyzing the data using EX IQ, Stancill found the majority of injured teens were hurt in restaurants and grocery stores. The most common injuries were sprains and cuts, but there were also amputations and deaths.\n\nThe figures provided Stancill with a solid foundation, but she needed to talk with the injured teenagers themselves. She started by interviewing high school teachers and guidance counselors to help her track down students who suffered injuries or long hours at their jobs.\n\nAfter interviewing some students Stancill found many overworked and injured teens. \"Children were being persuaded -- forced -- into working more and more hours at risk to their grades,\" Stancill said.\n\nOne teenage girl Stancill interviewed quit her job in a grocery store because she was working up to 48 hours a week. Her new job, where she works 35 hours a week and often closes the store alone, is little relief.\n\nOften when teenagers were hurt, their employers minimized their injuries and pressured them to continue working. One teenager, burned while working in a fried chicken restaurant, was given a little salve for his burn and told to continue working. The next day he went to see his school nurse who told him he had a \"serious second degree burn.\"\n\nOne follow-up story centered upon the 115 injuries to child farm workers. Stancill found migrant children as young as six years old working in the onion fields of the Rio Grande Valley. Often the children suffered headaches and vomiting after being sprayed with pesticides.\n\nThe Texas legislature drafted bills to strengthen child labor laws, citing the figures from Stancill's analysis as one motivation.\n\nA law to prevent businesses from hiring minors as strippers or topless dancers is also being pushed to the legislature. Currently the strippers can be as young as 17 years old.\n\nLegislators also want to increase the penalty for violators of the child labor law, and create a database to analyze injuries to working children. But Stancill noted that the prospect of an inspection program to investigate child labor complaints is still doubtful, due to lack of funds.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-junejuly-reporters-find-femas-secret-funding-of-fallout-shelter","slug":"1993-junejuly-reporters-find-femas-secret-funding-of-fallout-shelter","url":"https://dwillis.github.io/uplink/article/1993-junejuly-reporters-find-femas-secret-funding-of-fallout-shelter.html","year":1993,"month":"June/July","monthNum":6,"headline":"Reporters find FEMA's secret funding of fallout shelter","kicker":"","author_name":"Matt Reavy","author_title":"","affiliation":"MICAR","summary":"Cox News Service reporters uncovered a secret $2.9 billion FEMA nuclear war program through computer-assisted reporting and data analysis, revealing that FEMA spent $12 on nuclear contingency planning for every dollar spent on natural disaster response. The investigation succeeded where other media outlets failed because the reporters knew which specific questions to ask and had obtained GSA contract data that exposed the hidden program.","keywords":["FEMA","investigation","nuclear"],"topics":["Investigative Methods","Government & Politics","Public Safety"],"technologies":[],"full_text":"With every major news organization in the nation expending hundreds of hours to cover the Federal Emergency Management Agency (FEMA), you might wonder how $2.9 billion could slip through the cracks.\n\nLarry Lipman and Elliot Jaspin of the Cox News Service found out earlier this year when their recent computer-assisted reporting venture uncovered that for every dollar spent responding to natural disasters, FEMA has quietly shelled out $12 on plans to keep the government running during a nuclear war.\n\nIt wasn't that other media outlets hadn't gone to the right people, said Jaspin. They just didn't know which questions to ask.\n\nOnly after he and Lipman confronted a source with information they had obtained during a six-month analysis of computer tapes from the General Services Administration (GSA) did they learn what only a few FEMA officials and 20 members of Congress had known all along -- that for the past 11 years, FEMA has spent billions on a top-secret nuclear war program set up with the help of former White House national security aide Oliver North.\n\n\"It was like putting a key in a lock,\" said Jaspin. \"There was the great beauty and value of what I've been babbling about for years.\"\n\nThe story grew out of a suggestion from Cox member newspaper The Palm Beach Post. In the wake of Hurricane Andrew, the newspaper contacted Lipman and asked for a story on FEMA.\n\nLipman and Jaspin got to work shortly thereafter, downloading FEMA payments from a tape of government contracts above $50,000 handed out over the past four years. The tape is available for $100 annually from the GSA.\n\nIn reviewing the data, they noticed a number of payments to the Harris Corp. and several other national security contractors.\n\n\"At that point I was really baffled,\" said Jaspin. \"I didn't know what FEMA would be doing with national security contracts.\"\n\nThe pair went to FEMA with the information they had uncovered and a request to view the contracts. Agency officials rejected the appeal, citing national security concerns.\n\nBut one of Lipman's FEMA sources agreed to talk with the two reporters. \"It was enough,\" said Jaspin.\n\nThe source detailed a massive secret plan involving hundreds of high-tech communications vehicles, including one truck weighing 24 tons, all designed to make the government a moving target during a nuclear war.\n\nAlthough sources from other news organizations had been covering FEMA for years, none had discovered the secret operation. Part of the reason lies in the fact that budgets for the operation totalling more than a billion dollars have appeared annually as a single line - \"submitted under a separate package.\"\n\nLipman and Jaspin also learned that nearly a third of FEMA's employees operate within the top-secret project, three times as many as work on natural disaster programs.\n\nAccording to the article, which ran over the Cox News Service wire in February, FEMA spent $1.3 billion on equipment, support facilities and personnel between 1982 and 1991 in order to provide communications for government leaders during a nuclear war. The agency spent an additional $1.6 billion on related national security programs.\n\nDespite the obvious benefits the secret communications program could have in helping victims of natural disasters, no complete mobile unit has ever been used in such fashion. Most FEMA employees assigned to deal with Hurricane Andrew last year didn't even know about the agency's capacity.\n\nAfter hunting down other sources to verify the information they had obtained, Jaspin and Lipman called FEMA again and requested an opportunity to see the Mobile Emergency Response Support units, known as MERS. To their surprise, officials approved the request, as well as a full scale briefing of the project.\n\nProgram details provided by FEMA officials filled in the gaps and gave the pair enough to publish their story on February 22.\n\nOn March 1, one week after the Post published Lipman and Jaspin's account, a blue-ribbon panel recommended that FEMA's top-secret national security functions be turned over to the military and that a new domestic crisis unit be created to handle catastrophes.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-junejuly-nine-track-tape-drive-provides-easy-and-dependable-backup","slug":"1993-junejuly-nine-track-tape-drive-provides-easy-and-dependable-backup","url":"https://dwillis.github.io/uplink/article/1993-junejuly-nine-track-tape-drive-provides-easy-and-dependable-backup.html","year":1993,"month":"June/July","monthNum":6,"headline":"Nine-track tape drive provides easy and dependable backup","kicker":"Tech tips","author_name":"Elliot Jaspin","author_title":"","affiliation":"Cox News Service","summary":"The author describes using a nine-track tape drive with Computer Logics software to back up computer data after experiencing system failure, finding the reliable Save and Load utilities more effective than the built-in tape backup unit despite their basic command-line interface.","keywords":["backup","tape","software"],"topics":["Technology","Data & Databases"],"technologies":["9-track tape","mainframe"],"full_text":"My computer crashed. It was not really a crash. It was more like the big chill. One moment the computer chips were merrily processing data. The next instant they were suspended in cyberspace the electronic equivalent of chicken suspended in an eternal aspic.\n\nThe little tape backup unit that had come with the computer was no help. Despite hours of coaxing and several software upgrades, the tape backup unit continued to view its job with the glacial disdain of a headwaiter.\n\nThe answer to my woes was the nine-track tape drive. While we view these drives as a way to shovel mainframe data on to our PCs, they are able to perform several other valuable functions.\n\nThe software that comes with the drives sold by Computer Logics, for example, includes utilities that can backup a hard disk and copy tapes. Both programs are relatively easy to use and very dependable.\n\nTired of listening to the internal tape back up unit hoot, hum and whistle to no great effect, I put a blank tape on my nine-track drive and switched to my directory of utilities. A program, aptly named \"Save\", copies the contents of my hard drive to magnetic tape. In an era of graphical interfaces, mouse buttons and dialog boxes, it has all the appeal of a battered Ford truck with a loose muffler.\n\nOn the command line, you type \"Save c:\\*.*/s\". This tells the program to copy all the contents of the root directory. The \"/s\" switch says to copy all the subdirectories. When you need to restore the files to disk you type in the companion program, \"Load\", and the data comes flooding back. It may not be pretty but it has one major advantage. It works and it works. In the time it took to format a tape cartridge, I had copied all the files from my 660 megabyte hard disk to three reels of tape.\n\nThe cognoscente will point out a number of caveats. Reels of tape are awkward compared to a little cartridge. The Save and Load utilities do not handle data compression. And there is that ugly command line interface. All true and completely beside the point. Tape backup programs can be many things but they must above all else be super reliable and reasonably fast. And that is where these utilities shine.\n\nThe other essential are two utilities Dupin and Dupout. If you need to copy a tape, this is about as simple as you can get. Put the tape you want to copy on the drive, type \"Dupin\" followed by a file name, and an exact duplicate of what is on the tape will be copied to your hard drive.\n\nWhen Dupin is done, mount a blank tape on the drive with the plastic read/write ring installed and type Dupout and the name of the file. The program copies the disk file back to the tape and a duplicate tape is made.\n\nThe only cautionary note is to check the amount of free space on your hard disk. If you don't have room to hold the contents of the tape, housecleaning is in order.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-junejuly-enthusiasm-for-computer-assisted-reporting-is-keen-although-broadcasters-are-leery-of-cost-vs-return","slug":"1993-junejuly-enthusiasm-for-computer-assisted-reporting-is-keen-although-broadcasters-are-leery-of-cost-vs-return","url":"https://dwillis.github.io/uplink/article/1993-junejuly-enthusiasm-for-computer-assisted-reporting-is-keen-although-broadcasters-are-leery-of-cost-vs-return.html","year":1993,"month":"June/July","monthNum":6,"headline":"Enthusiasm for computer-assisted reporting is keen although broadcasters are leery of cost VS. return","kicker":"","author_name":"Michelle Smawley","author_title":"Associate Producer","affiliation":"The Crusaders, Los Angeles","summary":"Mary Field, a TV producer trained in computer-assisted reporting (CAR), struggles to implement the technology at her Chicago station due to budget constraints and lack of broadcaster adoption, highlighting the challenges of integrating CAR into broadcast journalism despite its proven investigative value.","keywords":["broadcast","technology","investigation"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Mary Field was excited by the possibilities. The investigative TV producer had just spent an intensive week at MICAR's computer-assisted reporting training seminar. When she returned to her station, WGN in Chicago, Field came up with a story idea: to examine the windy city's property taxes.\n\nBut there was one thing Field didn't have: the hardware and software necessary to do the story. Field's news director was not convinced installing CAR technology would fit in with the station's priorities under a tightening budget. So she had to look elsewhere.\n\nAfter weeks of searching, Field found a former academic interested in her subject. She had the data. He had the ability to download the information, so the two began a partnership.\n\nOf the 134 journalists who have attended the seminar, only four have been broadcasters.\n\nIt is not surprising that Field had to go through so much effort to use the skills she acquired at the MICAR seminar. Of the 134 journalists who have attended the seminar, only four have been broadcasters. Although major newspapers have experimented with computer-assisted journalism for several years, few broadcast outlets presently use computer-assisted reporting.\n\nSome attribute the lack of interest in CAR to the visual element required of the broadcast medium. \"The thinking by broadcasters is that computer-assisted stories are computer stories and they're not,\" Field said.\n\nShe also credited her knowledge of computer-assisted reporting with expanding the way she visualized stories. \"There have been other stories that I've done that I wouldn't have done before because of the way I now know the government stores its information.\"\n\nThe station put together a piece on Chicago's efforts to catch those who failed to register their cars. The city spent roughly $20,000 to employ an army of patrols which would walk the streets searching for offenders and then ticket them. WGN reported that the data was stored on tapes, and all the city had to do was run the two tapes against each other, a move they reported would save taxpayers thousands of dollars. The city changed their policy as a result of the story.\n\nBut the reality of the technology is costly by some standards. It runs about $14,000, including hardware, software, training and access to the database.\n\nBrooks Jackson, a correspondent with CNN's special assignment unit, can relate to the cost limitations. Jackson recently shared a sample of stories researched using CAR methods with CNN management. He also proposed they buy the appropriate equipment and hire a CAR specialist to run it. The result? Management was very enthusiastic but so far there has been no money committed to the project. \"International expansion is hot, so every spare nickel is going in that direction. For the moment everything domestically is on hold,\" said Jackson.\n\nThere's also the matter of time. Television stories run much shorter than print pieces and usually require less time to put together. Jackson said that, in an industry where every second counts, computer-assisted stories may appear to be too time consuming. \"Because TV is so demanding logistically, correspondents spend all their time traveling and doing ten takes in front of the Capitol. The ratio of producers to correspondents is one-to-one. We're already stretched thin doing the kind of stories we know are there.\"\n\nBut the future holds promise. In fact, at a recent seminar for the Radio and Television News Directors Association, broadcast news executives appeared very excited about the technology. One news executive, who said he'd like to keep his identity secret because he'll be the first in his market to use computer-assisted reporting, proclaimed a bright future for CAR in television.\n\n\"Electronic journalism is the frontier for TV investigative reporting,\" he said. \"There's almost an infinite amount of information which is available to the public and the media which we have in the past not even tapped. We are going to be able to do stories we have never been able to do in the past.\"\n\nThe news executive purchased the equipment almost as soon as he returned to work. He said he hopes to have a new computer-assisted reporting program on line by September.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-junejuly-ire-and-n-and-o-host-car-workshop-october-21-24","slug":"1993-junejuly-ire-and-n-and-o-host-car-workshop-october-21-24","url":"https://dwillis.github.io/uplink/article/1993-junejuly-ire-and-n-and-o-host-car-workshop-october-21-24.html","year":1993,"month":"June/July","monthNum":6,"headline":"IRE and N & O host CAR workshop October 21 - 24","kicker":"Bits, bytes and nibbles","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Investigative Reporters and Editors and The News & Observer will host a four-day computer-assisted reporting conference in October 2024 in Raleigh, bringing together 50 expert journalists and 400-500 peers to demonstrate and teach skills in using computers for data gathering, analysis, and storytelling.","keywords":["conference","journalism","computers"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["FoxPro","Paradox","CD-ROM","Visual Basic","Nine Track Express","Windows/DOS"],"full_text":"Investigative Reporters and Editors and The News & Observer in Raleigh, N.C., will show reporters and editors this fall how they can do their jobs with more precision, depth and power through computer-assisted reporting. About 50 top journalists and other experts who use computers to gather and sort information and write stories will demonstrate their skills and share their ideas with 400 - 500 peers during a four-day conference devoted exclusively to computer-assisted reporting. The conference will be Oct. 21 - 24 at the Radisson Plaza Hotel in downtown Raleigh.\n\nThe program will mix beginning and advanced sessions. Sessions will include:\n\nEquipment and software -- what's best and what to budget for\nDatabases -- what's out there, where to find them and how to negotiate for them\nGoing on-line\nIntroduction to CD-ROM and Hypertext\nAdvanced word processing techniques\nParsing, scanning, mapping\nVisual Basic and other application development tools\nSpecific story ideas to take home and do\nPolling and statistics\nIdeaware -- covering a variety of useful\nHow to set up computer-assisted reporting classes in the newsroom\n\nHands-on training classes will cover:\n\nWindows/DOS\nSpread sheets\nNineTrack Express\nCreating and manipulating databases, analyzing data and programming in FoxPro and Paradox\n\nFor more information about the conference please write or call: Investigative Reporters & Editors 100 Neff Hall UMC School of Journalism Columbia, MO 65211 (314) 882-2042.\n\nTop journalists from the News & Observer, the St. Petersburg Times, the Washington Post, New York Newsday, USA Today and U.S. News & World Report and experts from the FEC, the Poynter Institute and the National Library on Money and Politics will demonstrate their computer-assisted reporting skills at October conference","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1993-augustseptember-violence-in-tacoma","slug":"1993-augustseptember-violence-in-tacoma","url":"https://dwillis.github.io/uplink/article/1993-augustseptember-violence-in-tacoma.html","year":1993,"month":"August/September","monthNum":8,"headline":"Violence in Tacoma","kicker":"","author_name":"Adam Berliant","author_title":"Computer-assisted reporter","affiliation":"The News Tribune","summary":"An in-depth examination of violent crime in Tacoma using a database provided by the Law Enforcement Support Agency, highlighting areas of high crime and trends in violence.","keywords":["Tacoma","crime","database"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Like one of Murphy Brown's new secretaries, I started at The News Tribune in Tacoma as just another punchline to a running gag.\n\nI was the third in a line of computer-assisted reporters who was supposed to perform mind-boggling amounts of database research for The News Tribune's most ambitious project in years - an eight-day, 32-page series on violent crime in one of the nation's most violent cities.\n\nThe special projects team had already been working on the project for a year and, as with Murphy Brown's new secretaries, my arrival in the newsroom was met with more of an \"oh great, another one\" attitude than with optimism.\n\nBut, within two months, not only was the series published, but it was easily one of the most detailed looks at violent crime ever produced by a newspaper. It would not have been possible without computers.\n\nAt the foundation of the series was a database from Tacoma's Law Enforcement Support Agency (LESA), a police support facility that, in addition to answering 911 calls, compiles city and county police incident reports. The database we obtained from LESA contained every reported violent crime filed in Tacoma and Pierce county since 1986--almost 200,000 records.\n\nI strongly urge you to pursue your city's equivalent to the LESA database. Once again, LESA's database was nothing more than a compilation of data from everyday police incident reports, such as the type of crime, the age, sex and race of each victim and perpetrator, the location and time of the incident and so on. These are the same reports journalists frequently use for breaking crime stories, but on an individual basis these reports shed no light on a city's overall crime problems.\n\nIn your city, find the agency responsible for reporting crime statistics to the FBI and you will be well on your way to finding an equivalent database. I suggest starting at the local level first, and call state agencies as a last resort. Don't give up -- the LESA database yielded such a mammoth amount of intriguing and relevant information about crime in our city and county that even eight days and 32 pages seemed constricting.\n\nUsing the LESA database The News Tribune:\n\nIdentified the areas of Tacoma where the most and worst violence occurs, including several residential and retail areas where the extent of crime problems were unknown even to the police.\nRe-established that the area in the heart of the city that officials claimed was improving dramatically in terms of violent crime was still the worst part of town by far, despite specific efforts on the part of the police department to reduce crime there.\nExplicitly described trends in violence, such as bothersome increases in aggravated assaults committed by young people and reported rapes.\nShowed the enormous increase in the popularity of handguns for violent crimes, even in schools and domestic violence cases.\nIdentified specific schools with crime problems, something the police department had never done.\nClosely examined certain types of violent crime, such as an astonishing amount of domestic violence, which occurs as often as all other types of violent crimes combined.\nCompared crime statistics with U.S. Census data to connect social and economic factors with violence.\nDebunked the notion that the city's East Side, large, low-income, minority populated region of the city where the police had focused most of their personnel and administrative changes, was the worst part of Tacoma in terms of violent crime.\n\nThe only information LESA excluded from our electronic files were the event descriptions - lengthy paragraphs of text, filled out by the reporting officer about the events leading up to the crime - which would have been nice for our project but were too verbose for LESA's data processors.\n\nOnce again, chances are very good that your city has a similar database. Before you start your violence project, some words of computer-assisted wisdom based on The News Tribune's experiences:\n\n1.) Prepare For The Worst: It seems like crime databases of any kind, be they court records or prison records, tend to bring out the worst in government agencies. I wasn't around when The News Tribune negotiated for the LESA database, but my colleagues tell me they got their first taste of what most Uplink readers have dealt with for years - stingy government officials whose unwillingness to provide a computer database was surpassed only by their phobia of the media.\n\nLESA officials were confident that our newspaper would devote most of its time and energy to bashing the police department -- as if violent crime were somehow the fault of the police, which as my editor put it, would be like blaming garbage collectors for garbage.\n\nI predict this is an obstacle you will need to overcome. Prepare for legal disputes over access. In our case, we had to sign a contract that said we pledge scout's honor to abide by the federal and state laws that restrict researchers to \"statistical analyses\" with this kind of data (as if we intended to make a mailing list out of assault victims).\n\n2.) Work With The Cops: I wouldn't bank on uninhibited police cooperation for your violence project, and it certainly isn't a prerequisite. But at The News Tribune, our decision was that if we can show the police that we were a) not out to get them, b) truly interested in helping out the community with violence statistics, and c) coming up with new information that they could use, the project would run more smoothly both before and after publication.\n\nFor example, we invited police department analysts to stop by and do their best to debunk our findings before publication: Not only could they not find fault with our analyses, but by the time these meetings were over, they were busy telling us what a great job we were doing and asking if they could make copies of our reports.\n\nWhen our series was finally published, we knew that the police were not only aware of our results and our methods, but also were convinced of their validity. We didn't receive a single angry phone call.\n\n3.) Get Good Back-Up Data: I think any time you have government employees making judgment calls on official documents, you end up with screwy data.\n\nFor example, the LESA database had a field for domestic violence, where the officer would enter an \"X\" if he or she determined the crime to be domestic. But in our analyses of domestic violence, we found hundreds upon hundreds of cases where the perpetrator was the victim's husband, brother, daughter, etc. and where the crime occurred in the home, but the domestic violence field was not X'd.\n\nIt helped greatly to have FBI reports, official agency reports and good agency sources with which we could check our results.\n\n4.) Check Your Data Integrity EARLY!: The News Tribune wasted a lot of time generating and working with erroneous reports because some basic computer-assisted reporting rules were ignored early on.\n\nWhen I arrived, my first unpleasant task was explaining to the editors and reporters how code changes they didn't know about had sent them on wild goose chases. Later, I had the additional privilege of showing them the huge chunk of missing data that was altering all of our results.\n\nSpend days, not hours, checking data integrity - it's worth it.\n\nPlease do not hesitate to call me if you want a reprint of our violence series or if you have any questions. I can be reached at (206) 597-8258, and please, ask for Adam, not \"that new computer guy.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-augustseptember-plugging-in-the-morning-call-allentown-pa","slug":"1993-augustseptember-plugging-in-the-morning-call-allentown-pa","url":"https://dwillis.github.io/uplink/article/1993-augustseptember-plugging-in-the-morning-call-allentown-pa.html","year":1993,"month":"August/September","monthNum":8,"headline":"Plugging in. The Morning Call, Allentown, Pa.","kicker":"","author_name":"David Washburn","author_title":"Reporter","affiliation":"The Morning Call","summary":"The Morning Call newspaper has successfully used computer-assisted reporting to produce major investigative stories, including an exposé on a troubled local bank and analysis of mortgage lending discrimination, while working to build a library of local databases despite bureaucratic obstacles to public records access.","keywords":["reporting","databases","access"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Nine Track Express","XDB","Paradox","MapInfo","PCensus","9-track tape","CD-ROM"],"full_text":"Computer-assisted reporting is still relatively new at The Morning Call, but since January the staff has generated two major stories that would have been impossible to do without the help of computers.\n\nLast winter The Morning Call profiled a small local bank under state and federal investigation for dangerous and illegal lending practices. Since the 1980s, the bank had made one bad loan after another and was sitting on a mountain of foreclosed real estate.\n\nMorning Call reporters built a flat-file database from 1,300 hard copy court records which allowed them to quickly identify who was doing business with the troubled bank.\n\nThe result was a four-part series on the bank that \"couldn't have been done without the computer,\" said reporter Tim Darragh.\n\nAnother Morning Call story looked at mortgage lending in the Allentown area. The newspaper purchased the Federal Reserve's 1991 Home Mortgage Disclosure Act data on nine-track tape. Reporters analyzed more than 20,000 mortgage applications from the Allentown area, and found that blacks seeking mortgage loans were rejected more than twice as often as whites, and significantly more than Hispanics, even when they reported similar income.\n\nApart from putting together those two projects, we've been negotiating for several local databases. We've also subscribed to more than three dozen on-line services and bulletin boards. Morning Call reporters now have access to databases on everything from federal bankruptcy courts to the Toxic Release Inventory.\n\nOne of our goals is to build a library of local databases and then set up front-end programs that will make it easy for all reporters to use the computer on daily stories. We still have a ways to go before our goal is realized, but Morning Call reporters and editors are confident that the newspaper is on the right track.\n\n\"There have been no hoops to jump through to get funding for the program,\" said city editor David Erdman. \"Our executive editor Larry Hymans is very committed to computer-assisted reporting and new technology in general.\"\n\nSince January, The Morning Call has purchased two 486 computers with tons of memory, a nine-track tape drive, a CD-ROM driver and several software packages, including Nine Track Express, XDB, Paradox and the MapInfo/PCensus package.\n\nLocal bureaucrats, however, are doing everything they can to keep us from putting our new equipment to good use. We have run into several roadblocks in pursuit of local databases. Examples include one county wanting $5,000 for its tax assessment file, and another requiring that we tell them exactly how we plan to use their voter registration database.\n\n\"Bureaucrats like to describe this issue as one in which the press is seeking special privileges,\" Erdman said. \"All we're fighting for is basic access to public information.\"\n\nUnfortunately, the law has not always been on our side. Pennsylvania access law, which is notoriously weak, defers to other state statutes. For example, the open records law says the state Department of Motor Vehicles database is public. However, we can't get DMV files because the state's vehicle code forbids the mass interrogation of vehicle and drivers' records.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1993-augustseptember-programming-as-easy-as-sql","slug":"1993-augustseptember-programming-as-easy-as-sql","url":"https://dwillis.github.io/uplink/article/1993-augustseptember-programming-as-easy-as-sql.html","year":1993,"month":"August/September","monthNum":8,"headline":"Programming? As easy as SQL","kicker":"Techtips","author_name":"Richard Mullins","author_title":"","affiliation":"National Library on Money & Politics","summary":"Journalists already have foundational programming knowledge through SQL and database queries, and can now learn FoxPro programming to automate repetitive data tasks that go beyond what query languages can accomplish.","keywords":["programming","database","journalists"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["FoxPro","SQL","XBase"],"full_text":"OK, this one's about programming, a word that may clash with the usual job description of a reporter. Just like the ideas of data and computers and SQL once did.\n\nAssimilating SQL required something of an attitude adjustment from reporters about their life's calling and how they were to go about fulfilling their mission. This, in spite of the fact that SQL is all about Queries, the Prime Mover of Reporting.\n\nWell, scribblers, while you were cultivating the intellectual traits necessary for writing sharp, database-piercing queries, fertile ground was prepared for programming. Yes, programming. It's been growing up there, inside your skull, though you probably weren't aware of it.\n\nSince you already know so much about programming, this article would be as easy as SELECT FROM BidRiggers.\n\nI'm going to use the FoxPro environment to talk about programming since it integrates (with a few compromises) SQL and a Fox's version of a standard database programming language called XBase.\n\nYou may have also used some FoxPro commands by typing them in the command window: LOCATE, GOTO BOTTOM, INDEX ON, etc. If so, then you know all you need to know to de-mystify Fox Programming. A Fox program is just an ordinary text file with several commands that execute in order when you run the program.\n\nOne of the benefits of SQL is that it is a Query Language, not a programming language. That means you use SQL to specify what information you want extracted from the database; you don't have to tell the computer how to do the fetching. But there are things that you may need to do with your data that can't be done with a query language. That's when it's time for programming. Many repetitive data cleanup tasks can be programmed, for example.\n\nThe program can take user input, and then proceed based on the answers. A program can save the frustration of remembering the commands and the syntax and typing them correctly. Look them up and type them once, then run the program over and over.\n\nI'll end this part with a very shortened version of a program you can use to cut down, for example, a 500,000 record database to a copy with only 1,000 records. The program extracts or samples every 500th record (500,000 divided by 1,000).\n\nIn this shortened form, the program is a slight improvement over typing a few lines in the command window. It would be more useful if you didn't have to go into the program to change the three variables. I'll add this next level -- getting the variables from the user - in a second part. It would also be more useful if it figured out the skip interval for you, based on your input of how many records you wanted extracted.\n\nThe last line needs some explaining. The program language lets you copy ALL records or records satisfying a certain condition. The condition is introduced by the keyword \"FOR\". Fox and Dbase files keep a record number for every record in a database.\n\nTo satisfy the condition for getting copied to the target database, the number that we set as the skip interval (that's 'SkipInt' in the program) should divide evenly into the record number ('recno()' in the program). The \"MOD\" function does the division and reports what the remainder is; if it's ZERO, that record is extracted to the target database. (This method is not the only way this could be done. It's the one I thought of when I wrote the program, ad hoc, two years ago.)\n\nUse the built-in text editor in Fox to type in this program. To create or edit a program file: In the command window, type 'modify command xtract'. If the file XTRACT.PRG doesn't already exist, it is created.\n\nTo run the program, type 'DO xtract' in the command window.\n\nPROGRAM LISTING: XTRACT.PRG\n\nIn this simple version, you have to fill in these three variables each time you use the program:\nSkiplnt = 500 && This number sets the extraction interval\nSourceDB = u 29 && Name of DB to extract from\nTargetDB = u 29 && Name of DB to copy extracted records\nuse (SourceDB) in 1 alias source\nmake an empty copy of the source DB copy structure to trim (TargetDB)\nnow put the extracted records in it copy to (TargetDB) for mod( recno(), Skiplnt ) = 0","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-march-uplink-update","slug":"1994-march-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-march-uplink-update.html","year":1994,"month":"March","monthNum":3,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"Uplink","summary":"Uplink newsletter is expanding to 12 annual issues with more pages and content, and is asking readers to contribute computer-assisted reporting stories, tips, software recommendations, and training insights.","keywords":["journalism","reporting","software"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":[],"full_text":"We're on our way to a new and improved Uplink, but we'll need our readers help. We have increased the number of issues to 12 a year. We plan to jack up the number of pages. We want to pack more information and tips into each issue. It's on the more information and tips' part that we'll look for your assistance. If you do a computer-assisted reporting story that you're happy with, send it to us. Attach a note telling us how you did it. If you see a story you didn't do but like, call us and tell us about it. If you find a new piece of software that makes data analysis easier, let us know. If you find a new online or offline database you think would be valuable, give us the tip. If you figure out a good way to train journalists in a CAR technique or to bring CAR into the newsroom, tell us. We're looking forward to hearing from you. Uplink's editors","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-march-so-what-is-nicar","slug":"1994-march-so-what-is-nicar","url":"https://dwillis.github.io/uplink/article/1994-march-so-what-is-nicar.html","year":1994,"month":"March","monthNum":3,"headline":"So what is NICAR?","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR is a new national institute jointly operated by Investigative Reporters & Editors and the Missouri School of Journalism that provides training, data resources, and expertise in computer-assisted reporting to journalists and news organizations across the country.","keywords":["journalism","data","training"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":[],"full_text":"So what is NICAR?\n\nIt's the National Institute for Computer-Assisted Reporting. It's a joint venture of Investigative Reporters & Editors Inc. and the Missouri School of Journalism - started with a generous grant from the Freedom Forum. It is the logical continuation of the Missouri Institute for Computer-Assisted Reporting, which was begun by Elliot Jaspin five years ago. And it is the most ambitious project in the country for training journalists and educators, for providing data and data analysis, and for acting as the clearinghouse for information about computer-assisted reporting. In the coming months, NICAR plans\n\nIncrease the number and kind of seminars at Missouri.\nOffer seminars and conferences across the country.\nBuild up a data library that can provide information inexpensively to news organizations.\nHelp smaller papers deal with large amounts of data.\nDo data analysis on complicated projects.\nOffer services online.\n\nThe staff has a varied background, which will enable it to provide a wide range of services.\n\nThe managing director of NICAR is Brant Houston, who worked on computer-assisted stories at The Hartford Courant since 1988, where he was database editor. The training director is Jennifer LaFleur, who comes from the San Jose Mercury News and knows computers, statistics and mapping software. NICAR's academic adviser is Richard Mullins, who was at the National Library of Money & Politics. Three graduate research assistants, Matt Reavy, Drew Sullivan, and Gwen Carleton, will help in the day-to-day operations. The institute also will draw on the resources and experts at the school of journalism. NICAR's next on-campus, weeklong seminars will be May 15-20 and Aug. 14-19. The fees charged by the NICAR will be as low as possible, but high enough to support the expansion of its staff, equipment and other resources. The institute welcomes donations of tapes and diskettes and advice on how to improve its services. NICAR's phone numbers are 314-882-0684 and 314-882-9491.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-march-coming-events","slug":"1994-march-coming-events","url":"https://dwillis.github.io/uplink/article/1994-march-coming-events.html","year":1994,"month":"March","monthNum":3,"headline":"Coming Events","kicker":"NICAR on the road","author_name":"","author_title":"","affiliation":"","summary":"A calendar of upcoming journalism training events including NICAR seminars and IRE conferences scheduled throughout 2024 across various U.S. locations.","keywords":["training","journalism","conferences"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"May 15-20 NICAR Training Seminar Columbia, Missouri June 16-19 IRE National Conference St. Louis, Missouri August 14-19 NICAR Training Seminar Columbia, Missouri October 6-9 IRE Computer-Assisted Reporting Conference San Jose, California","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-march-dining-on-eatery-data","slug":"1994-march-dining-on-eatery-data","url":"https://dwillis.github.io/uplink/article/1994-march-dining-on-eatery-data.html","year":1994,"month":"March","monthNum":3,"headline":"Dining on eatery data","kicker":"Post-Dispatch opens inspection larder","author_name":"Dan Browning","author_title":"projects editor","affiliation":"St. Louis Post-Dispatch","summary":"The St. Louis Post-Dispatch uses computer-assisted reporting to analyze restaurant health inspection data, revealing that despite high average scores, critical violations are not uncommon and can pose health risks.","keywords":["restaurant","health","inspection"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"You might win public service awards for studies of giant government databases on highway contracts, creative campaign financing or the redlining practices of mortgage lenders, but to twist an old saw, the way to your readers' hearts is through their bellies. Everybody eats. The health departments in many metropolitan areas have computerized their restaurant inspections, opening a larder of data to news organizations with the nerve to peek inside the kitchen - or under the dumpster.\nLast spring, Jan Paul, a projects editor at the St. Louis Post-Dispatch, assigned three reporters to examine restaurant inspections after the public became concerned about increasing reports of E. coli bacteria poisonings in the Pacific Northwest and hepatitis outbreaks in St. Louis.\nIn March, the newspaper got computerized data from St. Louis County and the city of St. Louis listing 103,493 inspection reports on 8,544 restaurants, delis, snack bars, institutions, corporate cafeterias, mobile kitchens, schools, nursing homes, daycare centers and retail bakeries. (St. Louis County later updated the records, adding 8,524 inspections to the mix.)\nRestaurants that get Bs or Cs must correct their problems within two days or they can be shut down. Similarly, restaurants that get hit with critical violations must correct them within a couple of days - but the public might never know there was a problem, because a restaurant could have up to three critical violations and still get to post the \"A\" rating in its window.\nIn most places that use the FDA's model inspection form, restaurants start an inspection with 100 points (a perfect score). Inspectors look for 29 categories of minor problems, which cost the restaurant either one or two points each, and 13 categories of \"critical violations\" - those with the greatest potential to have an adverse effect on public health - which cost the restaurant four or five points each.\nWhen we dug into the data, it seemed that the obvious way to measure relative performance on inspections was to examine average scores. We did that, and we found that establishments got As 98 percent of the time. In fact, they scored above 90 points in 88 percent of the cases. Ranking restaurants based on their overall scores seemed futile.\nThis analysis, based on 44,000 inspections at 3,200 eateries, found 'A' rated restaurants could still pose a risk.\nWe figured that we should look at the frequency of critical violations on the grounds that it might only take one incident to make someone sick. And we decided we should look at the intensity of critical violations - expressed as the average number of critical violations per inspection - on the grounds that the more of them that existed, the greater the chance that someone might get sick.\nIn St. Louis and St. Louis County, restaurants get letter grades to post in their windows. An \"A\" means the restaurant scores higher than 84 points. A \"B\" can go as low as 70 points, and a \"C\" goes as low as 50.\nUnfortunately, the databases we got were so badly designed that the health departments admitted they had never been able to use them. And the data was so \"dirty\" with entry errors that it took us about six weeks to clean it up so that we could group the restaurants.\nOnce we could use the data, we decided to limit our examination of the county data to only those inspections since Jan. 1, 1990, so that it would parallel the city data. We purged places that had closed or places that had changed ownership in the study period. And we limited our analysis to establishments that had been inspected at least six times so that the data would not be distorted by anomalous inspections.\nWe ranked all establishments based on their average inspection scores and the frequency and intensity of their critical violations. We looked at retail eateries as a class (restaurants, delis, snack bars, salad bars), schools and daycare centers, retail operation run by grocery chains, nursing homes and hospitals. We examined scores based on ethnicity. And we examined the scores of all the individual health inspectors to see whether any patterns emerged that might indicate hanky-panky.\nBecause of the number of ties in critical violations, the range of possible places in our ranking system was dramatically reduced. The critical violations lost their emphasis when we merged them with average inspection scores, which had a greater range of possible rankings. Thom Gross, an investigative reporter, suggested converting all three rankings to percentiles, then averaging them together. We did that and it worked. Several public health officials around the country said our methodology seemed sound.\nTwo reporters took a list of eateries that scored in the bottom one percent in our lists, and began verifying every single hard copy of their inspections. We found only two errors in several hundred records, and they did not affect totals. We corrected those, and decided the overall data was statistically reliable.\nWe came under attack from several quarters. The director of the city's health department said our study was no more valid than ranking restaurants based on their addresses, because no link has been established between foodbourne disease and restaurant inspections. But thanks to a search I did on CompuServe before we met with him, we were prepared. We directed his attention to an FDA study in Seattle-King County, which concluded that restaurants with low scores and critical violations were directly correlated to outbreaks of foodbourne disease.\nThe final challenge to our project came from the Missouri Restaurant Association, which argued that the inspection system itself was flawed. We agreed, but for opposite reasons. The industry association argued that any restaurant that was open was safe. Our articles showed that the inspection system failed to communicate anything to the consumer about critical violations, that inspectors often give a break to restaurant owners if they are cooperative and that the letter grade system used in St. Louis can be misleading.\nWhen we finally published the articles, our analysis was based on about 44,000 inspections covering 3,200 eateries.\nOur restaurant inspection report was a hit with our readers. Callers are praising us for it. Although it can be time-consuming, it's a project that can be done virtually any place that has a health department. But beware: The project was not a hit with the advertisers. We were told that after the project, an editorial and a cartoon ran, advertisers cancelled a special section on dining out.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-march-computer-conference-set-for-oct-6-9-1994","slug":"1994-march-computer-conference-set-for-oct-6-9-1994","url":"https://dwillis.github.io/uplink/article/1994-march-computer-conference-set-for-oct-6-9-1994.html","year":1994,"month":"March","monthNum":3,"headline":"Computer conference set for Oct. 6-9, 1994","kicker":"Beginning and advanced skills highlighted","author_name":"Jonathan Krim","author_title":"","affiliation":"San Jose Mercury News","summary":"The second annual IRE conference on computer-assisted journalism will be held in Silicon Valley in October, offering hands-on training in various computer applications and tools for reporters and editors. The event will feature instruction from top practitioners, panel discussions on legal and ethical issues, and demonstrations of journalism technology.","keywords":["conference","journalism","training"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["DOS","Windows","Internet"],"full_text":"The second annual national IRE conference devoted to computer-assisted journalism will be held in the heart of Silicon Valley this October. Whether your publication or broadcast outlet is just beginning to use computer-assisted techniques or is ready for more sophisticated approaches, this year's conference will have something for you. Some of the top practitioners nationwide will serve as instructors, offering classroom training in which each participant will work at his or her own computer. Conference participants will receive intensive, hands-on training in a variety of computer applications, including:\n\nBasic PC use, including DOS and Windows\nHow to use various spreadsheet programs\nHow to use relational database software to further manipulate information\nHow to download government tapes\nHow to use on-line services, including Internet\nHow to map data\nHow to use statistical analysis software\n\nThe conference, beginning Oct. 6 at 8 a.m. and ending at noon on Oct. 9, also will feature panel discussions on issues affecting computer-assisted journalism, such as legal matters, ethics, and using computers for beat reporting and quick-hit stories.\n\nBuilding on the strengths of the computer-assisted conference in Raleigh, the planners intend to expand advanced parts of the program while keeping a basic track for those who are just learning these techniques.\n\n\"This conference will be a great opportunity to learn to use the most basic and necessary tools for reporting,\" said Brant Houston, IRE's new computer-journalism director.\n\n\"Reporters and editors can no longer afford to ignore this area. After all, most information we will seek is or will be in electronic form.\"\n\nHosted by the San Jose Mercury News, the conference will also feature displays of products for computer-assisted journalism and a special demonstration of Mercury Center, the Mercury News' on-line newspaper service. A luncheon banquet, included with registration, will feature an exciting keynote speaker.\n\nThe conference site, the Westin Hotel in Santa Clara, features many amenities, including free parking, courtesy transportation from the San Jose airport and light-rail access to and from downtown San Jose.\n\nFor those who want to bring their families, the hotel is one block from the Great America theme and amusement park and is close to activities in San Jose, San Francisco and the rest of the Bay area.\n\nSpecial conference hotel rates are available, but registration will be limited. So, register now.\n\nAct quickly because registration in the hands-on classes is limited.\n\nHotel Registration Westin Hotel, Santa Clara\n\nWhen you call, say you are an IRE participant. Call (800) 228-3000 or (408) 986-0700 to make your hotel reservation.\nReservations made after Sept. 9 will be on a space-available basis, and rooms may cost more.\nHotel rate is $89, single; $99 double and $109 triple.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-march-eeoc-data-is-goldmine-for-job-stories","slug":"1994-march-eeoc-data-is-goldmine-for-job-stories","url":"https://dwillis.github.io/uplink/article/1994-march-eeoc-data-is-goldmine-for-job-stories.html","year":1994,"month":"March","monthNum":3,"headline":"EEOC data is goldmine for job stories","kicker":"Analysis covers 35,242 companies, 40 million workers","author_name":"Thomas Monnay","author_title":"","affiliation":"Wall Street Journal","summary":"A Wall Street Journal investigation of EEOC employment data from over 40 million workers revealed that Black workers were the only ethnic group to suffer net job losses during the 1990-91 recession, losing 59,479 jobs while other groups gained employment. The analysis, which required extensive FOIA requests and verification calls to 400 major federal contractors, has prompted civil rights leaders and a congressional investigation into the companies involved.","keywords":["employment","recession","discrimination"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["9-track tape"],"full_text":"The 1990-91 recession left the impression that many jobs simply disappeared, but a recent computer analysis by the Wall Street Journal shows that among ethnic groups only blacks suffered a net job loss.\nJournal staffers Ed Soldessy and Rochelle Sharp examined Equal Employment Opportunity Commission (EEOC) reports from 35,242 companies covering more than 40 million workers.\nThe law requires that businesses with 100 or more employees annually report the number of women and minorities they hire. Businesses with 50 or more employees must also file reports if they have a government contract of $50,000 or more.\nThe Journal's computer analysis of these records showed that blacks lost a net 59,479 jobs during the study period. Asians and Hispanics gained 55,104 and 60,040 jobs respectively, while whites gained 71,444 jobs.\nWhen the news hit Washington, angered civil rights leaders and members of the Black Caucus pressed for an immediate investigation into the companies involved.\nThe article itself capped hundreds of hours spent locating the tapes, analyzing the data and verifying reports.\nThe Journal originally obtained 9-track computer tapes containing the EEOC data, but found that the fields containing company names had been deleted. Department of Labor officials claimed the data could not be released without permission from the companies involved.\nWhen contacted by the Department, most companies objected. The Journal appealed the decision, but still no companies responded.\nAfter filing three separate Freedom of Information Act (FOIA) requests, the Journal succeeded in obtaining the names, addresses and summary employment filings of the nation's 400 largest federal contractors in terms of market capitalization. The appeal process on these filings alone lasted six months.\n\"I certainly didn't expect that,\" said Sharpe. \"But it was worth sticking with.\"\nAfter Souldessy and Sharpe analyzed the data, a team of reporters began the time-consuming process of calling all 400 companies to verify the data.\nDuring verification Sear, Roebuck & Co. indicated the firm had provided incorrect data to the government. Although the company said the mistake exaggerated blacks' proportion of job loss, the revised data still showed blacks losing jobs at a disproportionate rate.\nAs a result of the story and the resulting response among minority activists, Congressman Julien Dixie of California has vowed to launch an investigation into the matter.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-march-how-to-keep-data-straight-by-adding-an-id","slug":"1994-march-how-to-keep-data-straight-by-adding-an-id","url":"https://dwillis.github.io/uplink/article/1994-march-how-to-keep-data-straight-by-adding-an-id.html","year":1994,"month":"March","monthNum":3,"headline":"How to keep data straight by adding an ID","kicker":"Tech Tips","author_name":"Richard Mullins","author_title":"","affiliation":"University of Missouri","summary":"A guide to eliminating duplicate rows in databases by creating a unique identifier field using the REPLACE ALL command with RECNO() function in FoxPro or dBase.","keywords":["duplicates","identifier","database"],"topics":["Data & Databases","Technology"],"technologies":["FoxPro","dBase"],"full_text":"Every data table you work with should have rows that are unique. Duplicates can be confusing, for one thing, because you never know if the duplicate row belongs there. (Did Richard Mullins bounce a check twice, or did a data clerk just type it in twice?)\n\nThis uniqueness rule is not just a good idea, it's the law, at least according to relational theory. Most often this is a number, in a column called ID, (or Casenum, Case# or CaseNo).\n\nBut we all know that there are lots of databases and data creators that overlook this. You may have done it too, in your young and foolish days.\n\nHere's how to right the wrong and be sure every row in a table is uniquely identified. The commands are for FoxPro or dBase.\n\nAfter you've created a new field to hold the number (make sure it's NUMERIC and big enough to hold the largest number that might be put there), type this command in the Command Window: (The field or column name here is ID)\n\nREPLACE ALL id WITH RECNO( )\n\nThat's it. This one-line command works because Fox stores internally a number for every row, or record, in the table. The REPLACE. WITH command puts that number (supplied by RECNO( ), a built-in function) in a field where you can see it. The command word ALL makes the replacement on every row.\n\nHere's one more twist on this command, if you want to start the ID numbers with a larger number. You can add another number, say 1000, to the internal record number. Then your ID numbers would start at 1000, instead of one. Here's the command:\n\nREPLACE ALL id WITH RECNO( ) + 1000","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-march-akron-completes-yearlong-car-series","slug":"1994-march-akron-completes-yearlong-car-series","url":"https://dwillis.github.io/uplink/article/1994-march-akron-completes-yearlong-car-series.html","year":1994,"month":"March","monthNum":3,"headline":"Akron completes yearlong CAR series","kicker":"Bits, Bytes & Barks","author_name":"Matt Reavy","author_title":"Graduate Research Assistant","affiliation":"Akron Beacon Journal","summary":"The Akron Beacon Journal completed a year-long investigation examining racial disparities across multiple dimensions including crime, race relations, living conditions, health, education, and economic progress, utilizing computer analysis of census data and various government databases.","keywords":["investigation","race","data"],"topics":["Investigative Methods","Demographics & Census","Crime & Justice"],"technologies":[],"full_text":"The Akron Beacon Journal recently wound up its year-long analysis of blacks and whites in the 90s with a four-part series about crime and punishment.\n\nOther series during the course of the investigation dealt with race relations, living conditions, health and education, and economic progress.\n\nBob Paynter, assistant metro editor-projects, indicated that the staff used computers to examine census data, efficiency test scores, police reports, and a variety of county and school district databases.\n\nReprints of the story are available from the Beacon Journal","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-march-a-shaggy-dog-story","slug":"1994-march-a-shaggy-dog-story","url":"https://dwillis.github.io/uplink/article/1994-march-a-shaggy-dog-story.html","year":1994,"month":"March","monthNum":3,"headline":"A shaggy dog story?","kicker":"Bits, Bytes & Barks","author_name":"Brad Goldstein","author_title":"","affiliation":"The Eagle-Tribune","summary":"The Eagle-Tribune analyzed dog registration data from 20 communities in New Hampshire and Massachusetts, discovering that Max was the most common dog name overall and uncovering a Plaistow woman with 33 Chihuahuas named Chico.","keywords":["dogs","database","analysis"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["FoxPro","9-track tape"],"full_text":"Imitation is the best form of flattery. The Eagle-Tribune, Lawrence, Mass., wanted a quick computer assisted story that would give a reporter a chance to learn databases as well as grab the reader. So we borrowed a story that was done by Steve Doig at the Miami Herald.\n\nWe requested dog registrations for 20 communities in southern New Hampshire and northern Massachusetts. Some towns used FoxPro and provided dog registrations, including name, breed, owners, address, color and sex, on floppy diskettes. Others gave us the data on nine-track tapes. Some towns required the reporter to input data. In the end, we had 18,000 dogs. We crunched them down by most common breed per town, per state, and most common name per breed. The winner: Max, the mutt.\n\nThe payoff. A Plaistow, N.H. woman who had 33 Chihuahuas - each with its own bed, living in her home. They all ate at 3 p.m., and she never let them out of her home. Oh yes, the most common Chihuahua's name? Chico, of course.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-march-census-bureau-phasing-out-mainframes","slug":"1994-march-census-bureau-phasing-out-mainframes","url":"https://dwillis.github.io/uplink/article/1994-march-census-bureau-phasing-out-mainframes.html","year":1994,"month":"March","monthNum":3,"headline":"Census Bureau phasing out mainframes","kicker":"Bits, Bytes & Barks","author_name":"Matt Reavy","author_title":"","affiliation":"","summary":"The U.S. Census Bureau is planning to replace its mainframe systems with Digital Equipment Corporation VAX minicomputers by 1996 in preparation for the 2000 census, while also exploring digital imaging and internet capabilities.","keywords":["mainframe","census","technology"],"topics":["Demographics & Census","Technology"],"technologies":["mainframe","VAX","9-track tape","Internet"],"full_text":"Census Bureau phasing out mainframes\n\nThe U.S. Census Bureau has announced plans to phase out its mainframe system by 1996, just in time for the 2000 decennial census.\n\nThe bureau plans to utilize its 18 Digital Equipment Corporation VAX minicomputers to pick up the bulk of the load. DEC terminals or PCs emulating DEC terminals are used to connect to the VAX\n\nPlans also call for construction of a new computer center at the University of Maryland's Technology Center in Bowie, Md. by 1996.\n\nThe bureau is reportedly examining what will become of its archive, which currently reside on 9-track computer tape, as well as the possibility of getting into digital imaging and moving onto Internet.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-april-uplink-update","slug":"1994-april-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-april-uplink-update.html","year":1994,"month":"April","monthNum":4,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"The journalism newsletter encourages reader submissions and shares updates from the National Institute for Advanced Reporting conference, with plans to apply insights at an upcoming IRE conference in St. Louis focused on computer-assisted reporting.","keywords":["conference","reporting","databases"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Uplink update\n\nWe asked for your help and ideas in our last issue, and we're starting to get the phone calls, clips, faxes, and electronic messages we had hoped for. Please keep them coming. (Several of our readers noticed that we failed to include our electronic addresses in our masthead. We included them in this issue).\n\nWe attended the recent annual conference at the National Institute for Advanced Reporting in Indianapolis, heard some terrific speakers, met with Jim Brown, NIAR's honcho, and picked up many ideas on where our endeavors at NICAR should head.\n\nWe hope to apply those ideas at the upcoming national IRE conference in St. Louis, June 16-19. If you want to attend, call IRE at (314)882-2042. The program for the traditional optional day on Thursday, June 16, will focus on online databases and computer-assisted reporting for daily and beat stories. The NICAR staff will be there in force, running a demonstration room, helping with panels, and providing advice.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-april-deciphering-welfare-fraud","slug":"1994-april-deciphering-welfare-fraud","url":"https://dwillis.github.io/uplink/article/1994-april-deciphering-welfare-fraud.html","year":1994,"month":"April","monthNum":4,"headline":"Deciphering welfare fraud","kicker":"","author_name":"Bradley Goldstein","author_title":"","affiliation":"The Lawrence Eagle-Tribune","summary":"The article discusses the efforts and challenges faced by The Eagle-Tribune in Lawrence while conducting a computer-assisted investigation into the Massachusetts Welfare System. The series uncovered a lack of preventative measures against welfare fraud, resulting in increased public interest and legislative action.","keywords":["welfare","fraud","investigation"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["dBase","ASCII","DSALVAGE","FoxPro"],"full_text":"Do you think readers would like to know if a well-paid foreign consul general also receives welfare benefits? Or if a sanitation worker walked into a Boston bank and used his Massachusetts welfare card to cash a $25,000 disability check? And never declared the outside income to welfare? Or if a man is legally entitled to receive a $29,000 Social Security check while serving a two-year prison sentence? Or what welfare dependency means to two women who have collected public assistance for nearly 20 years each? Stories like these and a dozen others helped bring the complex subject of welfare and welfare abuse into focus when The Eagle-Tribune in Lawrence published a five-part computer-assisted series on the Massachusetts Welfare System in September. The series found Massachusetts welfare officials doing little or nothing to prevent unscrupulous people from stealing tens of millions of dollars from taxpayers and the poor. Daily circulation rose 1,000 papers above normal during the series. And requests from across the country keep trickling into the newsroom. But the juicy stories were just one small part of a long and complicated project.\n\nThe names of welfare recipients have been protected under state and federal privacy laws since the Great Depression. So when two years of welfare rolls for the Greater Lawrence area landed in our laps we started to computerize them.\n\nComplicating matters was a 50-year-old law that makes it a criminal offense for anyone to possess welfare rolls or use them to derive statistics, and the task of deciphering the welfare department's code.\n\nTwo sets of lawyers told us the law was unconstitutional. And when a file layout was sent to us in the mail, the code was easily understood.\n\nWelfare lists contain names, social security numbers, addresses, dates on and off public assistance, the grant amount, the type of assistance, the case workers' names and the reasons for being on welfare.\n\nWhat the reader didn't see were the hundreds of hours spent inputting reams of data, the panic when software problems ate thousands of records or the cajoling that went into getting government bureaucrats to hand over computer information free of cost.\n\nBecause the lettering was too small to scan - we tried - we had to spend hundreds of hours inputting the data into a DBASE 4, version 1.5, file. We have been using DBASE for the past three years.\n\nOn at least four occasions, I was entering the 500th name, when all of a sudden, the computer froze while in the middle of the date field. Cold panic struck. The only way out was to hit the control/alt/delete buttons and reboot the system. We lost all 500 records and hours of work.\n\nWhen this problem first happened, we thought it was the keyboard. So we changed to a new one. It happened a second time in the same location: The date field. Again the records were lost.\n\nWe switched computers, thinking it was a hardware problem. Finally, after the fourth time we called the Borland Hotline to see if it was a software problem. Borland purchased DBASE from Ashton Tate.\n\nThey suggested we had a \"corrupt header\" - the computer equivalent of scrambled eggs. The code that tells the computer where the fields are within the database had somehow become corrupted, throwing everything out of whack.\n\nThe only way around the problem was to convert the file from DBASE into ASCII format. Once that was done, the file had to be transferred back into DBASE. We also purchased a program called DSALVAGE, designed to clean up problems like the one we kept experiencing.\n\nThis problem did not always happen when I was entering data into the computer. It struck again when we transferred 500,000 records from the newsroom 386. At first the data was fine. But as we began to cut and slice, we appeared to lose half our files.\n\nFortunately, we backed our data up on a tape and were able to try again. Borland has still not been able to explain the inputting problem. We are now looking at other databases like FoxPro for Windows.\n\nOnce we had the welfare lists computerized we were able to match it with other databases. We knew from a previous computer-assisted project that people collecting workers' compensation often failed to report the income to the welfare department. As a result, they could earn a very good wage tax free at the expense of the taxpayers.\n\nI spent four months negotiating with the Industrial Accident Board Commissioner to give me a computerized tape of all workers' compensation cases.\n\nAt first, the state wanted to give me just the names, dates of birth, insurance company, business, accident date, board identification number and settlement of everyone who filed a claim at the IAB. The names were worthless. I wanted the social security numbers. They were the one unique field I needed to conduct matches.\n\nI argued that the registry of state motor vehicle uses social security numbers on most drivers' licenses. And many banks put social security numbers on deeds.\n\nThey agreed to give me the social security number if it did not come with the name and date of birth. Why not? We had all the names we needed in our welfare database. It worked.\n\nOnce the match was done, I gave the state a list of names and board identification numbers and asked for paper copies to check my findings. They had no idea what I was trying to find. The paper records gave us something to fall back on and double check our findings.\n\nWe found Greater Lawrence businesses paid more than $1.5 million in settlement claims in the last two years to welfare recipients. We also found 99 percent never declared the outside income to the welfare department as required by law.\n\nWhen we asked the welfare commissioner why the state never performed such matches with workers' compensation records, he told us the two computers were not compatible. The state compensation was not where most fraud occurred.\n\nAfter our series came out, the state started performing matches and found thousands of double dippers trying to beat the system.\n\nAnother ripe area was prisons.\n\nWe knew from the Essex County Sheriff that inmates in the country jail were collecting welfare benefits behind bars on a somewhat routine basis.\n\nFour years ago, an inmate told The Eagle-Tribune how easy it was to collect welfare while serving a prison sentence. The state keeps track of prisoners through the Criminal History Systems Board. But records cost $25 per inmate, making that cost exclusive.\n\nWe did have, however, our own criminal systems history board. Several years ago, I asked our court reporter to save disks. I converted the files into a database making a match possible.\n\nIn two separate interviews both the Massachusetts welfare commissioner and the state's top fraud prosecutor told us that computer matches were impossible. They both cited the lack of a central registry. Our series showed them how to perform a match.\n\nThe fallout from the series has been tremendous. U.S. Senator Herbert Kohl, D-Wisconsin, asked for a copy. Two members of Congress are proposing legislation banning inmates from collecting Social Security while behind bars.\n\nGov. William F. Weld came to the newsroom and revoked a law making it illegal for welfare officials to provide U.S. Immigration and Naturalization with information to apprehend criminally convicted aliens. And welfare reform has a good chance of getting through the state legislature.\n\nIf you would like more information on deciphering welfare codes, welfare data in general, privacy issues we dealt with, the pitfalls of DBASE or a copy of our series, please call me at The Eagle-Tribune in Lawrence. My phone number is (508) 685-1000, ext. 189.\n\nDo you think readers would like to know if a well-paid foreign consul general also receives welfare benefits? Or if a sanitation worker walked into a Boston bank and used his Massachusetts welfare card to cash a $25,000 disability check? And never declared the outside income to welfare? Or if a man is legally entitled to receive a $29,000 Social Security check while serving a two-year prison sentence? Or what welfare dependency means to two women who have collected public assistance for nearly 20 years each? Stories like these and a dozen others helped bring the complex subject of welfare and welfare abuse into focus when The Eagle-Tribune in Lawrence published a five-part computer-assisted series on the Massachusetts Welfare System in September. The series found Massachusetts welfare officials doing little or nothing to prevent unscrupulous people from stealing tens of millions of dollars from taxpayers and the poor. Daily circulation rose 1,000 papers above normal during the series. And requests from across the country keep trickling into the newsroom. But the juicy stories were just one small part of a long and complicated project.\n\nThe names of welfare recipients have been protected under state and federal privacy laws since the Great Depression. So when two years of welfare rolls for the Greater Lawrence area landed in our laps we started to computerize them.\n\nComplicating matters was a 50-year-old law that makes it a criminal offense for anyone to possess welfare rolls or use them to derive statistics, and the task of deciphering the welfare department's code.\n\nTwo sets of lawyers told us the law was unconstitutional. And when a file layout was sent to us in the mail, the code was easily understood.\n\nWelfare lists contain names, social security numbers, addresses, dates on and off public assistance, the grant amount, the type of assistance, the case workers' names and the reasons for being on welfare.\n\nWhat the reader didn't see were the hundreds of hours spent inputting reams of data, the panic when software problems ate thousands of records or the cajoling that went into getting government bureaucrats to hand over computer information free of cost.\n\nBecause the lettering was too small to scan - we tried - we had to spend hundreds of hours inputting the data into a DBASE 4, version 1.5, file. We have been using DBASE for the past three years.\n\nOn at least four occasions, I was entering the 500th name, when all of a sudden, the computer froze while in the middle of the date field. Cold panic struck. The only way out was to hit the control/alt/delete buttons and reboot the system. We lost all 500 records and hours of work.\n\nWhen this problem first happened, we thought it was the keyboard. So we changed to a new one. It happened a second time in the same location: The date field. Again the records were lost.\n\nWe switched computers, thinking it was a hardware problem. Finally, after the fourth time we called the Borland Hotline to see if it was a software problem. Borland purchased DBASE from Ashton Tate.\n\nThey suggested we had a \"corrupt header\" - the computer equivalent of scrambled eggs. The code that tells the computer where the fields are within the database had somehow become corrupted, throwing everything out of whack.\n\nThe only way around the problem was to convert the file from DBASE into ASCII format. Once that was done, the file had to be transferred back into DBASE. We also purchased a program called DSALVAGE, designed to clean up problems like the one we kept experiencing.\n\nThis problem did not always happen when I was entering data into the computer. It struck again when we transferred 500,000 records from the newsroom 386. At first the data was fine. But as we began to cut and slice, we appeared to lose half our files.\n\nFortunately, we backed our data up on a tape and were able to try again. Borland has still not been able to explain the inputting problem. We are now looking at other databases like FoxPro for Windows.\n\nOnce we had the welfare lists computerized we were able to match it with other databases. We knew from a previous computer-assisted project that people collecting workers' compensation often failed to report the income to the welfare department. As a result, they could earn a very good wage tax free at the expense of the taxpayers.\n\nI spent four months negotiating with the Industrial Accident Board Commissioner to give me a computerized tape of all workers' compensation cases.\n\nAt first, the state wanted to give me just the names, dates of birth, insurance company, business, accident date, board identification number and settlement of everyone who filed a claim at the IAB. The names were worthless. I wanted the social security numbers. They were the one unique field I needed to conduct matches.\n\nI argued that the registry of state motor vehicle uses social security numbers on most drivers' licenses. And many banks put social security numbers on deeds.\n\nThey agreed to give me the social security number if it did not come with the name and date of birth. Why not? We had all the names we needed in our welfare database. It worked.\n\nOnce the match was done, I gave the state a list of names and board identification numbers and asked for paper copies to check my findings. They had no idea what I was trying to find. The paper records gave us something to fall back on and double check our findings.\n\nWe found Greater Lawrence businesses paid more than $1.5 million in settlement claims in the last two years to welfare recipients. We also found 99 percent never declared the outside income to the welfare department as required by law.\n\nWhen we asked the welfare commissioner why the state never performed such matches with workers' compensation records, he told us the two computers were not compatible. The state compensation was not where most fraud occurred.\n\nAfter our series came out, the state started performing matches and found thousands of double dippers trying to beat the system.\n\nAnother ripe area was prisons.\n\nWe knew from the Essex County Sheriff that inmates in the country jail were collecting welfare benefits behind bars on a somewhat routine basis.\n\nFour years ago, an inmate told The Eagle-Tribune how easy it was to collect welfare while serving a prison sentence. The state keeps track of prisoners through the Criminal History Systems Board. But records cost $25 per inmate, making that cost exclusive.\n\nWe did have, however, our own criminal systems history board. Several years ago, I asked our court reporter to save disks. I converted the files into a database making a match possible.\n\nIn two separate interviews both the Massachusetts welfare commissioner and the state's top fraud prosecutor told us that computer matches were impossible. They both cited the lack of a central registry. Our series showed them how to perform a match.\n\nThe fallout from the series has been tremendous. U.S. Senator Herbert Kohl, D-Wisconsin, asked for a copy. Two members of Congress are proposing legislation banning inmates from collecting Social Security while behind bars.\n\nGov. William F. Weld came to the newsroom and revoked a law making it illegal for welfare officials to provide U.S. Immigration and Naturalization with information to apprehend criminally convicted aliens. And welfare reform has a good chance of getting through the state legislature.\n\nIf you would like more information on deciphering welfare codes, welfare data in general, privacy issues we dealt with, the pitfalls of DBASE or a copy of our series, please call me at The Eagle-Tribune in Lawrence. My phone number is (508) 685-1000, ext. 189.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-april-coming-events","slug":"1994-april-coming-events","url":"https://dwillis.github.io/uplink/article/1994-april-coming-events.html","year":1994,"month":"April","monthNum":4,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A listing of upcoming journalism training and conference events for 2024, including NICAR seminars, IRE conferences, and computer-assisted reporting workshops scheduled across multiple U.S. locations.","keywords":["conferences","training","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nMay 15-20 NICAR Training Seminar Columbia, Missouri June 16-19 IRE National Conference St. Louis, Missouri August 14-19 NICAR Training Seminar Columbia, Missouri October 6-9 IRE Computer-Assisted Reporting Conference San Jose, California","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-april-lets-educate-the-public","slug":"1994-april-lets-educate-the-public","url":"https://dwillis.github.io/uplink/article/1994-april-lets-educate-the-public.html","year":1994,"month":"April","monthNum":4,"headline":"Let's educate the public","kicker":"Letter from an alumnus","author_name":"Adam Berliant","author_title":"Reporter","affiliation":"The Tacoma News Tribune","summary":"Journalists face growing resistance and new barriers to accessing government databases, with officials citing privacy and ethical concerns to delay or deny requests that were previously routine under freedom of information laws.","keywords":["databases","access","privacy"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"Reporters don't want to admit it, but there is a growing uneasiness among record-keepers, legislators and worst of all, the general public, that a person's ability to obtain government databases is not the greatest thing in the world.\n\nThis uneasiness, which goes way beyond legal issues and FOIA technicalities, could soon turn into a crisis for computer-assisted reporters.\n\nTwo examples from here in Washington: I requested court records from our Office of the Administrator of the Courts (OAC), which makes individual court records available without difficulty in every corner of this state. Court records are so readily available, you can access them from your home on-line!\n\nBut when I requested a copy of the raw data, new laws suddenly started materializing. The OAC denied our original request based on judges' right to privacy (\"huh?\" did you say?). The OAC's data dissemination board backed away from this absurd notion later. They said what their denial really meant was that our access to the entire database raises new ethical and privacy dilemmas not considered by the creators of the original freedom of information laws. An entire database is different, they claimed, and new dissemination policies have to be created. (P.S.: They also said they worried about the commercial use of the data.)\n\nSo now the OAC's administrators are talking to record-keepers, record-keepers are talking to judges, judges are talking to lawyers, lawyers are talking to other lawyers, meetings are being held, advisors are being called, new policies are being written and in the meantime court officers are scrambling for any reason they can conjure up to delay my access to the information.\n\nThis has gone on for months and despite a great deal of rigorous support from my newspaper, our attorneys, local advocacy groups and even other news organizations (including the Seattle Times), no progress has been made.\n\nThe OAC insists the delays and tactics are all in the public's interest. The people don't want you to have this data, they claim.\n\nEven though I think they're just trying to cover their behinds, upon casually discussing this situation with friends over beers, basketball games, lunches, etc., I am forced to agree with the OAC - some people do see access to computer databases as a threat. Some really think a new set of dissemination ethics is needed, even though they recognize that the same information is already available in conventional formats. Reporters, me included, fall into shock upon hearing such a thing.\n\nAnother example: I obtained a copy of Washington's gun registration database. This file contained demographic and product information about every gun owner in the state. We used the data to run a two-part series on the rise of gun popularity in Washington. Beyond the usual disgust from firearm-rights advocates, whose contempt we would expect even if we had a headline reading 'Handguns are Tops!', there was another, more disturbing development.\n\nIn a world where people are inundated with junk mail, telephone marketing and other computer-generated hassles, many find computerized information despicable.\n\nUpon reading the series, a state representative proposed new legislation in Olympia to specifically prevent the further release of the database. Crooks, the representative said, would obtain the database to get the names and addresses of people who own the particular brand and caliber of firearm they would most like to steal. He called the database a 'shopping list' for hoods who would break into your house for the purpose of obtaining your gun (did I hear you say \"huh?\" again?).\n\nOnce again, the representative was claiming that 'the people' need to be protected. The representative was not laughed out of the state, nor was his proposal. In fact, the House included portions of his proposal in its crime bill, and it passed. Now we're crossing our fingers for a governor's veto.\n\nThe point is this: There are politicians who are crying out for new public information restrictions designed specifically for computerized information.\n\nAnd what's most disturbing is that I think if they ever asked for public support - if there were a vote tomorrow - I think restrictions would win.\n\nReporters, meanwhile, continue to act very sure of themselves - these are records, they are public records, they are available under FOI laws. We act as though because the laws currently say we can get away with it, there is public support for the notion.\n\nBut are we really sure the rest of society agrees with us? I can tell you, many don't. In a world where people are inundated with junk mail, telephone marketing, electronic billing and other computer-generated hassles, many find computerized information despicable.\n\nSo what are news organizations doing about it? These days we are so busy arguing that current FOI laws protect our rights, we've forgotten to ask who is protecting the current FOI laws.\n\nIt's true that groups like the Reporter's Committee for Freedom of the Press are busy lobbying in government. But for what? At best, they argue for the continued acknowledgement that FOI laws are meant to include computerized information - or some version of that point. All of this 'politics' occurs without much consideration about what the public actually wants or thinks is fair.\n\nI think what we need is a good, honest, hard-hitting PR campaign.\n\nThe most a news organization ever does to tout the values of computer-assisted reporting is to mention it in a story: \"A (your news organization here) computer analysis of (your database here), a computer file containing hundreds of thousands of records concerning (fill-in-the-blank) showed that (nut graph).\"\n\nNot only is this line always included in computer-assisted reporting stories, but usually, we put in as if the reader will respond, \"Gee, aren't computers great!\"\n\nNICAR's Uplink newsletter, which I helped create, is a worst case example of this. We constantly congratulate ourselves over our new found resourcefulness. But we're not the ones that need convincing.\n\nIt's pointless to continually stress that stories were made possible with the use of computerized information. What we need to stress instead is that the stories would have been impossible without it - and we need to do this in places other than our own newsprint or broadcasts.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-april-eeoc-data-offers-another-nugget","slug":"1994-april-eeoc-data-offers-another-nugget","url":"https://dwillis.github.io/uplink/article/1994-april-eeoc-data-offers-another-nugget.html","year":1994,"month":"April","monthNum":4,"headline":"EEOC data offers another nugget","kicker":"Bits, Bytes and Barks","author_name":"Gwen Carleton","author_title":"Staff","affiliation":"Wall Street Journal","summary":"The Wall Street Journal obtained complete EEOC employment data through multiple FOIA requests, revealing that Black workers experienced unique job losses during the 1990-91 recession and that Black women's corporate advancement significantly outpaced Black men's between 1982 and 1992.","keywords":["employment","discrimination","workforce"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":[],"full_text":"Persistence is paying off for the Wall Street Journal. Last month we reported that, after the paper requested Equal Employment Opportunity Commission (EEOC) data, the reports arrived incomplete. It took three separate Freedom of Information Act requests to obtain the missing information. But with the completed files, Journal reporters found that among ethnic groups, only blacks suffered a net job loss in the 1990-91 recession. In a story once again using EEOC data, the Journal recently reported that black women's gains in corporate America dramatically outstripped those of black men between 1982 and 1992. The information was gleaned from an analysis of 1.7 million EEOC reports, which included 1992 reports covering about a third of the civilian workforce. Considering that the EEOC reports include data on the race, gender and type of job held by 42 million U.S. workers, this probably isn't the last we've seen of them.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-april-bulletin-board-surfing-pays-off","slug":"1994-april-bulletin-board-surfing-pays-off","url":"https://dwillis.github.io/uplink/article/1994-april-bulletin-board-surfing-pays-off.html","year":1994,"month":"April","monthNum":4,"headline":"Bulletin board surfing pays off","kicker":"Bits, Bytes and Barks","author_name":"Mike McGraw","author_title":"","affiliation":"","summary":"An investigation into FDA mishandling of contaminated food from a warehouse fire in Kansas City reveals that officials awarded special citations to the very employees responsible for allowing unsafe food to reach consumers worldwide.","keywords":["FDA","contamination","scandal"],"topics":["Investigative Methods","Health & Environment","Government & Politics"],"technologies":["FedWorld Gateway"],"full_text":"We had just spent six months looking into a classic federal government screw-up. The FDA was supposed to insure that all food contaminated in a toxic warehouse fire got sent to local landfills in Kansas City, Kan. Instead, thousands of pounds of food that was either clearly contaminated or most likely contaminated was sent to consumers worldwide. What's more, the FDA allowed laboratories hired by food salvagers to guarantee that the food they released was safe. After analyzing how the FDA mishandled the matter, we found the story was still missing something. Then, as I was rummaging around inside the FDA's electronic bulletin board (which reached through the FedWorld Gateway), I found what we needed. \"In Kansas City, 18 people received the Commissioner's Special Citation for their salvage efforts during an underground cave fire FDA Kansas district office investigators reviewed reconditioning plans presented by firms whose products were adulterated in the fire. So turned out that the guys who screwed up medal for the mess. Hell of a kicker. They wrote it for us.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-april-car-reporters-are-getting-more-popular","slug":"1994-april-car-reporters-are-getting-more-popular","url":"https://dwillis.github.io/uplink/article/1994-april-car-reporters-are-getting-more-popular.html","year":1994,"month":"April","monthNum":4,"headline":"CAR reporters are getting more popular","kicker":"Bits, Bytes and Barks","author_name":"Uplink editors","author_title":"editor","affiliation":"Uplink","summary":"Several journalism staffing changes occur as reporters move between major newspapers, including Dan Browning's move to the St. Paul Pioneer Press and David Armstrong's transfer to the Boston Globe, while The Hartford Courant expands its computer-assisted reporting program.","keywords":["journalists","newspapers","relocations"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Dan Browning (author of last month's food inspection article) has headed north from the St. Louis Post-Dispatch to the St. Paul Pioneer Press. David Armstrong has gone crosstown from the Boston Herald to the Boston Globe and his partner, Alan Levin, took Brant Houston's spot at The Hartford Courant. Actually, The Courant has expanded its CAR program by adding John Moran, a long-time reporter there, as its online reporter. (This issue's quiz: How many online reporters are there in the country? Find out next issue.) And, of course, Jennifer LaFleur of the San Jose Mercury News joined NICAR March 28.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-uplink-update","slug":"1994-may-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-may-uplink-update.html","year":1994,"month":"May","monthNum":5,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"National Institute for Computer-Assisted Reporting","summary":"The National Institute for Computer-Assisted Reporting announces expansion efforts including upgrades to its data library, planned seminars, and a computer-assisted day at the upcoming IRE conference in St. Louis. The newsletter invites journalists to submit CAR-inspired stories and offers resources on data storage and banking investigations.","keywords":["reporting","data","seminars"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":[],"full_text":"We're forging ahead at the National Institute for Computer-Assisted Reporting as equipment, data and requests for seminars pour in.\n\nIn this expanded issue we will report on new developments at the institute that include the upgrading our data library, plans for the optional computer-assisted day at the national IRE conference in St. Louis in June and our increasing number of on-the-road seminars.\n\nThis issue also has an overview of how to store all that data that some of you are accumulating, a tech tip from our academic adviser Richard Mullins and two looks at probing banking practices.\n\nIn the past few weeks, we've received a number of excellent CAR-inspired stories. Please send us your articles or videos as soon as they appear so we can put them in our resource center and use them at our seminars and conferences.\n\nAnd if you have any CAR-related questions or suggestions, call us at (314)882-0684 or e-mail us at jourjbh@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-new-advanced-seminars-planned-road-shows-launched","slug":"1994-may-new-advanced-seminars-planned-road-shows-launched","url":"https://dwillis.github.io/uplink/article/1994-may-new-advanced-seminars-planned-road-shows-launched.html","year":1994,"month":"May","monthNum":5,"headline":"New advanced seminars planned / Road shows launched","kicker":"","author_name":"Jennifer LaFleur","author_title":"Training Director","affiliation":"NICAR","summary":"NICAR is expanding its training programs by offering on-site computer-assisted reporting seminars across the country, in addition to its Missouri-based courses, while also providing data analysis services and developing advanced training seminars.","keywords":["training","data","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["9-track tape","CD-ROM","spreadsheet","floppy disk"],"full_text":"NICAR has taken its show on the road. In addition to its week-long seminars held at the University of Missouri, NICAR is offering on-site training for news organizations throughout the country.\n\nNICAR kicked off its traveling program in the great white north with one-day training programs for the Alaska Press Club, the Juneau Press Club and the Anchorage Daily News in April. We've also conducted training overviews for a regional conference in La Crosse, Wisconsin, and the Kansas City Star.\n\nOur next stops include Los Angeles, San Francisco and Sacramento with more penciled on our calendar, including appearances in Philadelphia, Nashville, Atlanta, and Evansville.\n\nThe on-site training ranges from a basic overview on computer-assisted reporting to hands-on training for a variety of software, including databases, spreadsheets, and statistics. The program can be tailored to your specific training needs. The cost of the program depends on its complexity and whether we provide hands-on training. Smaller news organizations or associations may pool resources to bring a seminar to their area.\n\nFor those of you in warmer climates, please consider having NICAR visit November through March as we are not yet used to Missouri winters.\n\nFor more information about our on-site training programs, call NICAR training director, Jennifer LaFleur at (314) 882-0684.\n\nData library and analysis\n\nWith the arrival of two number-crunching computers, new storage devices, and a CD writer, NICAR is ready to do data analysis and serious slicing and dicing of data for news organizations, large and small.\n\nFor a small fee, we can download data from a nine-track tape, carve out just the information you need, and send it to you on diskettes. We also can help with data analysis and technical problems. At the same time, we already are collecting data and increasing the catalog of information we have. Give us a call if you think we may have the data you need. If we don't, we may be able to acquire it inexpensively and get you the information quickly.\n\nAdvanced seminars\n\nNICAR is starting to develop advanced seminars. We are hoping to offer one in early August and are considering covering how to use statistics for news stories, mapping, advanced cleaning of data, and some basic programming. Please let us know what you think is needed for such a course.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-coming-events","slug":"1994-may-coming-events","url":"https://dwillis.github.io/uplink/article/1994-may-coming-events.html","year":1994,"month":"May","monthNum":5,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A listing of upcoming journalism conferences and training events including the IRE National Conference, NICAR Training Seminar, and IRE Computer-Assisted Reporting Conference scheduled across multiple cities in 2024.","keywords":["conferences","journalism","training"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nJune 16-19 IRE National Conference St. Louis, Missouri August 14-19 NICAR Training Seminar Columbia, Missouri October 6-9 IRE Computer-Assisted Reporting Conference San Jose, California","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-data-only-part-of-hmda-story","slug":"1994-may-data-only-part-of-hmda-story","url":"https://dwillis.github.io/uplink/article/1994-may-data-only-part-of-hmda-story.html","year":1994,"month":"May","monthNum":5,"headline":"Data only part of HMDA story","kicker":"Making sense of it all","author_name":"Seth Hamblin","author_title":"NICAR Staff","affiliation":"NICAR","summary":"Reporters across the country are using computer-assisted analysis of Home Mortgage Disclosure Act data to uncover significantly higher loan-denial rates for minorities than white applicants, though bankers dispute the findings and attribute disparities to credit and financial factors rather than discrimination.","keywords":["lending","discrimination","mortgages"],"topics":["Investigative Methods","Business & Economy","Data & Databases"],"technologies":["SPSS","FoxPro"],"full_text":"Reporters across the country are turning up the heat on banks with computer-assisted stories that show a much higher loan-denial rate for minorities than caucasians. But the tale told by Home Mortgage Disclosure Act data isn't as clear as black and white, reporters say, and bankers are claiming it's more a matter of green.\n\nIn St. Louis, a Jan. 10, 1993, HMDA analysis by George Landau of the Post-Dispatch, who used SPSS statistical software, found that blacks were being rejected for home-purchase loans 2.5 times more often than whites. In Detroit, Dan Gillmor of the Free Press reported in a Jan. 17 story that the loan denial ratio was 2.1 to 1.\n\nAnd figures for the entire country show only a slightly lower lending gap.\n\nThe analysis is so important and so easy to do, Gillmor said, that it should be attempted every year in every community. \"The banks know the stories are being done every year, and they're definitely making more of an effort towards minorities,\" he said. Due to media attention and stronger home mortgage regulations, Detroit lenders are now reviewing denials of minority applicants, hiring minority loan officers and opening offices in black neighborhoods, Gillmor said.\n\nHowever, banks say these numbers don't tell the whole story. They claim their decisions are based on wise banking practices, not discrimination. A higher percentage of black applicants are rejected because they have poorer credit histories, fewer assets and less job stability than white applicants, lenders say.\n\n\"We deny any overt discrimination,\" said a banker in Gillmor's story. \"The data tell us there's a problem, but reasonable people disagree as to the extent and reason for the problem.\"\n\nBut Orange County Register reporter Ronald Campbell said banks are contradicting themselves when they deny claims of racism.\n\n\"They're saying, it's not as bad as you claim, but we'll do better next year,\" he said. Campbell said that with affluent blacks and Hispanics getting turned down more often than poorer whites in Orange County, Calif., the numbers clearly point to lender discrimination. His Nov. 29, 1992, analysis showed that blacks and Hispanics with high incomes ($75,000 or greater) have denial rates of 26 and 27 percent, respectively, while whites with much lower incomes (under $30,000) have a denial rate of only 23 percent.\n\nAlthough the database includes fields for an applicant's race and income as well as loan amount, lenders aren't required to fill in a field that gives a denial reason, and in many cases, don't. Because of this and other inconsistencies, reporters must recognize the limitations of the data, Gillmor said.\n\n\"What this data is good for is raising questions,\" he said. \"It's not a smoking gun, but it gives community activists something to work with.\"\n\nHMDA records are made available from the Federal Reserve every November and also can be acquired through NICAR. A nationwide database, HMDA is huge, with last year's data containing more than 12 million records from 9,072 lending institutions. But HMDA can easily be whittled down to a smaller geographic area using fields for county code and census track numbers. From the 1992 data, Gillmor chose seven counties in the Detroit area with about 229,000 loan applications.\n\nHMDA has two datasets, one with loan information and the other with a list of lenders and their ID codes. Gillmor used FoxPro for Windows to combine the two datasets and produce smaller more manageable subsets from the more than 30-megabyte database. He then imported the data into SPSS for windows in order to use the program's more sophisticated cross-tabbing functions.\n\n\"It was really simple, but there are a couple of little tricks,\" Gillmor said. Numbers in the census tract field came through as six-digits without a decimal. Gillmor had to create a new field and divide by 100 in order to get an accurate tract number.\n\nIn his story, Campbell combined census tract and HMDA data using FoxPro to look for instances of redlining, the practice of drawing a line on a map around poor or minority areas and denying loans on property within the boundary.\n\nNowhere did Campbell find clear-cut instances of redlining, where banks denied lending entirely. However, he did find that 18 of the 21 census tracts with denial rates greater than 30 percent were in mostly Hispanic areas.\n\nCampbell also ran into an unusually high denial ratio in a type of area he wasn't looking for. In ritzy Newport Beach, where houses had sold for $750,000 during a 1989 housing boom and two years later dropped in value by one-third, lenders refused a large number of home-purchase and refinancing loans. This could be a story in itself, Campbell suggested.\n\nLooking at St. Louis records, Landau used a HMDA field that identified loan type to isolate a subset just for home-improvement loans. He found that in 14 of the city's 400 census tracts, more than half the black applicants were turned down. In only one of the tracts did the rejection rate for whites top 50 percent.\n\nThe numbers demonstrate that banks often refuse to lend money to renovate low-value property in minority areas, Landau said. This creates a viscous circle, where the neighborhoods that most need improvement dollars can't get them.\n\n\"It can have a devastating effect on a neighborhood,\" he said. \"It's like cutting off life support.\"\n\nHMDA records for home-improvement loans may also be a good place to jump into stories that don't center on race. Landau suggested doing a trend story detailing where the most new homes are being bought and where most restoration is being undertaken.\n\n\"You could use the data to see what neighborhoods are making a comeback,\" he said.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-hmda-data-pitfalls","slug":"1994-may-hmda-data-pitfalls","url":"https://dwillis.github.io/uplink/article/1994-may-hmda-data-pitfalls.html","year":1994,"month":"May","monthNum":5,"headline":"HMDA data pitfalls","kicker":"","author_name":"David Washburn","author_title":"reporter","affiliation":"Allentown Morning Call","summary":"Reporter David Washburn outlines key pitfalls to avoid when analyzing HMDA mortgage lending data, including proper data cleaning, appropriate tools, and verification strategies.","keywords":["HMDA","analysis","verification"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["spreadsheet"],"full_text":"While HMDA is in many ways a simple database, there are still many things to watch out for, said Allentown Morning Call reporter David Washburn. Here are a few of the suggestions he made for avoiding pitfalls:\n\nUse an application that can generate crosstabs. Spreadsheets and database managers don't have enough power to create quality crosstabs for HMDA analysis.\nA field called \"edit status\" tells whether there is an error in the record, but it doesn't say where the error is. Records that are marked as having errors should be thrown out.\nSome of the records have N/A in the fields for race or income. These records must also be thrown out.\nSeparate one-to-four family dwellings from multifamily and commercial property when generating percentages.\nWhen combining HMDA and census data, if your census records are broken down into blocks, you will have to spend some time adding them together into tracts in order to compare them with HMDA records.\nLook out for different banks with the same ID number. If two banks merge after the reporting period, they will be reported as one institution although they were separate entities at the time the data were generated.\nDouble check your numbers with each bank. If their numbers are different, make sure they're breaking down the data the same way you are. For instance, if you're looking at just one-to-four family homes, make sure their numbers don't reflect multifamily dwellings as well.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-mapping-reveals-data-trends","slug":"1994-may-mapping-reveals-data-trends","url":"https://dwillis.github.io/uplink/article/1994-may-mapping-reveals-data-trends.html","year":1994,"month":"May","monthNum":5,"headline":"Mapping reveals data trends","kicker":"Pictures better than words","author_name":"Ronald Campbell","author_title":"","affiliation":"The Register","summary":"A journalist uses computer mapping and database analysis to investigate whether banks are underserved in poor neighborhoods, discovering that statistical data initially contradicted their intuition until they employed geocoding techniques to visualize the distribution patterns.","keywords":["banks","mapping","neighborhoods"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["FoxPro","MapInfo"],"full_text":"Databases are wonderful tools, but sometimes you've got to have a picture to tell the story. When The Register out to document whether bankers were serving poor neighborhoods, the raw numbers told us they were. Intuition told us they weren't. A mapping program proved our intuition correct.\n\nFor years my editor and I have speculated about the paucity of banks in Santa Ana, a heavily Hispanic city of 300,000. Orange County has hundreds of banks; in some neighborhoods they are clustered as thickly as gas stations once were. But not in Santa Ana.\n\nWe both thought a computer mapping program could help us document the pattern. But then we ran into a barrier, and it stopped us cold. The problem was data entry. It wouldn't be enough to locate all the branches in Santa Ana; to make a valid comparison, we had to find all the branches in neighboring cities. Finding the raw data was easy enough: The state banking and savings-and-loan associations each printed annual directories. But some poor schmuck (most likely me) would have had to type all those names and addresses into a database. Then I (no alternatives this time) would have had to plot the branches on a computer map, a process known as geocoding.\n\nThe idea of geocoding several hundred banks was so terrifying that I stopped thinking about it for a year.\n\nBut late in 1993, banking writer Dawn Yoshitake discovered a database maintained by Sheshunoff Information Services. It also let us look beyond Orange County to examine statewide patterns. Finally, and maybe most importantly, it sidestepped geocoding.\n\nWith Sheshunoff, I did not have to worry about geocoding. I could locate banks by their census tract or zip code.\n\nGeocoding is a messy business. Most databases that I have used contain a cacophony of conflicting address styles. Is it \"625 N Grand Ave\" or \"625 North Grand Avenue\" or maybe \"625 N. Grand Ave.\"? To the database user, it hardly matters. But in mapping software, the distinction between \"N\" and \"North\" or \"Ave\" and \"Avenue\" is a vital one; every address must match the internal style of the computer map. And since computer maps often omit valid addresses, even a correctly styled address may not geocode.\n\nMy first step was to look for statistical patterns. I loaded the Sheshunoff database into FoxPro. After hand-eliminating hundreds of credit unions (listed only once in the database, regardless of the number of branches), I used FoxPro to count the number of bank branches in each of Orange County's 484 census tracts. Then I compared that with income data. I assigned each tract to an income range (low, low-moderate, upper-moderate and upper), based on that tract's median family income. Finally I compared the number of bank branches serving neighborhoods of a given income range with the number of people living in those tracts.\n\nSurprise! It appeared that bankers were about as likely to set up shop in a lower-middle class barrio as in a glitzy neighborhood overrun with BMWs. The numbers in Orange County looked especially strange after I reviewed the statewide numbers. The statewide analysis, done by zipcode instead of census tract, showed that bankers avoided lower-middle-class areas.\n\nThen I dumped the data into my mapping program, MapInfo, and the county pattern suddenly became clear. Bankers willingly put branches in poor or minority neighborhoods - but only if those neighborhoods bordered wealthier, whiter neighborhoods. When MapInfo shaded census tracts by the number of bank branches, the resulting map had a gaping hole in the middle. The hole corresponded with Orange County's largest and poorest barrio.\n\nThe map told a lot. Graphic artist Danny Sullivan found a way to make it tell even more. MapInfo census tables contain population and housing data; draw a line around several census tracts, and MapInfo will add the numbers together to produce statistics for the region. Danny joined the basic MapInfo table with a database table I had created, which included the median income, ethnicity and population for every census tract. Then he circled five areas on the county map. MapInfo calculated the average income (actually, the average of median family income for each tract in the selected area), as well as population and the number of bank branches for each of those areas.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-where-to-find-banking-info","slug":"1994-may-where-to-find-banking-info","url":"https://dwillis.github.io/uplink/article/1994-may-where-to-find-banking-info.html","year":1994,"month":"May","monthNum":5,"headline":"Where to find banking info","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Sheshunoff Information Services offers Bancpen, a comprehensive database containing geographic and financial data on all bank branches, savings-and-loans, and credit unions, available by state or national coverage at varying price points.","keywords":["database","banking","financial"],"topics":["Data & Databases","Business & Economy"],"technologies":["Bancpen"],"full_text":"Sheshunoff Information Services' Bancpen database includes geographic and financial summaries on every bank branch, every savings-and-loan, every credit union.\n\nFields include company name, branch number, street address, city, Zip code, county, state, Metropolitan Statistical Area (MSA) code number and census tract number. In addition, the database lists deposits (broken down between deposits by public agencies and deposits by individuals, partnerships and corporations) year by year for the past five years.\n\nSheshunoff sells the database on diskette for $595 per state. Tape versions go for $700. Orders for two or more states qualify for a 20-percent discount. The national database is sold for $5,000 on diskette or $5,500 on tape. To order, call (800) 456-2340.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-computer-conference-set-for-oct-6-9-1994","slug":"1994-may-computer-conference-set-for-oct-6-9-1994","url":"https://dwillis.github.io/uplink/article/1994-may-computer-conference-set-for-oct-6-9-1994.html","year":1994,"month":"May","monthNum":5,"headline":"Computer conference set for Oct. 6-9, 1994","kicker":"Beginning and advanced skills highlighted","author_name":"Jonathan Krim","author_title":"","affiliation":"San Jose Mercury News","summary":"The second annual IRE computer-assisted journalism conference will be held in Silicon Valley in October, offering hands-on training in various computer applications and technologies for reporters and editors of all skill levels.","keywords":["conference","journalism","training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["DOS","Windows","spreadsheet","Internet"],"full_text":"The second annual national IRE conference devoted to computer-assisted journalism will be held in the heart of Silicon Valley this October.\n\nWhether your publication or broadcast outlet is just beginning to use computer-assisted techniques or is ready for more sophisticated approaches, this year's conference will have something for you.\n\nSome of the top practitioners nationwide will serve as instructors, offering classroom training in which each participant will work at his or her own computer.\n\nConference participants will receive intensive, hands-on training in a variety of computer applications, including:\n\nBasic PC use, including DOS and Windows\nHow to use various spreadsheet programs\nHow to use relational database software to further manipulate information\nHow to download government tapes\nHow to use on-line services, including Internet\nHow to map data\nHow to use statistical analysis software\n\nThe conference, beginning Oct. 6 at 8 a.m. and ending at noon on Oct. 9, also will feature panel discussions on issues affecting computer-assisted journalism, such as legal matters, ethics, and using computers for beat reporting and quick-hit stories.\n\nBuilding on the strengths of the computer-assisted conference in Raleigh, the planners intend to expand advanced parts of the program while keeping a basic track for those who are just learning these techniques.\n\n\"This conference will be a great opportunity to learn to use the most basic and necessary tools for reporting,\" said Brant Houston, IRE's new computer-journalism director.\n\n\"Reporters and editors can no longer afford to ignore this area. After all, most information we will seek is or will be in electronic form.\"\n\nHosted by the San Jose Mercury News, the conference will also feature displays of products for computer-assisted journalism and a special demonstration of Mercury Center, the Mercury News' on-line newspaper service. A luncheon banquet, included with registration, will feature an exciting keynote speaker.\n\nThe conference site, the Westin Hotel in Santa Clara, features many amenities, including free parking, courtesy transportation from the San Jose airport and light-rail access to and from downtown San Jose.\n\nFor those who want to bring their families, the hotel is one block from the Great America theme and amusement park and is close to activities in San Jose, San Francisco and the rest of the Bay area.\n\nSpecial conference hotel rates are available, but registration will be limited. So, register now.\n\nSan Jose Registration\n\n(First-come, first served. Limited registration)\n\nName\n\nAffiliation\n\nAddress\n\nwork\nhome\n\nCity, State, Zip\n\nPhone\n\nPlease check:\n\n$150 for entire conference (I am an IRE member.)\n$180 for entire conference (I am not an IRE member, but I understand the fee includes the $30 membership cost.)\n$165 student fee for entire conference (I am not an IRE member, but I understand the fee includes the $15 membership cost.)\n$180 for entire conference (I am an IRE member but my membership will expire before the conference. Therefore, I would like to renew my membership at this time.)\n\nTOTAL\n\nAs much basic hands-on training as possible will be provided, but we cannot guarantee it to everyone. IRE limits overall registration to the conference.\n\nDo you want hands-on training? Yes No\n\nMake checks payable to IRE. Please mail to IRE/NICAR, University of Missouri School of Journalism, 100 Neff Hall, Columbia, Missouri 65211.\n\nHotel Registration\n\nWestin Hotel, Santa Clara\n\nWhen you call, say you are an IRE participant. Call (800) 228-3000 or (408) 986-0700 to make your hotel reservation.\nReservations made after Sept. 9 will be on a space-available basis, and rooms may cost more.\nHotel rate is $89, single; $99 double and $109 triple.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-the-trouble-with-storing-data","slug":"1994-may-the-trouble-with-storing-data","url":"https://dwillis.github.io/uplink/article/1994-may-the-trouble-with-storing-data.html","year":1994,"month":"May","monthNum":5,"headline":"The trouble with storing data","kicker":"","author_name":"Drew Sullivan","author_title":"Staff","affiliation":"NICAR","summary":"The article examines different data storage options for news organizations, comparing traditional tape storage with newer technologies like DAT cartridges and magneto-optical drives, discussing the trade-offs between cost, capacity, access speed, and reliability.","keywords":["storage","data","technology"],"topics":["Data & Databases","Technology","Journalism & Newsroom"],"technologies":["9-track tape","magnetic tape","Digital Audio Tape","DAT","Magneto-optical","WORM","CD-ROM"],"full_text":"The trouble with storing data\n\nBy Drew Sullivan, NICAR Staff\n\nIt's always difficult to decide what to do with all that great government data you've worked so hard to collect. If only we all had a 500GB hard drive. But a storage plan can be devised using new technologies that make it possible to store data safely and efficiently while still providing quick access to the most needed data. This involves deciding between magnetic, optical, magneto-optical and chip-based storage media. What's best for your organization depends on what trade-offs you are willing to make between money, space and time.\n\nGovernment agencies still store most data on 9-track tapes or 3480 tape carts. Tapes are inexpensive relative to their storage capacity (generally about 200 MB), but they can present some serious problems, especially for smaller news operations.\n\nFor example, tape are sensitive to temperature, magnetic fields and other environmental phenomena.\n\nTapes also are prone to \"read\" errors. If the machine that wrote the data has misaligned heads, your tape machine might not read it. Everyone who has used a 9-track machine is familiar with the problem of the tape spinning back and forth over one damaged or poorly written spot.\n\nA new form of tape cartridge on the market called the Digital Audio Tape (DAT) elimates many of the problems of 9-tracks and carts. A DAT is about the size of a standard audio cassette, costs a little more than an 9-track tape and can hold 8 GB or more of data.\n\nBecause DAT cartridges use a digital rather than analog storage, the alignment of heads isn't a problem. But DATs suffer from some of the same disadvantages of other tapes. The data is accessed serially. They are unstable, so copies should be made every couple of years. Also, damaging a small part of the tape will destroy large amounts of data.\n\nNevertheless, DAT's offer inexpensive storage for massive amounts of data. And if you don't mind slow access to your data, they make a good archival medium. DATs, at between $10 and $25 each, are cheap enough to make extra copies for offsite storage. DAT drives are available for about $2,500.\n\nMagneto-optical (MO) drives, a somewhat more exotic storage medium, show great promise for data storage. The drives use a rewriteable glass disk that resembles a large compact disk (CD). For about $130 these disks can store 1.2 GB of data. Although MO disks are more expensive than tapes, they can be accessed almost as quickly as a hard drive (19 milliseconds). An MO acts almost like a removeable hard drive. Data is not read serially, meaning the computer goes straight to the data you want.\n\nData on an MO disk is virtually uncorruptable and the disks themselves are hard to damage if handled correctly. Also available are WORM (write once read many) disks. Once written, data on WORM disks cannot be changed. This makes them useful for network systems where users can access data without the ability to alter or damage it.\n\nCost remains the biggest disadvantage of MOs. Drives sell for about $3,000 and the disks are expensive for long-term data storage. If you can afford it, it's the way to go.\n\nThe tapes themselves are large and difficult to store. If you don't have money for a climate-controlled room or even a large, safe storage area, 9-tracks constitute an inefficient long-term storage option.\n\nWriteable CDs also have gained ground as a viable medium for archiving data. Although they are not rewriteable, CDs are inexpensive (about $22), hold a lot of data (680 MB) and are virtually uncorruptable. With triple and quadruple speed CD drives on the market, access time has increased nearly to that of MOs.\n\nIn addition to space considerations, retrieving data from tapes is time-consuming. Computers access tape-stored data \"serially.\" If you envision your drive like a book, serial retrieval means the computer would have to read every page prior to the page you are interested in before it can access that data. Even with a good 9-track reader, you might have to wait 20 to 40 minutes to read the data you want.\n\nOnce a \"master\" has been created using a writeable CD drive, copies can be made for about $5 each, depending on the number needed. Still, CD writer drives are expensive ($3,500 - $5,800) but they are more cost-effective than they were a few years ago. CD \"jukeboxes,\" which operate much like their musical counterparts, allow ten or more CD's to be online at any one time providing an exceptional network data library.\n\nFlopticals are 3.5-inch floppies but they hold 21 MB of data on an optical disk. Access time is similar to that of a floppy. The drives are inexpensive (less than $350) but the disks are still pricey at $21.\n\nIn deciding among the above options, most organizations should consider the levels of storage needed. Frequently accessed data should be stored on hard drives, MOs or CDs. DATs or 9-track tapes are good for off-premises archival storage. By combining the two, a news outlet can continue to make use of the data they've collected and still have room to work.\n\nHave you had some good experiences with the hardware discussed here? Had some nightmares? Share your experiences by calling, e-mailing or writing us at NICAR.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-getting-your-program-to-talk-back","slug":"1994-may-getting-your-program-to-talk-back","url":"https://dwillis.github.io/uplink/article/1994-may-getting-your-program-to-talk-back.html","year":1994,"month":"May","monthNum":5,"headline":"Getting your program to talk back","kicker":"Tech tips","author_name":"Richard Mullins","author_title":"","affiliation":"Uplink","summary":"This article explains how to modify a database extraction program to prompt users for input instead of requiring manual code editing, making it more accessible to less experienced users.","keywords":["database","programming","queries"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["FoxPro"],"full_text":"Getting your program to talk back\n\nBy Richard Mullins\n\nOur last tech tip illustrated something you can do with a simple database program that you can't do with a query: jump through a table at a certain interval, say every 11th or 531st record, and then copy that record to another table. The extracted version would be a small version of the original. And if the records in the parent table were unordered, then the extracted records would be a genuinely random sample. When I made the program, I just needed a small slice of a large table. I wanted to test my queries on the small table, since they would execute faster, then do them on the larger table when I got them right. A shorter version of the program was printed in Uplink in the September 1993 issue. It worked fine, but you had to edit the variable parts of it - the Skip Interval, the Source Table Name, the Target Table Name - every time you wanted it to slice a different table to a different size and a different name. That can be just a slight inconvenience for a knowledgeable user, but a barrier to less experienced users. So we'll add some code to the program so it will ask the human at the keyboard what database to extract from and where to put the extracted records. The lines beginning with asterisks are comments. The program ignores them, so you don't need to type them for the program to work, but they're important to you. They help you remember how the program was written, how it works, and anything you'll need to know when you look at the program code a month or a year later.\n\nTo create a program file, go to the command window and type: modify command xtract\n\nThis opens up the Fox text editor. Type in the program as it's printed here. When you're done, pull down the FILE menu, choose CLOSE, and answer YES to the Save Changes Dialog Box.\n\nThe program begins with defining the variables to be used. We give them an empty value (zero, or a string of spaces) to start with:\n\nSet up the variables\nnExtract = 0\nTargetDB = space(30)\nSourceDB = space(30)\n\nThe next part puts up a prompt, a space to accept the answer and then stores that answer. The command to get user input is the SAY GET READ\" sequence. This and all other Fox commands can be looked up in the manual or in the on-screen help directory. But here is a brief explanation:\n\nThe At-sign (@) sets the screen position (row, column).\n\nThe part in quotes is the prompt.\n\nThe name after the keyword \"get\" is where the program stores the answer.\n\nThe two answers are plugged into the command to open the databases.\n\nThe \"alias\" lets you give the databases a temporary nickname.\n\n* Ask user for input\n@ 8,5 say \"Source DB? \" get SourceDB\nread\n* Open the Source database\nuse (SourceDB) in I alias source\n@10,5 say \"Target DB? \" get TargetDB\nread\n* create an empty Target Database\ncopy structure to (TargetDB)\n\nThe next part looks complicated, but it helps the user. Before asking how many records to extract, it helps to know how many records there are. Make sure you get all the nested parentheses typed in.\n\n* Now put up info for user and get input\n@12,5 say \"There are \" + alltrim(str(reccount(\"source\"))) + ;\n\" records in \" + SourceDB\n\nThe program works by skipping over a defined number of records, copying one, skipping the defined number, copying one, and so on to the end. Here the program displays the record count, then asks the user how many records should be extracted. Based on the answer, it figures the Skip Interval to yield that number.\n\n@13,5 say \"Extract how many records?\" get nExtract\nread\nnSkipInt = int ( reccount(\"source\") / nExtract)\n\nNow everything is ready to roll. It helps to let users know that something is really happening, so clear the screen and put up a message.\n\n@8,0 clear to 15,70\n@12,5 say \"Extracting records...\"\n* Here's where the work is done\nselect source\ncopy to (TargetDB) for mod( recno(),nSkipInt )=0\n\n@12,5 say \"Done.\"\n\nThe \"copy to\" line, roughly translated, says: \"Copy records to the Target database where the record number is evenly divisible by the Skip Interval the user typed in.\" Here's how the arithmetic works: The mod() function finds the remainder from dividing two numbers; in this case it's the internal record number divided by the Skip Interval. If the remainder is zero, then the record is copied to the Target table.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-bits-bytes-and-barks","slug":"1994-may-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1994-may-bits-bytes-and-barks.html","year":1994,"month":"May","monthNum":5,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"American University is offering a seminar on computers and federal government reporting, while NICAR highlights computer-assisted reporting stories and discounted software for journalists as the government pushes electronic mail adoption and proposes e-mail archiving rules.","keywords":["journalism","technology","government"],"topics":["Journalism & Newsroom","Technology","Government & Politics"],"technologies":["Nine Track Express"],"full_text":"Bits, Bytes and Barks\n\nAmerican U. plans CAR seminar\n\nJournalists covering the federal government may want to look into a seminar entitled \"Journalists, Computers and the Federal Government\" July 17-22 at American University. For more information contact Wendell Cochran at (703)276-5805 or via email at WC9923a@american.edu\n\nCAR stories in the NICAR library\n\nAfter reading Dan Browning's Uplink story (March issue) on St. Louis restaurant inspections, Devin Smith of The Olympian in Olympia, WA., sent us his work using a county restaurant inspection. Clips of both pieces are available from NICAR. Among other articles, NICAR has received CAR stories from the Charlotte Observer on food inspections, criminals in the classroom, and foster care. The Roanoke Times & World-News sent us stories on parking tickets and the Wisconsin State Journal mailed more stories on crime from its excellent ongoing series called \"City Of Hope.\" We also have the Akron Beacon Journal's Pulitzer Prize winning series \"A Question of Color.\"\n\nAttention NICAR shoppers\n\nNICAR is working to make software less expensive for journalists. Our first offer is Nine-Track Express, which we will sell for $250, a discount of $149. We will let you know about other bargains as we arrange them.\n\nGovernment continues infotech push\n\nJournalists who have yet to go online will fall even farther behind during the next few years if the federal government follows recommendations to give all its employees access to electronic mail by 1997. In a report issued last month, the Government E-Mail Task Force also advised agencies to provide outside access to their networks and publish their e-mail addresses by the end of this year.\n\nNARA proposes archiving e-mail\n\nThe National Archives Records Administration (NARA) has proposed new rules for treating electronic mail as public records that must be preserved, according to a recent article in Government Computer News. The proposal comes on the heels of a federal court ruling that ordered NARA to preserve all e-mail records along with agency documents. Although not all e-mail messages may qualify as public records, those created in the conduct of agency business do. Personal messages unrelated to agency business would not be included under the proposed rules.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-may-arch-investigations-june-16-19-1994","slug":"1994-may-arch-investigations-june-16-19-1994","url":"https://dwillis.github.io/uplink/article/1994-may-arch-investigations-june-16-19-1994.html","year":1994,"month":"May","monthNum":5,"headline":"Arch Investigations June 16-19, 1994","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE/NICAR conference in St. Louis on June 16, 1994 offers an optional day focused on computer-assisted reporting, featuring sessions on online research, databases, spreadsheets, and specialized topics like government records, environmental data, and mapping software.","keywords":["reporting","database","computer"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["modem/BBS","Internet","spreadsheet","GIS"],"full_text":"Arch Investigations June 16-19, 1994\n\nHere's a look at the developing line-up for the optional day, June 16, at the St. Louis National IRE conference. This day we will focus on computer-assisted reporting. For more information on the conference, please call IRE/NICAR headquarters at (314) 882-2042.\n\nThursday Optional day\n8-8:30 Online on the beat: 30 ideas in 30 minutes\nBill Dedman, Associated Press\n9:20-9:50 Data analysis on the beat: 30 ideas in 30 minutes\nShawn McIntosh, Dallas Morning News\n\n8:40-9:50 Specialized session: Business\nRob Wells, Associated Press\nPenny Loeb, U.S. News & World Report\nPrimer on online searching: Power up, modem in, log on. Everything you need to know to get started\nBeth Marchak, Cleveland Plain Dealer\n10-11:10 Creating and recycling databases: When to use a database and which ones to keep available for use on deadline\nJeff Kummer, St. Paul Pioneer Press\nTeresa Leonard, Raleigh News & Observer\nJames Derk, Evansville Courier\nGovernment and social sciences\nDan Browning, St. Paul Pioneer Press\nRich Robertson, Arizona Republic\nShawn McIntosh, Dallas Morning News\nThe mystery and power of Internet\nDan Woods, Raleigh News & Observer\nDan Gillmor, Detroit Free Press\n11:20-12:30 Using spreadsheets and national tabular data to show where your town or state ranks\nWilliam Casey, Washington Post\nRich Gordon, Miami Herald\nChris Schmitt, San Jose Mercury News\n\nEnvironment on computer\nDave Davis, Plain Dealer\nJane Kay, San Francisco Examiner\nWendell Cochran, USA Today\nOnline on the cheap\nNora Paul, Poynter Institute for Media Studies\nMike Berens, Columbus Dispatch\n1-2 Brown Bag lunch: Attribution and accuracy on the information highway\nNora Paul, Poynter Institute for Media Studies, moderator\n2:10-3:20 Deadline online: Using bulletin boards to find eyewitnesses and experts and profiling people\nDeborah Crowe, Alameda Newspaper Group\nMatt Reavy, NICAR\nMapping software on deadline: How to plot census, crime and gerrymandering information\nSuwan Brown, St. Louis Post-Dispatch\nJennifer LaFleur, NICAR\n\nMedicine and science\nBill Allen, St. Louis Post-Dispatch\nPaul Overberg, USA Today\nBeth Marchak, Plain Dealer\n3:30-4:40\n\nAccess to federal, state and local online records: Success stories and pending battles\nSandra Scott, Missouri School of Journalism\nMike Casey, Dayton Daily News\nStatistics can be your friend: Five statistical tools you need to know\nJon Schmid Sotomayor, Raleigh News & Observer\nPhil Meyer, University of North Carolina\nThe electronic newsroom\nCorky Johnson and Karl Dsvoog, WPCO-TV, Cincinnati\nMark Nichols, Indianapolis Star\nNeil Reisner, Bergen Record\n7:30-9 p.m. Getting Started\nNICAR staff","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-uplink-update","slug":"1994-june-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-june-uplink-update.html","year":1994,"month":"June","monthNum":6,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Uplink explores practical online data sources for journalism, featuring guides to Fed World, CompuServe, the World Wide Web, and ProfNet. The National Institute for Computer-Assisted Reporting plans to launch an Internet mailing list for journalists.","keywords":["data","journalism","online"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["CompuServe","World Wide Web","Internet","FTP"],"full_text":"In the past few months, Uplink has dealt mostly with data that you bring in-house. However, in this issue we're looking at the online world of data with help from some of the better journalistic users.\n\nWhile it can be fun to play in cyberspace, your news organization will be much happier if you return with instant information for a story rather than wide-eyed tales of electronic trips to arcane libraries in Barcelona.\n\nThus, we've tried to tilt our articles toward the practical end with features on Fed World, CompuServe, the World Wide Web, and ProfNet. We don't pretend to do more here than dip our toe in the big online ocean so don't hesitate to send us advice.\n\nMeanwhile, in the spirit of onlining, the National Institute for Computer-Assisted Reporting intends to set up a listserv (mailing list on the Internet) in the next month. See you on the Net.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-quick-queries-make-reporters-life-easier","slug":"1994-june-quick-queries-make-reporters-life-easier","url":"https://dwillis.github.io/uplink/article/1994-june-quick-queries-make-reporters-life-easier.html","year":1994,"month":"June","monthNum":6,"headline":"Quick queries make reporter's life easier","kicker":"Bits, Bytes and Barks","author_name":"Kenton Robinson","author_title":"Reporter","affiliation":"The Hartford Courant","summary":"Kenton Robinson shares his experiences using online services and ProfNet to streamline reporting, emphasizing the efficiency and breadth of information available for journalism.","keywords":["online services","ProfNet","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["ProfNet","CompuServe","modem/BBS","Internet"],"full_text":"Coming back to reporting after several years of editing, I cannot remember how I ever managed to do a reporter's job without using online services. Certainly, I was nowhere near as fast and thorough, and my stories were nowhere near as deep and broad. And I'm not talking hard news or investigative stories here; I'm talking features. Example: a restaurateur in Ohio sues some guy for welching on a reservation. Breach of contract, he argues. Not so, the welcher says, there was no contract. But there was, wasn't there? Whether or not a restaurant reservation is a legally binding contract is for the courts to decide, but there is no question there was a social contract, a promise which basic etiquette requires us to honor, and, if we cannot, to have the courtesy to cancel. My editor asked me to turn around a quick story exploring the issue of social contracts and the question: what does it say about our society when people feel they have to go to law to enforce the social contracts people used to honor out of common courtesy? Since this was a question that touched on issues of law and social psychology, I crafted a quick query which I sent out on ProfNet. \"Help!\" I said, \"I need experts on this question.\" Forums: \"Anybody out there heard of any similar instances of this sort of thing?\" Then left an open message on the CompuServe Legal and Social Issues. While my bait drifted in the electronic current, I did some old-fashioned reporting too: Got the number for Judith Martin, for example, and arranged a phoner; hunted down a newly-wedded couple (from the wedding announcements in our own paper) to ask them about RSVPs (another good example of a social contract); dropped into the local bookstore and picked up a copy of Rousseau's \"The Social Contract\" and read it again for the first time since college. When I went back to check my lines, I had some bites: A couple of social psychologist types ready to talk to me about the main question, and an intriguing note from a lawyer on the Legal Forum. \"There was a case, I think it was in Florida a few years ago,\" he wrote. \"Where a mother sued her daughter's prom date for standing her up.\" He couldn't remember anything more about it, but that was all I needed. Online again, and a quick dip into the electronic archives of the Miami Herald, searching on the words \"lawsuit\" and \"prom\" and BINGO! I've got two stories on the case. I interview Miss Manners et al., and I'm ready to write. My story on the role of etiquette in holding together the social fabric and these signs of its deterioration was picked by the L.A. Times-Washington Post Wire and run by papers all over the country. And I can only wonder how I ever could have done it without the use of online information services. How long would it have taken me to find the people I needed to talk to and the information I needed? There is no question in my mind I would not - within the bounds of my deadline, anyway - ever have found much of it. There is hardly a story I do that I don't use my modem. It's like having a giant library-telephone directory at your desk, with information available to you almost instantaneously. And people! You can find experts in everything from American Indian iconography to the language acquisition of children to modern tattoo ology. No matter how arcane your subject, somewhere out there there's somebody who's made a study of it, or, if not, at least has enough knowledge of related subjects to give it some intelligent perspective. Many reporters I talk to say they think using ProfNet is somehow cheating; it makes it too easy to find the right person to talk to. But there is an art to the query, as it were. You must make it as precise (and concise) as possible and make clear how close your deadline looms. Then, when you put the query on the net, it goes to hundreds of institutions where flaks cull through their staff lists for someone who might be a good match, and sometimes they stretch things a bit, so you have to do some culling yourself. And as for tracking down non-academic people, when you have only a bit of information about them, online services can't be beat. For example, when I recently had to locate the \"Worm Woman of Kalamazoo\" for a story I was doing on vermicomposting, I had only to go to Compuserve's Phone-File and search on her last name, Appelhof, in the Kalamazoo area to get her number and street address. When I recently did a story on the human vomeronasal organ and pheromones, I dipped into the National Library of Medicine to pick up the dozen most recent articles on this and related subjects in the medical journals (and with them the names of researchers, of course). Sure, I could have gone to a library and spent a day digging for this stuff; instead, I spent less than 5 minutes and never left my desk. Other people at my newspaper have dumb terminals on their desks, which allow them to do nothing but write stories on the system. I have a PC, with a modem, which allows me to travel the world. Here's my list of most used online resources. These are all sources I use virtually every week, without leaving my desk. All I simply do is dial into the local access number for Compuserve. 1) ProfNet. Need to find an expert on Civil War memorial statuary? Or the TV habits of blacks and whites? If there's anyone out there who knows about these things, a query to ProfNet is your fastest way to track him or her down. (See also Dissertation Abstracts below.) 2) Phonefile. Every listed phone number in the United States is at your fingertips in instants. If you only have a hunch as to the general geographic area your subject inhabits, you can simply search by surname. 3) NewsUSA or the Newspaper Library. Electronic morgues of all the major and a lot of smaller newspapers across the country. Searchable by subject, keywords and byline. 4) Dissertation Abstracts. If you want to find out whether anybody has ever written a dissertation on the skateboard subculture, the abstracts will tell you who did when and give you a summary of the thesis. Also, there's an order number, so you can get a copy of the whole thing if you want it. 5) Books in Print. Has anyone ever written a book about your subject? This gives you all the information including author, title, publisher, and ISBN number. 6) Magazine Database Plus. Many major and not-so-major magazines and their articles. 7) Academic American Encyclopedia. This may sound like a silly one, but it's handy to be able to look up some general point of reference quickly. And, needless to say, it's also much faster than walking back to the library and searching through the dusty stacks. 8) SIGs or Forums. There are dozens of Special Interest Groups (SIGs) and professional forums where people correspond on their subject, be it lawyers or dinosaurs. These are good places to find people to talk to about the subject you're writing about. You can contact Kenton Robinson at his CompuServe address, 75050,2773.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-june-online-sources-you-cant-do-without","slug":"1994-june-online-sources-you-cant-do-without","url":"https://dwillis.github.io/uplink/article/1994-june-online-sources-you-cant-do-without.html","year":1994,"month":"June","monthNum":6,"headline":"Online sources you can't do without","kicker":"","author_name":"Kenton Robinson","author_title":"","affiliation":"","summary":"A journalist describes eight essential online research resources accessed through CompuServe, including ProfNet for expert sourcing, newspaper archives, dissertation databases, and specialized forums that allow reporters to conduct comprehensive research without leaving their desks.","keywords":["research","sources","journalism"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["CompuServe"],"full_text":"Here's my list of most used online resources. These are all sources I use virtually every week, without leaving my desk. All I simply do is dial into the local access number for Compuserve.\n\n1) ProfNet. Need to find an expert on Civil War memorial statuary? Or the TV habits of blacks and whites? If there's anyone out there who knows about these things, a query to ProfNet is your fastest way to track him or her down. (See also Dissertation Abstracts below.)\n\n2) Phonefile. Every listed phone number in the United States is at your fingertips in instants. If you only have a hunch as to the general geographic area your subject inhabits, you can simply search by surname.\n\n3) NewsUSA or the Newspaper Library. Electronic morgues of all the major and a lot of smaller newspapers across the country. Searchable by subject, keywords and byline.\n\n4) Dissertation Abstracts. If you want to find out whether anybody has ever written a dissertation on the skateboard subculture, the abstracts will tell you who did when and give you a summary of the thesis. Also, there's an order number, so you can get a copy of the whole thing if you want it.\n\n5) Books in Print. Has anyone ever written a book about your subject? This gives you all the information including author, title, publisher and ISBN number.\n\n6) Magazine Database Plus. Many major and not-so-major magazines and their articles.\n\n7) Academic American Encyclopedia. This may sound like a silly one, but it's handy to be able to look up some general point of reference quickly. And, needless to say, it's also much faster than walking back to the library and searching through the dusty stacks.\n\n8) SIGs or Forums. There are dozens of Special Interest Groups (SIGs) and professional forums where people correspond on their subject be it lawyers or dinosaurs. These are good places to find people to talk to about the subject you're writing about.\n\nYou can contact Kenton Robinson at his CompuServe address, 75050,2773.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-fed-or-med-you-can-find-it-online","slug":"1994-june-fed-or-med-you-can-find-it-online","url":"https://dwillis.github.io/uplink/article/1994-june-fed-or-med-you-can-find-it-online.html","year":1994,"month":"June","monthNum":6,"headline":"Fed or Med? You can find it online","kicker":"","author_name":"Beth Marchak","author_title":"","affiliation":"The Plain Dealer","summary":"FedWorld, a government online database launched in 1993, has become a useful resource for journalists covering the White House and federal issues, while the Internet also offers specialized medical newsletters on topics like RSI, Lyme Disease, and cancer that can help reporters develop stories.","keywords":["database","journalists","medical"],"topics":["Technology","Investigative Methods","Health & Environment"],"technologies":["Internet","World Wide Web","gopher","FedWorld"],"full_text":"FedWorld, the government's one-stop snooping on-line computer databases, is evolving into a reference for those covering the White House. The database, which was launched in March 1993, was connected to the Internet in August. It contains more than 130 bulletin boards and databases. It's the easiest place to find White House information. The only problem is it's on a different menu, which makes it hard to find. To get there, Type \"W-house\" at the prompt, follow the menu and find access to White House speeches, press releases, appointments, and major releases like the health care bill. Journalists and others who dial into various World computers don't complete access to all the information that employees do. In some cases that means the database/bulletin board is almost worthless. In other cases, like Quick Facts! Information on Alcohol Abuse and the District of Columbia Court of Appeals, the information is a great start on a story. The messaging bulletin boards are not as lively as on CARR-L or The Well, but they are useful for contacts for government issues like the National Performance Review where federal workers love to jaw about who ought to pay for their business cards. Fed World is still evolving like its commercial counterparts. In late May, for example, it added several trade documents which it promoted on the welcome screen. The Internet has a vast array of medical resources. Here are three that could prove useful to reporters this summer: RSI: Repetitive Stress Injury newsletter tracks studies, medical literature and new treatments for those suffering from RSI, Some of the stories in the newsletter deal with the psychological studies of the illness, lawsuits, the ergonomic issues and advances that minimize the disease's impact. To subscribe, send a message entitled \"RSI subscription\" to dadadata@ world.std.com or call Craig O'Donnell at 617/776-2809 for more information. Lyme Disease: This little known, underreported disease is the subject of an online newsletter published 26 times a year. One way to get it is to gopher to NIH, then to medical publications. The newsletter contains information about studies and outbreaks. The newsletter, which in part promotes awareness of disease, now contains listings of patient and physician events. It also contains a jargon index and medical abstracts, sightings, treatments book reviews and stories about lobbying Congress for money and research. Some of the information is clearly too technical for newspapers but it gives sources of outbreaks and those involved in studying them. It's stated purpose is to bring \"physicians, researchers and support group leaders together electronically to put pieces of the LD puzzle together.\" To subscribe and get more information message lymenet-1@lehigh.edu. CancerNet: Several cancer-related topics are covered here, including AIDS, adult and childhood cancers and drug information summaries, reports on drug trials and some information from Medline. Gopher to NIH and start digging into the sub menus. A few highlights: The AIDS Daily Summary contains a roundup of local and national and international stories which appears to be available the same day. Statistics, resources, a bibliography and a reference guide for clinicians is also available. On the same menu is the Morbidity and Mortality Weekly Report, which recently highlighted the increase in rabies cases, examining incidents state by state. The report also had 1991 stats for breast cancer and listed several doctors studying each issue. Depending on the information sought, the results are available either online or by fax. To sign up, send a mail message to cancernet@icib.nci.nih.gov. Type the word help in the body of the message. Contact cheryl@icib.nih.gov with questions or problems. The FedWorld messaging bulletin boards are not as lively as on CARR-L or The Well, but they are useful for contacts for government issues like the National Performance Review where federal workers love to jaw about who ought to pay for their business cards. You can contact Beth Marchak at Marchak@digex.com or Marchak@digex.net","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-june-insights-into-mosaic","slug":"1994-june-insights-into-mosaic","url":"https://dwillis.github.io/uplink/article/1994-june-insights-into-mosaic.html","year":1994,"month":"June","monthNum":6,"headline":"Insights into Mosaic","kicker":"","author_name":"Wallace Winfrey","author_title":"Staff","affiliation":"NICAR","summary":"The World Wide Web, developed at CERN, organizes the Internet through hypertext and hypermedia links, with the Mosaic browser making it accessible through a user-friendly graphical interface that can display text, graphics, sounds, and videos.","keywords":["Internet","hypertext","browser"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["World Wide Web","FTP","Telnet","HTML","Hypercard","Mosaic","Visual Basic","FoxPro"],"full_text":"The Internet - it's big, it's cool, it's really confusing for most people. With terms like ftp, telnet, and finger, it's not exactly intuitive either. One could spend years just trying to figure out the syntax to send e-mail and transfer files. But if things continue to move in the direction of something called the World Wide Web (w3), this could cease to be a problem.\n\nWhat is the World Wide Web? Simply put, it's an effort to logically organize the Internet. Developed at CERN laboratories in Geneva, Switzerland, it was originally designed to enable scientists there to share their research on high-energy physics. It quickly became obvious that w3 would be a good thing to implement on the Internet as well.\n\nThe Web creates a system of \"logical\" interconnected links between various types of information using hypertext. Even if you haven't heard the term \"hypertext,\" you've probably already used it. Windows help files are a form of hypertext, as is Hypercard on the Macintosh.\n\nHere's how it works: say you are reading an article about the Cuban revolution and you come across the words Che Guevera. In a hypertext environment, clicking on the underlined words would take you to a document about Che, perhaps a biography or a commentary of sorts. However, text can get really confusing as you jump from one document to the next. Besides, who wants to limit their information to text only? With hypermedia you can click on underlined text and go not only to another text document, but to a graphics file (a photograph of Che Guevera), a sound (a speech by Guevera), or even a digitized movie.\n\nThe Web is organized around the concept of \"pages.\" These pages are technically called HTML (\"HyperText Markup Language\") documents. HTML is a low-level programming language that defines how that page will look and where the links (the underlined words) go to when that page is viewed.\n\nUntil last year, there really was no good application to browse the Web. Almost all \"browser\" applications were text-based, and while it was possible to link other forms of media, you had to download it first, and then open it up with the appropriate program. Then came Mosaic.\n\nMosaic represented a breakthrough in w3 browsers. With Mosaic, you are no longer confined to looking at a command-line using arrow keys to navigate. You can now point-and-click your way around using a mouse. Mosaic also has the ability to display graphics within a document, and even define text size and appearance.\n\nAll this gives w3 pages a hitherto-unrealized capacity to present information in a way that is not only logical, but even visually appealing. This might account for the jump in w3 use of over 300,000 percent in the last year alone.\n\nMosaic also lets you listen to the linked sound files and view the linked movies while you are browsing a document. While it relies on other applications to accomplish this, the operation is smooth enough that it's more or less invisible.\n\nOther applications can also be linked to Mosaic, letting you use telnet or tn3270 or e-mail seamlessly. Mosaic also lets you input data into a page. This is extremely useful if you want to send or receive feedback on the appearance of a page.\n\nHowever, feedback is only one small application of data entry onto pages. HTML documents (Mosaic pages) can also be used as a front end for database files. Just as many use Visual Basic to customize a front-end for their FoxPro files, HTML can be used as well.\n\nThis brings up some very interesting possibilities, especially in the realm of computer-assisted reporting. By putting a database up and customizing a front-end for it using HTML, it now becomes possible to deliver that data to anyone in the world, in the form they want it.\n\nSay you are working on a story about bridges (to lift from a recent CARR-L thread) and you want to know how many bridges in the United States last year didn't pass initial inspections. You also want to know, out of those that didn't, how many were closed down. You would then go to the appropriate w3 page, enter in the variables (bridges, inspection failure and closings), define the form of your output (as a .dbf or .txt file) and then press enter.\n\nThe computer at the other end does its crunching and then generates a customized document on-the-fly containing the relevant information. No more waiting weeks for a 9-track tape. No more trying to convert the data into a readable form. Just data, as you want it, when you need it.\n\nMosaic does require a certain kind of connection to the Internet to work properly. To use it, you have to have what is known as a TCP/IP connection. Local area networks connected to the Internet sometimes have this kind of connection.\n\nIf you are confined to a modem, you need a dial-up TCP/IP connection to a SLIP server. What these are demands a whole other article to explain. But if you have the right connection, you can access the Web fairly easily.\n\nTo download the latest version of Mosaic, ftp to ftp.ncsa.uiuc.edu and look in the directory called Mosaic. I should mention however that there are other browsers besides Mosaic. For example, just this morning, I downloaded a browser for the Macintosh called MacWeb. In its defense, though, Mosaic is the only browser to have been implemented on the Mac, Windows, the Amiga and XWindows.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-fixing-data-in-foxpro","slug":"1994-june-fixing-data-in-foxpro","url":"https://dwillis.github.io/uplink/article/1994-june-fixing-data-in-foxpro.html","year":1994,"month":"June","monthNum":6,"headline":"Fixing data in FoxPro","kicker":"Tech Tips","author_name":"Richard Mullins","author_title":"","affiliation":"University of Missouri","summary":"This article explains how to use FoxPro commands like REPLACE and STRTRAN to standardize and clean up data fields such as addresses and city names, removing inconsistencies like periods and inconsistent capitalization.","keywords":["FoxPro","database","standardization"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["FoxPro","SQL","XBase"],"full_text":"When you're cleaning up data or making names, addresses, and other details consistent, some changes can be done on a field as a whole. For example, if the city field has \"St. Louis\" or \"Saint Lewis\" you can issue a command to change it to \"ST LOUIS\" in every case. The SQL command for this is UPDATE. FoxPro, while it supports SQL, does not include the UPDATE command, since the XBase command language that FoxPro also uses already had an equivalent command for UPDATE. In FoxPro, the command for this task is:\n\nREPLACE ALL city WITH \"ST LOUIS\";\nFOR city = \"St. Louis\" OR city = \"Saint Lewis\"\n\nThe FoxPro command words are capitalized in this example for clarity. The semicolon allows you to continue the command to the next line.\n\nYou'll notice that not only did this example fix a gross misspelling and standardize on the abbreviation for \"Saint,\" it also capitalized every letter and got rid of the period in the abbreviation. Both of these steps are good standards for address information and are used by the Federal Election Commission and the Postal Service.\n\nTo carry out this standard of no periods in an address field, we need a command to search and replace inside a field. Here's the FoxPro command:\n\nREPLACE ALL address WITH STRTRAN (address, )\n\nThe STRTRAN() function does the work inside the fields. The syntax for STRTRAN() looks like this:\n\nSTRTRAN <FieldName>, <SearchFor>, <ReplaceWith>)\n\nThe function takes its working instructions from the stuff you put inside the parentheses. The computer word for \"stuff inside the parentheses\" is \"arguments.\" The first argument says \"Here's where to do the work.\" A comma signals the second argument, which says \"Look for this string of characters.\" The third argument says \"Replace all the strings with this string.\" If the third argument is missing, then the search string is replaced with nothing. In this case, the periods are removed and nothing is put in their place. Here's a before-and-after picture:\n23 Elm St. N.W.\n23 Elm St NW\nTo capitalize everything in the address field:\nREPLACE ALL address with UPPER (address)\nMullins can be reached at Jourram@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-new-looks-for-the-net","slug":"1994-june-new-looks-for-the-net","url":"https://dwillis.github.io/uplink/article/1994-june-new-looks-for-the-net.html","year":1994,"month":"June","monthNum":6,"headline":"New looks for the Net","kicker":"GUIs on the horizon","author_name":"Jennifer LaFleur","author_title":"NICAR Staff","affiliation":"NICAR","summary":"A reporter covers the 1994 Spring Internet World Conference in San Jose, exploring the latest Internet technology and graphical user interfaces (GUIs) designed to help users navigate the early Internet.","keywords":["Internet","conference","technology"],"topics":["Technology","Journalism & Newsroom"],"technologies":["World Wide Web","gopher","FTP","Telnet","Windows","Macintosh","DOS","modem/BBS","UUCP"],"full_text":"Silicon Valley - Net surfers from around the country came to test the waters in San Jose June 1 and 2 at the 1994 Spring Internet World Conference, sponsored by Meckler Media, Inc., publisher of Internet World magazine. The conference provided Internet training and a look at the latest technology for \"getting wired.\" My search through this maze of vendor stands and demo rooms provided a living model of the Internet itself - there was just so much \"stuff.\"\n\nBefore coming to this conference, I was fairly confident in my nerd-abilities. I was humbled when I entered rooms filled with heavy users for whom the discussion of network protocols is mere small-talk. After being knocked aside by a fellow conference attendee who swept by me on roller blades and clutched a fist full of demo disks, I wound my way through the massive web of hardware suppliers, GUIs, booksellers, software vendors and online services suppliers. GUIs, or Graphic User Interfaces were the hot topic at the conference. These are programs that provide a point-and-click tool for navigating through the sea of information available on the Internet.\n\nKeep in mind that most GUIs do not provide access to the Internet - you have to do that through an Internet service vendor. Rather, these programs provide an electronic roadmap to the Internet through your desktop PC or Macintosh. Here are a few of the GUIs that were on display at Internet World:\n\nInternet in a Box (O'Reilly & Associates, Inc and SPRY, Inc.) This is a box containing all the tools for PC users to connect to the Internet: optional automatic connection to the Internet via a service provider (for those who don't have access); Global Network Navigator, an interactive guide to the Internet's resources; AIR Series Internet applications for e-mail, Usenet, file transfer, TELNET and Gopher; and two books describing how to use the system.\n\nWinGopher (NOTIS Systems, Inc) This is a graphical tool for searching GopherSpace on the Internet through the Windows environment. GopherSpace is a collection of publically available resources that use a protocol designed by the University of Minnesota. This system does not handle World Wide Web (full-text indexing) or e-mail. For more information: wingopher@notis.com.\n\nCameleon (NetManage). This program provides complete desktop communications for the Windows environment. Cameleon provides a graphical interface for accessing GopherSpace; Netnews, an Internet news group manager; Whols, an application for identifying people on the Internet. In addition, Cameleon provides other network communications utilities such as e-mail, file and printer-sharing and terminal emulation. For more information, call (408) 973-7171.\n\nWorldLink for Macintosh and DOS (Intercon) WorldLink is a accessing program and an online service that allows Internet access for individuals. With this program, you use a modem to access information that is available through electronic mail, network news groups and online services obtainable via anonymous FTP (file transfer protocol). Users pay a monthly fee, plus connect-time fees. For more information: info@intercon.com\n\nMKS Internet Anywhere (Mortice Kern Systems) provides an interface for the Windows environment to e-mail and network news on the Internet. Users must already have a UUCP network connection to the Internet. For more information: mksia@mks.com.\n\nNetCruiser (NetCom) NetCruiser is a user interface for the Windows environment the system also provides access to the Internet through NetCom's service. Access includes: electronic news, ARCHIE (keyword search), File Transfer Protocol for downloading software and files, e-mail, International Relay Chat (real-time chat), TELNET access to other computers on the Internet, UseNet, Gopher and VERONICA (search by subject). For more information: info@netcom.com.\n\nYou can contact Jennifer LaFleur at Jourjal@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-beyond-the-rolodex","slug":"1994-june-beyond-the-rolodex","url":"https://dwillis.github.io/uplink/article/1994-june-beyond-the-rolodex.html","year":1994,"month":"June","monthNum":6,"headline":"Beyond the Rolodex","kicker":"ProfNet","author_name":"Dan Forbush","author_title":"ProfNet System Operator","affiliation":"State University of New York at Stony Brook","summary":"The State University of New York's Internet Demonstration Project launched ProfNet, a cooperative network that connects journalists with expert sources at universities, research institutions, and corporations worldwide through email distribution lists and online directories.","keywords":["journalists","network","experts"],"topics":["Technology","Journalism & Newsroom"],"technologies":["Internet","Email"],"full_text":"Beyond the Rolodex\n\nBy Dan Forbush\nProfNet System Operator\n\nReporters throughout the U.S. and around the world have come to increasingly rely upon a unique source of information that allows them to connect to sources on a wide range of topics via the Internet - the State University of New York's Internet Demonstration Project (IDP).\n\nThe University at Stony Brook launched the IDP in Fall, 1992 with two goals: 1) to create on the Internet global community of public information officers, linking the world's major educational, scientific and cultural institutions; and 2) to use the Internet to increase the frequency, immediacy and quality of communication between journalists and college and university faculty members. The project has two parts:\n\nProfNet, a cooperative of public information officers linked via Internet to give journalists and authors quick and convenient access to expert sources;\n\n\"PIOs Online,\" a directory of academic PIOs who can be reached by email.\n\nProfNet\n\n\"ProfNet\" (\"Professors Network\") is centered in Stony Brook's Office of News Services. At ProfNet's core is an e-mail distribution list that enables our news staff to send journalists' queries simultaneously to public information officers throughout North America and overseas. PIOs respond directly to the inquirer by e-mail, fax, phone or hard mail.\n\nAs of May, ProfNet's membership included 1,023 PIOs representing 600 colleges, universities, and a wide range of academic, government, and corporate entities oriented to scholarship and research. Most of the Ivy League and Big Ten are represented, as are such institutions as Stanford, Berkeley, MIT, CalTech, Johns Hopkins and University of Chicago.\n\nAlso represented are:\n\nFederal entities, such as the National Science Foundation, National Academy of Sciences, National Academy of Engineering, National Research Council, National Institute of Standards and Technology, Office of Naval Research, Smithsonian Institution, Argonne National Laboratory, and others;\n\nCorporate research labs, such as AT&T Bell Labs and IBM's Thomas J. Watson Research Center and Almaden Research Center;\n\nMedical centers, such as the Howard Hughes Medical Institute and centers at Stanford, University of Michigan, University of Alabama at Birmingham, Vanderbilt, University of Utah, University of Pennsylvania, University of Southern California, and the University of Texas at Houston;\n\nSchools of business and public policy, including the University of Chicago's Graduate School of Business, Dartmouth's Amos Tuck School, Harvard's John F. Kennedy School of Government, and Princeton's Woodrow Wilson School of Public and International Affairs.\n\nProfNet has a rapidly expanding contingent of members in Australia, Europe, Africa and Asia. Among them: Bond University, University of Sydney, Macquarie University, University of New South Wales, Queensland University and Griffith University (Australia); University of Heidelberg and University of Bayreuth (Germany); Lund University and Swedish University of Agricultural Sciences (Sweden); University of Bergen (Norway); University of Jyvaeskylae (Finland); Center for Economic Policy Research, University of Southampton, and University of Stirling (United Kingdom); University of St. Gallen (Switzerland); Universite catholique de louvain (Belgium); University of Milan (Italy); Rhodes University (South Africa); and Beijing Agricultural University (China).\n\nAmong other ProfNet resources in the U.S. are the American Association for the Advancement of Science, Association of American Universities, Brookings Institution, National Opinion Research Center, MacArthur Foundation, Scientists' Institute for Public Information, International Food Policy Research Institute.\n\nPIOs Online\n\nThe Stony Brook Guide to PIOs Online provides email addresses and other contact information (phone, fax and hard mail) for 770 public information officers who had joined ProfNet as of December 1, 1993. Any academic PIO with an email address is eligible to be listed in future editions.\n\nTo order: Make out check in amount of $20 to Research Foundation of SUNY. Send to: c/o Dan Forbush, Office of University Affairs, University at Stony Brook, Stony Brook, NY 11794-0605.\n\nTo Make a ProfNet Query\n\nIn submitting queries, please specify your news organization, the nature of your project, the nature of the expertise you seek, your time frame, and the manner in which you wish PIOs to respond. The more detailed your query the more lead time you give us - the more successful your search will be. There is no fee for conducting ProfNet searches, nor is there any limit to the number or frequency of your queries. You may deliver queries as follows: By CompuServe: 73163,1362 or 76550,750 - By Internet: profnet@sunysb.edu - By fax: (516) 632-6313 - By phone: (800) 776-3638. ProfNet queries typically are distributed to the membership within 12 to 24 hours, though you may request a same-day rush query if your deadline demands it. Rush queries should be submitted before 11:30 a.m. Eastern time and are best made by phone. If you have a concern about maintaining exclusivity on an idea, you may cloak your query so as to conceal both your identity and that of your news organization. Cloaked queries are best made by phone.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-canadian-govt-launches-internet-project","slug":"1994-june-canadian-govt-launches-internet-project","url":"https://dwillis.github.io/uplink/article/1994-june-canadian-govt-launches-internet-project.html","year":1994,"month":"June","monthNum":6,"headline":"Canadian govt. launches Internet project","kicker":"Bits, Bites and Barks","author_name":"","author_title":"","affiliation":"","summary":"Canada's federal government launched the Open Government Project, an internet-based pilot program designed to increase public accessibility to government information, officials, and Members of Parliament through multiple web and gopher sites.","keywords":["government","internet","accessibility"],"topics":["Government & Politics","Technology"],"technologies":["World Wide Web","gopher","Internet"],"full_text":"Canada's federal government last month launched a pilot project to evaluate an Internet-based service designed to inform citizens about governmental operations. \"The Open Government Project\" aims to increase accessibility to government information, officials and Members of Parliament, according to Industry Minister John Manley. Although available in several formats the new project has been designed to take advantage of the World-Wide-Web, a multimedia, hypertext application. Anyone interested in accessing the pilot project can do so at the Industry Canada host site or the Canadian Center for Inland Water (Environment Canada). The universities of New Brunswick, Montreal, Calgary, and British Columbia offer Internet access to the project. SITES: WWW Mother Site: http://debra.dgbt.doc.ca/opengov WWW Mirrors: http://www.culturenet.ucalgary.ca/opengov http://www1.cciw.ca/opengov http://www.unb.ca/opengov http://www.emr.ca/opengov http://www.kaapeli.fi/opengov/ http://www.cs.ubc.ca/opengov http://bri3e.ere.umontreal.ca:6764/opengov Gopher Mother Site: debra.dgbt.doc.ca Gopher Mirrors: gopher.emr.ca","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-epa-improving-online-access","slug":"1994-june-epa-improving-online-access","url":"https://dwillis.github.io/uplink/article/1994-june-epa-improving-online-access.html","year":1994,"month":"June","monthNum":6,"headline":"EPA improving online access","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The EPA is expanding its Seattle-based bulletin board system due to a significant increase in public information requests, with plans to add a new phone line and potentially offer Internet access or an 800 number. The system provides nationwide access to Superfund reports, hazardous waste data, and daily EPA news briefings.","keywords":["EPA","bulletin","expansion"],"topics":["Health & Environment","Technology","Government & Politics"],"technologies":["modem/BBS","Internet"],"full_text":"The U.S. Environment Protection Agency (EPA) has decided to expand its Seattle-based bulletin board system after public information requests more than quadrupled in recent months. The agency will be adding a new phone line to the system soon. Additional plans call for the possibility of Internet access or an 800 number to eliminate user phone bills. Although the EPA currently offers some documents on Internet, those materials are not updated regularly, said system operator Thomas Denning. The Seattle BBS site handles nationwide Superfund reports, Resource Cost Recovery Act reports, and hazardous waste data among other things. The site also offers a daily \"news conference\" of EPA news and information.\n\nThe BBS can be accessed at (206) 553-2241. New users are required to complete a brief questionnaire before accessing the system.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-state-land-records-going-on-cd","slug":"1994-june-state-land-records-going-on-cd","url":"https://dwillis.github.io/uplink/article/1994-june-state-land-records-going-on-cd.html","year":1994,"month":"June","monthNum":6,"headline":"State land records going on CD","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Eastern States Bureau of Land Management is digitizing federal land titles from 13 states onto CD-ROM to preserve historic documents, with a project completion target of 2000 and reported conversion accuracy rates exceeding 99 percent.","keywords":["digitization","preservation","patents"],"topics":["Data & Databases","Government & Politics","Agriculture & Land"],"technologies":["CD-ROM"],"full_text":"Federal land titles from 13 states are being moved to CD-ROM as part of a project designed to preserve the historic documents, according to a recent article in Government Computer News.\n\nThe task, which the Eastern States Bureau of Land Management expects to complete by the year 2000, involves states west of the Ohio River where the government sold off public land to repay Revolutionary War debts.\n\nSome 260,000 land patents from Alabama are due to be published next month. Arkansas, Florida, and Louisiana have already been completed. Others on the list are: Illinois, Indiana, Iowa, Michigan, Minnesota, Missouri, Mississippi, Ohio, and Wisconsin.\n\nThe records have a reported conversion accuracy rate of 99 percent with 99.5 percent index accuracy.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-uc-berkeley-offers-mosaic-site","slug":"1994-june-uc-berkeley-offers-mosaic-site","url":"https://dwillis.github.io/uplink/article/1994-june-uc-berkeley-offers-mosaic-site.html","year":1994,"month":"June","monthNum":6,"headline":"UC Berkeley offers Mosaic site","kicker":"","author_name":"","author_title":"","affiliation":"UC Berkeley","summary":"UC Berkeley's graduate journalism school has launched a new Mosaic website featuring resources of interest to journalists, including student photos and school information.","keywords":["journalism","Berkeley","Mosaic"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Mosaic","World Wide Web"],"full_text":"Journalists with access to Mosaic may want to check out a relatively new site at the University of California-Berkeley (http://128.32.158.47). The graduate school of journalism there has created a Mosaic page featuring places of interest for journalists. It also contains student photos and some information about the school itself.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-usenet-for-those-with-e-mail-only-access","slug":"1994-june-usenet-for-those-with-e-mail-only-access","url":"https://dwillis.github.io/uplink/article/1994-june-usenet-for-those-with-e-mail-only-access.html","year":1994,"month":"June","monthNum":6,"headline":"Usenet for those with e-mail only access","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A guide for accessing Usenet newsgroups through an alternative method via listserv at a Belgian university email address using the /NNHELP command.","keywords":["Usenet","listserv","Netnews"],"topics":["Technology"],"technologies":["Usenet","Email","Listserv"],"full_text":"Anyone who feels a desperate need for Usenet but can't find a feed may want to try getting Netnews via a listserv. For more info send an email message to listserv@ccl.kuleuven.ac.be In the first line type the command \"/NNHELP\" and you should be on your way.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-june-internet-guide-to-govt-sources-available","slug":"1994-june-internet-guide-to-govt-sources-available","url":"https://dwillis.github.io/uplink/article/1994-june-internet-guide-to-govt-sources-available.html","year":1994,"month":"June","monthNum":6,"headline":"Internet guide to govt. sources available","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A 50-page Internet guide to government business and economic sources has been made available through the University of Michigan Libraries, accessible via multiple platforms including FTP, Gopher, and Mosaic.","keywords":["government","internet","guide"],"topics":["Government & Politics","Technology","Investigative Methods"],"technologies":["Internet","gopher","FTP","Mosaic"],"full_text":"Internet guide to govt. sources available\n\nThose of you using Internet to find sources of information from or on the government may want to check out the Internet Guide to Government Business and Economic Sources.\n\nThe 50 page guide (122K) can be found at the Clearinghouse of Subject-Oriented Internet Resource Guides on the University of Michigan Libraries gopher.\n\nAnonymous FTP: host: una.hh.lib.umich.edu path: /inetdirsstacks/govdocs:tsangaustin\nGopher: via University of Minnesota list of gophers menu: North America/USA/Michigan/Clearinghouse of Subject-Oriented Internet Resource Guides\nMosaic: gopher://una.hh.lib.umich.edu/00/inetdirsstacks/govdoc%3atsangaustin.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-july-uplink-update","slug":"1994-july-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-july-uplink-update.html","year":1994,"month":"July","monthNum":7,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The National Institute for Computer-Assisted Reporting and Investigative Reporters and Editors launched two listservs to facilitate discussions on reporting techniques.","keywords":["NICAR","listserv","reporting"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","Listserv"],"full_text":"For surfers of the Internet, we have good news. The National Institute for Computer-Assisted Reporting and Investigative Reporters and Editors have entered the online world.\n\nLast month we set up two listservs (discussion group areas) through the University of Missouri. One listserv is NICAR-L and the other is IRE-L. You can join the NICAR listserv by sending a message to listserv@mizzoul.missouri.edu. In the text of the message type: subscribe nicar-1 your name. For example, Jackie Smith would type: subscribe nicar-1 jackie smith. Do exactly the same for IRE, except type ire-l. Once you do that, every message sent to the listserv will land in your e-mail box.\n\nIn the NICAR listserv we plan to have discussions on computer-assisted reporting and on IRE on investigative reporting, computer and non-computer related.\n\nIn the coming months, we hope you will see other NICAR and IRE resources put online.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-july-are-your-bridges-falling","slug":"1994-july-are-your-bridges-falling","url":"https://dwillis.github.io/uplink/article/1994-july-are-your-bridges-falling.html","year":1994,"month":"July","monthNum":7,"headline":"Are your bridges falling?","kicker":"Bits, Bytes and Barks","author_name":"Paul D'Ambrosio and Rick Linsk","author_title":"The Asbury Park (NJ) Press","affiliation":"The Asbury Park Press","summary":"An investigation into bridge safety using the National Bridge Inventory database found that one in six bridges in the state needs major repairs costing $4.8 billion, with reporters discovering serious structural issues hidden beneath temporary surface-level fixes.","keywords":["bridges","infrastructure","repairs"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["9-track tape"],"full_text":"When you drive over a bridge, you assume it's safe. After all, the truck right in front of you made it across okay, so your little Honda should do just fine. Don't be so sure. To find out how bridges in our state were doing, we turned to the National Bridge Inventory list, a database detailing all inspections of bridges in the United States with a length of more than 20 feet. In a story published in February, we found that one of every six bridges in the state needs major repairs. The bill for those repairs totals $4.8 billion - more than ten times what the state spends on bridge maintenance. The NBI database only told us part of the story. Once we located bridges in questionable shape, we took a look at them. What we found were bridges with fractured foundations, one that had been struck by a train (but remained in use), and one that had state officials so scared they inspected it daily.\n\nWe discovered that temporary repairs on bridges are usually done to the road surface so motorists won't feel the bumps. The real decay, which can cause a bridge to be shut down, occurs underneath.\n\nOne tell-tale sign of decay was wood slate on the underside of the bridge. Highway crews put the slate there to catch falling concrete so motorists below don't end up with part of the bridge in their laps.\n\nAt first glance, the NBI code guide can be intimidating. Its 100 pages provide mostly technical detail on 116 fields. But you only have to use a few of these fields to get a good idea of what shape a bridge is in. To understand how your area's bridges work, talk with a local engineer or county bridge inspection department. It doesn't make much sense to spend a lot of time looking at the \"scour critical\" field (dealing with water erosion under support piers) if most of your state's bridges are over desert. In New Jersey we used Item 67, structural evaluation, as our divining rod. A rating of 0, 2, or 3 (1 is not used) told use that a bridge was in pretty bad shape. Keep in mind that this database was designed by engineers, who tend to be pretty conservative creatures by nature. Nothing in this list will tell you if a bridge will fall down or how many more years of use it has. The latter used to be a field, but was discontinued.\n\nOnce we compiled a list of bridges, we examined the actual inspection reports at the state and county offices. These documents gave valuable insight into every problem with the bridge and up-to-date repair estimates.\n\nNot every bridge is examined at the same time and the NBI data isn't always current. Bridges should be inspected every two years, but depending on how vigilant the inspection crews are, this can range from once a week to once a decade. Before printing a list of troubled bridges, double check with the inspection department to see if any repairs have been done.\n\nAn application that uses crosstabs can be helpful in grouping the ages of your state's bridges. We found that many of the bridges in New Jersey were built in the 1930s and 1950s, which means that most of them will have to be replaced in the next 10 to 20 years.\n\nThe NBI list also contained a lot of faulty locations for bridges, which can make mapping a nightmare. We found that most of the bridges in our areas were either in the wrong towns or used the wrong street names. Once again, we had to rely on the inspection department for proper locations. We later spent a lot of hours cleaning up the addresses.\n\nTo obtain a copy of the NBI list on 9-track tape for $250, contact the Federal Highway Administration at (202) 366-0660. NICAR also has the NBI list available for the entire U.S. ($160) or by state ($75). Also, ask for a copy of the \"Recording and Coding Guide for the Structure Inventory and Appraisal of the Nation's Bridges.\"\n\nAdvice for working with NBI data\n\nTo get the most out of the National Bridge Inventory list, you have to know what to look for in the 116 fields. If you examine the data well in advance, you will probably end up using only about a dozen of those fields.\n\n- In Item 5, use only bridges with a code of \"1.\" This means it's an actual bridge. Any other code means a roadway passes under the bridge.\n- The NBI only deals with bridges more than 20 feet in length. If your favorite bridge isn't on the list, it may not meet the minimum requirement.\n- Don't get hung up on one rating number. There is no single data field that will tell you if a bridge is good or bad. There are separate ratings for the deck, superstructure, and sub-structure. The closest you will come to an overall safety rating is Item 67, Structural Evaluation.\n- Don't get too excited about the last item, Sufficiency Rating. This relies on a complex formula that takes into account how well traffic flows over a bridge, which has no bearing on the bridge's soundness.\n- Because the government is still paranoid about nuclear missile attacks, it makes mapping almost impossible. The latitude and longitude of each bridge will be deleted from your copy of the tape (for national security reasons). No amount of begging has convinced the government that journalists are not a security threat.\n- The database contains Federal Information Processing Standards (FIPS) codes rather than town names. You will have to join the FIPS with your own Census table to determine town names.\n- Beware of typos and miscoding in the location field. In our area, 20 percent of the bridges either had a wrong code or no code at all.\n\nEditor's Note: The full text of the Asbury Park bridge article is available from the IRE Resource Center. Also available are a Cox Newspapers article and NBC Dateline transcript, both of which came from a joint NBI analysis by Elliott Jaspin and David Hinchman in December (#9718). You can contact Paul D'Ambrosio and Rick Linsk at this e-mail address: pmd@cscns.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-july-coming-events","slug":"1994-july-coming-events","url":"https://dwillis.github.io/uplink/article/1994-july-coming-events.html","year":1994,"month":"July","monthNum":7,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article announces upcoming journalism training events including a NICAR seminar in Columbia, Missouri and an IRE computer-assisted reporting conference in San Jose, California.","keywords":["training","journalism","conferences"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nAugust 14-19 NICAR Training Seminar Columbia, Missouri October 6-9 IRE Computer-Assisted Reporting Conference San Jose, California","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-july-why-internet-is-mostly-cool","slug":"1994-july-why-internet-is-mostly-cool","url":"https://dwillis.github.io/uplink/article/1994-july-why-internet-is-mostly-cool.html","year":1994,"month":"July","monthNum":7,"headline":"Why Internet is (mostly) cool","kicker":"","author_name":"Gwen Carleton","author_title":"NICAR Staff","affiliation":"NICAR","summary":"Internet offers journalists powerful tools for gathering information and finding sources during crises and routine reporting, though its evolving nature and accessibility challenges remain obstacles for some news organizations.","keywords":["internet","journalism","sources"],"topics":["Technology","Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","PROFNET","Usenet","FedWorld"],"full_text":"To hear some people tell it, Internet is a prodigious trove. Unfortunately, the treasure's behind a locked door - and the lock keeps changing.\n\nBy now, almost everyone knows what's in there: government documents and regulations, discussion groups, libraries, experts on everything. But knowing how to find what you need, and how to get it, consistently - that's the sticking point. For journalists with little patience and even less time, the result can be frustration.\n\nYet as the network continues to grow, so do its uses. And the successes of the net's most consummate users are giving their warier colleagues reason to take another look.\n\nInternet's usefulness for gathering information in a crisis is unparalleled. It buzzed with live reports during the Tiananmen Square uprising, the Gulf War and the Soviet coup attempt to name a few examples. Michael Meiners, database specialist for the Sun-Sentinel, utilized the system's capabilities to contact people involved in the recent Indian rebellion in southern Mexico.\n\n\"We were able to contact local people and get first-hand accounts as well as a perspective that was not available through conventional media resources,\" he said. \"The reporter was able to conduct interviews over the Net and ended up writing an excellent column.\"\n\nOf course, Internet has proven useful during American crises as well, providing contact with people when all other lines of communication were down. After January's earthquake in Los Angeles, Neil Reisner, database editor for The (Hackensack, N.J.) Record, went online and logged some real-time conversations with survivors. Reisner also went a step further, checking an Internet site that tracks earthquakes around the world. The result was an immediate story with an unexpected international perspective.\n\nDuring political crises and natural disasters, Internet is an obvious source for information. But advocates say the network's more mundane applications are its greater strength.\n\nMeiners described how a reporter at The Tampa Tribune utilized the net to arrange a meeting with a source in Cuba who would have been unreachable by phone or mail. Entire interviews via Internet are becoming more common daily.\n\nAnother strength unique to Internet is its people-finding capability.\n\n\"I use PROFNET all of the time; it's one of the greatest tools I know,\" Reisner said. The database gives reporters access to the public relations departments of scores of universities nationwide.\n\n\"People use PROFNET to find out who can talk to you intelligently on a certain topic,\" explained Barbara Semonche, news librarian at the University of North Carolina, \"The public information officers can check and see if they have someone who is an expert in this area. The reporter can then contact them directly.\"\n\nUSENET newsgroups, which most Internet users can tap into, provide access to an even larger pool of people. According to the latest statistics, there are now more than 7,800 USENET groups worldwide. Through them, reporters can locate people with almost any agenda, hobby or experience, discover local sources or just lay low in hopes of picking up story ideas.\n\nAlthough much of the information on the net is also available through online databases and through printed sources, Internet offers some unique added conveniences - assuming users know what they're doing.\n\n\"Believe it or not,\" said Meiners, \"one of the most successful uses I have found was to find complete texts of song lyrics or poetry.\"\n\nThe net offers help for weightier queries, as well. Favorite databases among journalists include FedWorld (a gateway to hundreds of federal agency bulletin boards), the Library of Congress, FEC campaign finance records, SEC filings, White House schedules, speeches, news briefings, U.S. Supreme Court decisions and census data.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-july-picking-the-right-storage-tools","slug":"1994-july-picking-the-right-storage-tools","url":"https://dwillis.github.io/uplink/article/1994-july-picking-the-right-storage-tools.html","year":1994,"month":"July","monthNum":7,"headline":"Picking the right storage tools","kicker":"Tech Tips","author_name":"Drew Sullivan","author_title":"NICAR Staff","affiliation":"NICAR","summary":"Nine-track tapes remain the dominant storage medium for computer-assisted reporting, but alternative technologies like CD-ROMs and tape cartridges are challenging their dominance, each with different advantages based on budget and organizational needs.","keywords":["storage","technology","data"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["9-track tape","CD-ROM","mainframe","magnetic tape"],"full_text":"Nine-track tapes continue to be the backbone of computer-assisted reporting. For 30 years, these trusty old reels have been the storage medium of choice for both mainframe and minicomputer users.\n\nBut, while nine-track reels remain popular, other media including CD-Roms, tape carts and others are now challenging their dominance. Knowing what to buy depends on your budget and your needs.\n\nCD-ROMS\n\nThere is no reason why any news organization should not have a CD drive. They are cheap and easy to use. Double-speed drives cost about $160 and CDs hold 680 MB of non-corruptible data. Quadruple-speed drives available for about $900 can move data at a respectable rate of 600 KB/sec.\n\nSome government data providers (notably the Environmental Protection Agency and the Census Bureau) are releasing data on CDs and more will be available in the future. In addition, private companies offer national phonebooks, business databases, industry directories, and other useful reference materials on CD.\n\nBut don't expect to get away with just CDs. Government agencies use them as consumer products rather than data storage, says long-time CAR reporter Elliot Jaspin of Cox Newspapers.\n\n\"CDs are terrific to distribute large amounts of data cheaply,\" Jaspin says. \"But there are other kinds of data the government does not want to distribute freely and those will always be on some kind of mainframe storage tape.\"\n\nTape carts\n\nA number of government and commercial users in the mainframe world have opted for 18-track 3480/3490 tape cartridges (carts) instead of nine-track reel tapes. Carts hold more data - depending on the data block size - and are smaller in size.\n\nLarger organizations prefer them because they can be used with automated storage retrieval systems. But, except for the most well-supported CAR programs, not many people can afford their $9,000-plus price ticket.\n\nPrices will inevitably come down, and Jaspin says he believes carts will start to supplant tape reels over the next two or three years.\n\nIn fact, tape reels may never really go away completely because of the large amounts of data stored on existing tapes. Like the transition from 5.25\" to 3.5\" floppies, CAR reporters may need both tape and cart machines for the next eight or nine years, Jaspin says.\n\nOf course, there are alternatives to buying. In any medium to large city, many commercial firms offer to transfer data from carts to reel tapes at a cost of about $1 per megabyte (see your local computer magazines). Mail order services are also available through many companies.\n\nNine-track drives\n\nThough they are admittedly waning, nine-track tapes are still a staple of the mainframe market and should be for the next century. But again, there are alternatives to rushing out and buying a new drive. If you've just started doing computer assisted projects, you might want to hold off investing the $4,000 it takes to buy an entry level drive until you've got a few projects under your belt. NICAR and a number of commercial houses will transfer data from nine-track reel to other media.\n\nHowever, if you do decide to buy a nine-track drive, there are a few things to consider: For PC systems, a manual loading nine-track tape drive, controller card, cables and software cost around $3,800. Though there are cheaper models, they do not handle the high density reels (6250 bits per inch) that have almost become an industry standard.\n\nSpeeds vary significantly from baseline systems reading 10 inches/sec. to high end systems ($7,000 to $15,000) that move 200 inches/sec.\n\nMake sure you have room on your PC to mount the controller card.\n\nBecause reel tapes come from the IBM-dominated mainframe world, it is not surprising they are more compatible to the PC than the Mac platform. For Apple computers, the choices are more limited and more expensive.\n\nA manual loading baseline nine-track tape drive and software for a Mac starts at $4100. The drives use a standard SCSI interface and don't require installing a controller card.\n\nOne problem is that the system only supports one software program called Qutape.\n\nNICAR is now offering CD mastering services, allowing news organizations to move their most used databases like drivers license or voter registration records onto the non-corruptible media. Also, NICAR can transfer data between 3480/3490 tapes and most media at below market price.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-july-pennsylvania-suit-seeks-to-treat-tapes-public-records","slug":"1994-july-pennsylvania-suit-seeks-to-treat-tapes-public-records","url":"https://dwillis.github.io/uplink/article/1994-july-pennsylvania-suit-seeks-to-treat-tapes-public-records.html","year":1994,"month":"July","monthNum":7,"headline":"Pennsylvania suit seeks to treat tapes public records","kicker":"","author_name":"Bob Warner","author_title":"","affiliation":"Philadelphia Daily News","summary":"The Philadelphia Daily News is suing Pennsylvania over access to state payroll records, arguing it's unreasonable for the state to force citizens to purchase expensive paper copies when the information is maintained in a computer database.","keywords":["access","records","database"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["9-track tape"],"full_text":"For year and years, Pennsylvania had treated its state payroll as a public record, providing the names, job titles and salaries of its 100,000 or so employees.\n\nBut there's a catch. To check on any employee, you must visit the state library in Harrisburg or buy your own paper copy of the records for what state officials describe as \"a very nominal charge\" (most recently $540 for 2,160 pages).\n\nLast October, the Philadelphia Daily News asked for the same information on a 9-track tape. The governor's office turned down the request. In May, the newspaper sued.\n\nPennsylvania's Right to Know Law was written in 1957 and doesn't mention computers. But it does provide that any citizen shall have the right to make copies of public records, and leaves it to the custodians of those records to adopt and enforce \"reasonable rules\" to govern the making of such copies.\n\nThe Daily News lawsuit contends it is unreasonable for the state to force the public to deal with paper records when the state itself maintains the information in a computer database. The Casey administration has not yet responded to the suit, which was filed in Commonwealth Court.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-july-editors-note","slug":"1994-july-editors-note","url":"https://dwillis.github.io/uplink/article/1994-july-editors-note.html","year":1994,"month":"July","monthNum":7,"headline":"Editor's Note","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"An editor's note directing readers to resources about an Asbury Park bridge investigation, including articles from Cox Newspapers and an NBC Dateline transcript resulting from a joint NBI analysis.","keywords":["bridge","investigation","resources"],"topics":["Investigative Methods"],"technologies":[],"full_text":"Editor's Note: The full text of the Asbury Park bridge article is available from the IRE Resource Center. Also available are a Cox Newspapers article and NBC Dateline transcript, both of which came from a joint NBI analysis by Elliott Jaspin and David Hinchman in December (#9718). You can contact Paul D'Ambrosio and Rick Linsk at this e-mail address: pmd@cscns.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-august-uplink-update","slug":"1994-august-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-august-uplink-update.html","year":1994,"month":"August","monthNum":8,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR is expanding its seminar schedule across multiple locations and growing its database library capabilities, offering data conversion services while encouraging readers to attend the CAR Trek conference in October.","keywords":["seminars","database","conference"],"topics":["Journalism & Newsroom","Data & Databases","Investigative Methods"],"technologies":["9-track tape","CD-ROM"],"full_text":"Our schedule at NICAR is becoming ever busier, with seminars this fall in Canada, California, Pennsylvania, Florida, Connecticut, Wisconsin, and Oregon. We also plan to add a seminar to our on-campus program in November or December. We will post a note on our listserv, NICAR-L, as soon as the dates are set.\n\nIn addition, the size of our database library increases weekly. We are slicing and dicing data for news organizations across the country. We can convert 9-track tapes into diskettes or CD-ROM for easier use and we can deal with those difficult tape cartridges that are landing on our desks more often.\n\nAs always, please give us a call if you have suggestions for Uplink articles, data acquisition, or anything else in the realm of computer-assisted reporting.\n\nAnd don't forget to be in Silicon Valley Oct. 6-9 for the CAR Trek conference.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-august-canadian-foi-structure-threatens-car","slug":"1994-august-canadian-foi-structure-threatens-car","url":"https://dwillis.github.io/uplink/article/1994-august-canadian-foi-structure-threatens-car.html","year":1994,"month":"August","monthNum":8,"headline":"Canadian FOI structure threatens CAR","kicker":"","author_name":"Bill Doskoch","author_title":"Health and Environment Reporter","affiliation":"The Leader-Post","summary":"A discussion on the challenges faced by Canadian reporters in accessing government records compared to their American counterparts, focusing on the limitations imposed by the current Freedom of Information laws in Canada.","keywords":["Canada","FOI","CAR"],"topics":["Investigative Methods","Government & Politics","Technology"],"technologies":["9-track tape","CD-ROM"],"full_text":"Ah, America! A magical land where the roads are paved with 9-track tapes, where all government information is available at the snap of a reporter's finger. At least that's the way most Canadian reporters see it, especially relative to their condition. Unfortunately, it's an attitude that could halt the advancement of computer-assisted reporting (CAR) in your friendly neighbor to the north before it ever really gets rolling. Interest is building in Canada. The CAR Network of the Canadian Association of Journalists (CAJ) now has 73 members and is hosting a major professional development session on Aug. 27, using trainers from NICAR. But most Canadian reporters despair of ever having the same access to records as their American counterparts, and so they aren't even trying. That's not quite true. The Montreal Gazette won a two-year court battle in 1992 to win access to five years worth of court records stored on computer tape.\n\nHowever, the paper is a branch of Southam, Canada's biggest (in terms of circulation) newspaper chain. Southam owns a company called InfoMart, which is a business online information service. When it heard about the tapes, it asked for them, ostensibly to develop some type of information product with them. That's the last the newsroom ever saw of them - and so away went the dream of setting up a major database on Quebec's courts.\n\nIt should be noted that Quebec has the best freedom-of-information law in the country, and so the victory might be tough to duplicate elsewhere. In my home province of Saskatchewan (perched over North Dakota and Montana), provincial FOI legislation doesn't specify one can have access to government records in computer format - instead, it specifies the government may supply paper transcripts. If anyone knows of a decision where the court said the government should honor requests for electronic records, please let me know about it. On the other hand, I have been making requests for certain types of information already made public on paper - the public accounts, which list amounts of money paid to individuals by government; workers' compensation injury summaries by occupation - in electronic format, and while the bureaucrats have been cautious (\"why would you want it that way?\"), they've also said it should be possible. Cost has been a big barrier in some ways. One reporter for the Ottawa Citizen asked for the tax assessment records for a particular municipal region and was told the information would cost $1,500.\n\nThat's about the same cost as a Statistics Canada CD-ROM containing information from the 1991 census just for Saskatchewan. A national set? $7,500. To give some comparison to the U.S. situation, here's an example given by Peter Calamai - editorial page editor of the Ottawa Citizen and a principal in the Open Government Campaign - during a January 29 speech to journalists:\n\nPurchasing a CD-ROM with about 450 megabytes of information from the U.S. Bureau of the Census would cost $99 U.S. Comparable data from Statistics Canada would cost $320,000 U.S.\n\nThat's because there are policies in place in the United States that allow the government only to charge the cost of transferring the information.\n\nCalamai said the government here looks upon its information as a tradeable commodity from which a profit can be made or costs recovered. The federal government, and some provincial ones, are licensing the distribution of their information through private companies.\n\nIn Ontario, journalists have applied for government records under that province's FOI legislation, only to be told that they can purchase that information from licensed vendors. Currently, the fight here is seen as a political one, not a legal one; hence the Open Government Campaign.\n\nThere is some hope on the horizon. New FOI acts were passed in British Columbia and Alberta recently, each more liberal than any of their predecessors.\n\nIn a speech to journalists at the CAJ's annual convention in April, Marcel Masse said the government recognizes the importance of making the phrase \"open government\" meaningful. Masse is the federal minister of intergovernmental affairs and minister responsible for public service renewal in the new Liberal government (the old Liberals introduced the country's first Access to Information legislation in 1983).\n\nBut ominously, he also talked of \"how to balance the costs in providing information to special audiences with fiscal fairness to all taxpayers.\" He didn't say whether news organizations were a \"special audience.\"\n\nSo there is a fight to be fought before Canadians join their American colleagues at the highest levels of CAR. Here's hoping that fight actually takes place, and that news organizations don't concede victory to governments in the battle for access to public information, fearing they will never win.\n\nBill Doskoch is a health and environment reporter with The Leader-Post in Regina, SK. He edits the newsletter for the CAJ's CAR Network and has written a 76-page primer on CAR.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-august-coming-events","slug":"1994-august-coming-events","url":"https://dwillis.github.io/uplink/article/1994-august-coming-events.html","year":1994,"month":"August","monthNum":8,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A listing of upcoming journalism and computer-assisted reporting conferences scheduled for August and October 1994, including a NICAR seminar in Columbia, Missouri and a CAR Trek conference in Silicon Valley, California.","keywords":["conferences","journalism","1994"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nAugust 14-19, 1994 NICAR Seminar Columbia, Missouri\n\nOctober 6-9, 1994 CAR Trek Conference Silicon Valley, California","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-august-data-services-at-nicar","slug":"1994-august-data-services-at-nicar","url":"https://dwillis.github.io/uplink/article/1994-august-data-services-at-nicar.html","year":1994,"month":"August","monthNum":8,"headline":"Data services at NICAR","kicker":"Transfer and data at reasonable prices","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers affordable data transfer services for journalists using various media formats and provides access to numerous government databases including FEC contributions, FAA data, and mortgage disclosure records.","keywords":["data","journalists","databases"],"topics":["Data & Databases","Technology"],"technologies":["floppy disk","9-track tape","magnetic tape","CD-ROM","magneto-optical disk"],"full_text":"NICAR is now offering inexpensive data transfer for journalists and news organizations.\n\nThe Institute can transfer data to and from \"floppy\" diskettes, 9-track tapes, 3480/3490 cartridge tapes, Trakker 120 and 250 mb backup tapes, compact disks, and some 1.3 gb magneto-optical disks (as well as some 650 mb MO disks).\n\nNICAR also offers a number of government databases including:\n- Federal Elections Commission contributions data\n- Federal Aviation Administration data\n- Home Mortgage Disclosure Act records\n- General Services Administration federal procurement data\n- Alcohol, Tobacco and Firearms gun dealer records\n- National Bridge Inventory System data\n\nCall the Institute for an up-to-date list of prices and available data. The Institute also has FAA service difficulty reports from 1988 to 1994 on compact disk for $125. The disk includes the airmen directory. Call NICAR at (314) 882-0684 for more information.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-august-car-trek-oct-6-9-1994","slug":"1994-august-car-trek-oct-6-9-1994","url":"https://dwillis.github.io/uplink/article/1994-august-car-trek-oct-6-9-1994.html","year":1994,"month":"August","monthNum":8,"headline":"CAR Trek: Oct. 6-9, 1994","kicker":"Panels set for fall conference","author_name":"Jennifer LaFleur","author_title":"NICAR Training Director","affiliation":"NICAR","summary":"IRE and NICAR are hosting CAR Trek, a major computer-assisted reporting conference in Silicon Valley from October 6-9, 1994, featuring over 110 expert panelists and hands-on training courses for 500 attendees.","keywords":["conference","journalism","technology"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["SPSS","SAS","Atlas GIS","MapInfo","9-track tape","CD-ROM"],"full_text":"CAR Trek: Oct. 6-9, 1994\n\nPanels set for fall conference\n\nPick out your favorite pocket protector and plan to head west this fall when IRE, NICAR, and the San Jose Mercury News present CAR Trek: Computer-Assisted Reporting in the 90s, Oct. 6 - 9 in Silicon Valley, Calif.\n\nThe schedule is shaping up for what should be the largest gathering of computer-assisted reporters ever. More than 110 experts in computer-assisted reporting will share their experience and provide practical tips for doing CAR. In addition, 500 attendees will be able to take three hands-on training courses - so register early to ensure a spot.\n\nHere are some of presentations planned for CAR Trek 1994:\n\n- Introduction to Computer-Assisted Reporting\n- The Power of Computer-Assisted Reporting: Examples of what others have done\n- What you need: Software and hardware\n- Introduction to spreadsheets: Using them quickly on the beat\n- Introduction to relational database programs: How to matchmake\n- Introduction to the online world\n- How to get the most out of your searches\n- Linking to your library\n- Introduction to the Internet\n- Number crunching for journalists\n- CAR on a shoestring: It doesn't have to be expensive to do CAR\n- Old fashioned shoeleather: How to get out from behind that computer\n- Beat reporting and CAR\n- Turning online information into stories\n- One project-soup to nuts: The planning and execution of a CAR project\n- Telling the story: Writing a compelling story from numbers is no easy task.\n- Data and the law\n- California access\n- California government online\n- Cyber-law: It's a brave new on-line world, you reprint information from the Net?\n- Government Databases\n- CAR Trek - The Next Generation: The CAR pioneers take a look into the future\n- Surgical strikes with medical databases\n- Workplace safety and databases\n- Inequality: Gender and racial disparity\n- Welfare and housing information\n- Exposing immigration abuses\n- Databases for business reporting\n- Crime in the community: Using data to go deeper\n- Using CAR for education probes\n- Covering the environment with CAR\n- CAR for feature stories\n- The Census\n- Campaign contributions\n- Federal election financing\n- Bad data: How to spot data that might be filled with errors and what to do about it\n- CAR and Macs: See how Macintosh computers can be used for projects\n- Mapping with Atlas GIS and MapInfo\n- CAR and graphics\n- Statistical analysis with SPSS and SAS\n- Nine-Track and cartridge technology\n- Organizing notes and files: Several computer applications for organizing and tracking information.\n- Advanced programming\n- Advanced statistics\n- From Hieroglyphics to data: Techniques for taking data on paper or in ASCII format and putting it into a database manager, using scanners and/or special software applications\n- CAR for broadcast\n- Preach, pitch and pray: Selling the benefits of CAR to the bosses and persuading them to buy the equipment you'll need\n- News of the future: Where is the news business headed?\n- Look who's talking: This session looks at who's typing away at the other end of the line, where the information may reveal far less than what's really out there\n- Surfing the Internet\n- Point and shoot: Here's how you can set up easy-access databases without knowing the ins and outs of software\n- How to set up CAR training\n- Keeping your data all shaped up\n- Building databases\n- CD-ROMs\n\nSpecial sessions\n\nTour of Mercury Center, the San Jose Mercury News' online, phone, and fax service\nDemonstration Room: Ongoing demonstrations of the latest technology including a session on the wireless/portable office\nSpecial editor and publisher program: How to bring CAR into your newsroom\nSuper-session on ethics\nHands-on training in spreadsheets, database management programs, statistical analysis software, mapping, online databases, the Internet, nine-track technology, and building databases\n\nFor more information about this conference, call IRE/NICAR at (314) 882-2042","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-august-the-car-lot-munster-ind","slug":"1994-august-the-car-lot-munster-ind","url":"https://dwillis.github.io/uplink/article/1994-august-the-car-lot-munster-ind.html","year":1994,"month":"August","monthNum":8,"headline":"The CAR lot: Munster, Ind.","kicker":"The state of CAR at a small newspaper","author_name":"Carol Napolitano","author_title":"","affiliation":"The Times","summary":"A reporter describes the frustrating challenges of computer-assisted reporting at a mid-sized newspaper, particularly their 18-month struggle to obtain electronic records from Lake County officials who resist transparency and even suggested withholding public information from the public.","keywords":["journalism","transparency","bureaucracy"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":[],"full_text":"The county data board's attorney actually suggested they might give us electronic records if we promised not disclose any of this public information to the public. That's true, I swear. There were witnesses.\n\nThis is a story of inspiration and frustration. It is about living every day in a world of archaic laws and petty bureaucrats, distracted bosses, and tightening purse strings.\n\nIt is about courting governments, educating editors, and changing the way people think.\n\nWelcome to the world of computer-assisted reporting. It isn't easy to ease your way into CAR, especially if you are a small or mid-sized newspaper.\n\nThe Times is one of those. Our circulation dances around the 70,000 mark. But, unlike many newspapers our size, we are luckier than some because our publisher and many of our editors have vision.\n\nBut that vision often is obscured by financial realities and deadline pressures. And that, unfortunately, is the way things are just about everywhere.\n\nFew papers give their reporters a year to analyze data, or spend $30,000 routinely to keypunch information. Yet, sometimes, it seems those are the only stories we hear about CAR programs. And we ask ourselves, \"Gee, why can't it be so wonderful at my paper?\"\n\nWell, I'm here to share some CAR stories that aren't so wonderful. But, I hope they will be inspirational to reporters who struggle each day to get a CAR program off the ground.\n\nThose pesky data requests\n\nThe Times has the misfortune of covering Lake County, Indiana, where corruption and secrecy are a way of life in government and too many bureaucrats think we should be eternally grateful for the mere fact that they show up for work.\n\nA lot of records here are not computerized at all. And for the ones that are, county officials have hired a private agency, Cenifax, to oversee them.\n\nWe've been pestering the county for almost 18 months now to get records - any records in any electronic format. Officials' responses have ranged from ignoring us to contacting their private attorneys, viewing our requests as some kind of personal legal threat.\n\nOur paper even sat down with the county Data Board at one of its public meetings to ask for some kind of policy or procedure for acquiring electronic records.\n\nThe board's attorney actually suggested they might give us electronic records if we promised not disclose any of this public information to the public. That's true, I swear. There were witnesses.\n\nWhen the meeting was over, the Data Board added one more insult. It refused to give us written minutes or a copy of the tape recording they made of that public meeting. In March, when Rebecca Buckman of the Indianapolis Star did a story about access to electronic records, she asked the head of Cenifax why his agency and county officials were so uncooperative. His answer: \"The reason for that is because the files would be obsolete in a matter of days and therefore any statistical abstractions drawn from the data\" would be outdated. \"Paper gets obsolete too,\" he said. But with computer tapes \"you can manipulate (data) in any manner you see fit, therefore deriving any statistics that you want...\" Gee, I don't remember any law that allows government to withhold public records because they are worried someone out there might misconstrue them.\n\nSpeaking of the law\n\nSpeaking of the law, Indiana's public records access law dates back to 1200 B.C., I think. For instance, death records are private! The state figures revealing someone's death certificate is violating their medical privacy. How can a dead person have privacy?\n\nIn fact, to get death records you have to convince the state registrar that there is a \"direct tangible and legitimate public interest\" at stake, according to the law.\n\nSo, when Lake County became embroiled in vote fraud scandal, we figured we had a good case to get a copy of some death records. Already we had uncovered cases of \"dead\" people voting and things were getting so hot the county prosecutor got in on the act.\n\nHere's what the state registrar said, in response to our request:\n\n\"Clearly, you have no direct interest in the matter recorded on the death certificates. In addition, I do not believe that your request shows an extraordinary case that is a direct tangible and legitimate public interest.\"\n\nIn addition to public officials who don't consider election fraud to be of public interest, we must contend with a state law that gives those same officials the choice as to whether to provide us with electronic data or paper data. You can pretty much guess what option they take.\n\nRecently, the Hoosier State Press Association, working with journalists from around the state, drafted a new records access law that would require government to give people records in the format requested, if they had the ability to produce that format.\n\nThe law also would have required agencies to charge people no more than the direct cost of reprogramming computers to produce the public record.\n\nThe bill never got out of committee.\n\nBattles and luxuries on the homefront\n\nWhen we're not fighting government, we're fighting technology and time.\n\nThe Times cannot afford the luxury of letting me work on CAR projects full time. It can't afford to hire a computer whiz.\n\nAnd our own technical people have to keep 18 papers in our chain running. They just don't have the time for me or my program.\n\nSo, being a reporter who stumbled into computers, I grope my way through the world of CAR blindly, but with hope.\n\nI absorb manuals that often leave me asking more questions than they answer. I get on the computer and learn by making mistake after mistake.\n\nI swallow my pride and mutter endless apologies each time I find myself calling that elite group of CAR geniuses that you often hear speaking at conferences.\n\nBut, after 18 months of talking to editors and pleading my cases, I am lucky to have acquired a good variety of CAR tools - We have dBase for Unix on all terminals; FoxPro on our Macs and on my laptop; Quattro Pro for Windows on my laptop and Lotus on a few Macs. My paper has not been stingy when it comes to buying software.\n\nBut, like most other papers out there, I don't get a lot of time to work with my techno-toys. Deadline pressures demand I spend a lot of my time writing daily and weekend stories.\n\nIt's hard to convince an editor to give you a few weeks to diddle around with the census data or just let you sit there and learn the software they've bought you. Often, you hear CAR gurus say the way they got more time was to produce some quick hitting stories. That works, if you live in a county or state where you can get data for quick hitters - like marriage licenses for a story on the most popular day to get married, or property records for a story on who owns the most land or who has the most expensive homes in towns. But in this state and this county, those kinds of records either aren't computerized, or aren't public, or government just won't give them to you electronically. So these popular quick hits often aren't an option for us here.\n\nOne small step for CAR, one giant step for the Times\n\nAs a result, most of the dozen or so stories we have done in the past year have come from homemade databases.\n\nWe created a database on gun permits to do a story about what kind of people are carrying guns. We conducted polls on hot local issues, like riverboat gambling, and used our CAR software to analyze results.\n\nWe compiled a database of all murders in Gary, Indiana, which had the highest per capita murder rate in the nation at the time, and used it to write a week-long series on homicide in that city.\n\nWe gave students in the area a geography test, entered their scores into a database, and wrote a story about geography knowledge.\n\nAnd most recently, we used surveys filled out by police departments to take a look at our law enforcement resources and whether they were adequate to combat crime in the area.\n\nWe did invest in one database - the census on CD-ROM and got a great little quick hit story out of that. We used the data to find heavily Irish neighborhoods to spruce up our St. Patrick's Day story.\n\nSo, even without the cooperation of government and without endless financial resources and without the unwavering editorial support I'd love to have, we have managed to breathe life into a CAR program here.\n\nAnd, recently, we got our first electronic records from Lake County, thanks to one county official who still understands that public officials are there for the public.\n\nSee? There are miracles.\n\nIn this business there always will be people who want to know why you can't produce that Pulitzer-quality CAR series in a few days; who don't understand why it can take weeks, or even months, to make data usable.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-august-choosing-the-right-computer","slug":"1994-august-choosing-the-right-computer","url":"https://dwillis.github.io/uplink/article/1994-august-choosing-the-right-computer.html","year":1994,"month":"August","monthNum":8,"headline":"Choosing the right computer","kicker":"Need a kicker","author_name":"Drew Sullivan","author_title":"NICAR Staff","affiliation":"NICAR","summary":"Strategies employed by the Philadelphia Daily News to obtain campaign contribution records electronically from gubernatorial candidates, highlighting the hurdles and eventual successes.","keywords":["campaign data","negotiation","electronic records"],"topics":["Technology","Journalism & Newsroom","Investigative Methods"],"technologies":["PC","Pentium","Power PC","OS/2","IDE","SCSI","ISA","EISA","VESA Local","PCI"],"full_text":"Judging from the number of calls NICAR has been getting on computer hardware, news organizations have begun putting more money into CAR programs. There are several special requirements needed for a good computer system to do this kind of work. Here are some things to consider.\n\nGo for processor speed\n\nThe faster the processor, the faster programs are going to run. But if you're going to cut corners, this is a good place to do so. That's because in CAR, there is more concern about storage and data transfer speeds than about the CPU (central processing unit). Much of the time wasted is spent going to peripherals (such as the hard drive or a nine track machine) and reading the data.\n\nStill, a fast CPU is a wonderful thing. If you can afford them, there are a number of crunchers out there that can smoke. At a minimum, I would suggest a full 486 machine (not an SX) with speed of at least 33 MHz. Currently, 50, 66, 90 and 100 MHz machines are on the market in the 486 variety.\n\nPentium machines are a step up. The Pentium is the next generation CPU chip after the 486 chip. Prices for Pentiums are falling very quickly so they make a lot of sense. If you opt for the 486, get one that is easily upgraded to a Pentium.\n\nThe future lies in the Power PC. It is a jet engine compared to the Pentium's propeller. It will be cheaper, better and faster within a year. Not a lot of software written for the chip takes advantage of its speed yet. Keep a close eye on them.\n\nHow many megs of RAM?\n\nEasy answer here. Get a minimum of 16 mb of Random Access Memory (RAM). Not enough RAM will slow your system down. Think of it as desk space. If you don't have enough desk space to work on, you'll have to keep swapping files in and out of your storage area. Plenty of RAM lets you lay it all out and not make wasteful trips. Make sure your RAM is expandable to a reasonable amount (about 128 mb).\n\nShopping for disks\n\nFor a discussion of peripheral storage systems, see the two previous issues of Uplink.\n\nMake sure you have enough hard disk space. It's usually the first thing you run out of. Programs are getting big (OS/2 takes up 80 mb of disk space), and you'll need a big work space for your databases. If you can afford it, don't buy less than 1 gigabyte (a gigabyte is equal to 1,000 mb). Better to buy a 2 or 2.4 gb drive. It'll be worth it in the long run.\n\nIf you go with an Integrated Drive Electronics (IDE) system, you're stuck with only two drives and a maximum partition of 528 mb each. The alternative is a Small Computer System Interface (SCSI) controller. SCSI can handle up to seven peripherals (compared to four for IDE). It also can handle any size drive on the market.\n\nSCSI controllers are more expensive and a little more complicated to set up. They're also much faster. Fast and wide SCSI hard drives can handle up to 40 mb/sec compared to mb/sec for IDEs. But look for the enhanced IDE drives coming out in the near future. The new standard will support up to 8.4 gb drive partitions.\n\nTake the local bus\n\nA bus is an electronic transportation system on your motherboard that allows you to transfer data at higher rates. I strongly suggest getting a computer with a high-speed local bus.\n\nIf you buy a PC at Sears, it's going to have the old-fashioned Industry Standard Architecture (ISA) buses, which have a 16-bit processor and a 8 MHz speed.\n\nIn 1988, Extended-ISA (EISA) came along. It had a 32-bit processor but still operated at 8 MHz. Now VESA Local (VL) and Peripheral Component Interconnect (PCI) buses operate up to 32 bits and 64 bits with speeds up to 66 MHz. While IDE can transfer data at 8 mb/sec, the new buses can move data at 140 mb/sec with bursts up to 320 mb/sec.\n\nThe faster access allows you to get the maximum speed out of your peripherals. But be careful. These buses can create some potential compatibility problems if you don't spec out your system carefully.\n\nBasic Computer System: 486-33 MHz system, 430 megabyte hard drive, VL-local bus, 8 mb RAM. Approx. cost: $2,200.\n\nHigh-End System: Pentium 90 MHz system, 2 gigabyte hard drive, PCI bus, 16 mb RAM. Approx. cost: $4,400.\n\nPrices include monitor, keyboard, videocard, tower case, mouse.\n\nIf you have any questions or if you've discovered something you would like to share with other journalists, call Drew Sullivan at (314) 882-0684 or better yet, e-mail me at C621897@mizzou1.missouri.edu","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-august-tackling-database-disk-hogging","slug":"1994-august-tackling-database-disk-hogging","url":"https://dwillis.github.io/uplink/article/1994-august-tackling-database-disk-hogging.html","year":1994,"month":"August","monthNum":8,"headline":"Tackling database disk-hogging","kicker":"Tech Tips","author_name":"Chris Feola","author_title":"Reporter","affiliation":"Waterbury Republican American","summary":"Database management programs like Paradox often require temporary disk space equal to several times the size of tables being processed, and can fail when configured to use a full drive instead of one with available space, a remnant of 1980s DOS limitations.","keywords":["database","storage","software"],"topics":["Technology","Data & Databases"],"technologies":["Paradox","FoxPro","Access","WordPerfect","DOS"],"full_text":"Computer hardware tends to run a couple of generations ahead of software. This is sort of natural for programmers; there's not a lot of sense in writing a program for equipment that doesn't exist.\n\nIt can be a royal pain for users, though, when your screaming high-tech machine won't work because your software was designed for the latest and greatest hardware - of 1988.\n\nA case in point: many database management programs have notorious problems with free disk space.\n\nThe problem lies in the need for large stretches of open disk space for temporary tables. Although this article deals with the problem in Paradox, the theories apply to FoxPro, Access, and other database management programs as well.\n\nAs a rule of thumb, Paradox needs free disk space equal to three times the size of the tables you want to use. If you want to sort a 30 megabyte table, for example, you need 120 megabytes of disk space: 30 for the table, and 90 for the temporary tables.\n\nThe temporary tables are built by Paradox as it reformats a table. It's the digital equivalent of running off 10 copies of a 50-page report on a copier, then spreading everything out on a table to assemble 10 sets of reports with the pages in the right order.\n\nKnowing that database management programs tend to be disk pigs, I keep a 550-megabyte drive empty for database work.\n\nSo imagine my surprise one day when Paradox gave me an \"Out of disk space\" error message while rekeying a 12-megabyte table.\n\nAs you can guess, I was somewhat puzzled. Even if Paradox needed open disk space equal to 10 times the table, I was still good to go with 550 megabytes of free space.\n\nBeing geniuses, we did what we always do: We tried it again, on the always-popular theory that computers are odd and will sometimes not work for no particular reason.\n\nNo go. We got the same message: \"Out of disk space.\"\n\nWe still needed the table sorted, however, so we thought of a different tack. Instead of rekeying the table (keys are fields used to automatically sort a database table), we decided to try manually sorting the table with the SORT command.\n\nThis proved unsuccessful, but provided a solution. The sort failed, producing an \"Out of disk space\" error message. However, unlike the error messages that came up while trying to rekey the table using the table restructure command, the SORT error message offered the name of the offending table: C:TEMP$$$AF123.TMP.\n\nNow we were getting somewhere. Paradox, in its infinite wisdom, was trying to shoehorn its temporary tables into my jammed C: drive, rather than my wide-open I: drive. This is a throwback to the early days of DOS and PCs. A decade ago, you could plunk down several hundred dollars and get an enormous 10-megabyte hard drive. DOS was built to treat the drive as a single object - after all, who could need more than one drive or partition? But as far back as DOS 3.3 - around 1988 - DOS started to provide the ability to handle multiple partitions for large drives. And when your system gets up into the gigabyte range, dividing your drives up into partitions is a basic rule of disk management. Many large systems are now using a separate partition for the operating system - an arrangement I favor. Modern system design reserves drive letters A: and B: for floppy drives; C: is usually the boot partition on the hard drive. Setting up a separate C: partition has lots of advantages. This arrangement makes it easy to switch operating systems, for one; a switch I make about once a month it seems lately. My setup uses a 150-megabyte C: drive. My work drive for Paradox is the empty 550 megabyte I: drive. But instead of working on I:, Paradox was putting the temp files on C:.\n\nParadox is putting its temp files on C: because that is where the system pointer is set. But the system pointer is supposed to be for system files, rather than program files. WordPerfect, for example, lets you set the pointer for that program's temp and backup files; if you don't set anything, the program defaults to its home directory. Once we knew what Paradox was up to, the problem was easy to solve. We reset the system pointer to the I: drive, and Paradox cranked happily away.\n\nTips for conquering the disk problem:\n\nSet up a temp directory. Pick a drive with lots of open space and make sure you don't let it fill up. Make a temp directory using the Make Directory command. \"MD TEMP\" will make a directory called \"TEMP.\"\n\nSet the system pointer to your new temp directory by putting the following line in your temp directory: \"SET TEMP=C:TEMP\". That line points to the TEMP directory on the C drive; obviously, substitute the correct drive and directory.\n\nPeriodically clean out your temp directory. There are lots of ill-behaved programs that don't clean up their temp files. You can run out of disk space real quick - and for no reason - if you have a program leaving around large temp files.\n\nNever clean out your directory when Windows is running!! If you shell out to a DOS prompt and erase temp files while Windows is running, you will crash your system and lose data from any programs that are running. Always exit to DOS and exit all programs before cleaning out the temp directory.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-august-how-to-negotiate-for-campaign-data","slug":"1994-august-how-to-negotiate-for-campaign-data","url":"https://dwillis.github.io/uplink/article/1994-august-how-to-negotiate-for-campaign-data.html","year":1994,"month":"August","monthNum":8,"headline":"How to negotiate for campaign data","kicker":"Begging for records in Pennsylvania","author_name":"Bob Warner","author_title":"","affiliation":"Philadelphia Daily News","summary":"A Pennsylvania newspaper persuaded eight of ten major gubernatorial candidates to provide campaign finance reports on diskette rather than paper, overcoming technical and political resistance to create an electronic database for investigative reporting on campaign contributions and special interests.","keywords":["campaign","database","candidates"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":["Paradox","Quattro Pro","Monarch"],"full_text":"Facing record field of 12 candidates for Pennsylvania governor, and wanting to get their campaign contributions into an electronic database, we tried a new tack: begging. We asked all the candidates to give us their campaign finance reports on diskette instead of on paper.\n\nWe thought this was a pretty creative way to deal with our data entry budget, roughly $12.95 for the year. But creatively speaking, it was nothing compared to the answer we got from the candidates and their handlers.\n\nFour told us it was technically impossible. (The simple response: it isn't. If they can get their computers to spit out reports for the state election bureau, they can create an electronic copy of the file for you and your newspaper).\n\nTwo of them told us they couldn't give us the diskettes because they used proprietary software. (Tell them you don't want the software, just the data).\n\nHalf of them told us we were asking them to do our work for us. (Not exactly. They'd already done the work, by using the computers for their campaigns. If they were still using paper and pencil for their internal record keeping, as two fringe candidates were, we certainly didn't expect them to create computer files for us).\n\nThe bottom line: After several months of persistent phone calls, not-so-gentle prodding and a few arguments, eight of the 10 major candidates provided their contributions on diskette.\n\nIt helped that, on the Republican side, the race was so close that no one wanted to take any unnecessary shots from the media; eventually, all the GOP candidates gave us their stuff electronically.\n\nOn the Democratic side, Lt. Gov. Mark Singel had a solid lead all the way. We got nothing from Singel before the May 10 primary. Later, with the fall campaign looking close, he decided to send us a diskette.\n\nThe data came to us in various formats. We converted it to Paradox, generally using the conversion tools provided by Paradox and Quattro Pro. Our only significant conversion problems involved changing text files into data files; for that we used Monarch software, first brought to our attention (through Appliance) by Tom Boyer of the Norfolk Virginian-Pilot and Ledger-Star. So far, we've used our database:\n\nto identify 170 individuals, law firms, lobbyists, and other special interests giving $1,000 or more to multiple candidates;\n\nto describe how one candidate used connections at the Philadelphia Parking Authority to secure nearly $100,000 in campaign donations from authority contractors and employees;\n\nto keep track of cumulative contributions, making it easier to identify the candidates' top backers.\n\nAnd whatever stories we produce during the fall campaign will become even more valuable after the next governor takes office.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-august-bits-bytes-and-barks","slug":"1994-august-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1994-august-bits-bytes-and-barks.html","year":1994,"month":"August","monthNum":8,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"Reporter","affiliation":"The Olympian","summary":"The article highlights a CAR award won by The Olympian for a restaurant inspection database story, announces Illinois criminal justice system profiles for all 102 counties, and reports the launch of Congressional Quarterly's new gopher providing access to legislative documents and congressional information.","keywords":["reporting","database","legislation"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["gopher"],"full_text":"CAR story wins regional SPJ award - Reporter Devin Smith and The Olympian recently won an award from the regional Society of Professional Journalists for a database story on restaurant inspections. The article took first place for consumer reporting in the under 50,000 circulation category.\n\nIllinois offers county justice system profiles - The Illinois Criminal Justice Information Authority (ICJIA) has recently completed profiles of the criminal justice system for each of Illinois 102 counties. Sal Perri, research analyst with ICJIA, said the reports contain analyses of drug and violent crime data for each Illinois county, comparing trends experienced over the past decade across counties with similar population characteristics. For more information about the reports, contact the Authority's Information Center at (312) 793-8550 or send an e-mail message to Perri at 73244.2726@compuserve.com\n\nCongressional Quarterly Gopher now up - Congressional Quarterly has announced the opening of its new gopher, which will allow users on the Net to explore current and archival CQ documents. Files now accessible via the Net include:\n\n- Lead stories of the current CQ Weekly Report, as well as an archive of past stories.\n- A weekly news brief from the CQ Researcher.\n- Status of appropriations bills and other major legislation.\n- Results from key 1993-1994 roll call votes.\n- Updates on congressional and gubernatorial elections.\n- Information on members of Congress, including committee and caucus rosters, leadership positions, room numbers, and telephone numbers.\n\nTo access the gopher, connect to gopher.cqalert.com. For more information on the CQ gopher, contact Melissa Henderson at (800) 432-2250, ext. 578, or e-mail her at mhenderson@cqalert.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-september-uplink-update","slug":"1994-september-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-september-uplink-update.html","year":1994,"month":"September","monthNum":9,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"NICAR is expanding its online presence through new platforms like JForum and CompuServe while preparing for the upcoming CAR Trek conference in Silicon Valley. The organization continues to build its database library and receives growing submissions of computer-assisted reporting stories.","keywords":["conference","database","journalism"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["CompuServe","JForum"],"full_text":"While gearing up for the October computer-assisted reporting conference in Silicon Valley (CAR Trek), we're keeping busy with new online activities and an ever-expanding database library. Our latest foray into cyberspace involves Compuserve and JForum. NICAR now has a place in JForum, and IRE member and dedicated CAR journalist Neil Reisner has graciously volunteered to help Our presence on JForum will be a happy addition to our lists at the University of Missouri. We also continue to receive excellent examples of CAR stories from around the country, and the submission of articles to Uplink increases each month.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-car-uncovers-urban-ills","slug":"1994-september-car-uncovers-urban-ills","url":"https://dwillis.github.io/uplink/article/1994-september-car-uncovers-urban-ills.html","year":1994,"month":"September","monthNum":9,"headline":"CAR uncovers urban ills","kicker":"Computers help expose crime, poverty","author_name":"Andy Hall","author_title":"investigative reporter","affiliation":"Wisconsin State Journal","summary":"A Wisconsin newspaper describes how it used computer-assisted reporting and traditional investigative methods to examine urban problems in Madison through a multi-part series called 'City of Hope,' covering issues like juvenile crime, racism, poverty, and housing shortages.","keywords":["journalism","investigation","technology"],"topics":["Investigative Methods","Data & Databases","Crime & Justice"],"technologies":["Excel","FoxPro","SPSS","CD-ROM","9-track tape"],"full_text":"On Oct. 11, 1992, the Wisconsin State Journal published a disturbing front-page headline: \"Crisis hits home front. Creeping urban ills threaten our special city.\" We came up with an upbeat title, though, for the new series - \"City of Hope.\" Nearly two years later, the \"City of Hope\" logo continues to accompany our reports examining Madison's urban problems. And while reporters have rotated in and out of the project, one member of the reporting team our newsroom's personal computer - has played an increasingly valuable role. This project is an excellent example of how computers can become an important, inexpensive and innovative tool. Our newspaper, after all, had only one PC (a Zeos 386 purchased in 1991 for less than $2,000). Software costs (Excel spreadsheet, FoxPro for database analysis, SPSS for statistical analysis) have totaled about $1,000. Data costs are low - about $350 for Madison police records, $150 for census on CD-ROM. A decent CD-ROM reader costs about $350.\n\nNone of our reporters was trained in computing. We learned while doing the project. (That was a mistake. Get some decent training up front. It saves time and frustration). And we shamelessly scrounged for free help from in-house and university experts, as well as journalists who have tackled other computer-assisted reporting projects.\n\nFirst, a summary of what we've done. In ten major packages, we've outlined the problems - and potential solutions - regarding juvenile crime, racism, the safety net, repeat offenders, a neglected neighborhood that became violent, crime patterns, links between crime and socioeconomic factors, a shortage of jobs that pay livable wages, and a lack of low-cost housing. We've tried to show how the problems, and their solutions, are connected to one another.\n\nOur project is an unusual blend of traditional investigative reporting methods, lengthy periods spent out in the community in residents' homes, and computer-assisted journalism. In addition, before we published most packages, we convened group community leaders and experts on that topic to discuss solutions to the problems. This helped cause residents and, frankly, our editors, to buy into the project. Findings and techniques include:\n\n- Quality threatened: (Oct. 11, 1992) A dramatic increase in violent crime and poverty raises concerns Madison could lose its high quality of life - and its national reputation as a great place to live. We used interviews and a took a very basic look at crime and demographic data.\n\n- Juvenile crime skyrockets: (Nov. 8-11, 1992) The juvenile justice system is a revolving door for hard-core offenders, with troubled teens bouncing from one program to another while their crimes multiply. This time we used interviews and a manual review of court records.\n\n- Racism widespread: (Dec. 20, 1992) Blacks suffer from frequent racism despite the city's liberal image and widespread residential integration. We conducted a public-opinion survey designed, conducted, and analyzed by reporters and in-house polling workers, with advice from university experts. An in-house marketing director with expertise in SPSS, a statistical analysis program, helped us analyze the findings.\n\n- Fixing fragmented system: (April 18-27, 1993) Local leaders are trying to transform a human services system as fragmented and troubled as the community it serves.\n\n- Repeaters spur crime: (June 20, 1993) Repeat criminals contribute to dramatic increases in crime here. People charged with committing more than one crime accounted for nearly 85 percent of criminal charges from 1988 to 1992. We used a computer to analyze data obtained from county court officials. A computer expert who runs our company's billing and subscriber operations helped us transfer the data from a 9-track tape into the newsroom PC. (He has a tape reader, worth $15,000, so the newsroom didn't need to purchase one.)\n\n- Worthington's pain: (Sept. 26-27, 1993) Three decades of neglect by apartment owners and city officials left the Worthington Park neighborhood on Madison's near East side vulnerable to crime and decay. Increased police patrols in other city neighborhoods pushed drug dealers into Worthington. We spent time in the neighborhood to witness problems and become acquainted with the residents and police, manually tallied microfiche building-inspection reports and used FoxPro to compile census and police data for the neighborhood. The police department, at our request, compiled an analysis of its data, too. The census data was on a $150 compact disc of Summary Tape File 3A (commonly called STF 3A) that has proven its value many times over.\n\n- Crime concentrated: (Feb. 20-22, 1994) While violence in Madison stabilized last year, most serious crime continues to be concentrated in several neighborhoods. Using help from an in-house computer expert, we transferred 1992 and 1993 police data from 9-track tapes to our PC. We analyzed calls for service - not just incidents officially designated as crimes - because they give a richer portrait of neighborhood strife. We focused on 10 types of violent incidents. Intensive time spent in the city's most violent neighborhood brought the story to life and spurred ideas, too, for the data analysis. We compared demographics to violence patterns.\n\n- Unemployment, crime linked: (March 20-21, 1994) A natural follow-up to the previous package. An analysis of police and U.S. Census data shows that the areas of highest unemployment, large numbers of idle youths, and highest numbers of blacks are the areas with highest violence. Findings suggest policymakers might focus on better jobs and education to lower violence in the community. Poverty experts from the University of Wisconsin helped us use our crime and census data in a regression analysis - A method researchers commonly rely on to explain social factors. We paired this story with an in-depth comparison of two families, one poor and one well off. We also used PUMS - the census' Public Use Microdata Sample - to get a closer look at who's poor.\n\n- Job market faulty: (March 27, 1994) Underlying problems in the Dane County job market are pushing poor residents into a swelling number of low-paying, dead-end jobs. Interviews, census data, and employment data told most of the story. But a simple half-day exercise - typing job-training and pay data for the fastest-growing jobs into FoxPro - gave the analysis some real bite: \"An analysis of Dane County's 40 fastest-growing occupations reveals that most newly created jobs won't pay enough for poor families to climb out of poverty.\"\n\n- Shortage of low-cost housing: (May 1-2, 1994) Affordable housing is largely segregated and confined to certain urban neighborhoods in Madison, while fast-growing suburbs have little affordable housing. And the gap is expanding. We used FoxPro and census data to determine how many housing units in each Dane County community were within reach of poor households. Housing experts, who'd long wished for such an analysis, helped us decide precisely what to look for. Graphics and maps played a key role in making housing and crime packages understandable. We also used the Comprehensive Housing Affordability study, which is free from HUD. The study is also available on CD.\n\nThe results are pouring in. Police patrols have been moved and a neighborhood center is being opened in Worthington Park. The police now refer some public inquiries about crime to the State Journal. A \"jobs summit\" is being held this summer to find ways to improve the local labor market. A county task force has been formed to increase the supply of low-cost housing.\n\nWe've begun incorporating the lessons of \"City of Hope\" into our daily beat reporting. Reporters are asking developers about the affordability of new housing projects, and they're finding out whether new jobs pay livable wages.\n\nAnd in the newsroom, we're replacing our mid-1970s Harris mainframe with a PC-based system, meaning every reporter will have a 486 computer. We are committed to using these powerful machines not just for writing stories, but for finding the real stories hidden between the numbers.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-coming-events","slug":"1994-september-coming-events","url":"https://dwillis.github.io/uplink/article/1994-september-coming-events.html","year":1994,"month":"September","monthNum":9,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Listing of upcoming journalism conferences including the CAR Trek Conference in Silicon Valley (October 1994) and the NICAR Seminar in Columbia, Missouri (January 1995).","keywords":["conferences","journalism","events"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nOctober 6-9, 1994 CAR Trek Conference Silicon Valley, California\n\nJanuary 8-13, 1995 NICAR Seminar Columbia, Missouri","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-tapes-for-those-who-missed-it","slug":"1994-september-tapes-for-those-who-missed-it","url":"https://dwillis.github.io/uplink/article/1994-september-tapes-for-those-who-missed-it.html","year":1994,"month":"September","monthNum":9,"headline":"Tapes for those who missed it","kicker":"","author_name":"Rosemary Armao","author_title":"Executive Director","affiliation":"IRE and NICAR","summary":"IRE's popular Optional Day program has become increasingly focused on computer-assisted reporting (CAR) training, with record attendance at the St. Louis conference, prompting debate about integrating CAR panels into the main conference schedule. The article describes available resources including handouts, tapes, and a CAR guide for journalists who missed the training sessions.","keywords":["training","journalism","technology"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["modem/BBS"],"full_text":"We gotta change the name. For years the little-something-extra program staged before the traditional Friday start of IRE's national conference has been referred to as the Optional Day. But the fare of that program changed two conferences from such traditional staples as ethics, libel and writing to computer-assisted reporting, and ever since, the Optional Day has been more like Mandatory Day.\n\nHands-on training classes were filled and closed nearly as soon as the sign-up sheets were hung in New York City in 1993. And in St. Louis this June a record crowd of 450 crammed an Optional Day devoted to computer skills and on-line resources.\n\nIRE/NICAR's low prices ($35 for a full-day computing seminar at St. Louis) were part of the draw, but no more so than the array of talent and topics. We've begun debating whether it's not time to start integrating CAR panels into the traditional conference program. That would be consistent with our philosophy that while CAR is the new, hot technique among journalists, it is, in the end, not much different than conquering the telephone or the VDT in order to produce better journalism. Still, the numbers persuade us there's a special demand to focus on CAR training. If you didn't make it to St. Louis, we invite you to look over what you missed and catch up. For little more than the cost of copying and mailing, you may order handouts from any of the Optional Day panels. Copies of \"Computer Assisted Research: A Guide to Tapping Online Information\" by Nora Paul are on sale for $3 plus shipping and handling. And audiotapes of the sessions are available for $8 each by ordering directly from Sound Images Inc., P.O. Box 460519, Aurora, CO 80046. Also available are tapes of all the sessions from IRE's first computer-training boot camp, a three-day, mind-wringing immersion in CAR last fall at Raleigh. To be honest, tape sales of the computer-assisted sessions don't sell as well as those from other advanced-reporting panels. We suspect there's a fear that audio without the attendant visual component of screens and demos don't work.\n\nThey do. Ask for reviews of the tapes you're interested in from IRE/NICAR staffers who listen to nothing else while driving around Columbia and off on summer vacations. Here's a listing of the St. Louis tapes. Call (314) 882-2042 if you'd like more information or the entire list of Raleigh tapes.\n-Online on the Beat: 30 Ideas in 30 Minutes (Bill Dedman)\nPrimer on Online Searching: Power Up, Modem In, Log On. Everything you Need to Know to Get Started (Beth Marchak)\n-Specialized Session on Business Reporting (Rob Wells, Penny Loeb)\n-Data Analysis on the Beat: 30 Ideas in 30 Minutes (Shawn McIntosh)\n-The Mystery and Power of Internet (Dan Woods, Dan Gillmor)\nCreating and Recycling Databases: When to Use a Database and Which Ones to Keep Available for Use on Deadline (Jeff Kummer, Teresa Leonard, James Derk)\n-Specialized Session on Government and Social Sciences (Dan Browning, Rich Robertson, Shawn McIntosh)\n-Online on the Cheap (Nora Paul, Mike Berens)\n-Using Spreadsheets and National Tabular Data to Show Where Your Town or State Ranks (William Casey, Rich Gordon, Chris Schmitt)\n-Environment on Computer (Dave Davis, Jane Kay, Wendell Cochran)\n-Attribution and Accuracy on the Information Highway (Nora Paul)\n-Deadline Online: Using Bulletin Boards to Find Eyewitnesses and Experts and to Profile People (Deborah Crowe, Matt Reavy)\n-Mapping Software on Deadline: How to Plot Census, Crime and Gerrymandering Information (Susan Brown, Jennifer LaFleur)\n-Specialized Session on Medicine and Science (Bill Allen, Paul Overberg, Beth Marchak)\nAccess to Federal, State and Local Online Records: Success Stories and Pending Battles (Sandra Scott, Mike Casey)\n-Statistics Can Be Your Friend: Five Statistical Tools You Need Know (Jon Schmid, Phil Meyer)\nThe Electronic Newsroom (Corky Johnson, Karl Idsvoog, Mark Nichols, Neil Reisner)\nGetting Started (NICAR staff)","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-train-with-best-in-car","slug":"1994-september-train-with-best-in-car","url":"https://dwillis.github.io/uplink/article/1994-september-train-with-best-in-car.html","year":1994,"month":"September","monthNum":9,"headline":"Train with best in CAR","kicker":"Panels range from beginning to expert","author_name":"","author_title":"Panels range from beginning to expert","affiliation":"","summary":"CAR Trek, a four-day computer-assisted reporting conference in Silicon Valley from October 6-9, offers panels and hands-on training from pioneering journalists and top CAR practitioners, with registration at $150.","keywords":["conference","reporting","training"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet"],"full_text":"If you can attend just one computer-assisted reporting event this year, make it to CAR Trek in the Silicon Valley, Oct. 6-9. The people who pioneered these reporting techniques, the top practitioners of the craft, and the best instructors of CAR will talk and teach for four intense days of panels and hands-on training at the Westin Hotel (800-228-3000 or 408-986-0700 for reservations) in Santa Clara. Registration is $150. Call IRE at 314-882-2042 for details. CAR panels will range across every kind of computer-assisted reporting. Hands-on training will cover beginning and advanced topics, including the Internet. You'll be able to snag speakers for one-on-ones in the hallways and in the computer demonstration room where you can go into greater detail about what interests you. Sponsored by IRE/NICAR and the San Jose Mercury News, the conference also will showcase a host of vendors showing off their databases and online wares. CAR Trek has a terrific team of journalists from newspapers and broadcast. You may have e-mailed them or talked to them on the phone - now come and meet them.\n\nConference registration is $150. Call IRE at (314) 882-2042 for details and forms.\n\nHere's a partial line-up\n\nScott Anderson, Ft. Lauderdale Sun-Sentinel\nDave Armstrong, Boston Globe\nAdam Berliant, Tacoma News Tribune\nNeill Borowski, Philadelphia Inquirer\nTom Boyer, Virginian-Pilot & Ledger-Star\nJames Brown, Indiana University\nRon Campbell, Orange County Register\nPete Carey, San Jose Mercury News\nMorgan Cartwright, Contra Costa Times\nWilliam Casey, Washington Post\nRose Ciotta, Buffalo News\nRussell Clemings, Fresno Bee\nWendell Cochran, American University\nSarah Cohen, Tampa Tribune\nAlan Cox, WCCO-TV\nTom Curran, New York Newsday\nDave Davis, Cleveland Plain Dealer\nBill Dedman, Associated Press\nJames Derk, Evansville Courier\nDavid Dietz, San Francisco Chronicle\nSteve Doig, Miami Herald\nJacqueline Doubinis, National Library on Money and Politics\nJohn Freed, New York Times\nDan Gillmor, San Jose Mercury News\nRich Gordon, Miami Herald\nTom Heslin, Providence Journal Bulletin\nElliot Jaspin, Cox Newspapers\nTom Johnson, San Francisco State University\nGeorge Jordan, New York Newsday\nJillian Kaskey, Money Magazine\nJeff Kummer, St. Paul Pioneer Press\nJennifer LaFleur, NICAR\nGeorge Landau, St. Louis Post Dispatch\nAlan Levin, Hartford Courant\nPenny Loeb, US News & World Report\nBeth Marchak, Cleveland Plain Dealer\nCarla Marinucci, San Francisco Examiner\nRich Mauer, Anchorage Daily News\nShawn McIntosh, Dallas Morning News\nTed Mellnik, Charlotte Observer\nPhilip Meyer, University of North Carolina\nTom Moore, Congressional Quarterly\nDwight Morris, Los Angeles Times\nRichard Mullins, NICAR/University of Missouri\nCarol Napolitano, Munster Times\nRichard O'Reilly, Los Angeles Times\nPeter Overby, Common Cause Magazine\nBob Paynter, Akron Beacon Journal\nTom Philp, Sacramento Bee\nBob Port, St. Petersburg Times\nMatt Reavy, NICAR\nGreg Reeves, Kansas City Star\nNeil Reisner, Bergen Record\nKenton Robinson, Hartford Courant\nRichard Robertson, Arizona Republic\nSteve Ross, Columbia University\nRicardo Sandoval, Center for Investigative Reporting\nAnne Saul, Gannett Newspapers\nJon Schmid, Raleigh News & Observer\nChris Schmitt, San Jose Mercury News\nDavid Stoeffler, Wisconsin State Journal\nHai Strauss, Atlanta Journal & Constitution\nDanny Sullivan, Orange County Register\nDrew Sullivan, NICAR\nScott Thurm, San Jose Mercury News\nLisa Van Asch, Raleigh News & Observer\nPatrick Weiland, Eye to Eye, CBS\nDan Woods, Raleigh News & Observer\nKevin Woodward, Santa Cruz County Sentinel","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":false}},{"id":"1994-september-internet-robust-but-unreliable","slug":"1994-september-internet-robust-but-unreliable","url":"https://dwillis.github.io/uplink/article/1994-september-internet-robust-but-unreliable.html","year":1994,"month":"September","monthNum":9,"headline":"Internet robust but unreliable","kicker":"Would-be cyberjournalists beware!","author_name":"Gwen Carleton","author_title":"Staff","affiliation":"NICAR","summary":"Journalists express concerns about the Internet's reliability and verifiability as a news source, citing issues like unverifiable information, inconsistent access, and the need for traditional reporting methods despite the network's growing usefulness.","keywords":["verifiability","Internet","journalists"],"topics":["Technology","Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","Usenet","gopher","FedWorld","Jughead","Veronica","Mosaic","World Wide Web"],"full_text":"Despite the recent fascination with the Internet, there is a reason that much of the world is hanging on to its telephones and fax machines. In fact, there are several.\n\nFor journalists, one of the net's worst failings is its lack of verifiability.\n\n\"In USENET newsgroups, there's a barrage inaccurate stuff posted all the time,\" said Dan Gillmor, a reporter for the San Jose Mercury News. \"People put up opinions, and people forge opinions from someone else.\"\n\nMuch of the \"official\" information flowing through Internet must be double-checked as well. The federal government takes no responsibility for mistakes are transmitted through FedWorld, for example.\n\nResearchers see similar weaknesses in the system. Gopher, a popular application that organizes access to Internet resources, helps information seekers to pursue specific topics throughout the vast network.\n\n\"To me, the problem with Gopher is, you don't know what it's indexing on,\" said Nora Paul, library director for the Poynter Institute. \"You in a word, but what has it searched? There's so much reliance on someone being reliable somewhere.\"\n\nAnother serious obstacle for journalists is the fluid nature of the network itself.\n\n\"I have been at budget meetings and promised to obtain information I knew was available on the net, only to get back to the office and find that for some reason or other the information was unavailable at the time,\" said Michael Meiners, database specialist for the Ft. Lauderdale Sun-Sentinel. Internet vagaries, from disappearing databases to access overloads, are all part of the experience. But according to Beth Marchak, a reporter for the Cleveland Plain Dealer, that's no excuse for staying unplugged. \"You don't wait until 4 in the afternoon to put out a PROFNET request,\" she said. \"You do it first thing in the morning.\"\n\nPrograms such as jughead, veronica, gopher and mosaic have been developed to make the organization and retrieval of information on the Internet simpler. Even so, users agree that Internet adventurers must know what they're looking for. As Marchak pointed out, \"there is no one-stop shopping.\"\n\nThe network is no substitute for reporting. \"It doesn't replace anything else; it's a supplement,\" Gillmor said. \"You still have to get out of the office.\"\n\nHowever, for all its quirks and limitations, the Internet is a resource to watch. And it rewards those who learn to use it well.\n\n\"The Internet offers many possibilities, several successes and thousands of disappointments,\" Meiners said. \"I think it has a long way to go before it is a reliable resource for directed research, but I still seem to end up using it at least once a day.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-drive-by-shootings-in-tacoma","slug":"1994-september-drive-by-shootings-in-tacoma","url":"https://dwillis.github.io/uplink/article/1994-september-drive-by-shootings-in-tacoma.html","year":1994,"month":"September","monthNum":9,"headline":"Drive-by shootings in Tacoma","kicker":"Mean Streets","author_name":"Adam Berliant","author_title":"","affiliation":"The News Tribune","summary":"The News Tribune analyzed drive-by shooting data from Tacoma and Pierce County police using database reporting techniques, discovering approximately 200 incidents over a year rather than the commonly assumed three to four per night, demonstrating how data journalism can produce accurate enterprise stories.","keywords":["database","shootings","reporting"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["XDB","FoxPro","Access","VAX cartridges"],"full_text":"Guns, gangs and drive-by shootings. For obvious and unfortunate reasons, violence, the media buzz-word of the '90s, quickly brings these thoughts to mind. But few would deny that among them, there's something particularly disturbing about drive-by shootings.\n\nYou can choose not to own a gun. You can choose not to belong to a gang. But drive-by shootings seem beyond our control. Even worse, we're hearing increasingly how innocent people end up stopping the bullets.\n\nYet year, when The News Tribune ran its eight-day series on crime, \"Beyond Violence,\" Tacoma and Pierce County police could barely provide us with even the vaguest clue about how many drive-by shootings there were in our city, much less how likely it was for bystanders to actually get shot.\n\nThe \"Beyond Violence\" project was time-consuming, titanic in scope and extremely thorough (see Uplink, pt.1993). The series was made possible by a police database containing seven years' worth of violent crime incident reports, about 250,000 records. We proudly squeezed that data until there was nothing left.\n\nFortunately, the feeling was mutual - police were just as eager to track drive-bys as we were. Only a few months before the violence project was published, LESA started tracking drive-bys. By the time the violence project's one year anniversary arrived, the 16-month-old drive-by database was ripe for analysis.\n\nThe News Tribune obtained the drive-by data and also bought an update of the original violent crime database. The total cost was around $200, and the data was supplied in compressed files on four diskettes. (The original, seven-year database came on VAX cartridges.)\n\nAfter spending about a week updating the findings from the previous year, we discarded our original idea to produce a large, follow-up violence project. There simply wasn't much new information we could derive from one year's data that seven years' worth hadn't already made clear last summer. It was hard to get excited about publishing a dozen \"Fill-in-the-blank: One Year Later\" stories when they could probably all read \"ditto.\"\n\nThe drive-by stories were XDB's dying gasp here at the News Tribune. I've always liked XDB even if it was slow, but when I found out the absurd price for upgrades and additional licenses, and when I finally persuaded my editors to let me experiment with FoxPro for a few days, I put good ol' XDB to sleep. Now I'm desperately trying to master FoxPro in time for Access to take over the market. May the wheels of progress continue to smash us flat!\n\nInstead, we used the data to prove, once again, that database reporting can lead to excellent, relatively short enterprise stories.\n\nWe started by determining once and for all that there weren't three or four drive-by shootings every night in Tacoma. The database showed there were 200 incidents over a full year - about one every other night.\n\nThat's why it was so bothersome that we couldn't get real numbers for drive-by shootings. The Law Enforcement Support Agency (LESA), the record-keeping mechanism for both the city and county police that provided The News Tribune with the database, simply didn't bother to categorize drive-by shootings separately. Instead, drive-bys were recorded as assault, vandalism or reckless endangerment.\n\nSo, we were stuck with anecdotes. One police official told us there were so many drive-by shootings in Tacoma that people didn't even bother to report them anymore. Patrol officers guessed that there were as many as four drive-by shootings every night, a shocking figure for any city, but even more so in a town the size of Tacoma.\n\nThe stories and numbers seemed hard to believe, and we were disappointed (to put it nicely) that better information about drive-bys wasn't available.\n\nMore interesting for our readers, though, was that drive-bys weren't the random acts of violence they were perceived to be, and that drive-bys presented little threat to most people. The data proved this in three ways.\n\nFirst, it showed that strangers in strange places were not of much interest to drive-by shooters. The targets of 1 in 3 drive-bys were houses that had been targeted previously. And if the target wasn't one of these \"hot houses,\" then it was the house next door, or some other house on the block.\n\nIn other words, the records showed that the majority of drive-bys occurred on certain streets, in particular parts of town. And, police told us, these parts coincided exactly with the areas of greatest known gang and drug activity.\n\nThe conclusion made by both police and The News Tribune was that if you weren't in a gang, involved with drugs or living next door to someone who was, you didn't have much to worry about.\n\nSecond, the data suggested that drive-by shootings were not increasing, as many people feared. Although we didn't have enough data to report with complete certainty that there was no trend, a month-to-month analysis of what data we did have showed that '94, so far, wasn't any worse than last year.\n\nFinally, the police records indicated that a large portion of drive-by shootings didn't target people at all. We provided a break-out box showing how many of the drive-bys were categorized secondarily as assaults, homicides, reckless endangerment, etc. It showed that 30 percent of all drive-bys were determined by police to be acts of vandalism (40 percent outside city limits). Police sources had a different story than they did a year earlier. Officials were now saying the odds of getting shot by a drive-by were less than being struck by lightning. One police source called the public \"paranoid about getting shot by gang members.\"\n\nIt took only a few hours to complete all the reports necessary to reach these conclusions. Patti Epler, who wrote two stories using the drive-by data, completed her reporting in less than two weeks. We were also able to use the data to include a standard database answer dump: where are the worst parts of town for drive-bys, what's the worst season for drive-bys and what time do the shootings take place.\n\nCall anytime (206)597-8258, or e-mail Berliant@Tribnet.Com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-nicar-offers-inexpensive-data-transfer","slug":"1994-september-nicar-offers-inexpensive-data-transfer","url":"https://dwillis.github.io/uplink/article/1994-september-nicar-offers-inexpensive-data-transfer.html","year":1994,"month":"September","monthNum":9,"headline":"NICAR offers inexpensive data transfer","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers data transfer services using various media formats and provides access to numerous government databases including FEC contributions, FAA data, and mortgage disclosure records.","keywords":["databases","government","data"],"topics":["Data & Databases","Technology"],"technologies":["9-track tape","magnetic tape","Trakker 120","Trakker 250","CD-ROM","magneto-optical disks"],"full_text":"The Institute can transfer data to and from floppy diskettes, 9-track tapes 3480/3490 cartridge tapes, Trakker 120 and 250 mb backup tapes, compact disks, and some 1.3 gb magneto-optical disks (as well as some 650 mb MO disks). NICAR also offers a number of government databases including:\n- Federal Elections Commission contributions data\n- Federal Aviation Administration data\n- Home Mortgage Disclosure Act records\n- General Services Administration federal procurement data\n- Alcohol, Tobacco and Firearms gun dealer records\n- National Bridge Inventory System data\n\nThe Institute also has FAA service difficulty reports from 1988 to 1994 on compact disk for $125. The disk includes the airmen directory. Call the Institute for an up-to-date list of prices and available data.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-japanese-equipment-prices-rising","slug":"1994-september-japanese-equipment-prices-rising","url":"https://dwillis.github.io/uplink/article/1994-september-japanese-equipment-prices-rising.html","year":1994,"month":"September","monthNum":9,"headline":"Japanese equipment prices rising","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Computer equipment prices from Japanese manufacturers are increasing due to dollar weakness, with Fujitsu raising prices on tape cart readers by $600, while American-made 9-Track drives remain unaffected.","keywords":["prices","dollar","equipment"],"topics":["Technology"],"technologies":["9-track tape"],"full_text":"Alan Warzecha of Computer Logics called to warn CAR journalists that prices for Japanese computer equipment are on the rise because of the weakness of the dollar.\n\nWarzecha said Fujitsu has raised prices $600 on low-end tape cart readers to about 9,400. He also noted that 9-Track drives are American made and therefore should not be directly affected.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-nicar-and-ire-hit-the-net-with-two-new-lists","slug":"1994-september-nicar-and-ire-hit-the-net-with-two-new-lists","url":"https://dwillis.github.io/uplink/article/1994-september-nicar-and-ire-hit-the-net-with-two-new-lists.html","year":1994,"month":"September","monthNum":9,"headline":"NICAR and IRE hit the Net with two new lists","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE, NICAR, and the University of Missouri have launched two new email discussion lists (NICAR-L and IRE-L) for investigative reporters and journalists using computers for news gathering and analysis.","keywords":["journalism","discussion","investigation"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","Email"],"full_text":"IRE, NICAR and the University of Missouri have launched two new lists aimed at investigative reporters and journalists who use computers for newsgathering and analysis. NICAR-L and IRE-L aim to provide a discussion forum for journalists and educators interested in investigative and computer-assisted reporting.\n\nTo subscribe to NICAR-L, send an e-mail message to \"listserv@mizzoul.missouri.edu and include the words \"SUBSCRIBENICAR-Lyou name\" providing the first and last name your would like to use on the list. To subscribe to the IRE list, simply substitute \"IRE-L\" for \"NICAR-L\" on the message line.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-gillmor-goes-west","slug":"1994-september-gillmor-goes-west","url":"https://dwillis.github.io/uplink/article/1994-september-gillmor-goes-west.html","year":1994,"month":"September","monthNum":9,"headline":"Gillmor goes West","kicker":"","author_name":"","author_title":"technology editor","affiliation":"San Jose Mercury News","summary":"Dan Gillmor has joined the San Jose Mercury News as technology editor and is helping organize Internet training at the CAR TREK conference in October.","keywords":["journalist","technology","training"],"topics":["Technology","Journalism & Newsroom"],"technologies":["Internet"],"full_text":"Dan Gillmor, Internet surfer extraordinaire, has packed his bags and moved to the San Jose Mercury News where he will serve as technology editor.\n\nHe is also helping to put together Internet training at the CAR TREK conference October 6-9.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-jobs-jobs-jobs","slug":"1994-september-jobs-jobs-jobs","url":"https://dwillis.github.io/uplink/article/1994-september-jobs-jobs-jobs.html","year":1994,"month":"September","monthNum":9,"headline":"Jobs, jobs, jobs","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR is receiving increased requests from news organizations seeking Computer-Assisted Reporting (CAR) professionals, and interested candidates can contact Brant Houston to be considered for positions.","keywords":["jobs","NICAR","CAR"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Jobs, jobs, jobs\n\nSpeaking of job switches, NICAR is getting more and more requests from news organizations looking for CAR reporters and editors.\n\nIf you want your name quietly suggested as a possibility, give Brant Houston a call at (314) 882-1984. We hope to put together a job bank later this year.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-paper-marks-decade-of-car","slug":"1994-september-paper-marks-decade-of-car","url":"https://dwillis.github.io/uplink/article/1994-september-paper-marks-decade-of-car.html","year":1994,"month":"September","monthNum":9,"headline":"Paper marks decade of CAR","kicker":"The CAR lot: Charlotte, North Carolina","author_name":"Ted Mellnick","author_title":"","affiliation":"The Charlotte Observer","summary":"The Charlotte Observer pioneered computer-assisted journalism in 1984 by using mainframe computers to analyze political contributions, and has since expanded this capability to investigate disparities in property taxes, lending, and employment through sophisticated data analysis on newsroom PCs.","keywords":["data","journalism","analysis"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["mainframe","DOS","Windows","OS/2","HP9000","Unix","9-track tape","CD-ROM","Quattro Pro","FoxPro","dBase","Word","XyWrite","Atlas GIS","SAS","SPSS","CalComp DesignMate","ArcInfo"],"full_text":"The Charlotte Observer was an early pioneer in computer-assisted journalism. Back in 1984, the newspaper used an IBM mainframe to catalog political contributions to 267 candidates for the North Carolina legislature.\n\nHere's a look at our hardware & software\n\nPCs: Four 486 IBM compatibles, 8mb RAM, color monitors, +300mb hard drives, three CD readers. PC operating systems: DOS, Windows and OS/2. Mini/mainframe: Newsroom has access to an HP9000 with Unix operating system and 9-track tape drive. Printers: Lasers, dots and access to a CalComp DesignMate. Spreadsheet: Quattro Pro for Windows. Database: FoxPro, dBase IV. Text & word processing: Word for Windows, Xywrite. Mapping & GIS: Atlas GIS/DOS 21, Atlas GIS Windows 2.0. Statistical analysis: SAS, SPSSPC/DOS, SPSS/Windows.\n\nThe stories showed the raw power of special interest money ($2 of every $3 contributed), and the links between gifts and lawmakers' votes on issues like recycling, pollution control and business fees.\n\nThat project, which has continued and expanded for a decade, also shows how the newsroom's use of computers has gotten bigger, more sophisticated and more self-sufficient.\n\nBack in '84, a computer programmer developed the mainframe applications, and a market research analyst handled printed reports, queries and analysis. Now, the project has moved from a mainframe to a newsroom PC.\n\nInstead of a couple thousand, there are tens of thousands of contributors on file. And one newsroom staffer has handled programming, application development and report design.\n\nBesides state and federal campaign finance, computer-related stories have reported: Inequities in property taxes, disparities in mortgage lending, and differences in OSHA inspection rates and workers compensation injury rates.\n\nWe've used Census data to look at growth trends, poverty patterns and racial disparity in education and income.\n\nMy two favorite census stories were based on the Public Use Microdata Sample (PUMS) file. The analysis was done in the statistical package SAS, running in UNIX on an HP9000.\n\nOne story involved a comparison of earnings of men, childless women and mothers in the Carolinas. It showed that women make less, even when compared with men of the same age, education, or, in most cases, the same job.\n\nThe other was a study of commuters. It revealed that inbound commuters play a much larger than expected role Charlotte's workforce, constituting 40 percent of the city's full-time workers. The published census estimate, which included part-time workers, was 30 percent. The story also showed the income benefits various kinds of workers derive from their longer drive time into the city.\n\nThe computer base map of street blocks was derived from data compiled by local government and university staffers in the GIS package ARC/INFO. The newspaper did the address matching in Atlas GIS/DOS. The resulting reports and maps plotted on a CalComp 8-pen plotter gave our reporters their first definitive view of the city's high crime areas, their first rankings of neighborhood violent crime rates and their first detailed maps of the exact locations of violent crimes. This material has been the starting point for a year-long series of neighborhood profiles. The computer side of the crime project also has included plotting more than 12,000 home break-ins and estimating neighborhood break-in rates, plotting the home addresses of thousands of victims and profiling suspects and victims.\n\n\"Criminals in the Classroom,\" a series published in March, began with The Observer matching a file of 50,000 teachers against a file of 300,000 prison inmates. The state doesn't require background checks for public school teachers. Matching revealed that people convicted of rape, incest, child molesting and assault still teaching.\n\nOur current projection on neighborhood crime involves plotting, by address, 18,000 violent crimes reported to Charlotte police in 1992-93.\n\nNow that our newsroom has developed a few basic computer-assisted reporting skills, the goal is to spread those skills among the staff. There are now four PCs on the newsroom floor - up from one a year ago - loaded with database, spreadsheet and communications programs. Last May, eight newsroom staffers went through three days of in-house training on FoxPro. More training is planned.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-september-heres-a-look-at-our-hardware-and-software","slug":"1994-september-heres-a-look-at-our-hardware-and-software","url":"https://dwillis.github.io/uplink/article/1994-september-heres-a-look-at-our-hardware-and-software.html","year":1994,"month":"September","monthNum":9,"headline":"Here's a look at our hardware & software","kicker":"","author_name":"","author_title":"","affiliation":"The Charlotte Observer","summary":"A newsroom's technology inventory from the early 1990s detailing computer hardware, operating systems, and software applications including PCs, Unix systems, and various productivity tools for journalism work.","keywords":["technology","newsroom","software"],"topics":["Technology"],"technologies":["DOS","Windows","OS/2","HP9000","Unix","9-track tape","CalComp DesignMate","Quattro Pro","FoxPro","dBase","Word","XyWrite","Atlas GIS","SAS","SPSS","CD-ROM"],"full_text":"PCs: Four 486 IBM compatibles, 8mb RAM, color monitors, +300mb hard drives, three CD readers. PC operating systems: DOS, Windows and OS/2. Mini/mainframe: Newsroom has access to an HP9000 with Unix operating system and 9-track tape drive. Printers: Lasers, dots and access to a CalComp DesignMate. Spreadsheet: Quattro Pro for Windows. Database: FoxPro, dBase IV. Text & word processing: Word for Windows, Xywrite. Mapping & GIS: Atlas GIS/DOS 21, Atlas GIS Windows 2.0. Statistical analysis: SAS, SPSSPC/DOS, SPSS/Windows.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-uplink-update","slug":"1994-october-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-october-uplink-update.html","year":1994,"month":"October","monthNum":10,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This month's Uplink newsletter showcases diverse applications of computer-assisted reporting (CAR) in journalism, including aviation data analysis, science reporting, voter fraud investigation, and government computer oversight.","keywords":["journalism","reporting","computers"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"This month's Uplink offers a potpourri of information that reflects the wide range of uses for computer-assisted reporting in journalism.\n\nIn addition to the cover story on aviation data, Gwen Carleton's piece explores the use of CAR in science journalism; Paul D'Ambrosio of the Asbury Park Press writes about exposing voter fraud; and David Bloom of the Los Angeles Daily News gives advice (in part one of two) about how your mastery of computer lingo allows you to cover government mishandling of computers and computer contracts.\n\nNext month we plan to report on the leading experts at CAR TREK, this year's national conference on computer-assisted reporting.\n\nIn the meantime, don't forget to send us your clips and tips.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-faa-database-provides-quick-history","slug":"1994-october-faa-database-provides-quick-history","url":"https://dwillis.github.io/uplink/article/1994-october-faa-database-provides-quick-history.html","year":1994,"month":"October","monthNum":10,"headline":"FAA database provides quick history","kicker":"","author_name":"Brant Houston","author_title":"","affiliation":"NICAR","summary":"The National Institute for Computer-Assisted Reporting provided Federal Aviation Administration data on aircraft mechanical problems to multiple news organizations following US Air Flight 427's crash, demonstrating the value of accessible databases for journalism while revealing limitations in the data itself.","keywords":["database","aviation","journalism"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"When US Air Flight 427 crashed Sept. 8, the new data library at the National Institute for Computer-Assisted Reporting got its first emergency test. From the results, it appears the institute passed. More than a dozen news organizations, ranging from small to large, requested and received data from NICAR on mechanical problems that the Boeing 737 had experienced prior to the crash. While the data did not point to a cause, it provided context for print and broadcast journalists throughout the country. For several years, the Institute has kept Federal Aviation Administration records on service difficulty reports reports filed when an airplane experiences mechanical problems or when inspectors find problems. During the last six months the Institute has reorganized the data, which goes back to 1988, so that it is more easily accessible and can be distributed on different kinds of media. The Institute not only gets monthly updates of the data, but it also is working on a project to include data covering the 15 years prior to 1988.\n\nWhile some news organizations have purchased the data themselves, and others have accessed the data through commercial vendors online, many journalists still seek the data from NICAR. This includes those who can't afford or do not have access to commercial databases, want the data in a form in which they can do wider searches or want to work with NICAR to understand the data better. After the crash, NICAR staffer Drew Sullivan, with the help of graduate student Padraic Cassidy, fielded calls from such organizations as WDIV-TV in Detroit, the Beaver County Times in Pennsylvania and the Chicago Sun-Times. Other organizations - The Boston Globe and WCCO-TV in Minneapolis had already purchased a CD-ROM from NICAR that contains the FAA database and did their searches on those CD-ROMs. The service difficulty reports are not without flaws. After all, every database has imperfections and is often only a starting point for a story. In fact, Cox Newspapers began a database analysis that ended up as a story that revealed shortcomings with the data and the inspection system. Larry Lipman, a reporter in the Cox Newspapers' Washington, D.C., bureau, worked with Elliot Jaspin (former director of the Institute and now at Cox) on the stories. He said Cox used the FAA data effectively for a story on service difficulty reports. Then, they started an overall story using the data for safety comparisons within the industry.\n\nWhen they began doing interviews on their findings, however, air safety experts warned them off their conclusion that 737s had a high rate of problems.\n\n\"Everybody told us not to use the numbers,\" said Lipman, who writes for the Palm Beach Post.\n\nNonetheless, Lipman said the data was useful in the initial story and led Cox to their critical review of the inspection and report system.\n\n\"Service Difficulty Reporting is uneven,\" NICAR's Sullivan said. \"Airlines do most of the reporting themselves and government oversight has been diluted. But overall, for a particular airplane, it is a valuable insight into the plane's repair history.\"\n\nUsing the database itself can be tricky. A first search usually is based on the tail number of the plane. But the tail number can change. So, during the first search, you must note the serial number of the plane and do a second search.\n\nSullivan compares the tail number to a car's license plate and the plane's serial number to a car's vehicle identification number. In the case of flight 427, NICAR's on-the-road trainer Jennifer Fleur (who happened to be doing a seminar in Harrisburg, Pa.) alerted Sullivan to the crash and later supplied the tail number.\n\nSullivan turned up 26 service difficulty reports based on the tail number and a 27th report based on the serial number. Each report has a severity rating, ranging from 1 (low) to 5 (high). Of the 27 reports, two were in the serious categories: a faulty fuel valve and a blown tire.\n\nThe FAA database has about 30 fields and 10 pages of codes. NICAR charges about $20 for a search and can send data for a particular plane by electronic mail or other online services such as the Internet, by fax, or by diskette. The CD-Rom for 1988 to present is $125 and NICAR hopes to have a file transfer site running soon for updates.\n\nNICAR also has a database on pilot licenses and a database on aircraft registration.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-coming-events","slug":"1994-october-coming-events","url":"https://dwillis.github.io/uplink/article/1994-october-coming-events.html","year":1994,"month":"October","monthNum":10,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A listing of upcoming journalism conferences and seminars including the CAR Trek Conference in Silicon Valley in October 1994 and the NICAR Seminar in Columbia, Missouri in January 1995.","keywords":["conferences","journalism","events"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nOctober 6-9, 1994\nCAR Trek Conference\nSilicon Valley, California\n\nJanuary 8-13, 1995\nNICAR Seminar\nColumbia, Missouri","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-faa-reports-considered-suspect","slug":"1994-october-faa-reports-considered-suspect","url":"https://dwillis.github.io/uplink/article/1994-october-faa-reports-considered-suspect.html","year":1994,"month":"October","monthNum":10,"headline":"FAA reports considered suspect","kicker":"When the numbers look too good to be true","author_name":"Larry Lipman","author_title":"Washington correspondent","affiliation":"The Palm Beach Post","summary":"A journalist investigating USAir Flight 427 discovers potentially alarming safety data showing two-thirds of USAir's 737s had reported level 5 service difficulties, but consultation with aviation experts reveals the statistics are misleading due to inconsistent reporting practices across airlines and the minuscule percentage of actual problem flights.","keywords":["data","aviation","statistics"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["FoxPro","Email"],"full_text":"Sometimes the numbers look just too good to be true. That's when it pays to be suspicious. Two days after the mysterious crash of USAir Flight 427 near Pittsburgh, Elliot Jaspin and I were looking for a common thread in the Federal Aviation Administration's service difficulty reports on 737s. Elliot had obtained a five-year file of SDRs by e-mail from the National Institute for Computer-Assisted Reporting. Unzipped, it took up more than 30 megabytes. Using FoxPro, he sorted the records by level of severity. The FAA ranks its SDRs on a scale of 1 to 5, with 1 having a \"seldom relationship\" to an accident and 5 having \"frequent relationship\" to an accident. We decided to just look at the level 5 reports. He then sorted the level 5s by the type of problem, looking for some pattern of difficulty with the 737s. By far, the biggest area of difficulty was \"Other.\" We looked at all the other areas, but no significant pattern emerged. We looked closer at \"Other,\" and, again, no pattern emerged. Then Elliot sorted the data by airline and by unique plane numbers. Suddenly, it appeared that we had a pattern - and a helluva story. Dividing the number of SDR 5s for each airline into the number 737s in its fleet over the past five years gave us a percentage. From this calculation, it appeared that two-thirds of USAir's planes had reported a level 5 SDR. That was among the highest percentage of the major airlines. It looked like we'd made a significant finding. I called USAir for their reaction to our hypothesis that two-thirds of their planes had experienced major service difficulties. They said they weren't sure what the data showed and defended their record as a safe airline.\n\nWe decided to run the data past aviation safety experts. I posted an e-mail query on Profnet. Within a few hours I had the numbers of several aviation safety experts affiliated with universities, and they led me to others in the field.\n\nAfter a day of talking to experts, I was convinced our hypothesis was wrong. The numbers were accurate; they were also misleading. Every expert I spoke with said that publishing a story along the lines I suggested would be unfair. They gave two main reasons:\n\n- Not all airlines report SDRs consistently. At least one expert noted that USAir, which has a relatively young fleet of 737s, was reporting SDRs twice the rate of two airlines with substantially older 737s. That might indicate that USAir was simply more scrupulous about reporting its SDRs than the other airlines.\n\n- While the number of SDRs compared with the number of planes gave us a percentage, it was meaningless. When you looked at the thousands of times each airline's planes had flown over the past five years, the percentage of SDRs to flights was minuscule. To say that two-thirds of the planes had problems, when far less than 1 percent of the flights had problems, would have been misleading and sensationalist.\n\nFaced with such comments by aviation experts not affiliated with any airline, we decided to scrap the story. But we still had something; we had information that a data system used by the FAA wasn't very reliable or meaningful. We decided to find out what the FAA did with this data and here, ultimately, is where we found our story. It turns out that the FAA admits the data from the airlines is neither consistent nor used as well as it might be for statistical analysis of either the airline's performance or safety trends. Virtually all the analysis is done manually because the FAA doesn't have the staff or the computer resources to better handle the data. That's the story we ran. - Larry Lipman is the Washington correspondent for The Palm Beach Post in the Cox Newspapers Washington Bureau. Elliot Jaspin is Systems Editor for Cox Newspapers Washington Bureau. They can be reached by e-mail at jaspin@access.digex.net.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-voters-cast-ballots-from-grave","slug":"1994-october-voters-cast-ballots-from-grave","url":"https://dwillis.github.io/uplink/article/1994-october-voters-cast-ballots-from-grave.html","year":1994,"month":"October","monthNum":10,"headline":"Voters cast ballots from grave","kicker":"Asbury Park Press exposes voting fraud","author_name":"Paul D'Ambrosio","author_title":"","affiliation":"Asbury Park (NJ) Press","summary":"A New Jersey news investigation used the Social Security Administration's Death Master File to compare against voter registration records, discovering over 400 deceased individuals still on voting rolls, including three who continued voting after death. The findings highlight the need for better voter roll management as new federal Motor Voter law requires states to maintain computerized, statewide registration databases.","keywords":["voting","deceased","database"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["9-track tape","Nine Track Express","FoxPro"],"full_text":"Egads! The dead are still voting in New Jersey.\n\nSince the invention of the ballot box, politicos around the country have always found clever ways of stuffing it with illegal votes. Rumors abound that Kennedy won the 1960 presidential election with votes from the silent majority resting in the graveyards of Hudson County, N.J., and Chicago.\n\nNow, with proposed reforms that will make registering to vote as easy as signing a form, the Press wanted to see if the dearly departed were still actively involved in earthly politics. But we encountered one major obstacle: Death records kept by New Jersey, as in some other states, are private.\n\nWe had the voter registration tapes. But without death records, no comparison could be made.\n\nThe Social Security Administration came to our rescue.\n\nThe SSA has developed a \"Death Master File\" to record everyone with a Social Security number who has died since 1937. That translates into 40 million dead, enough records to fill 19 6,250 bpi 9-track tapes.\n\nWhile the list is not perfect, it is the best there is if you can't get the full records from your state's vital statistics department. In fact, the U.S. General Accounting Office this summer praised the DMF as the most accurate collection of death records in the United States. Other departments, such as defense, routinely use the DMF to remove the dead from pension records.\n\nI pulled out New Jersey's death records from the DMF using Nine-Track Express and compared the names and dates of birth with our voter registration tapes.\n\nIt took a 486/66 an hour to process the information with FoxPro, but it delivered gold: We found more than 400 deceased still on the voting rolls. More importantly, we found some - three in one county alone - who were continuing to vote from the grave. One man's political party even changed from Democrat to Republican after his death.\n\nI examined the findings in conjunction with information on the National Voter Registration Act that will go into effect Jan. 1.\n\nThe act, better known as the Motor Voter law, makes registration easier, but mandates that states keep their voting rolls as clean as possible. The law is forcing states to develop statewide, computerized registration lists so they can purge those who have died or moved.\n\nNew Jersey is spending $4 million to develop such a database. For now, each county controls its own voter rolls. Checking for deaths is as low-tech as reading the obit page. However, after I told one county how we found their dead voters, officials there said they would begin using the DMF.\n\nWhile the DMF only has eight fields (SSN, last name, first name, date of death, date of birth, state, zip of last residence and zip of lump sum payment), there were some tricks to massaging the information.\n\nThe DOB uses century, a format not known to Nine-Track Express. It looks like 12/12/960. To get around this, I told Nine-Track to set up four separate fields: month, day, century and year.\n\nThis meant I had to split the voter's date of birth into two new fields: month and year.\n\nAfter pulling out names of the deceased by their zip codes, I joined the two files using last name, first name, month and year of birth. I excluded the day of birth since the SSA sometimes used \"00\" to denote an unknown day. Out of 1.3 million deaths and 450,000 voters, I only got a handful of duplicate matches on common names.\n\nThat was the easy part. In one county, about 10 percent of the voters listed as dead by DMF were still alive. Apparently the local SSA office had a habit of killing off living spouses the same day their husband or wife died. This became a sidebar.\n\nTo be absolutely certain a voter was dead, I obtained hard copy death certificates from the state, checked voting signature books and called the next of kin before confronting election officials.\n\nOfficials tried to claim that votes by the dead were clerical errors until they saw the signature sheets showing that someone had signed in after the voter's death. Investigations are now pending.\n\nPaul D'Ambrosio can be reached at pmd@app.com or by phone at (908)922-6000, ext. 4261.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-science-stories-and-car","slug":"1994-october-science-stories-and-car","url":"https://dwillis.github.io/uplink/article/1994-october-science-stories-and-car.html","year":1994,"month":"October","monthNum":10,"headline":"Science stories and CAR","kicker":"Tapping into the power of databases","author_name":"Gwen Carleton","author_title":"NICAR Staff","affiliation":"NICAR","summary":"Science writers are using computer data analysis to investigate complex stories, as demonstrated by California reporters who analyzed toxic chemical releases in the Central Valley using EPA databases and created an award-winning series on agricultural pesticide pollution.","keywords":["pesticides","investigation","data"],"topics":["Investigative Methods","Health & Environment","Technology"],"technologies":["9-track tape","FoxPro","PC"],"full_text":"Everyone's heard about the computer-assisted story that rocked city hall. But what if you prefer physics to politics? As science writers around the country are discovering, the right computer data can bring perspective, insight and scope to stories large or small.\n\nToxic chemical stories\n\nReporters from the Bee newspapers in Fresno, Modesto and Sacramento recently used computers to analyze the amount and location of toxic chemicals in California's Central Valley. Their six-month investigation resulted in the series \"Dirty Air,\" which appeared in the Fresno Bee in early November, 1993. The series illustrated how tons of agricultural pesticides are released in central California every year, adding to the Valley's already severe air pollution.\n\nReporters Chris Bowman, Russell Clemings and Alvie Lindsay discovered that more than 130 million pounds of toxic pesticides and industrial chemicals were released in the area in 1990. By analyzing data on 9-track tapes from the California EPA's Pesticide Use Report and the federal EPA's Toxic Release Inventory, they also determined where the largest releases of the most toxic chemicals occurred.\n\nClemings came up with the idea in the mid '80s after he discovered the California pesticide database. \"I thought it would be nice to take that data and sum up the amount of pesticide out there. But it was too big a task for a PC at that time it was a database of mind-boggling size.\"\n\nThe reliability of the California records, which included only the most toxic pesticides, also was a barrier. But in the late '80s state legislators increased the number of regulated chemicals, making the records more comprehensive.\n\nBy 1993, a more advanced computer system and editorial support for an investigative project also were in place. Armed with a new IBM 486, the California team was ready to go.\n\nInstead of summarizing pesticide amounts, the reporters decided to rate each chemical by its LD-50 value, a standard toxicity measure.\n\n\"They shovel pesticides into mice until half are dead, and then call that the LD-50,\" Clemings said. \"It wasn't perfect, but toxicologists said that something toxic by an oral route was probably toxic by an inhalation route.\"\n\nWith the help of Foxpro software, they discovered the 10 most common toxic chemicals in the Valley's air, the 10 primary sources of those chemicals and the 10 areas that were most affected. Bowman, Clemings and Lindsay also created a full-page color map, enabling readers to identify the major toxins in their communities.\n\nThe study found the farming industry was responsible for many of the toxic hot spots, the majority of which were on the east side of the Central Valley, adjacent to growing cities. Although state legislative reaction to the series was lukewarm, Clemings said he is always ready for similar computer-assisted stories.\n\n\"Knowing what I know now, this would take about one month or six weeks - the computer analysis part,\" Clemings said. \"There's lots of data on the environmental beat. Those agencies generate data better than they clean up the environment.\"\n\nFederal and local databases such as the EPA's Toxic Release Inventory can be a gold mine for journalists, often providing information for numerous stories. But federal and state computer records are just the beginning.\n\nUniversity medical school stories\n\nUniversities, corporations and non-profits all keep important data in electronic form the type of thing that can make or break a story. That is, if you can get it.\n\nWhen Maura Lerner and Joe Rigert of the Minneapolis Star Tribune decided to investigate their local university's medical school, they found out just how difficult it can be to take on a local sacred cow.\n\n\"In the beginning, they gave us quite a bit of stuff, but after the first article they closed up,\" Rigert said. \"We had to fight for everything we got.\"\n\nOne of the few items they acquired without legal pressure was a pile of computer disks containing information on thousands of university research projects.\n\n\"We got disks with all the research grants for five years,\" Rigert said. \"They included dates, money involved, titles of researchers, principal and secondary investigators.\"\n\n\"Using that, we were able to get all sorts of breakdowns on what companies certain professors were getting money from, what companies were doing lots of research at the university,\" he said. \"Every conceivable variation, it was all there.\"\n\nThe disks, which Lerner and Rigert analyzed using Paradox database management software, added significantly to the two-year investigation. \"Money vs. Mission at the University of Minnesota,\" which ran between May 1992 and December 1993, revealed 20 years of wrongdoing in a drug program, misuse of medical-practice funds, get-rich research schemes, kickbacks and research misconduct.\n\nThe Star Tribune investigation resulted in the resignations of four top university administrators, an FBI investigation, an overhaul of the school's secretive private practice system and one conviction for fraud.\n\nBut all computer-assisted science stories need not produce dramatic results. Huge amounts of data that once existed only on paper - or didn't exist at all - now are available in electronic form. For journalists interested in the big picture, the results can be significant.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-some-databases-produced-by-the-federal-government","slug":"1994-october-some-databases-produced-by-the-federal-government","url":"https://dwillis.github.io/uplink/article/1994-october-some-databases-produced-by-the-federal-government.html","year":1994,"month":"October","monthNum":10,"headline":"Some databases produced by the federal government","kicker":"","author_name":"","author_title":"","affiliation":"The Federal Database Finder, Third Edition","summary":"The article describes several federal databases maintained by the EPA and Department of Health and Human Services that track toxic chemical releases, hazardous waste, asbestos use, health data, and food safety information.","keywords":["environmental","health","database"],"topics":["Health & Environment","Data & Databases"],"technologies":[],"full_text":"Environmental Protection Agency\n\nTRI (Toxic Chemical Release Inventory): Contains information on the annual estimated releases of toxic chemicals to the environment. Hazardous Waste Data Base: Maintains information by the 5,000 facilities that treat, store, or dispose of hazardous waste and 165,000 handlers who generate or transport hazardous waste. Asbestos Information System: Assists the EPA in collecting, storing and analyzing data on the commercial and industrial uses of asbestos. Grants Information and Control System: Tracks the processing of all EPA grant applications and active grant projects. Both Washington, D.C., and regional programs are included.\n\nDepartment of Health and Human Services\n\nCancer Surveillance and Epidemiology in the U.S. and Puerto Rico, 1973-77: Contains information and demographics on cancer gathered from hospitals, clinics, private labs, autopsies, death certificates, etc.\n\nCholesterol and Smoking Data Base: Maintains information on blood cholesterol and smoking geared to the public, health professionals and issues pertaining to the workplace.\n\nBiologics Defect Reporting System: Contains information transcribed from reports of defects, errors and fatalities ascribed to the use of biological products.\n\nPriority-Based Assessment of Food Additives: Contains summaries of food additive safety profiles, ranking of additives based on safety concerns.\n\nSelected from The Federal Database Finder, Third Edition","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-covering-the-computer-beat","slug":"1994-october-covering-the-computer-beat","url":"https://dwillis.github.io/uplink/article/1994-october-covering-the-computer-beat.html","year":1994,"month":"October","monthNum":10,"headline":"Covering the computer beat","kicker":"The medium is the message","author_name":"David Bloom","author_title":"","affiliation":"The Los Angeles Daily News","summary":"Journalists should investigate how government agencies and businesses purchase, implement, and manage computer systems, as decision-makers often lack technical expertise and may waste millions on poor choices.","keywords":["computers","investigation","contracts"],"topics":["Investigative Methods","Technology","Business & Economy"],"technologies":["mainframe"],"full_text":"IRE, Journal and Uplink have run a lot of stories in recent years about using computers to improve your journalism, analyzing reams of data, finding trends, and pinpointing problems with computer-assisted techniques.\n\nWe don't often think to write about computers themselves, about the ways they are bought, improved, maintained, developed, used and misused by government agencies, business and others.\n\nBut we don't often think to write about computers themselves, about the ways they are bought, improved, maintained, developed, used and misused by government agencies, business and others.\n\nIt's a fertile field for good investigative pieces, for one simple reason: A lot of times the top decision makers who must pick computer systems aren't particularly good at it, often because they aren't any more comfortable with the arcana of computer systems than any other non-specialist.\n\nJust about any business, including (and sometimes especially) government, has top managers who don't know much about computers but now are making important decisions that may cost millions of dollars and cause uncounted problems if they are wrong.\n\nLuckily, if you've been trying to learn more about computer-assisted reporting, you've probably learned enough of the big-picture details about computing to at least understand what the experts are talking about on these stories. But even if you don't have that background, here are some ideas to get you going.\n\nThe Basics\n\nMany of these are basic story approaches that any veteran government reporter will recognize, only with a high-tech spin that often means they've been neglected both within the government agency and without.\n\nLook for the usual sources of dispassionate information, the kind of stuff The Reporter's Handbook by IRE talks about, such as audits, contracts, trade press reports, and professors of computer science or management information science and other specialists. Usually there are reports trying to explain what went wrong to governing bodies and investigative agencies such as grand juries. (This depends upon the state. California grand juries issue an annual watchdog report on government operations that often has useful information on these things).\n\nThere are also the somewhat less dispassionate sources, such as insiders and former insiders at the agency's data processing operations and competitors' lobbyists and marketing representatives (who are always willing to dish dirt on what's going bad).\n\nLooking to buy a big box\n\nHow are contracts structured and awarded? More than with most contracts for tangible goods, a craftily written computer hardware, software or services contract can be set up to nearly guarantee one company will win a lucrative bid. Why? Well, take one common situation: the county welfare agency needs a new software program to automate the form-bedeviled eligibility determination process. But say the welfare agency already is using an aging IBM mainframe. Because the agency already has IBM, the contract could specify only a program that runs with that company's mainframe and the operating system it is using. While this seems rational enough, does it mean the welfare agency is locking itself into an old technology and approach that ultimately will be more expensive and less effective? Specifying one computer and one kind of operating system for it generally means a lot of other companies that could provide workable alternatives for less cost are being shut out of the process from the beginning. The state of California, currently trying to develop a new welfare computer system, has trashed one of two approaches after spending millions on it. It is choosing to go with the other system, though it uses quite dated technology and is already behind schedule.\n\nContract Wars\n\nLook at the lobbying wars for computer contracts. What are companies doing to influence the writing of these requests?\n\nIn poking around Los Angeles County government, I noticed in lobbying reports that the computer companies were spending hundreds of thousands of dollars on all kinds of county officials. Those reports detailed how IBM, EDS, Unisys, Lockheed Information Management Systems and other big data companies were jockeying for millions of dollars in contracts of all kinds.\n\nThe companies didn't even focus their efforts on department heads and elected officials, though those top decision makers weren't ignored. Instead, the companies were targeting the mid-level bureaucrats who typically have to work with such systems and often are charged with the grunt work of acquiring new systems.\n\nNot everyone has such lobbying reports to tap for information. But maybe the agency requires officials to file campaign contribution reports, or annual conflict-of-interest reports, or gift reports that a reporter can look at.\n\nOne of the things I do is keep copies of all the county's contracts when they're approved. That's a great place to look when you're trying to figure out what is at stake for these companies. Follow these contracts over a few years and you'll find some interesting patterns of renewals and expanded payments.\n\nLook outward, angels\n\nMany governments are now looking at \"outsourcing\" their data processing operations, contracting virtually everything with one big provider, like EDS or Lockheed. Outsourcing is a hot trend in both the private and public sectors, but one laden with land mines. If your government agency is considering doing it, watch it closely, and watch who gets the contract.\n\nThese outsourcing contracts are notorious for their problems, particularly when the agency has unanticipated new computer demands. At this point, they often are vulnerable to exorbitant charges from their new \"partner.\" Having jettisoned their in-house specialists, the agency often has little choice but to go along, at a premium price.\n\nOutsourcing also raises issues of data integrity and security that just aren't present in the private sector.\n\nFor instance, while it's darned problematic for some private company if its financial system gets scrambled, it's not a community-wide data disaster like having birth certificate information, criminal or judicial information or property records fouled up, tampered with or destroyed. How secure is this information?\n\nThe Long Run\n\nLook at the maintenance contracts for these systems. Often they are more lucrative than the original purchase, and get less oversight. These contracts can run into the millions of dollars, yet are often routinely, even automatically awarded to the same vendor year after year without bids or contract reviews.\n\nLook at what decisions are being made about technology. Mainframes are no longer the sole answer and often are the worst one. Distributed systems, such as networks of PCs or work stations, sometimes connected to mini computers, often are cheaper, more flexible and can have new programs developed in substantially less time. Again, are government agencies making the best decisions, or even considering all the options?\n\nThe Diaspora\n\nAs systems have decentralized, so has the oversight. This trend drives central data processing operations bonkers. It also has implications for the agency in terms of controlling its spending, making sure systems are compatible and making sure they all can talk together.\n\nLos Angeles County, the nation's largest, spends hundreds of millions of dollars a year on computer-related expenses, but it can't tell you how many hundreds of millions of dollars a year. That's virtually a story in itself.\n\nA rule of thumb on computer-related costs is percent of sales in the private sector, 3 percent of expenditures in the public. That indicates a lot of money, but if they aren't spending in that ballpark, are they too slow in automating?\n\nThe county is now doing a study to find out what it's spending on computers; so far they know they have more than 700 different computer systems. Ask your agency what they spend. They probably can't tell you.\n\nBy the way, if they have done such a study, get it. It's a good resource for computer-assisted reporting projects. You'll have some idea of the databases the agency tracks, and who's in charge of them.\n\nCheck the priesthood\n\nYou should also look at what's happened to the agency's information services or data processing department. Frequently, it gets left behind, as other departments decide they can do without mainframes and the high internal charges the DP guys bill them for their services. But they are the organization's internal experts. Are they effective and underutilized, or are they out-of-touch dinosaurs wedded to old technology?\n\nIt's all timing. And money.\n\nWhen it comes to developing a new computerized system, you can almost guarantee any large project will come in late and over budget. An expert I interviewed routinely doubled all estimates of delivery time and price to get a realistic view of a project's development schedule.\n\nPart two will appear next month.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-calculating-medians-in-paradox","slug":"1994-october-calculating-medians-in-paradox","url":"https://dwillis.github.io/uplink/article/1994-october-calculating-medians-in-paradox.html","year":1994,"month":"October","monthNum":10,"headline":"Calculating medians in Paradox","kicker":"Tech Tips","author_name":"Ray Robinson","author_title":"Reporter","affiliation":"The Press of Atlantic City","summary":"When average values are skewed by outliers, the median provides a more representative number. This article explains a seven-step process for calculating medians in Paradox 4.0 database software, using mortgage income data as an example.","keywords":["median","database","Paradox"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Paradox"],"full_text":"Any database software can easily calculate the average of a numeric field. But sometimes a few extremely high values skew the average.\n\nWe ran into that with the income field of the Home Mortgage Disclosure Act data for New Jersey. Our MSA, due to some extremely high incomes, was showing the average loan applicant with an income of more than $84,000 per year.\n\nWhen the average is skewed, you can often get a more representative number by calculating the median. That's a little more difficult than calculating an average. Here's how it's done in Paradox 4.0:\n\n1) Query your table, marking the following fields: the field you want to group by; the field you want to calculate as a median (be sure to mark this field with Alt-F6, rather than just F6, so it will include duplicate values in the answer). Rename the answer table to save it.\n2) Sort the table in ascending order, making the median field the last sort field.\n3) Restructure the table, adding a numeric field called RECNO.\n4) Insert the record number Paradox has assigned to each record into the new field. Here's the code:\n\nEDIT \"Tablename\" MOVETO [RECNO] SCAN = RECNO() ENDSCAN DO_IT!\n\n5) Query the resulting table to mark the grouping field and do a CALCAVERAGE on the RECNO field. Rename the answer table to save it.\n\n6) The resulting table will have each value in the grouping field with an average in the RECNO field. Now you need to strip the decimal places off the averages in RECNO. Here's the code:\n\nEDIT \"Tablename\" SCAN IF ([Average of Recno] Int([Average of Recno])) > 0 THEN [Average of Recno] = t([Average of Recno]) COPYTOARRAY arecord DOWN INS COPYFROMARRAY arecord [Average of Recno] = [Average of Recno] + 1 ENDIF ENDSCAN DO_IT!\n\n7) Now set up a multi-line query as follows: In the new table, put an example element in the Average of Recno field. In the earlier table, put the same example element in the Recno field. Mark the grouping field in that table. And in the field you want to calculate as a median, type CALC AVERAGE AS MEDIAN.\n\nThe resulting table should display each value in the grouping field with a median value.\n\nYes, it's a headache. And if you only have one median to figure, it's obviously easier to sort the original table and scroll down to the record number that falls in the middle. But if you want to calculate the median for a number of different groups, it's worth it.\n\nRay Robinson, (609) 272-7273; Primary: 73203.3644@compuserve.com; Secondary: acpress@acy.digex.net","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-october-dambrosio-hill-win-njpa-award","slug":"1994-october-dambrosio-hill-win-njpa-award","url":"https://dwillis.github.io/uplink/article/1994-october-dambrosio-hill-win-njpa-award.html","year":1994,"month":"October","monthNum":10,"headline":"D'Ambrosio, Hill win NJPA Award","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"CAR coordinator","affiliation":"Asbury Park Press","summary":"Asbury Park Press writers Paul D'Ambrosio and Michelle Hill won the 1994 New Jersey Press Association's Business Writing Award for their investigative series on redlining titled 'The Mortgage Game,' which also earned runner-up status for the Enterprise Writing Award.","keywords":["redlining","journalism","investigation"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":[],"full_text":"Asbury Park (NJ) Press staff writers Paul D'Ambrosio and Michelle Hill have won the 1994 New Jersey Press Association's Business Writing Award. It is the group's top business award, which comes with a $1,000 check. D'Ambrosio and Hill won the award for their four-part series on redlining in New Jersey, entitled, \"The Mortgage Game.\" The computer and statistical work, as well as mapping, was done by D'Ambrosio, the paper's CAR coordinator. The series was researched and written by D'Ambrosio and Hill. The series was also named runner-up for the NJPA's Enterprise Writing Award.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-october-send-us-your-stories","slug":"1994-october-send-us-your-stories","url":"https://dwillis.github.io/uplink/article/1994-october-send-us-your-stories.html","year":1994,"month":"October","monthNum":10,"headline":"Send us your stories","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR is seeking submissions from journalists about their experiences with data acquisition and computer-assisted reporting projects, whether successful or challenging, for publication in Uplink and presentation at seminars.","keywords":["data","journalism","reporting"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Looking for someone to listen to your nightmare experiences in negotiating for data? Interested in sharing what you did well, and perhaps not so well, while pursuing your latest computer-assisted reporting venture? Drop us a line.\n\nNICAR is interested in publishing your stories in Uplink and sharing them in other forms through our on-the-road seminars.\n\nIf you'd like to submit an article for publication, simply e-mail it to Matt Reavy (c598895@showme.missouri.edu). Articles specifically not for publication should be sent to Jennifer LaFleur (jenster@aol.com). Either can be reached by phone at (314) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-uplink-update","slug":"1994-november-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-november-uplink-update.html","year":1994,"month":"November","monthNum":11,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A journalism newsletter covering highlights from the CAR Trek conference in Silicon Valley, including training advice and computer-assisted reporting examples, with upcoming coverage of internet databases and new programs.","keywords":["conference","reporting","databases"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet"],"full_text":"Although we are still recovering from the long days and nights at the CAR Trek conference in Silicon Valley, we wanted to share the high points with you before someone hits our delete button.\n\nThis issue includes an overview of the conference, advice from NICAR training director Jennifer La Fleur on what to do and other items of interest from an event that drew an intense and enthusiastic crowd.\n\nIn addition to CAR Trek news, we also have our usual bits and bytes and an article on the Orange County Register's work in computer-assisted reporting.\n\nNext month we have articles about our progress in making databases and tips available over the Internet. We'll also write about some new programs.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-life-after-car-trek","slug":"1994-november-life-after-car-trek","url":"https://dwillis.github.io/uplink/article/1994-november-life-after-car-trek.html","year":1994,"month":"November","monthNum":11,"headline":"Life after CAR Trek","kicker":"Where do we go from here?","author_name":"Jennifer LaFleur","author_title":"","affiliation":"NICAR","summary":"The CAR Trek '94 conference was a major success, bringing together over 420 journalists from around the world to learn computer-assisted reporting skills, including internet navigation and database management.","keywords":["CAR","conference","journalists"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Excel","Quattro Pro","FoxPro","Paradox","DOS","Windows","Macintosh","Internet","9-track tape"],"full_text":"They came, they saw, they computed. Journalists from around the country (and the world) found their way to San Jose for CAR Trek '94 to learn about computer-assisted reporting. Some were beginners who came to take their first byte of this new discipline, while others came to brush up on or learn more advanced techniques.\n\nMore than 420 journalists received hands-on training in a wide range of skills, from the beginnings of DOS to advanced statistical analysis. The most popular of the hands-on classes was the Internet training in which more than 300 participants throughout the conference took a test flight through cyberspace. Demand was also high for advanced techniques such as mapping, building databases, building front-ends and downloading from 9-track tape.\n\nAttendees learned how to build spreadsheets in Excel and Quattro Pro and database management in FoxPro and Paradox. In addition to the hundreds of participants in DOS and Windows classes, the Macintosh sessions were in higher demand than anticipated. As some newsrooms implement Macintosh systems, reporters want to learn how to use them for CAR.\n\nNone of the many sessions would have been possible without a dedicated team of coordinators, teachers and coaches who generously gave of their time and expertise to make sure every conference participant received hands-on training.\n\nWhere do we go from here?\n\nIf you were one of the attendees at CAR Trek '94, you might wonder what to do next. So you know how to make a subdirectory in DOS, you can analyze a city budget in a spreadsheet, and you've gotten a taste of your first database. Many of you have gone back to your news organizations to be crowned the CAR guru. You have been handed the tasks of immediately conducting a huge analysis project from which you will produce the series to end all series, building your entire datalibrary and training everyone else in the newsroom. Just stop. Here's a quote for your editors: \"Developing computer-assisted reporting skills takes practice and time. You should start with some small, short-range projects,\" says NICAR's Managing Director Brant Houston.\n\nTo start, the most important thing you can do is keep your hands on the keyboard. (Well, not all the time - that would make it very difficult to eat). But keep practicing. Get some real data and work with it and keep practicing. If you have questions, call someone you met at the conference or call NICAR. You don't even need to be doing data analyses to keep up your skills. Use a spreadsheet to do your expense report and transfer your Rolodex to a database - that will force you to use those new skills.\n\nOnce you're feeling more confident in your abilities, you might be ready to share your knowledge with others. However, suggest that those interested in learning CAR attend a training program because once you train them, you need to be available to help them as they get up to speed. You can either do computer-assisted reporting or teach it to everyone else. Doing both might kill you.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-november-coming-events","slug":"1994-november-coming-events","url":"https://dwillis.github.io/uplink/article/1994-november-coming-events.html","year":1994,"month":"November","monthNum":11,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A listing of upcoming NICAR (National Institute for Computer-Assisted Reporting) seminar dates scheduled for January and March 1995 in Columbia, Missouri.","keywords":["NICAR","seminar","training"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nJanuary 8-13, 1995 NICAR Seminar Columbia, Missouri\n\nMarch 12-17, 1995 NICAR Seminar Columbia, Missouri","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-selling-editors-on-car","slug":"1994-november-selling-editors-on-car","url":"https://dwillis.github.io/uplink/article/1994-november-selling-editors-on-car.html","year":1994,"month":"November","monthNum":11,"headline":"Selling editors on CAR","kicker":"Hey, Boss!","author_name":"Rosemary Armao","author_title":"","affiliation":"Investigative Reporters and Editors","summary":"The article discusses strategies for convincing newspaper editors and publishers to adopt computer-assisted reporting (CAR) programs, including hosting a special editors' program at a conference with demonstrations and speakers who emphasized CAR's importance as a journalistic tool and competitive advantage.","keywords":["journalism","editors","technology"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"How do you talk the bosses into starting and building a CAR program? This topic has become a staple at IRE/NICAR conferences and newsroom presentations.\n\nWe've developed strategies for reporters to deal with the inevitable questions from the bean counters about cost and we've coined lines for reporters to feed to skeptical editors about CAR being the future of journalism. But at the Silicon Valley conference, we did something really radical. We invited in top editors to see for themselves.\n\nIt took two members of the local planning committee in California, Jonathan Krim, assistant managing editor at the San Jose Mercury News and Tom Honig, executive editor of the Santa Cruz Sentinel, to come up with a notion that ought to have been as plain as an \"Abort, Fail, Retry\" message. We need to talk with the men and women who make decisions and set policies in newsrooms. Publishers, executive editors and managing editors are the people to convert. As reporters, we've wasted too much time whining among ourselves about their frugality and lack of vision.\n\nThe editor's program\n\nThe half-day editors program, even with a demonstration of Mercury Center, an open pass to the panels and hands-on training at the Westin and a FREELUNCH, was not entirely a smash. Top editors from many California papers, including some of the biggest, didn't come or canceled at the last minute pleading press of business and meetings. Still, that is only a symptom of the problem and it was a great start.\n\nThe editors who did come got a hard sell, starting with a welcome from Mercury News publisher Jay Harris, who saw the light about investigative reporting and computer-assisted reporting before a lot of the rest of us. A slide version of IRE/NICAR's traditional 60-Ideas-in-60-Minutes offered proof of what CAR already has done, suggested story ideas and a sense that CAR is important right now and not some prediction for a distant future.\n\nSpeakers help with the message\n\nLisa P. Van Asch, news research and training coordinator for the Raleigh News & Observer, told the editors that computers are as vital a journalistic tool now as telephones, not some luxury to be bestowed by management on the privileged. She told them CAR can level the playing field in the newspaper industry, supplying smaller papers with the resources to take on much bigger competitors.\n\nShe also advised them to \"arm everyone who can shoot,\" as CAR extraordinare Pat Stith has described the Raleigh program. Train anyone who shows an interest in CAR. That's the best way to safeguard a program against the loss of a star nerd. It also ensures that good stories will wind up in all sections.\n\nVirginia Fielder, vice president for research for Knight-Ridder's newspaper division, issued another wake-up call. Check out your readers, she told the editors. Count how many have car phones. TVs. Cable. PCs. The technology that will change how most people communicate and get their news already has taken root and begun growing.\n\nThe overriding theme of the editors' program, like that of CAR Trek overall, was that it's time to get over the fear of computing and start doing it.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-hooking-into-the-internet","slug":"1994-november-hooking-into-the-internet","url":"https://dwillis.github.io/uplink/article/1994-november-hooking-into-the-internet.html","year":1994,"month":"November","monthNum":11,"headline":"Hooking into the Internet","kicker":"Reflections on Net Classes","author_name":"Dan Gillmor","author_title":"","affiliation":"San Jose Mercury News","summary":"A journalist describes setting up an Internet training room in San Jose, detailing the technical challenges of connecting computers to the Internet via T-1 lines and teaching beginners about email, Usenet, Gopher, and the World Wide Web using Mosaic browsers.","keywords":["Internet","training","technology"],"topics":["Technology","Journalism & Newsroom"],"technologies":["Macintosh","T-1","Sparc20","Sparc","Pine","Usenet","gopher","Veronica","Mosaic","World Wide Web","Unix","Netra","Telnet","Lynx"],"full_text":"It was fun. At the time I have pulled this off any Silicon Valley, where a lot of expertise coalesced. Still, putting Internet Room at CAR-Trek drawing two cards and straight. We got lucky.\n\nApple Computer was lending Macintosh computers, each with networking connections, who heads the Mercury support area, said she and her a high-speed local-area provider had already T-1 connection - that's a its network and lend us some connect the T-1 line to the would handle the network. And a Silicon Valley company computer workstations, Sparc20 computer for a Andrew Zazzi put the Sun's Shane Sigler con- Sparc workstation. We tested the on Wednesday, the day before to begin. All was well, we to the outside didn't hours we witnessed a of buck-passing. The provider said, \"Not our fault.\" said, \"Not our fault.\" handled the wiring from the into the Internet room, three classes Thursday, re- later in the weekend. That holding a slimmed-down ver- Internet training, showing only e- of the Internet's World was unsatisfying for everyone. Thursday, however, we got repre- phone company, Internet pro- room together and asked together to find the problem.\n\nThey finally did, and by Friday morning we had the room up and running the way we'd hoped.\n\nNow we had a room that matched our ambition. The classes were designed to take beginners into the Internet in a basic way at first, but by the end have them surfing like veterans. We had folks log into the server, as if they were dialing by phone and modem to a local Internet provider that offered Unix shell access only. The point was to give people a taste of the low end of the Net food chain. Using Unix mail software called Pine, we had each person send e-mail to someone else and then send a reply. Next we showed people some Usenet newsgroups. We couldn't get everyone logged onto the newsgroups at once. Even on our fast network that was causing electronic traffic jams. The next step was to look at Gopher, an immensely useful Internet menu system. We used Gopher client software on the Macs to show people how they could tunnel around the world through various Gopher servers, and then search server menus using Veronica software.\n\nFar and away the most popular and entertaining part of the class was the Mosaic playtime, in which folks surfed the Internet's World Wide Web using a Mosaic browser on each Mac. Because we were attached to the outside on a T-1 line, Mosaic ran fast enough to allow reasonably quick loading of images.\n\nThe system worked wonderfully all day Friday and Saturday morning. There were no Internet classes Saturday afternoon. That evening, the Mosaic software had stopped connecting to the outside world, as had the Gopher software loaded on the Macs. I still have no idea what went wrong.\n\nOn Sunday night we all rested.\n\nLessons from San Jose\n\n1) If you're setting up a newsroom Internet server, don't try to do it overnight. This stuff still isn't easy. Make sure you have a Unix guru in-house. 2) Expect problems. One of Murphy's laws is that Murphy attends all live demonstrations, something we proved at CAR-Trek. 3) Look at new systems from Internet-savvy vendors such as Sun Microsystems Sun's Netra, in particular - if you're planning to bring the Internet into your building. Netra is billed as a server that makes it easy to connect PC and Mac clients.\n\nAlthough we were somewhat constrained by losing the Mac-based Gopher and Mosaic, we still were able to offer some solid Net instruction. We used the telnet capabilities of the server to log onto other computers in other cities, where we could run the Unix Gopher software on those remote systems. That alone demonstrated a vital feature of the Internet. For World Wide Web surfing we used a remote machine that ran a character-based Web browser called Lynx. It wasn't nearly as much fun as Mosaic, but it did work.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-november-lessons-from-san-jose","slug":"1994-november-lessons-from-san-jose","url":"https://dwillis.github.io/uplink/article/1994-november-lessons-from-san-jose.html","year":1994,"month":"November","monthNum":11,"headline":"Lessons from San Jose","kicker":"","author_name":"Dan Gillmor","author_title":"Computing Editor","affiliation":"San Jose Mercury News","summary":"A guide to setting up newsroom Internet servers, emphasizing the need for experienced Unix support, careful planning to avoid problems, and consideration of Internet-savvy vendors like Sun Microsystems' Netra system for managing PC and Mac clients.","keywords":["server","Internet","newsroom"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Unix","Internet","gopher","Mosaic","Telnet"],"full_text":"1) If you're setting up a newsroom Internet server, don't try to do it overnight. This stuff still isn't easy. Make sure you have a Unix guru in-house. 2) Expect problems. One of Murphy's laws is that Murphy attends all live demonstrations, something we proved at CAR-Trek. 3) Look at new systems from Internet-savvy vendors such as Sun Microsystems Sun's Netra, in particular - if you're planning to bring the Internet into your building. Netra is billed as a server that makes it easy to connect PC and Mac clients.\n\nAlthough we were somewhat constrained by losing the Mac-based Gopher and Mosaic, we still were able to offer some solid Net instruction. We used the telnet capabilities of the server to log onto other computers in other cities, where we could run the Unix Gopher software on those remote systems. That alone demonstrated a vital feature of the Internet.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-feds-against-releasing-data","slug":"1994-november-feds-against-releasing-data","url":"https://dwillis.github.io/uplink/article/1994-november-feds-against-releasing-data.html","year":1994,"month":"November","monthNum":11,"headline":"Feds against releasing data","kicker":"Electronic access not a sure thing","author_name":"David H. Morrissey","author_title":"","affiliation":"Colorado State University","summary":"Federal agencies continue to restrict public access to electronic records under the Freedom of Information Act despite a Clinton administration policy encouraging disclosure, with surveys showing little change in agency resistance to releasing computerized data.","keywords":["FOIA","electronic","agencies"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":[],"full_text":"By the year 2000, as much as 75 percent of the federal government's business will be conducted electronically. That's exciting news for computer-equipped reporters. Washington is the world's largest publisher. Reporters find it increasingly difficult to cover any beat without in some way relying on federal data.\n\nBut several recent surveys of federal agencies reveal many treat electronic information as a special class of data - outside the range of the Freedom of Information Act and often off limits to the public.\n\nThe Justice Department says it may rely on these agency views when crafting FOIA policy - unless Congress acts first.\n\nIn 1993 and again in 1994 I surveyed 70 federal agencies on their FOIA policies toward electronic records. Key questions asked agencies were:\n\n- Must agencies create or modify computer programs to search for and retrieve electronic records?\n- Must agencies create or modify computer programs to segregate the releasable portion of a record?\n- Do requesters have the right to specify the format in which information is released?\n\nMore than 75 percent of the agencies answered the first question \"no;\" about half answered the second two questions \"no.\" Most agencies not answering \"no,\" said that they hadn't yet dealt with the questions and had no opinion. Fewer than 5 percent answered any question \"yes.\"\n\nThe responses are significant for three reasons. First, they reveal strong opposition to increased FOIA access to computerized federal files. The implication of questions 1 and 2 is that information retrievable only through creation or modification of computer programs is not covered by the FOIA. Maybe the agency will release it; maybe not. Question 3 implies that an agency can release information in an undesired format - say, a 1,000-page printout instead of floppy disks.\n\nThe answers are also important because the Justice Department asked the same questions of agencies in a 1989 survey (receiving essentially the same answers.) The Justice Department says it may rely on these agency views when crafting FOIA policy - unless Congress acts first.\n\nFinally, the answers are important because they haven't changed. I resurveyed the same 70 agencies in 1994 - seven months after President Clinton announced a new FOIA policy, encouraging electronic disclosure. Clinton's policy was the first by a president specifically mentioning electronic information. It was also the first to emphasize the public has a right to information stored in computers. \"Each agency has a responsibility to enhance public access through the use of electronic information systems,\" Clinton said.\n\nDespite this emphasis, agency responses in 1994 remained virtually unchanged from 1989 and 1993, revealing a persistent bureaucratic hostility to FOIA requesters seeking electronic records. Not one agency surveyed in 1994, after the Clinton policy was announced, had drafted a new electronic record FOIA policy. Many said they were already practicing maximum disclosure, and thus no new policy was needed.\n\nThe surveys reveal agencies eager to retain control over \"their\" electronic records, using the ambiguity created by a new technology to limit public access. For instance:\n\n- Some agencies refuse to perform even simple computer programming to retrieve information. But if information exists only in an electronic form, retrievable only through programming, then an electronic black hole is created, containing information the public can't obtain.\n- While agencies must conduct reasonable searches for information specified in FOIA requests, some agencies say they won't use computers for these searches, although this might make searches quicker and more accurate.\n- Agencies are slow to tell the public what information they maintain electronically, although a database list could save time and money.\n- Other agencies say they will provide information in an electronic form, but only if requesters pay fees not required for paper documents.\n\nThese unclear and inconsistent policies appear to violate the spirit of the FOIA and perhaps the letter of the law. Worse, they have quietly shifted the burden of proof for disclosing information. Instead of the law's requirement that federal agencies prove why information should be withheld, a process forcing agencies to cite specific exemptions to overcome a presumption of openness, requesters seeking computerized data now increasingly find they must prove why information should be released in that electronic form.\n\nThe mandate of the FOIA - stating information is presumed to be releasable unless an agency can prove otherwise - appears in question. The new form in which information is kept allows agencies, by bureaucratic fiat, to overturn one of the more important reforms achieved by passage of the FOIA.\n\nThere is clearly a need for Congress to enact a detailed policy for federal electronic records. The Leahy-Brown Bill, which passed the Senate this year but died in the House, would have been a modest first step. Something on this order - or more comprehensive - should be considered next year.\n\nAs Congress considers legislation to bring the FOIA into the computer age, it can expect substantial agency opposition to changing the status quo. Four factors, however, can help Congress place that opposition in its proper perspective:\n\n- The federal government's use of computers continues to increase. At the same time, the federal government's reliance on paper records continues to decrease.\n- The power of personal computers continues to improve. This has given reporters and others requesting computerized records a greater ability to access, analyze, and use federal electronic records, when agencies allow this access.\n- The demand for federal computerized records continues to increase. Journalists, academics, lobbyists and lawyers have joined the list of those wanting electronic access. Requesters know computerized information is easier to use, and thus more valuable, than the same information printed on paper.\n- A significant number of organizations, from the American Bar Association to the Office of Technology Assessment, have studied the issues raised by electronic federal records and the FOIA and concluded that - despite agency protestations - increased access to these records is both legal and logical. Most also conclude that allowing this access is necessary if the intent and letter of the law are not to be violated.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-november-history-federal-computer-use","slug":"1994-november-history-federal-computer-use","url":"https://dwillis.github.io/uplink/article/1994-november-history-federal-computer-use.html","year":1994,"month":"November","monthNum":11,"headline":"HISTORY: Federal Computer Use","kicker":"","author_name":"David H. Morrissey","author_title":"Professor, Department of Technical Journalism","affiliation":"Colorado State University","summary":"This timeline documents the federal government's dramatic expansion of computer usage from 1946 to 1993, showing growth from a single ENIAC to millions of computers, while noting that early FOIA legislation failed to anticipate the importance of digital information.","keywords":["computers","government","FOIA"],"topics":["Government & Politics","Technology"],"technologies":["mainframe","PC"],"full_text":"HISTORY: Federal Computer Use\n\n1946 Congress passes Administrative Procedure Act- forerunner to FOIA. First computer (ENIAC) goes into operation.\n\n1955 Congress begins hearings on what will become the FOIA. Federal government owns 45 computers, all first-generation mainframes.\n\n1966 Congress passes Freedom of Information Act. Law doesn't mention computers because they didn't seem important enough to include. Federal government owned about 3,000 computers (roughly one computer for every 1,000 federal workers.)\n\n1974 Congress strengthens the FOIA, \"gives it teeth.\" Law still doesn't mention computers. Computers mentioned, as an aside, in the Senate report on the law. Federal government owns about 15,000 computers.\n\n1981 IBM introduces the PC. Federal government owns 17,723 computers. Mostly mainframes; some minicomputers.\n\n1990 Federal government purchases 1 millionth computer. Purchases of mainframes and minicomputers decline dramatically; PC purchases increase.\n\n1993 Federal government purchases 2 millionth personal computer. Total still increasing rapidly. Federal government will spend $25 billion this year alone for computer hardware, software, equipment, etc.\n\n(Statistics compiled by Prof. David H. Morrissey, Department of Technical Journalism, Colorado State University, Fort Collins, Colo. Sources from various federal agencies, including: U.S. Bureau of Standards; U.S. Bureau of the Census; General Services Administration, and the Office of Technology Assessment.)","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-second-generation-databases","slug":"1994-november-second-generation-databases","url":"https://dwillis.github.io/uplink/article/1994-november-second-generation-databases.html","year":1994,"month":"November","monthNum":11,"headline":"Second-generation databases","kicker":"The CAR Lot:","author_name":"Ronald Campbell","author_title":"computer-assisted reporting specialist","affiliation":"The Register","summary":"The Register's computer-assisted reporting program uses Dell computers running DOS/Windows with FoxPro databases and MapInfo mapping software, led by a single CAR specialist who has produced investigations into campaign donations, lending discrimination, and other data-driven stories.","keywords":["reporting","software","journalism"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["FoxPro","Excel","MapInfo","XDB","Aldus FreeHand","Crosstalk","PC Tools","Nine Track Express","SPSS","SAS","XyWrite","DOS","Windows","OS/2","CD-ROM","modem/BBS"],"full_text":"The Register launched its computer-assisted reporting program in January 1991. We're on our second computer and second generation of software. Personnel: Me. I'm supposed to spend 80 percent of my time on CAR, and occasionally I really do. Efforts to train others have been sporadic and short-lived Hardware: A Dell 466-T (486 computer, 66 megahertz top speed) with 16 megabytes of RAM and a gigabyte hard drive. I also have a Qualstar tape drive, a leftover from the bad old days, as well as a CD-ROM reader and a tape backup system. The Register slowly is replacing its ancient Atex terminals with Gateway 200-mb personal computers. As the PCs spread throughout the newsroom, I hope to find converts to CAR. Operating system: DOS and Windows 3.1. The Register is shifting to OS/2. Database: FoxPro 2.6 for Windows. I switched to FoxPro from XDB a couple of years ago and have never regretted the change. FoxPro is incredibly fast, easy to use, and flexible (also brave, clean, and reverent, but that's another story). Spreadsheet: Excel 4.0. Mapping: MapInfo 2.0 for Windows. MapInfo easily handles FoxPro tables. But for more than a year, we couldn't make publication-quality maps. We solved that problem by buying MapInfo for the Macintosh (the graphic artists' computer of choice). Now I create a map on my PC, transfer the map and accompanying workspace and tables to a diskette, and walk them across the hall to Graphics; the artist loads the diskette to the Macintosh version, dresses up the map, and transfers it to Aldus FreeHand for final touchups. Not a very elegant solution, but it works. Word processing: None, unless Windows Write counts. I'm getting Xywrite for Windows soon. Communications: Crosstalk software operating on a U.S. Robotics Sportster 14,400-modem. I recently began exploring the Internet through the Register's \"node.\" Utilities: PC Tools. Other: Nine-Track Express. This is essential for CAR and likely will remain so as long as government agencies resist the trend toward compact discs. Nine-Track is easy to use, though the documentation isn't much help when something goes wrong. (Hint: If you're saddled with a Qualstar, check that first when problems arise.) Notable stories: I've sliced and diced political campaign donations dozens of ways, establishing among other things the surprising role that female professionals played in the election of California's two U.S. Senators. I've also written about discrimination in home-lending by banks, toxics, and the census. Wish list: Statistical software, such as SPSS or SAS. A decent tape-drive. Ronald Campbell can be reached by telephone at (714) 664-5030 or by e-mail at rcampbell@ocrl.freedom.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-fbi-crime-stats-due-out-soon","slug":"1994-november-fbi-crime-stats-due-out-soon","url":"https://dwillis.github.io/uplink/article/1994-november-fbi-crime-stats-due-out-soon.html","year":1994,"month":"November","monthNum":11,"headline":"FBI crime stats due out soon","kicker":"Bits, Bytes and Barks","author_name":"Gwen Carleton","author_title":"","affiliation":"NICAR","summary":"The FBI will release its 1993 crime statistics in late November, with NICAR planning to be first to obtain the data across five databases covering homicides, property crimes, demographics, and arson. The statistics will be broken down by region, state, city, and police agency.","keywords":["FBI","statistics","crime"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"FBI crime stats due out soon\n\nThe FBI publishes its 1993 crime statistics in late November, and NICAR plans to be first in line for the new data.\n\nThe statistics are broken down into five databases: the Supplementary Homicide Report, Return A, Supplement to Return A, Police Data and Arson Data.\n\nEach of the first three databases explores a different aspect of crime. Return A, for example, gives the numbers of crimes in a variety of categories such as murder, rape, robbery and so on. Supplement to Return A presents a more detailed breakdown of the property crimes and gives the value of stolen and recovered property. SHR gives information about homicide victims and offenders and their relationship to one another.\n\nAccording to the FBI, another database called Age, Sex and Race will be included in the SHR database. ASR gives demographic information about the offenders involved in a variety of crimes.\n\nAll of the above data and the police and arson databases (these describe crimes against police officers and arson reports) are broken down by region, state, city, metropolitan statistical area, and the police agency that produced the report.\n\nFor more information, including prices and release dates, call NICAR at (314) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-car-trek-highlights","slug":"1994-november-car-trek-highlights","url":"https://dwillis.github.io/uplink/article/1994-november-car-trek-highlights.html","year":1994,"month":"November","monthNum":11,"headline":"CAR Trek highlights","kicker":"Audio tapes available","author_name":"Brant Houston","author_title":"","affiliation":"NICAR","summary":"Over 500 journalists attended CAR Trek '94, a major computer-assisted reporting conference in Santa Clara featuring 60 panels, 75 training classes, and demonstrations of the latest technology. The four-day event brought together journalism professionals from across the United States and several foreign countries to learn about digital reporting tools and techniques.","keywords":["journalism","technology","conference"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["CD-ROM","magneto optical drives","magnetic tape","Power PC","GIS","Internet"],"full_text":"More than 500 journalists gathered in Santa Clara in early October for CAR Trek '94, the most ambitious computer-assisted reporting conference ever. The four-day conference included 60 panels, 75 hands-on training classes and a constantly buzzing demonstration room where conference-goers got one-on-one tutorials. Attracted by the wide array of offerings and the presence of the best in the business, journalists came from more than 30 states and from several foreign countries, including Brazil, Australia, Sweden, Norway, the Netherlands and Costa Rica. Although nearly half the attendees were from California, some made a good showing from other states: 21 from Minnesota, 18 from Washington, D.C., 18 from Florida, 16 from New York and 13 from Virginia.\n\nWhile the conference built upon the success of last year's event in Raleigh, N.C., it quickly took on its own character. Among the highlights:\n\nInstructors found advanced classes that they wanted to take, whether it was in mapping software, building friendly front-ends, building databases, or statistics.\n\nThe Internet classroom stayed filled for the conference. Initially, the classroom threatened to become a netmare when it refused to come online, but a phone cable problem was solved by Thursday night and the 25 networked Power PCs kept the room warm and toasty until Sunday.\n\nA demonstration room that exhibited the latest technology such as magneto optical drives, CD-ROMS, Internet tutorials, tape drives and a variety of software.\n\nA high-energy ethics supersession led by Michael Josephson, a lively speaker who continually kept several panelists on their toes.\n\nMore than 130 classroom computers, including 50 Power PCs, 50 Acers, and 30 IBM miscellaneous compatible personal computers. All were set up and loaded within two days.\n\nA track for more than a dozen top editors brave enough to face a high-tech future.\n\nA keynote speech by John Batelle, managing editor of the hot new magazine Wired.\n\nA first-ever panel of some of the pioneers in the field: Elliot Jaspin of Cox Newspapers, Dwight Morris of the Los Angeles Times and Philip Meyer of the University of North Carolina.\n\nCAR TREK was sponsored by Investigative Reporters and Editors, the National Institute for Computer-Assisted Reporting (a joint venture of IRE and the Missouri School of Journalism) and the San Jose Mercury News.\n\nIf you want to hear the tapes of panels you missed or get panelists' electronic handouts on diskette, call IRE at 314-882-2042 or via e-mail at jourire@muccmail.missouri.edu. If you want more information about training, call NICAR at 314-882-0684 or via e-mail at nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-november-freedom-forum-nieman-foundation-online","slug":"1994-november-freedom-forum-nieman-foundation-online","url":"https://dwillis.github.io/uplink/article/1994-november-freedom-forum-nieman-foundation-online.html","year":1994,"month":"November","monthNum":11,"headline":"Freedom Forum, Nieman Foundation online","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Freedom Forum and Nieman Foundation have launched online presences through NandO.net, offering conference transcripts, reports, and multimedia content focused on public-interest journalism.","keywords":["journalism","foundation","online"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web","Internet"],"full_text":"For those of you who haven't noticed yet, the Freedom Forum and the Nieman Foundation have gone online. Both organizations are available on the Web via NandO.net an Internet gateway operated by the Raleigh News & Observer.\n\nThe Nieman Foundation page (http://www.nando.net/may/confpage.biml) offers full transcripts of discussions at its recent conference dealing with the role and challenges of public-interest journalism in an expanding information marketplace.\n\nThe Freedom Forum page (http://www.nando.net/prof/freedom/1994/freedom.biml) features some of its published reports (including some color photographs and graphics) and selected transcripts of seminars and audio \"sound bites.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-access-drug-street-names-online","slug":"1994-november-access-drug-street-names-online","url":"https://dwillis.github.io/uplink/article/1994-november-access-drug-street-names-online.html","year":1994,"month":"November","monthNum":11,"headline":"Access drug \"street names\" online","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Drugs and Crime Data Center has published an online document containing over 1,500 street terms related to drugs and drug trade activities, accessible via FTP.","keywords":["drugs","terminology","reference"],"topics":["Crime & Justice","Investigative Methods"],"technologies":["FTP"],"full_text":"The Drugs and Crime Data Center and Clearinghouse has made its document \"Street Terms: Drugs and the Drug Trade\" available online. The document contains more than 1,500 street terms that refer to specific drug types or drug activity, from \"A\" (LSD or amphetamines) to \"Zoomers\" (individuals who sell fake crack then flee). To access the document via the net, FTP For source information, contact the DCDCC at (800) 666-3332.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-atandt-puts-800-directory-on-web","slug":"1994-november-atandt-puts-800-directory-on-web","url":"https://dwillis.github.io/uplink/article/1994-november-atandt-puts-800-directory-on-web.html","year":1994,"month":"November","monthNum":11,"headline":"AT&T puts \"800\" directory on Web","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"AT&T has launched an online directory of toll-free 800 numbers accessible through the World Wide Web at http://att.net/dir800.","keywords":["AT&T","directory","toll-free"],"topics":["Technology"],"technologies":["World Wide Web"],"full_text":"AT&T's 800 Directory of toll-free numbers throughout the United States is now available on the World Wide Web (http://att.net/dir800).","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-covering-the-computer-beat","slug":"1994-november-covering-the-computer-beat","url":"https://dwillis.github.io/uplink/article/1994-november-covering-the-computer-beat.html","year":1994,"month":"November","monthNum":11,"headline":"Covering the computer beat","kicker":"The medium is the message - Part Two","author_name":"David Bloom","author_title":"Reporter","affiliation":"The Los Angeles Daily News","summary":"Computer-assisted reporters should investigate government computer systems for investigative story opportunities, including incompatible departments, poor data coordination, and public access barriers to information.","keywords":["government","systems","access"],"topics":["Investigative Methods","Government & Politics","Technology"],"technologies":["mainframe","floppy disk"],"full_text":"Editor's Note: This is the second of a two-part article offering advice to computer-assisted reporters who might cover or develop a computer beat. Part one ran in the October issue of Uplink.\n\nIn the last issue we talked about a variety of government operations that are ripe for good investigative pieces. You might also look at data coordination across departments. Frequently you will find a large government agency has numerous systems, none of which are compatible. Many times they can't even share e-mail.\n\nSometimes personal disputes can lead to computer problems. In Los Angeles County, for instance, the Treasurer-Tax Collector, Auditor-Controller and Assessor departments all are responsible for part of the property tax billing and collection process.\n\nWhen it came time for a new property tax system, however, the three departments didn't work together. The Assessor developed one system on its own, while the other two departments separately developed a second one that was supposed to hook up with the assessor's new computers.\n\nWhen a new Assessor got into a personal war with the Auditor, communications between the departments ceased. Months later, when the assessor's new computer system went on-line, it kicked out 15,000 parcels, just didn't recognize them. No one caught the problem in the Assessor's office. The department that DID catch the problem, the Auditor's office, didn't bother to tell the assessor for months, until the problem entries filled boxes and boxes.\n\nAnd when the Auditor-Treasurer system started running two years later, it also didn't work well with the assessor computer, causing another 26,000 problem parcels that took years, millions of dollars and dozens of additional staff to fix.\n\nThen there's the general problem of just getting information out of these computers. Some systems don't even allow you to get electronic versions of their records easily and inexpensively. When I sought a text-only dump of all the appointees to county boards and commissions, bureaucrats told me it would cost well over $200 for what should have been a 15-minute (or less) text dump onto a single floppy disk. The mainframe they use would require special programming to do even that basic task, they said. We passed on the opportunity, but there's no reason why it should have required anything like that kind of cost. Often, the information is in a proprietary format that can only be read by a single program, which in turn may cost thousands of dollars. How does the public get access in a reasonable way to this information? And what happens when the agency gets a new system and wants to convert the information to that program? Can it?\n\nThere are also a number of \"softer\" stories about computer systems that newspapers ought to run more often, spotlighting issues of public access to the information we've all paid so much for government to collect.\n\nMany revenue-starved governments are looking at ways to make money. The possibility of selling tapes of property tax records, voter registration rolls or other information at a premium is quite tempting, but often is a violation of the spirit if not letter of open records laws.\n\nInformation, please\n\nSociology 101\n\nComputers are changing the way we relate to each other, through e-mail, on-line chat services, and more. George Gilder, among others, has written interestingly about the changes being wrought.\n\nE-mail, for instance, causes people to communicate in different ways (more candid, sometimes painfully, or rudely so) than we do in person.\n\nIt's worth investigating as we network more and more to see what it means for streamlining government and improving public access. Politicians are increasingly seeing the Internet and other online capabilities as a great way to reach out and tell their side of things. That's worth following up. There are issues of children's access to the sometimes raw discussion channels and chat lines of adults in online services and bulletin boards. There are concerns about child molesters and pornographers using the lines to recruit new victims and trade their materials with each other. There have been occasional scandals afflicting some of the national on-line services in this regard in the past three or four years.\n\nThese ideas ought to get you started and many, in fact, can either come out of or lead to computer-assisted projects. Often, you'll get leads on problem systems when you try to get copies of interesting tapes of information. Additionally, you'll make important contacts within the bureaucracy who can point you to problems while you do the kind of small-talk kibitzing that leverages these tips out. Working on computer-assisted projects also will give you the background and lingo you need to intelligently explain what's going on in what can be a difficult and challenging area for readers and bureaucrats alike. Have at it.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-november-information-please","slug":"1994-november-information-please","url":"https://dwillis.github.io/uplink/article/1994-november-information-please.html","year":1994,"month":"November","monthNum":11,"headline":"Information, please","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Newspapers should cover technology stories addressing public access to government data, the internet's role in politics and communication, and emerging concerns about online safety including child protection issues.","keywords":["government","internet","children"],"topics":["Journalism & Newsroom","Technology","Government & Politics"],"technologies":["Internet","Email"],"full_text":"There are also a number of \"softer\" stories about computer systems that newspapers ought to run more often, spotlighting issues of public access to the information we've all paid so much for government to collect.\n\nMany revenue-starved governments are looking at ways to make money. The possibility of selling of tapes of property tax records, or\n\nIt's worth investigating as we network more and more to see what it means for streamlining government and improving public access. Politicians are increasingly seeing the Internet and other online capabilities as a great way to reach out and tell their side of things. That's worth following up. There are issues of children's access to the sometimes raw discussion channels and chat lines of adults in online services and bulletin boards. There are concerns about child molesters and pornographers using the lines to recruit new victims and trade their materials with each other. There have been occasional scandals afflicting some of the national on-line services in this regard in the past three or four years.\n\nComputers are changing the way we relate to each other, through e-mail, on-line chat services, and more. George Gilder, among others, has written interestingly about the changes being wrought.\n\nE-mail, for instance, causes people to communicate in different ways (more candid, sometimes painfully, or rudely so) than we do in person.\n\nCan it?","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-november-sociology-101","slug":"1994-november-sociology-101","url":"https://dwillis.github.io/uplink/article/1994-november-sociology-101.html","year":1994,"month":"November","monthNum":11,"headline":"Sociology 101","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article examines how computers and online technologies are transforming human relationships and communication, while exploring implications for government, politics, and child safety in digital spaces.","keywords":["technology","communication","society"],"topics":["Technology","Government & Politics","Journalism & Newsroom"],"technologies":["Internet","Email","modem/BBS"],"full_text":"Sociology 101\n\nComputers are changing the way we relate to each other, through e-mail, on-line chat services, and more. George Gilder, among others, has written interestingly about the changes being wrought.\n\nIt's worth investigating as we network more and more to see what it means for streamlining government and improving public access. Politicians are increasingly seeing the Internet and other online capabilities as a great way to reach out and tell their side of things. That's worth following up. There are issues of children's access to the sometimes raw discussion channels and chat lines of adults in online services and bulletin boards. There are concerns about child molesters and pornographers using the lines to recruit new victims and trade their materials with each other. There have been occasional scandals afflicting some of the national on-line services in this regard in the past three or four years.\n\nE-mail, for instance, causes people to communicate in different ways (more candid, sometimes painfully, or rudely so) than we do in person.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-december-uplink-update","slug":"1994-december-uplink-update","url":"https://dwillis.github.io/uplink/article/1994-december-uplink-update.html","year":1994,"month":"December","monthNum":12,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This Uplink newsletter highlights how computer-assisted reporting (CAR) can be used effectively for beat and deadline journalism, featuring examples from reporters at major news organizations who use databases to break stories quickly and cover ongoing beats.","keywords":["reporting","databases","journalism"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Many journalists still think of computer-assisted reporting as long-term, tree-eating (at least for newspapers) projects. But this month's Uplink highlights how computer-assisted reporting can be used for beat and deadline reporting.\n\nRalph Frammolino of the Los Angeles Times shows how quickly a database manager can help a reporter get the scoop and beat the competiton on a story everyone else is covering.\n\nPaul D'Ambrosio writes about the non-stop CAR stories the Asbury Park Press produces, while Bill Dedman of AP and Richard Mullins of NICAR recount the tribulations and triumphs of supplying AP bureaus nationwide with federal campaign finances.\n\nAnd, for the beat, Gwen Carleton offers a review of some of the year's best CAR crime stories.\n\nSee you on the net at nicar@muccmail.missouri.edu or on our listserv located at nicar-.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-december-car-and-oj-perfect-together","slug":"1994-december-car-and-oj-perfect-together","url":"https://dwillis.github.io/uplink/article/1994-december-car-and-oj-perfect-together.html","year":1994,"month":"December","monthNum":12,"headline":"CAR & OJ: Perfect together","kicker":"Newly acquired skills reveal jury profile","author_name":"Ralph Frammolino","author_title":"Reporter","affiliation":"Los Angeles Times","summary":"Ralph Frammolino discusses the application of computer-assisted reporting (CAR) in creating a detailed jury profile for the O.J. Simpson trial, highlighting the effectiveness of newly acquired skills.","keywords":["O.J. Simpson","jury","database"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Access"],"full_text":"Talk about instant gratification.\n\nThe Los Angeles Times recently invited NICAR to town to put me and nine other reporters, editors and researchers through a week-long computer-assisted journalism boot camp. The goal: To lighten the load on the paper's overworked in-house computer gurus by introducing at least a few newsroom types to the rudiments of transferring files and analyzing statistics on a spreadsheet or database program.\n\nOur mission-besides trying to complete a game of computer Solitaire during class breaks or when the instructor wasn't looking-was to overcome the fear of ASCII, DOS, ANSI and a handful of other tortured acronyms, and to eventually integrate the powerful tool of computer analysis into daily reporting or projects.\n\nBut no sooner had I walked back into the newsroom the next Monday morning, then my (gulp) newly acquired \"expertise\" was put to the test. And on no less a story than the O.J. Simpson murder trial.\n\nThe judge and attorneys on the case were close to making the first cut of the jury pool, and my editor was eager to turn a quick profile of those still in the running to determine the fate of the Hall of Fame running back.\n\nAll we had to go on were three pages of responses each member of the jury pool on questionnaires about their backgrounds, impressions about the case, what they had learned from the media and their own experiences with crime. Under normal circumstances, we'd either eyeball the responses or tally up a few categories-race, sex, educational background--and paint a broad-brush portrait of the panel. But the NICAR training helped us to dig deeper, be more precise and do more extensive comparison. We set this up by using the Microsoft Access program. After looking at the three-page forms, I decided to create about 30 fields to capture each juror's age, race, sex, educational level and home city, as well as key responses about media exposure, attitudes and personal habits that could be represented by symbols or a simple yes or no. Thus, I recorded whether each watched the infamous chase of the white Bronco and for how long; heard anything about Nicole Brown Simpson's previous 911 call to police alleging O.J.'s domestic abuse; had met Simpson in person; and had ever seen him play football.\n\nI recorded what their friends said about Simpson (guilty, not guilty, no opinion); what each potential juror thought about Simpson; whether each had experienced domestic abuse, resorted to domestic abuse or thought it should be treated differently than other assaults. I noted how each felt about discrimination against African Americans in Southern California (not serious, somewhat serious, very serious, etc.); whether each had been a victim of crime and the worst type of crime; who owned knives; who enjoyed detective novels or television shows; and who dated people of a different race. With a data researcher reading them off, it took me hours to type in the responses but we were able to have all 92 files in the Access database by about 4:15 p.m. I kept two files- those who made the cut and those who didn't- and kept adjusting them almost instantly as the judge and attorneys made their decisions. For the next 45 minutes, it was just a series of frantic Access drills-group by, count, descending order, difference divided by the first number - for each category of interest. By deadline, we could say that 61 percent of the jurors who had not been eliminated were black, 25 percent were white; 19 percent said they experienced domestic violence at home, and many more saw Simpson at his worst during the Bronco chase than ever saw him in his glory playing football. The effort on that first day enhanced our story and provided fodder for the graphic. Yet the biggest payoff came on Tuesday, when I used the database to perform a different analysis. Rather than concentrate on the jurors who made the cut, I decided to compare those who were still in the running with those who were weeded out. At first, I intended to look at characteristics such as race and geography of the two different pools when-thanks to Access-something unexpected jumped right out at us.\n\nEach panelist was asked to express an opinion about the reliability of DNA testing (somewhat reliable, very reliable, etc.), and the numbers showed that 39 percent of those who had been excused were likely to believe in it compared to 10 percent who remained in the pool.\n\nSince the prosecution's case rests overwhelmingly on DNA tests linking Simpson's blood to the crime scene, this seemed to be a significant difference and it became the lead of my front-page story the next day. A jury consultant I interviewed underscored the significance further for our readers by saying that the effect was a \"plus for the defense because there aren't people who have already made up their minds that it [DNA testing] is irrefutable evidence.\"\n\nThe story also noted other important differences that seemed to favor the defense in the jury pool. More than 80 percent of the rejected jurors had been crime victims and 31 percent had experienced domestic violence themselves - percentages much greater than those left in the jury pool. The second-day story was well received and it couldn't have been possible without the NICAR training. While it required far less knowledge and skills than the ambitious computer-assisted investigative projects featured in recent Uplink newsletters, it nonetheless shows how far a little computer work can go.\n\n-Ralph Frammolino can be reached by e-mail at frammol@news.latimes.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-december-coming-events","slug":"1994-december-coming-events","url":"https://dwillis.github.io/uplink/article/1994-december-coming-events.html","year":1994,"month":"December","monthNum":12,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"This is a schedule of upcoming NICAR (National Institute for Computer-Assisted Reporting) seminars being held in Columbia, Missouri during January and March 1995.","keywords":["NICAR","seminars","Columbia"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nJanuary 8-13, 1995 NICAR Seminar Columbia, Missouri\n\nMarch 12-17, 1995 NICAR Seminar Columbia, Missouri","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-december-cars-field-of-dreams-in-nj","slug":"1994-december-cars-field-of-dreams-in-nj","url":"https://dwillis.github.io/uplink/article/1994-december-cars-field-of-dreams-in-nj.html","year":1994,"month":"December","monthNum":12,"headline":"CAR's Field of Dreams in NJ","kicker":"The CAR Lot","author_name":"Paul D'Ambrosio","author_title":"CAR Coordinator","affiliation":"Asbury Park Press","summary":"The Asbury Park Press developed a computer-assisted reporting program providing newsroom-wide access to databases and analytical software, enabling investigations into local issues like tax delinquency and crime statistics despite ongoing staff adoption challenges.","keywords":["reporting","investigation","databases"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Nexis","CompuServe","Internet","America Online","PC","FoxPro","Excel","Mosaic","SPSS","Atlas GIS"],"full_text":"If you put a gun to my dog's head and forced me to write one of those movie comparison leads, it would probably read something like this: CAR at the Press is like \"Field of Dreams\" - If you build it, they will come.\n\nIn the last two years we developed a computer-assisted reporting program that is accessible to everyone. We have open access to NEXIS, CompuServe, Internet and America On-line. Anyone who wants to get involved with CAR can learn at their own pace. We have held several Internet classes, and this summer, we trained 19 staffers during a week of intensive CAR classes. We have installed 486/66 computers in each of our six bureaus, and at our sister paper, The Home News. The staff has access to FoxPro, Excel and Mosaic. SPSS for Windows and Atlas GIS are currently limited to one system because of software costs.\n\nThe Press' program is broken into two areas: projects of a state or national interest, and those dealing with local issues.\n\nTo understand our computer-assisted reporting program, you have to understand the Press' news area. We cover 83 towns in two counties. New Jersey has a strong home rule charter, which means most of the information we need is kept only at the municipal level.\n\nThere is no state or county agency that collects tax information or police records for us. In order to provide context for any regional story, we have to negotiate with 83 bureaucrats, all of whom have a different interpretation of the state's vague and anemic Open Public Records Act.\n\nThe only way to keep track of all that information is with PCs. In early 1994, we decided to take a look at delinquent property taxpayers in one county. The municipal reporters collected printed tax lists from each town and we had clerks build our own database of deadbeat taxpayer information.\n\nFour months and 50,000 names later, we ran a two-day, eight-part series. We not only outline the damage delinquent taxpayers were doing to the tax base, we also named names: a state judge, two political kingmakers (including the GOP's former national finance chairman) and an assortment of local and county politicos.\n\nOther stories we have done include:\n- Finding that the dead were still voting in one county,\n- Discovering that the governor's hiring freeze was anything but a freeze.\n- Exploring the world of sexual deviancy via the Internet and local bulletin boards. The resulting article ran on our Health page.\n- Showing that the state's crime rate hasn't changed in the last 10 years, despite a plethora of mandatory-minimum prison sentence laws and billions of dollars in new prison instruction.\n\nPerhaps the greatest hurdle at the Press is not money or management support, but what I call the \"15-Minute Syndrome.\" This is when a reporter, coat in hand, asks you to show him or her how to do CAR in 15 minutes.\n\nStaffers have learned to use NEXIS for quick research, but there is a reluctance by many staffers to get involved with databases and spreadsheets that may take more than a day to examine. I have yet to hit upon a way to get staffers involved with intense CAR on a regular basis.\n\nAs a result, many of the above stories were written by Rick Linsk and myself, or in conjunction with other staffers. What I have seen is a generation gap-new staffers who have come to the Press in the last year are using the PCs more frequently than staffers who have been at the paper longer.\n\nTo remedy this situation, I hope that a series of intensive, week-long training classes next year will close the gap and fulfill our \"Field of Dreams\"-reporters flocking to personal computers.\n\n-Paul D'Ambrosio is CAR Coordinator at the Press. He can be reached by phone at (908) 922-6000, ext. 4261, or via e-mail at pmd@app.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-december-ap-scores-with-big-fec-project","slug":"1994-december-ap-scores-with-big-fec-project","url":"https://dwillis.github.io/uplink/article/1994-december-ap-scores-with-big-fec-project.html","year":1994,"month":"December","monthNum":12,"headline":"AP scores with big FEC project","kicker":"Contribution charts cover all 50 states","author_name":"Brant Houston","author_title":"NICAR Managing Director","affiliation":"NICAR","summary":"An AP team led by Bill Dedman and Richard Mullins used computer-assisted reporting and FEC data to create 25 detailed reports on campaign contributions for 1,300 federal candidates before the November elections. The project pioneered the use of database management to analyze donor patterns and connect congressional actions to funding sources.","keywords":["data","elections","reporting"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["FoxPro","modem/BBS","CD-ROM"],"full_text":"Its creators may call it \"public service agate,\" but an Associated Press project recently demonstrated how computer-assisted reporting can strongly bolster routine election coverage.\n\nWith a lot of toil and ingenuity, Bill Dedman, director of computer-assisted reporting for AP, and Richard Mullins of the National Institute for Computer-Assisted Reporting gave AP bureaus up-to-date contribution data just before the November federal elections.\n\nAP bureaus in every state received 25 or more reports for federal candidates in their area. The charts used different categories to sort contributions from individuals and political action committees.\n\nFor example, one chart sorted contributions by city, another by top fundraising days and yet another by special interests. Altogether, Dedman and Mullins analyzed contributions to 1,300 federal candidates. The longest chart alphabetically listed all individuals who gave $500 or more.\n\nMullins, an assistant professor at the Missouri School of Journalism and an academic adviser for NICAR, did the slicing and dicing of Federal Election Commission data for AP. The FEC collects and distributes contribution data for candidates.\n\nIn an intensive effort that spanned two months, Mullins developed reports on a NICAR Texas Instrument laptop-a 486-50 megahertz machine with a 200 megabyte hard-drive. He used the FoxPro database manager to sort more than 60 megabytes of data that contained 300,000 individual contributions and 128,000 PAC donations.\n\nThat was the easy part.\n\nThe more difficult challenge was making the reports suitable for conversion to standard AP agate. Mullins used the FoxPro memo field to store the appropriate headers, footnotes, and queries.\n\nHe also had to ensure that no report was longer than 130 lines. If it was, he had to split it into multiple \"takes\" because long files could not be used by the smaller AP members. After a report was ready, Dedman used a modem to call Mullins' computer and download the report from Missouri to New York City. (Call Mullins at 314-882-2127 or send e-mail to him at jourram@muccmail.missouri.edu for more technical details.)\n\n\"We made available to AP staff about 12,000 tables,\" Dedman said. \"This allowed AP reporters to dive in, wade in or stick their toes in.\"\n\nDedman said almost every state AP bureau used the data for stories. For the project, AP also purchased special interest codes from the Center for Responsive Politics. Those codes identify PACs by economic or political interest because the PAC names generally offer little clue as to the industry or ideology they represent. Jacqueline Duobinis, a researcher at the National Library of Money & Politics, also assisted on the project with advice and tips. (The library is affiliated with the center.)\n\nThe effort, as Dedman and Mullins said, was a mass-produced assembly line approach that resulted in the most extensive campaign finance information and detail ever provided for AP bureaus for an election.\n\n\"It wasn't fancy, just universal,\" Dedman said.\n\nBefore election week was over, AP followed up the initial project with a quick analysis of the dollars spent by candidates for each vote. That analysis revealed that the most expensive winning and losing campaigns for the U.S. House and Senate.\n\n\"Oliver North spent the most money to lose a Senate seat,\" Dedman said. \"He spent $17 a vote.\"\n\nMullins said he hopes the effort might give helpful ideas to newspapers and broadcasters trying to distribute state-wide databases to reporters in their remote bureaus. For example, some news organizations are considering writing often-used data onto CD-ROMs and giving those CDs to bureaus or reporters who take road trips frequently.\n\nDedman also said that the real value of this project will not be the work on election eve, but during the next Congress when it will be possible to compare a Congressman's actions with the contributions.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-december-social-stats-on-the-net","slug":"1994-december-social-stats-on-the-net","url":"https://dwillis.github.io/uplink/article/1994-december-social-stats-on-the-net.html","year":1994,"month":"December","monthNum":12,"headline":"Social stats on the Net","kicker":"Bits, Bytes and Barks","author_name":"Dan Browning","author_title":"","affiliation":"compuserv.com","summary":"A guide to online newsgroups and mailing lists focused on statistics and social science research, including resources like sci.stat.consult, CUSSNET, STAT-L, and EVALTEN where journalists and professionals can find statistical expertise and discuss methodology.","keywords":["statistics","newsgroups","resources"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Internet","Usenet","Email","Listserv"],"full_text":"Social stats on the Net\n\nInterested in degrees of freedom? What about range values? Is multivariate data analysis your cup of tea? Let's be serious: How about a novice's questions about reliabilities?\n\nYou can find these topics and hundreds of other statistical bran muffins on an active newsgroup at the following e-mail address:\n\nsci.stat.consult. It's full of serious statisticians as well as us novices who need a helping hand now and then.\n\nHere's several other valuable resources on statistical conundra:\n1. CUSSNET, short for Computer Users in the Social Sciences. It's a discussion list devoted to social workers, counselors, human service workers and others interested in computer applications and research. To get on the list, send a message to LISTSERV@stat.com and in the body of the message type, SUBSCRIBE CUSSNET. To contribute an article, send it to CUSSNET@stat.com.\n\n2. STAT-L is an unlisted mailing list. It appeals to anyone interested in statistics. To subscribe, send e-mail to LISTSERV@MCGILLI.BITNET (or LISTSERV@VM1.MCGILL.CA). Digests of the contributions can be found in the newsgroup sci.stat.consul\n\n3. EVALTEN, short for Evaluation Center's Topical Evaluation Network on Methodology and Statistics, is another mailing list. This one primarily works with statistics and mental health. To subscribe, send e-mail to Majordomo@world.std.com. In the body of the message, type SUBSCRIBE EVALTEN. To contribute articles, send them to EVALTEN@world.std.com\n\nI am 95 percent certain that the average computer-assisted reporter out there will find something useful the online resources mentioned above. -Dan Browning (70343.2713@compuserv.com).","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-december-fun-with-the-dir-command","slug":"1994-december-fun-with-the-dir-command","url":"https://dwillis.github.io/uplink/article/1994-december-fun-with-the-dir-command.html","year":1994,"month":"December","monthNum":12,"headline":"Fun with the dir command","kicker":"Tech Tips","author_name":"Richard Mullins","author_title":"","affiliation":"University of Missouri","summary":"A DOS command technique using DIR with specific options can create a comprehensive, sortable inventory of all files on a hard drive and save it to a text file for easy viewing or printing.","keywords":["DOS","directory","files"],"topics":["Technology"],"technologies":["DOS","Windows"],"full_text":"There really is something useful that a DOS command can do for you, even if you use Windows all the time because you hate typing DOS commands.\n\nComputer hard drives can quickly become like attics or neglected closets. They fill up with so much stuff that we forget what's there and waste a lot of space with stuff we don't need any more or files that are duplicated unknown times in subdirectories all over the place.\n\nWhat if we could inventory all the files on a hard drive into one sorted text file that could be viewed with a text editor or printed? The DIR command, plus a few options, will do it. Many DOS users are familiar with the use of /p or /w after a DIR command. These switches, as they're called, pause the output of the DIR list every screenful or arrange the output horizontally. The following disk inventory command will use three options you may not have used before. Here's the command. Change to the root directory first (cd/), then type: dir /on /s /a-d > disklist.txt\n\nIt may take a minute to finish, and you won't see anything on your screen while it's working, but that's it. Here's the explanation: The first option /on - orders the directory output by name. Use /oe if you want to order files by extension, /od by date, /os by size. The second option /s - gets the directory output for all subdirectories. That's why you have to do the command from the root directory of the drive.\n\nThe third option /a-d - means show only files, not directory names, which you normally see in a DIR output. We exclude them here because they would be redundant. Every subdirectory in the output will have a full path heading.\n\nTo understand this one, remember that DOS considers directories as a special kind of file. Translated more exactly, this option means: include files that don't have the directory attribute. \"Slash A\" (/a) introduces the attribute option; \"Minus d\" (-d) means \"not directory.\"\n\nThe greater-than sign (>) is the redirect sign. If you leave this part off, you'd see the directory output rush by on the screen without stopping. Redirecting the output from the screen to a file (here called disklist.txt) gives you something a little more permanent you can view or print.\n\nOne more tip: If you want a quick reminder on any DOS command, don't fumble for your manual; the answer is already on your computer if you have DOS 5.0 or higher. You can find the short documentation on any DOS command by typing the command word, followed by /?\n\nFor example: copy /?\n\n- Richard Mullins can be reached at (314) 882-2127.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-december-milliron-joins-gannett","slug":"1994-december-milliron-joins-gannett","url":"https://dwillis.github.io/uplink/article/1994-december-milliron-joins-gannett.html","year":1994,"month":"December","monthNum":12,"headline":"Milliron joins Gannett","kicker":"","author_name":"","author_title":"Special Projects Editor for Computer-Assisted Reporting","affiliation":"Gannett News Services","summary":"David Milliron joins Gannett News Services as Special Projects Editor for Computer-Assisted Reporting, where he will coordinate database reporting efforts and provide training to USA TODAY and Gannett journalists.","keywords":["journalism","database","reporting"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"David Milliron has joined Gannett News Services as its Special Projects Editor for Computer-Assisted Reporting.\n\nMilliron - who comes from the Fort Myers (Fla) News-Press, where he was police/database reporter - will coordinate database reporting efforts for GNS and assist USA TODAY and Gannett journalists in their computer and data-gathering efforts. Milliron is available to conduct training at Gannett newspapers, and can be reached at (703) 276-5805. He is a graduate of the University of Florida, Gainesville.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-december-crime-tops-car-agenda-for-94","slug":"1994-december-crime-tops-car-agenda-for-94","url":"https://dwillis.github.io/uplink/article/1994-december-crime-tops-car-agenda-for-94.html","year":1994,"month":"December","monthNum":12,"headline":"Crime tops CAR agenda for '94","kicker":"This just in","author_name":"Gwen Carleton","author_title":"","affiliation":"NICAR","summary":"In 1994, CAR reporting concentrated on crime coverage, with notable investigations including the Wisconsin State Journal's examination of crime and socioeconomic factors in Madison, analysis of juvenile crime patterns in Southern California, and USA Today's study of prosecutor effectiveness disparities across cities.","keywords":["crime","investigation","enforcement"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Excel","FoxPro","SPSS"],"full_text":"Judging from the clips we've seen, computer-assisted journalists made a strong showing in 1994. Stories made possible by computer analysis appeared nationwide, tackling a multitude of topics. Not surprisingly in the year of the Brady Bill, the Crime Bill and \"Three Strikes, You're Out,\" crime was an especially popular topic.\n\nAs we reported in September's Uplink, the Wisconsin State Journal continued its award-winning \"City of Hope\" series into 1994. The series, which focuses on urban problems in Madison, looked at several aspects of crime this year: where the most crimes occur, how shortages in affordable housing aggravate the problems and the connection between unemployment and crime.\n\nStaff reporters did their analyses with the help of one personal computer running some inexpensive local police records through Excel, FoxPro and SPSS software. Yet, they have become such authorities on crime statistics that the police now refer some inquiries to the State Journal.\n\nMeanwhile in Southern California, Rebecca Fairley Raney took a close look at youth violence in her three-day San Bernardino Sun series, \"Too violent, too young.\" The series described how a system designed to reform truants is straining to keep juvenile murderers off the street.\n\nRaney analyzed thousands of San Bernardino arrest records. She found that, although the number of teens arrested remained level between 1982 and 1992, the severity of their crimes changed dramatically. Car theft, robbery and assault with a deadly weapon made the top ten list of juvenile crimes, displacing relatively minor offenses such as truancy and curfew violation. The numbers portray an increasing number of youths unfettered by discipline and desperate for respect, and a legal system that only can deal with the worst among them.\n\nUSA Today reporters Laura Frank and Sam Vincent Meddis looked at federal prosecutors across the nation in their special report, \"Unequal justice.\" They found that, although the number of U.S. attorneys nearly doubled during the last decade, convictions fell. The reporters compared different cities according to the number of prosecutors, the number of cases prosecuted and convictions per million. They discovered wide disparities.\n\nAccording to the USA Today analysis, different crime rates drain resources at different rates, and an avalanche of federal laws promote selective enforcement. As a result, neighboring districts can differ dramatically with regard to who they prosecute and how harshly they punish. The article also cites legislative efforts to increase oversight, at least one of them provoked by the newspaper's investigation.\n\nAnother analysis, this one by Jon Schmid and Joseph Neff of the Raleigh News & Observer, examined the probable impact of the celebrated \"Three Strikes, You're Out\" proposal in North Carolina.\n\nThe story, which appeared as the state legislature weighed a \"Three Strikes\" bill, found the law would do little for the state. Only one criminal in 2,000 current would have been affected, according to Schmid and Neff's analysis of FBI data and North Carolina Corrections Department data.\n\nOf the 177,505 people entering state prisons between 1972 and 1992, only 91 committed three violent crimes - and more than half of those criminals remain in prison. Schmid and Neff showed that only a tiny proportion of crimes might have been prevented by a \"Three Strikes\" law. Just five criminals are currently in jail for committing four violent crimes.\n\nThe reporters used charts and graphs to illustrate the age of most violent offenders, as well as the crimes third-time offenders committed. They also talked to politicians, criminologists and convicted felons concerning the crime bill's likely impact on the state.\n\nElsewhere, other papers went beyond police, FBI and prison statistics to look at more unconventional types of crime. Paul D'Ambrosio of the Asbury Park Press compared the Social Security Administration's Master Death List with voter records to see just how many of New Jersey's dead are showing on voting lists.\n\nD'Ambrosio found 370 dead people registered to vote in the area he examined; one even changed party posthumously.\n\nAn examination of local death records showed some of those listed as deceased by the election board actually are alive and well. D'Ambrosio found other instances where the election board probably passed a dead person's registration on to another voter with a similar name. However, scores of other posthumous votes remained a mystery.\n\nD'Ambrosio pointed out the Motor Voter law, which will go into effect Jan. 1 will make it more difficult to remove questionable voters from the rolls.\n\nReporters from the (Greensburg, Pa.) Tribune-Review examined crime in another specialty area - the rent-to-own industry. Reporters Sharon Santus and David Josar discovered rent-to-own stores in their area typically charge customers more than 200 percent interest for items such as TVs, furniture and refrigerators. Moreover, the stores use bullying repossession tactics when customers fall behind on their inflated bills.\n\nA computerized mapping system at the University of Pennsylvania's library revealed nearly two-thirds of the stores were located in areas with large minority populations and large percentages living below the federal poverty level. The computer analysis showed what consumer advocates had been trying to prove: that rent-to-own store operators target minority and poor sections of the community. The Pennsylvania Attorney General's Office intends to use the research to help negotiate a settlement with several rent-to-own operators in the state.\n\nThese stories are just a sample of the quality work we've seen throughout 1994. Thanks to all of you for your submissions - be sure to stay in touch in the new year.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1994-december-crime-data-available-at-nicar","slug":"1994-december-crime-data-available-at-nicar","url":"https://dwillis.github.io/uplink/article/1994-december-crime-data-available-at-nicar.html","year":1994,"month":"December","monthNum":12,"headline":"Crime Data available at NICAR","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to comprehensive FBI crime statistics from 1994 organized across five detailed databases, along with Federal Death records from the Social Security Administration and ATF firearms dealer information for investigative journalism purposes.","keywords":["FBI","databases","records"],"topics":["Crime & Justice","Data & Databases"],"technologies":["dBase"],"full_text":"NICAR now has 1994 FBI records, available either for the whole nation or individual states. The database is divided into five tables: Return A, listing numbers of crimes committed in a number of categories; Supplement to Return A, listing more details about the crimes (especially types and dollar amounts of property crimes); Supplemental Homicide Report, listing the victims and offenders in homicides (this also includes victim/offender data on other crimes); Police, listing assaults and homicides of police officers; and Arson, listing reported and proven instances of arson.\n\nNICAR also has Federal Death records from the Social Security Administration and data from the Federal Bureau of Alcohol, Tobacco and Firearms. SSA Death Records comprise almost 50 million deaths dating from 1937. The SSA collected the records from Social Security rolls, states, funeral homes and other sources. Data included are SSN, name, dates of birth and death, state code and zip code,\n\nThe ATF database contains the business and mailing addresses of firearms dealers across the United States. Please contact NICAR for more information and prices.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-december-department-of-justice-data","slug":"1994-december-department-of-justice-data","url":"https://dwillis.github.io/uplink/article/1994-december-department-of-justice-data.html","year":1994,"month":"December","monthNum":12,"headline":"Department of Justice Data","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The National Criminal Justice Reference Service (NCJRS) launched a new online gopher service providing access to information from multiple Office of Justice agencies including juvenile justice, victim services, and crime statistics resources.","keywords":["NCJRS","justice","gopher"],"topics":["Crime & Justice","Data & Databases","Government & Politics"],"technologies":["gopher"],"full_text":"The National Criminal Justice Reference Service (NCJRS) recently announced its new online service. The new gopher provides users with an overview of the NCJRS and Office of Justice agencies including: The Office of Juvenile Justice and Delinquency Prevention; the Office of Victims of Crime; the Bureau of Justice Statistics Clearinghouse; and the Drugs and Crime Data Center Clearinghouse. The URL is: gopher://ncjrs.aspensys.com:77","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1994-december-sharing-car-with-the-masses","slug":"1994-december-sharing-car-with-the-masses","url":"https://dwillis.github.io/uplink/article/1994-december-sharing-car-with-the-masses.html","year":1994,"month":"December","monthNum":12,"headline":"Sharing CAR with the masses","kicker":"Front-ends broaden computer use","author_name":"Alan Levin","author_title":"Computer-assisted reporting developer","affiliation":"The Hartford Courant","summary":"The Hartford Courant created a customized FoxPro application to make computer-assisted reporting resources accessible to non-technical reporters by simplifying database searching through user-friendly menus and point-and-click interfaces.","keywords":["databases","interface","accessibility"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["XDB","SQL","FoxPro","9-track tape"],"full_text":"As computer-assisted programs in newsrooms mature, there's an inevitable problem: How to distribute the wealth of computer resources to the masses of reporters who have little or no high-tech training.\n\nAt The Hartford Courant, where I began work earlier this year, we were bursting with information, but could not always easily pass it on to those who needed it.\n\nAs one of the first papers to aggressively use computers in news gathering, the Courant had filled several large hard drives and nine-track tape racks with data of all descriptions.\n\nLiterally hundreds of databases had been stored away as a resource during Brant Houston's tenure at the paper (which ended January 1994 when he became managing director at NICAR).\n\nThe data files were available for inspection on the old computer-assisted war horse, XDB, an SQL database program. But those files were inaccessible to many reporters and editors without a computer-trained guide. Though more than 30 reporters and editors had received training, the training did not stick unless they continued to work on computer-assisted stories.\n\nThe solution we chose was to create a customized application program in FoxPro for Windows that took advantage of the searching power of SQL, but allowed any reporter who could use a mouse and a pull-down menu the opportunity to search through dozens of files.\n\nOur customized application program in FoxPro for Windows grants access to selected criminal records, professional listings of lawyers and accountants, a simple state payroll, an ever-growing list of contributors to several gubernatorial candidates, some basic census information, a listing of all persons registered to own handguns in the state and a Hartford voters' list - among other files.\n\nWithout going into details of FoxPro programming language, a user enters the system by clicking on a special icon in the Windows Program Manager.\n\nOnce it starts, the program allows users to choose a database from a Windows-style, pull-down menu. After the database is selected, a simple search screen appears prompting a reporter to type in a name or some other search criteria. Clicking the mouse on a box starts the search. All the SQL language is embedded in the program and activated without any work by the searcher.\n\nThe menus also give access to a simple help file designed to guide a reporter through a search and include text files describing each of the databases.\n\nThe system is by no means perfect. Any real programmer who examined my crude code would probably drop dead of fright. Though most people who use it seem to navigate the system without problems, it could be further simplified.\n\nWe are also working to expand the types of data on the system with an eye toward adding to its functionality.\n\nOur next hurdle is to figure out a way to give reporters located in our extensive bureau system access to such a program without lugging hundreds of floppies out to each bureau (not to mention the cost of buying all those copies of FoxPro).\n\nBut what we found was that almost anyone with a spare computer and a little knowledge of FoxPro programming language (or a willingness to beg and cajole the wonderful technical support staff at Microsoft) can put together a similar system.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-january-uplink-update","slug":"1995-january-uplink-update","url":"https://dwillis.github.io/uplink/article/1995-january-uplink-update.html","year":1995,"month":"January","monthNum":1,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A journalism newsletter features multiple articles on computer-assisted reporting applications across different beats, including restaurant inspections, environmental reporting, parking violations, and data cleaning techniques.","keywords":["reporting","databases","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":[],"full_text":"We start the new year with articles that show the ways in which computer-assisted reporting can be used on different beats and subjects.\n\nMike Wendland, a TV reporter, recounts the intense and widespread interest that stories on poorly-maintained restaurants can create. (After all, everyone eats.) Seth Hamblin interviews some of the better environmental reporters on how to use electronic information. David Armstrong, a long-time Boston reporter, talks about the treasure trove of stories that a parking ticket databases offers.\n\nIn addition, George Landau, a guru of cleaning dirty data, volunteers yet another clever way of standardizing the bothersome ways in which names appear in databases.\n\nIn next month's issue, we plan to offer more variety by doing an overview of some of the best CAR stories that we saw in 1994.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-january-restaurants-on-car-menu","slug":"1995-january-restaurants-on-car-menu","url":"https://dwillis.github.io/uplink/article/1995-january-restaurants-on-car-menu.html","year":1995,"month":"January","monthNum":1,"headline":"Restaurants on CAR menu","kicker":"","author_name":"Mike Wendland","author_title":"head of the I-Team","affiliation":"WDIV-TV Detroit","summary":"This piece discusses a computer-assisted reporting project that investigated restaurant sanitation in metropolitan Detroit. The project identified hundreds of unsanitary establishments, including some high-profile eateries, through systematic data collection and analysis of inspection scores.","keywords":["restaurant","sanitation","Detroit"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["FoxPro","Profnet"],"full_text":"Viewer response and e-mail are still coming in, weeks after our CAR (computer-assisted reporting) project on metropolitan Detroit's dirtiest restaurants hit the air last fall. From start to finish, the project took about two months, with all but a week or so taken up by data acquisition and analysis. Fleshing out the people part of the stories - finding the food poisoning victims and confronting the restaurant owners - was the easy part. Restaurant inspections are handled by county health departments. In our case, we had three separate counties regulating restaurant sanitation and two municipalities operating their own health departments. That meant we had five agencies to deal with, none of which kept the data we needed in electronic form. What made the project feasible for us, however, was the standard inspection form each health department sanitarian filled out during the inspection. The form called for a rating to be given to each establishment. The rating was determined by subtracting standardized points for each violation from a perfect score of 100. Anything less than 70 was considered unsanitary. So we knew we had a measuring stick - something that could compare a restaurant in one jurisdiction to a restaurant in another. The problem was that there were too many \"others.\" More than 10,000 restaurants are in business in metropolitan Detroit. We wanted to find the worst.\n\nOur approach was to review the most recent inspection reports for each restaurant. We designed our own FoxPro table and took laptops into the offices to enter the data on each eating place that scored 70 or below. While gathering the raw data we went online and sent a query out on Profnet looking for experts in the field of restaurant sanitation. From that research, we learned that a single low score can sometimes be attributed to the inspector simply arriving at the wrong time on the wrong day. The only way to be fair, the experts said, was to look for patterns - consistently low scores over a period of time. We decided to revise the table and include the last three inspections, which covered an 18-month period. Back at the office, we later averaged and sorted our list of \"dirty\" restaurants. From just over 10,000 restaurants, our list of problem, or unsanitary establishments totaled 224. There were plenty of surprises. Top gourmet restaurants, trendy yuppie eateries in the suburbs and a popular downtown Cajun restaurant all made the list of dirty restaurants. So did plenty of Coney islands, Chinese restaurants and some of the well-known chains.\n\nAs we entered the data at the health department, we created a memo field in the FoxPro table, using it to record food-borne illness reports, customer complaints and notable problems like cockroaches floating in water glasses, bugs in food, sewerage backed up on the kitchen floor and other disgusting things you'll be glad we didn't mention if you're reading this at lunch time. That led us to the people stories that fleshed out the statistics.\n\nOur I-team series aired on the 11 p.m. news. We won the time period all four nights. The series was the talk of the town for weeks on the morning drive-time radio shows and the newspaper on-the-town columns.\n\nWe invited viewers to write in for a list. We received some 8,000 letters. We also made the lists available by e-mail and electronically sent out another 1,000. And then there was this postscript: The Detroit area restaurant with the absolutely lowest score - a 40 - was owned by the suburban chef who we did a weekly \"gourmet-on-the-go\" report on our noon news. I ended up interviewing him about his problems after he stepped off the set. \"Go\" he did. He is no longer working for the station.\n\nNote: For additional information on covering restaurants and health department data see \"Dining on eatery data\" by Dan Browning in the March 1994 issue of Uplink.\n\nMike Wendland is the head of the I-Team for WDIV-TV in Detroit. He attended the August 1994 NICAR \"boot camp\" for computer-assisted reporting. He can be reached by e-mail at mikew@wdiv.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-january-coming-events","slug":"1995-january-coming-events","url":"https://dwillis.github.io/uplink/article/1995-january-coming-events.html","year":1995,"month":"January","monthNum":1,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A journalism newsletter listing upcoming NICAR (National Institute for Computer-Assisted Reporting) seminars scheduled for March and May 1995 in Columbia, Missouri.","keywords":["seminar","NICAR","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nMarch 12-17, 1995\nNICAR Seminar\nColumbia, Missouri\n\nMay 14-19, 1995\nNICAR Seminar\nColumbia, Missouri","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-january-lessons-from-parking-database-never-expire-repaying-tickets-in-boston","slug":"1995-january-lessons-from-parking-database-never-expire-repaying-tickets-in-boston","url":"https://dwillis.github.io/uplink/article/1995-january-lessons-from-parking-database-never-expire-repaying-tickets-in-boston.html","year":1995,"month":"January","monthNum":1,"headline":"Lessons from parking database never expire Repaying tickets in Boston","kicker":"","author_name":"David Armstrong","author_title":"Reporter","affiliation":"The Boston Globe","summary":"The Boston Herald obtained a 3.5 million record parking ticket database and discovered the city had collected over $2 million in overpayments from violators due to errors and poor bookkeeping, leading to a public refund effort. The article details how reporters managed the large dataset and the technical challenges of analyzing nearly 1.7 gigabytes of parking violation data.","keywords":["parking","database","overpayments"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["9-track tape","FoxPro","Nine Track Express","PC"],"full_text":"Parking in Boston is often compared to a trip to the dentist: It can be painful and expensive. The city of Boston, not unlike scores of other cities across the country, dispatches meter maids to aggressively prowl the clogged city streets and hand out thousands of parking tickets each day.\n\nSince everybody hates parking tickets, the Boston Herald (where I used to work) decided a year ago to request a copy of the city's parking ticket database. We had a number of stories in mind when requesting the database. We wanted to know what areas of the city were ticketed most frequently, which areas were avoided by ticket writers and why, who were the scofflaws and who was having their tickets dismissed or taken care of.\n\nThe database of 3.5 million records on 269-track tapes proved helpful for those ideas and more.\n\nIn fact, the most surprising information on the database was something we had never considered. The city had collected more than $2 million in overpayments from unwitting parking violators. This information proved to be wildly popular with the motoring public. The newspaper published a list of those companies and individuals owed the most money by the city because of overpayments. The overpayments were the result of errors by the city in recording license plate numbers, overlapping overdue notices and poor bookkeeping by parking violators who paid for the same ticket twice. Embarrassed parking officials were forced to admit they knew about the overpayments for years, but did nothing to return the money. The same officials set up a hotline to refund the money after the story appeared. The mayor ordered that printouts of the 60,000 companies and individuals owed money for overpayments be placed in every library for public inspection. Two months after the first story was published, the city had returned $1 million to parking violators.\n\nThe database provided by the city of Boston actually came from a city vendor - Lockheed Information Services. Lockheed maintains the same records for many large and midsize cities across the country.\n\nThe database has fields indicating the location of the ticket, who wrote it, the amount, the amount of overdue charges, whether or not the ticket was dismissed, the registration number of the ticketed vehicle and more.\n\nA Lesson on obtaining data\n\nThe size of the file - 1.7 gigabytes - presented some problems. The data was analyzed on a Gateway 486 PC with an 800 megabyte hard drive. Using Nine-track Express, I heavily filtered the data to bring in only the information I absolutely needed.\n\nUsing FoxPro, I would look at 300 to 400 megabyte chunks. For instance, when looking at who had tickets dismissed, I brought in only the fields for dismissal, name, amount, location and date. This cut the record length more than 70 percent. The hardest part of this endeavor, it turned out, was getting the tapes in the first place.\n\nSpending three weeks analyzing and playing with the data also drove home another point: Reporters should always do their own computer work, even if it is with the assistance of a more experienced or skilled programmer. Several story ideas were born from the long and often painful task of learning the database and attempting to analyze it.\n\n- Widely hated meter maids, who endure the curses and shouts of angry ticket recipients, are major money makers for the city. We found one meter maid had written 15,210 tickets totalling $516,000 in just one year of work. The same meter maid was paid $433 a week for her work.\n\n- We were able to produce a chart showing readers the 40 locations in Boston that would most likely result in your car getting a ticket.\n\n- Rental car companies were the biggest scofflaws in the city. Two companies owed the city a combined $1 million for parking tickets amassed by rental drivers who never paid up.\n\nIt took three years for the Boston Herald to obtain a copy of the city's parking ticket database on 9-track tapes. The city only complied with the request when it was threatened with legal action by the state attorney general, who is charged with enforcing the state's public records law.\n\nIn repeatedly rejecting the Herald's request, the city maintained it had the right to decide the form of any response to a public records request. In that vein, the city offered to provide the parking records in paper form at great cost to the newspaper. The state's public records law, which is modeled after the federal FOIA, does not provide government agencies the option of responding this way.\n\nUntil recently, the city of Boston uniformly refused to comply with any requests for records in electronic form. In most of these cases, the city would offer a mile-long printout of paper as an alternative. In fact, city workers once delivered boxes of documents to the Herald newsroom after a reporter specifically requested the information on 9-track tape.\n\nIt should be noted that the city's top computer manager also served as the mayor's pollster. That combination of computer knowledge and political savvy may explain the city's reluctance to help the media and others gain access to computerized records.\n\nIn addition to the discovery that tens of thousands of motorists were overpaying the city for tickets, here are some of the other stories that resulted from analyzing the database:\n\n- The police officer assigned to drive former Mayor Raymond Flynn had every parking ticket issued to his personal car, a 1987 Jaguar, dismissed. The tickets totalled $355.\n\nMeanwhile, police officers and local business owners contributed anecdotes and color with their tales of furious motorists and creative scofflaws.\n\nWhether you're a CAR novice or an old pro, city parking ticket records offer a wealth of story opportunities.\n\nRon Brown of the Roanoke Times & World-News investigated his city's \"Car Wars\" in a series published earlier this year. (Call the IRE/NICAR resource center, 314-882-2042 for a copy.) With the help of computerized municipal records, Brown found how much Roanoke is earning from parking tickets, how many it dismisses, and the top 10 blocks for parking violations. Other statistics included the top grossing police officers and a list of the most common types of violations.\n\nBut even better, Brown also found a few surprises on his list of frequent offenders.\n\nAt the top of the list was Roanoke's Chief Assistant Commonwealth Attorney Betty Jo Anthony - someone, Brown wrote, who makes a living enforcing the rules.\n\nAnd before she could be identified, Times & World-News columnist Kathleen Wilson stepped forward with her own admission of excessive fines.\n\n(David Armstrong moved from The Herald to The Globe in 1994. He can be reached at 617-929-2539 or through e-mail at armstr@news.globe.com.)","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-january-a-lesson-on-obtaining-data","slug":"1995-january-a-lesson-on-obtaining-data","url":"https://dwillis.github.io/uplink/article/1995-january-a-lesson-on-obtaining-data.html","year":1995,"month":"January","monthNum":1,"headline":"A Lesson on obtaining data","kicker":"","author_name":"David Armstrong","author_title":"Reporter","affiliation":"Boston Herald","summary":"A journalist describes the technical process of analyzing a 1.7 gigabyte parking ticket database, highlighting the importance of reporters conducting their own computer work and detailing several story discoveries including high-earning meter maids, ticket hotspots, and rental car company violations.","keywords":["database","analysis","journalism"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["9-track tape","FoxPro","Nine Track Express","PC"],"full_text":"A Lesson on obtaining data\n\nThe size of the file - 1.7 gigabytes - presented some problems. The data was analyzed on a Gateway 486 PC with an 800 megabyte harddrive. Using Nine-track Express, I heavily filtered the data to bring in only the information I absolutely needed.\n\nUsing FoxPro, I would look at 300 to 400 megabyte chunks. For instance, when looking at who had tickets dismissed, I brought in only the fields for dismissal, name, amount, location and date. This cut the record length more than 70 percent. The hardest part of this endeavor, it turned out, was getting the tapes in the first place (see sidebar).\n\nSpending three weeks analyzing and playing with the data also drove home another point: Reporters should always do their own computer work, even if it is with the assistance of a more experienced or skilled programmer. Several story ideas were born from the long and often painful task of learning the database and attempting to analyze it.\n\n- Widely hated meter maids, who endure the curses and shouts of angry ticket recipients, are major money makers for the city. We found one meter maid had written 15,210 tickets totalling $516,000 in just one year of work. The same meter maid was paid $433 a week for her work.\n\n- We were able to produce a chart showing readers the 40 locations in Boston that would most likely result in your car getting a ticket.\n\n- Rental car companies were the biggest scofflaws in the city. Two companies owed the city a combined $1 million for parking tickets amassed by rental drivers who never paid up.\n\nIt took three years for the Boston Herald to obtain a copy of the city's parking ticket database on 9-track tapes.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-january-parking-tickets-offer-wealth-of-stories","slug":"1995-january-parking-tickets-offer-wealth-of-stories","url":"https://dwillis.github.io/uplink/article/1995-january-parking-tickets-offer-wealth-of-stories.html","year":1995,"month":"January","monthNum":1,"headline":"Parking tickets offer wealth of stories","kicker":"","author_name":"Gwen Carleton","author_title":"Staff","affiliation":"NICAR","summary":"A Roanoke Times investigation of city parking ticket records reveals surprising patterns, including numerous dismissed tickets for the mayor's driver and excessive violations by the Chief Assistant Commonwealth Attorney.","keywords":["parking","violations","investigation"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":[],"full_text":"Whether you're a CAR novice or an old pro, city parking ticket records offer a wealth of story opportunities.\n\nRon Brown of the Roanoke Times & World-News investigated his city's \"Car Wars\" in a series published earlier this year. (Call the IRE/NICAR resource center, 314-882-2042 for a copy.) With the help of computerized municipal records, Brown found how much Roanoke is earning from parking tickets, how many it dismisses, and the top 10 blocks for parking violations. Other statistics included the top grossing police officers and a list of the most common types of violations.\n\nBut even better, Brown also found a few surprises on his list of frequent offenders.\n\nThe police officer assigned to drive former Mayor Raymond Flynn had every parking ticket issued to his personal car, a 1987 Jaguar, dismissed. The tickets totalled $355.\n\nAt the top of the list was Roanoke's Chief Assistant Commonwealth Attorney Betty Jo Anthony - someone, Brown wrote, who makes a living enforcing the rules.\n\nAnd before she could be identified, Times & World-News columnist Kathleen Wilson stepped forward with her own admission of excessive fines.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-january-successes-with-epa-databases","slug":"1995-january-successes-with-epa-databases","url":"https://dwillis.github.io/uplink/article/1995-january-successes-with-epa-databases.html","year":1995,"month":"January","monthNum":1,"headline":"Successes with EPA databases","kicker":"Environment & CAR","author_name":"Seth Hamblin","author_title":"NICAR Staff","affiliation":"NICAR","summary":"Environmental reporters can leverage government databases for computer-assisted reporting stories, with examples including investigations into toxic chemical spills and endangered species funding allocation.","keywords":["environment","database","reporting"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["XDB","Paradox"],"full_text":"With government data available on everything from endangered birds to the quality of the air through which they fly, there's no limit to the enterprise stories an environmental reporter can craft. Some reporters say that because of the amount, variety and accessibility of these databases, the environment is better suited for computer-assisted reporting than any other beat. Their consistent success in using environmental databases shows they're probably right.\n\nToxic waste spills\n\nDave Davis asked to cover the environment when he joined the Plain Dealer in 1990 because he knew the beat would give him plenty of chances to do CAR stories. One of his first CAR stories for the Plain Dealer, \"Spills Chill Regulators,\" took about two weeks. Using XDB database manager software, Davis analyzed the 1991 Ohio emergency response database, which comprises 5,589 records on chemical spills. (NICAR has the U.S. emergency response database, ERNS, in its library.)\n\nThe database combines numerical fields with variable-length text fields. This mix helped Davis to flesh out statistics with colorful narratives.\n\n\"This was a perfect database for us because Ohio has a lot of chemical emergencies. A lot of plants blow up,\" Davis said.\n\nFor instance, one record explained how a railroad switchman lost communication with an engineer when batteries failed in their two-way radios just as the train was switching tracks. The engineer backed the car into a storage tank, which spilled 800 gallons of a toxic chemical. The chemical ended up being flushed into the city's sewers.\n\nDavis discovered that 121 of the chemical spills in Ohio had been severe enough to threaten lives and property. Rarely did the companies report the spills to the Ohio Environmental Protection Agency, as is required by law. Instead, mostly police or fire officials called in the accidents. Sometimes residents blew the whistle.\n\nEndangered species\n\nFresno (Calif.) Bee reporter Russell Clemings recommended that when reporters \"cast their net\" for environmental stories, they should cast it wide enough to include computer data. When Clemings was researching \"Struggle to Survive,\" a series on endangered species in California, he heard the 1987 amendment to the Endangered Species Act required the federal Fish and Wildlife Service to compile a list of how much money was being spent to preserve each endangered species. He asked for a copy of the report.\n\n\"I realized that what I was looking at was a computer printout,\" Clemings said. \"So I asked, 'Can you send me the disk, too?'\"\n\nUsing a 286 computer and a DOS version of Paradox, Clemings grouped federal and state spending on each class of endangered species. He found that most funding went to birds or mammals that are attractive and well-known, such as bald eagles and gray wolves. Preservation efforts discriminated against bugs, crustaceans and snails. Officials were particularly short-shrifting plants. They spent an average of 66 times more per species on birds than plants - an average of $604,260 and $9,050, respectively. Spiders had it the worst of all. Officials spent an average of $500 per endangered spider species.\n\nClemings said this database could be adapted for any state or region. Reporters could find out from their states' wildlife departments which species on the list inhabit their areas. The reporters then could focus on the species within the reporters' states or regions.\n\nWater pollution\n\nBefore joining the Plain Dealer, Davis was a reporter at the Dayton (Ohio) Daily News, whose readers live near the banks of the Great Miami River and its tributaries. Davis acquired a U.S. EPA database on water pollution discharge permits to find out how much sludge was being pumped into the waterways.\n\nNot only was it a whole lot of sludge, it was a whole lot more sludge than the factories could legally dump under their National Pollution Discharge Elimination System permits.\n\nThe records, compiled in the Permit Compliance System database, showed that state environmental regulators knew of the facilities' chronic violations but did little to punish the businesses. Some of the violators were dumping up to several thousand times the amounts that their permits allowed. Many companies racked up hundreds of violations without penalty.\n\nTo see if the computer data were reflected in the quality of the river's water, Davis canoed 110 miles of the Great Ohio. What he saw and smelled confirmed what the numbers were telling him.\n\n\"We were going by one factory, and this shocking pink water was coming out of the pipes, turning the river pink,\" he said. The wastewater from the factory near Hamilton, Ohio, altered the color of the river for a mile. Further downriver near Franklin, Ohio, a mix of black incinerator ash and detergent ran down a trough into the water to turn it into a sickly green.\n\nDatabases in dailies\n\nScott Thurm, a reporter for the San Jose (Calif.) Mercury News, says environmental databases can be as handy for dailies as they are for enterprise stories. During the early morning on July 25, 1993, General Chemical Corp. in nearby Richmond, Calif., developed a leak in a tank containing sulfur dioxide. More than 2,000 people sought treatment at hospitals for stinging eyes and sore throats.\n\nThurm called the California EPA to get data on the types of toxics General Chemical produced. By 11 a.m., Thurm had the entire Toxic Release Inventory profile of the company to provide background for the story.\n\nTo prepare for chemical accidents such as this one, Adam Berliant, a database editor for the Tacoma (Wash.) News Tribune, set up the Toxic Release Inventory on a computer in the newsroom. Any reporter can type in the name of a company and come up with the kinds of chemicals it releases and the manufacturing process in which it is involved. Unfortunately most reporters at the News Tribune fail to see its potential, and the database setup, while easy to operate, is rarely used, Berliant said.\n\nThurm suggested looking for state databases rather than federal ones whenever possible. TRI and ERNS, for instance, are first compiled by states and then sent to the U.S. EPA. Therefore, state databases are usually more up-to-date than their federal counterparts. Another good reason to look on the state level, Clemings said, is that there are lots of obscure but useful state environmental databases waiting for reporters to discover them. His rule of thumb: If you get a document that looks like a computer table, ask for it on disk.\n\nEPA sources\n\nFederal EPA databases can provide environmental reporters with countless enterprise stories. However, reporters who have used these databases say state and local databases often are easier to get and have more recent data than their federal counterparts.\n\nHere are some of the EPA databases environmental reporters have found useful. They can be acquired with a Freedom of Information Act request. If the cost is too high, call NICAR, 314-882-0684, and we might be able to offer it at a lesser price.\n\nEmergency Response Notification System: ERNS contains notifications of oil and hazardous chemical spills. Fields in the database include name of discharger, date of release; material released; cause of release; amount released; source of release; damages, injuries or deaths; and incident location. Sponsoring department: Emergency Response Division. Contact Dana Stalcup. Telephone: (703) 603-8735. Information line: (202) 260-2342.\n\nPermit Compliance System: PCS contains information on more than 65,000 industrial facilities that hold National Pollution Discharge Elimination System permits, which allow the facilities to discharge wastewater. Fields include facility name, permit limit, violation and enforcement actions. Sponsoring department: Office of Water. Contact: Mike Mundell. Telephone: (202) 564-5031. Information line: (703) 908-2680.\n\nAerometric Information Retrieval System: AIRS contains records on emissions and compliance data for industrial plants as well as ambient air quality data for geographical areas. Sponsoring department: Office of Air and Radiation. Contact: Andrea Kelsey. Telephone: (919) 541-5549.\n\n\"Access EPA,\" the source of this list, can be found at the gopher site: gopher.epa.gov.\n\nHow to deal with EPA data\n\nWith just an old XT, shareware and a free database, you can create great environmental stories for your readership area. The EPA now gives out state Toxic Release Inventory databases as if they were candy. To get a state's TRI database mailed to you, call TRI user support at (202) 260-1531. You will receive it within a couple of weeks. The data comes on 1/2-inch disks, so if you don't have this type of drive, swap the data onto 3 1/2-inch disks on another computer. Data jockeys at the EPA set up state TRI for easy import into almost any database program. If you don't already have a database program, you can get a shareware database, such as PC-File, from a computer-users group, BBS or university computing service. Once the data is imported, separate the records by zip code and add the amount of chemicals released by all the companies within each zip code. This will show you which areas in your county or region have reported the most industrial pollution. You can then look at the companies which fall within the zip code areas with the highest amount of pollution to see which companies are emitting the most and worst pollutants. Present this information in a datamap or create a list ranking the polluted areas and the biggest polluters. By calculating totals for the releases to air, water, on-site landfill, underground well injection and off-site hauling fields, you can show, in percentages, how industries in your area are disposing their waste. Each facility will have a field for its standard industrial code, which tells what sort of manufacturing they do, from making clothes to smelting metals. By totaling the amount of releases for each standard industrial code, you can rank the types of industries that pollute the most. Your reporting doesn't end with the data analysis. With the address of each facility included in the database, it is easy to find the residents nearest the factories. The companies also file the name and telephone number of a contact person with whom you can confirm the data and ask whether attempts at pollution reduction are being made.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-january-epa-sources","slug":"1995-january-epa-sources","url":"https://dwillis.github.io/uplink/article/1995-january-epa-sources.html","year":1995,"month":"January","monthNum":1,"headline":"EPA sources","kicker":"","author_name":"","author_title":"Staff","affiliation":"NICAR","summary":"Environmental reporters can leverage several EPA databases including ERNS, PCS, and AIRS to find enterprise stories about pollution, spills, and industrial compliance, though state and local databases often contain more recent and accessible data.","keywords":["EPA","databases","pollution"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["gopher"],"full_text":"Federal EPA databases can provide environmental reporters with countless enterprise stories. However, reporters who have used these databases say state and local databases often are easier to get and have more recent data than their federal counterparts.\n\nHere are some of the EPA databases environmental reporters have found useful. They can be acquired with a Freedom of Information Act request. If the cost is too high, call NICAR, 314-882-0684, and we might be able to offer it at a lesser price.\n\nEmergency Response Notification System\n\nERNS contains notifications of oil and hazardous chemical spills. Fields in the database include name of discharger, date of release; material released; cause of release; amount released; source of release; damages, injuries or deaths; and incident location. Sponsoring department: Emergency Response Division Contact Dana Stalcup Telephone: (703) 603-8735 Information line: (202) 260-2342\n\nPermit Compliance System\n\nPCS contains information on more than 65,000 industrial facilities that hold National Pollution Discharge Elimination System permits, which allow the facilities to discharge wastewater. Fields include facility name, permit limit, violation and enforcement actions. Sponsoring department: Office of Water Contact: Mike Mundell Telephone: (202) 564-5031 Information line: (703) 908-2680\n\nAerometric Information Retrieval System\n\nAIRS contains records on emissions and compliance data for industrial plants as well as ambient air quality data for geographical areas. Sponsoring department: Office of Air and Radiation Contact: Andrea Kelsey Telephone: (919) 541-5549\n\n\"Access EPA,\" the source of this list, can be found at the gopher site: gopher.epa.gov.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-january-how-to-deal-with-epa-data","slug":"1995-january-how-to-deal-with-epa-data","url":"https://dwillis.github.io/uplink/article/1995-january-how-to-deal-with-epa-data.html","year":1995,"month":"January","monthNum":1,"headline":"How to deal with EPA data","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Journalists can obtain EPA Toxic Release Inventory databases to analyze industrial pollution patterns in their regions by zip code and facility type using basic database software. The data can be used to create data maps, identify major polluters, track waste disposal methods, and contact facility representatives for verification and follow-up reporting.","keywords":["EPA","pollution","database"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["PC-File"],"full_text":"How to deal with EPA data\n\nWith just an old XT, shareware and a free database, you can create great environmental stories for your readership area. The EPA now gives out state Toxic Release Inventory databases as if they were candy. To get a state's TRI database mailed to you, call TRI user support at (202) 260-1531. You will receive it within a couple of weeks. The data comes on 1/2-inch disks, so if you don't have this type of drive, swap the data onto 3 1/2-inch disks on another computer. Data jockeys at the EPA set up state TRI for easy import into almost any database program. If you don't already have a database program, you can to get a shareware database, such as PC-File, from a computer-users group, BBS or university computing service. Once the data is imported, separate the records by zip code and add the amount of chemicals released by all the companies within each zip code. This will show you which areas in your county or region have reported the most industrial pollution. You can then look at the companies which fall within the zip code areas with the highest amount of pollution to see which companies are emitting the most and worst pollutants. Present this information in a datamap or create a list ranking the polluted areas and the biggest polluters works. By calculating totals for the releases to air, water, on-site landfill, underground well injection and off-site hauling fields, you can show, in percentages, how industries in your area are disposing their waste. Each facility will have a field for its standard industrial code, which tells what sort of manufacturing they do, from making clothes to smelting metals. By totaling the amount of releases for each standard industrial code, you can rank the types of industries that pollute the most. Your reporting doesn't end with the data analysis. With the address of each facility included in the database, it easy to find the residents nearest the factories. The companies also file the name and telephone number of a contact person with whom you can confirm the data and ask whether attempts at pollution reduction are being made. (Some of the concepts for this sidebar were derived from Dave Davis' story \"Industries Report Decreases,\" The Plain Dealer, December 11, 1991.)","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-january-editing-data-for-consistency","slug":"1995-january-editing-data-for-consistency","url":"https://dwillis.github.io/uplink/article/1995-january-editing-data-for-consistency.html","year":1995,"month":"January","monthNum":1,"headline":"Editing data for consistency","kicker":"Tech Tips","author_name":"George Landau","author_title":"Data editor","affiliation":"St. Louis Post-Dispatch","summary":"A guide to efficiently cleaning and standardizing inconsistent data entries in databases using FoxPro queries and keyboard macros, using employee name variations as an example.","keywords":["database","data","cleaning"],"topics":["Data & Databases","Investigative Methods"],"technologies":["FoxPro","SQL"],"full_text":"A lot of data analysis involves grouping and sorting - summing the expense reimbursements to a bunch of employees when you have the employees' names and addresses but not Social Security numbers, for example. Unless you're sure that Joseph Schmo's name is entered the same way for each reimbursement, and that his address is consistently spelled, you won't be able to get reliable totals.\n\nOne dependable but very painful way to edit this stuff is to list the various values in a field - names, for example - and manually do search-and-replace on the database until everything's consistent. In FoxPro or another SQL database, you could start like this:\n\nselect fullname, count(*) from expenses group by 1; order by 1\n\nThe query would return an alphabetized list of every way that a name is spelled in the entire database, along with the number of times each variation occurs. For example:\n\nFULLNAME / CNT\nLandau, George / 7\nLandau, George T / 3\nLandau, George T. / 3\nLandau, George Thomas / 2\n\nLet's say you're certain these are all the same George Landau (remember, we can check addresses, also). You want to standardize on the variation with the most info: \"George Thomas Landau.\" To edit these the hard way, you could start typing commands like this:\n\nreplace all fullname with \"Landau, George Thomas\" for fullname = \"Landau, George\"\nreplace all fullname with \"Landau, George Thomas\" for fullname = \"Landau, George T\"\nreplace all fullname with \"Landau, George Thomas\" for fullname = \"Landau, George T.\"\n\nFine, but there are 10,000 records in this database. You might not be done for another month. The absolute fastest way out of this mess involves some moderately complex FoxPro programming; if you really want to know, give me a call and I'll tell you how to do it. But without any fancy programming - with only two keyboard macros, really - you can get the job done with a minimum of pain.\n\nHere's how:\n\n1. For reasons that will become clear as you read on, we need to get a list of every unique name into a FoxPro program file (*.prg), which is just a text file that contains a list of commands that will be executed in bulk, as if you had typed them all at once in FoxPro's command box. A query like this will do the job:\n\nselect fullname from expenses group by 1; order by 1 [this line isn't actually necessary, since the 'group by' sorts alpha-ascending by default]\nto file CLEAN1.PRG nowait\n\nThe last line of this query puts the results into the specified text file, as ordinary ASCII text. The 'nowait' clause prevents the process from halting at every screenful and forcing you to press a key to continue.\n\n2. Now let's edit the \"program\" you just created by typing in the Command box:\nmodify command clean1\n\n3. Delete the first few lines, down to and including the line that contains the field name heading. You'll also notice that each line starts with two spaces of padding, and ends with a lot of padding. We want to get rid of those extra spaces, which we can do by using the Find function off the Edit menu.\n\nFirst, get rid of all the leading spaces by searching for \\r\\n (these blanks represent two spaces) and replacing with \\r\\n (\\r\\n is how you specify Carriage Return-Line Feed in FoxPro's Find dialog box). Then get rid of the trailing spaces by replacing \\r\\n with \\r\\n a few times; each time you get no matches, specify fewer trailing spaces until all are gone.\n\nThis search-and-replace step can be tricky, so don't panic if you don't get it right the first time.\n\n4. Now we've got a list of every name variation in the database, free of unwanted spaces. Our goal is to record two keyboard macros that will generate a REPLACE command for the names we want to change. Let's say you're looking at these entries again:\n\nLandau, George\nLandau, George T\nLandau, George T.\nLandau, George Thomas\n\nIf you wanted to make the last one (Landau, George Thomas) the standard entry, click the cursor anywhere on that line. Here's where we'll record our first macro. First press shift-F10 to start the macro recorder. Specify a keystroke to assign the macro (I like Ctrl-Z, 'cause I don't use it for anything else). Then click OK. Every key you press from now until you press shift-F10 again will be recorded - mouse movements and clicks will be ignored. Press these keys (things in curly braces represent special keys; don't actually type the braces or the word, just press the key or key-combo):\n\n[Home] {Shift+End} {Ctrl+C}\n\nNow click shift-F10 again and click OK to stop recording the macro. Now whenever you press Ctrl-Z (or whatever keystroke you specified for the macro), FoxPro will define whatever line the cursor's on and copy all of it to the clipboard.\n\nNow click the cursor on the first line you want to change, \"Landau, George.\" Here's where we'll record our second macro. First press shift-F10 to start the macro recorder. Specify a keystroke to assign the macro (try something physically close to Ctrl-Z or whatever you chose, like Ctrl-A). Then click OK. Once again, every key you press from now until you press shift-F10 again will be recorded - mouse movements and clicks will be ignored. Type these keys and characters (again, things in curly braces are special keys; don't actually type the braces or the word, just press the key or key-combo):\n\n{Home}replace all fullname with \"{Ctrl+V}\" for fullname = \"{End}\"\n\nThen press Shift-F10 again and click OK to stop recording the macro. Now whenever you press Ctrl-A (or whatever keystroke you specified for the second macro), FoxPro will create a command to replace that line's contents with whatever's in the clipboard.\n\n5. Before you get down to business, you should save the macros so that you won't have to re-record them next time you run FoxPro. Pull down the Program menu and choose Macros. You can either save these as part of the default macros, which are loaded whenever FoxPro is run, or under a new name that you can restore manually when you need to.\n\n6. Now for the truly thrilling part. To begin standardizing all those spellings, just position the cursor on an entry that you want to become the standard and click Ctrl-Z (the first macro you recorded). This copies the line to the clipboard. Then click on one of the lines you want changed to the standard, and click Ctrl-A (or the key for the second macro) to create a \"replace\" command. Do this until you've made it all the way through the file. Then go through and delete any entries that weren't made into a \"replace\" command (Hint: if you're comfortable going back and forth between text files and tables, as I described in the earlier part of this handout dealing with print images, you can append this program into a table and delete all the records that don't start with \"replace\", then copy it back to a text file).\n\n7. Before you go any further, make a backup copy of the table you're intending to edit.\n\n8. Then open up either the backup or the original and do the program you just created. If you called the file \"clean1.prg,\" you just need to type \"do clean1\" in the Command box. This program may take a while to run if the database is large, because each \"replace\" command forces FoxPro to scan through all of the data. At least you're free to do something else, though.\n\n9. When it's done, your data will be standardized. You can now do accurate grouping, summing and sorting.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-january-database-library-keeps-growing","slug":"1995-january-database-library-keeps-growing","url":"https://dwillis.github.io/uplink/article/1995-january-database-library-keeps-growing.html","year":1995,"month":"January","monthNum":1,"headline":"Database library keeps growing","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers journalists access to multiple government databases including FAA incident reports, FBI crime data, Social Security death records, federal contracts, mortgage disclosure records, OSHA inspections, and gun dealer information for investigative reporting.","keywords":["databases","journalism","investigation"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"NICAR keeps adding to its database library. A partial list includes:\n\nFederal Aviation Administration service difficulty reports that show the history of problems that a plane had before it crashed.\nFBI crime data that allows you to do detailed analysis of crime in your community and state.\nSocial security death records that you can use for stories that go beyond simple listing. The database also permits reporters to circumvent states that have closed their death records.\nFederal procurement contracts. This database can help you discover the impact of cuts in defense spending, or the amount of housing and office space federal agencies rent in your city, or the amount of toilet paper the feds purchased in your community, or the biggest federal contractors in your state.\nHome Mortgage Disclosure Act records that you use to look at disparities in minority lending.\nOSHA inspection safety records that show the most frequently cited employer in your state and what the work safety violations have been.\nFederal gun dealer records that show who is dealing in weapons or that a local police officer moonlights as a provider of guns to criminals.\n\nTo request data, call NICAR at 314-882-0684 or E-Mail us at nicar@muccmail.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-january-online-environmental-info-in-canada","slug":"1995-january-online-environmental-info-in-canada","url":"https://dwillis.github.io/uplink/article/1995-january-online-environmental-info-in-canada.html","year":1995,"month":"January","monthNum":1,"headline":"Online environmental info in Canada","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"","summary":"Environment Canada's Green Lane is a comprehensive online resource offering regional and national environmental information, including weather forecasts, publications, program details, expert contacts, and environmental legislation for Atlantic Canadians.","keywords":["environment","information","government"],"topics":["Health & Environment","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Environment Canada's Green Lane (http://www.ns.doe.ca/how.html) provides a wide variety of regional and national environmental information, including:\n- the latest regional weather forecasts, satellite photos, maps & radar images\n- the complete texts of many of its most popular national publications;\n- details about many regional and national initiatives and programs;\n- answers to some of the questions that Atlantic Canadians most frequently ask when they phone or write to the Department;\n- direct phone numbers of the Department's regional experts on numerous issues and initiatives;\n- details and documents regarding regional public consultation meetings;\n- news about regional public environmental events;\n- descriptions of the department's funding programs, including a guide on how to prepare funding proposals;\n- a list of the department's key publications available to order;\n- copies of its recent regional press releases and announcements;\n- the complete text of its environmental legislation, regulations, guidelines, codes of good practice, and related policies;\n- details about its key regional federal/provincial agreements; and\n- descriptions of its mission, roles, responsibilities and organizational structure.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-january-overview-of-1994-car-stories","slug":"1995-january-overview-of-1994-car-stories","url":"https://dwillis.github.io/uplink/article/1995-january-overview-of-1994-car-stories.html","year":1995,"month":"January","monthNum":1,"headline":"Overview of 1994 CAR stories","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"Uplink","summary":"Uplink highlights recent investigative journalism stories from 1994 that used public records and databases to uncover issues ranging from suicide and elevator safety to parking tickets, crime statistics, divorce delays, and unnecessary medical procedures.","keywords":["investigation","records","journalism"],"topics":["Investigative Methods","Data & Databases"],"technologies":["Uniform Crime Reports"],"full_text":"Uplink will do an overview of 1994's CAR stories in the February issue, but here's a sample of articles we've recently received:\n\n- A three-part series on suicide by (Tacoma, WA) News Tribune, using Washington state death records.\n- A three-part series on elevator safety in Massachusetts by The Boston Globe, using elevator inspection records.\n- An investigation by New York Newsday into parking tickets that found the city owed $18 million to more than 600,000 motorists, using a parking ticket database. (See parking ticket story on page two.)\n- An eight-part series on crime and punishment by The Miami Herald, using FBI Uniform Crime Report data, criminal justice databases, and 87 reports from the U.S. Bureau of Justice Statistics.\n- A four-part series, \"Starving the Wife,\" by The Charlotte Observer on how divorcing spouses take advantage of flawed state laws and poorly monitored courts. The Observer used court records to document long delays in cases.\n- An investigation by the The Plain Dealer in Cleveland, OH, into thousands of unnecessary C-sections, using Ohio birth records.\n\nCopies of these articles can be obtained through NICAR by calling 314-882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-february-uplink-update","slug":"1995-february-uplink-update","url":"https://dwillis.github.io/uplink/article/1995-february-uplink-update.html","year":1995,"month":"February","monthNum":2,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"Uplink","summary":"This month's Uplink newsletter features several examples of computer-assisted reporting (CAR) in journalism, including case studies from newspapers in Delaware, St. Louis, Cleveland, and Seattle that demonstrate how CAR is becoming essential for uncovering significant stories.","keywords":["reporting","database","journalism"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"This month's Uplink contains offerings from throughout the country. Merritt Wallick of the Wilmington News Journal in Delaware shows how a medium-size paper can tear up the state using computer-assisted reporting on property tax records.\n\nJim Mosley of the St. Louis Post-Dispatch talks about how his newspaper is conducting one the most ambitious CAR training programs in the country, using both inhouse and NICAR resources.\n\nAndrew Lehren, a new researcher at NICAR (fresh from the Philadelphia Business Journal), reviews the uses of a Federal Aviation Administration database. In one case, The Cleveland Plain Dealer, gets a hit by the significant omissions in the database, while in another, The Seattle Times, scores a nice story using details from the database.\n\nAltogether, it shows the continuing emergence of CAR in journalism as a sure path to good stories.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-february-coming-events","slug":"1995-february-coming-events","url":"https://dwillis.github.io/uplink/article/1995-february-coming-events.html","year":1995,"month":"February","monthNum":2,"headline":"Coming Events","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article announces upcoming NICAR seminar dates scheduled for March and May 1995, both held in Columbia, Missouri.","keywords":["NICAR","seminar","Missouri"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Coming Events\n\nMarch 12-17, 1995 NICAR Seminar Columbia, Missouri May 14-19, 1995 NICAR Seminar Columbia, Missouri","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-february-property-tax-records-disclose-wealth-of-stories","slug":"1995-february-property-tax-records-disclose-wealth-of-stories","url":"https://dwillis.github.io/uplink/article/1995-february-property-tax-records-disclose-wealth-of-stories.html","year":1995,"month":"February","monthNum":2,"headline":"Property tax records disclose wealth of stories","kicker":"Looking for haystacks in data","author_name":"Merritt Wallick","author_title":"Wilmington (Del.) News Journal","affiliation":"Wilmington News Journal","summary":"Investigation into Delaware's farmland preservation program reveals misuse by wealthy landowners and companies to avoid property taxes.","keywords":["property tax","Delaware","investigation"],"topics":["Data & Databases","Investigative Methods","Agriculture & Land"],"technologies":["9-track tape"],"full_text":"It was one of those times when the computer print-out makes you feel like a prospector staring at a gold nugget in your pan. This was supposed to be a list of farmers participating in a property tax abatement program intended to preserve farmland. Yet the list included the names of some of the richest people and the largest companies in the United States. It included the names of land developers who have played a major role in Delaware transforming farmland into housing tracts. Several sharpened queries and dozens of field trips to eyeball the 'farms' later we had the basis of our story. Only the interviews remained. Delaware's farmland preservation program had become a means for the owners of big country estates, oil and chemical companies and land developers to avoid paying property taxes. The state law governing the program was too weak to enforce and county officials had stopped trying. The farmland preservation program actually had the effect of helping to finance land development. In one case, a 400-acre parcel of land zoned for a regional shopping mall avoided $86,000 in property taxes by claiming to be a farm. Nothing save scrub pine trees grew there. The story drew one of the greatest responses from the public and the government of any we have ever published. Within days, governments at every level had appointed task forces to study the abuse. Legislators and county councilmen were deluged with calls from voters. Reform of the system is likely in the current session of the General Assembly. All this from a routine database. For general reference purposes, we had acquired property records on 9-track tape from each of Delaware's three counties. We also planned to do a story on whether property tax burdens are unfair. You answer that question straightforwardly, using a sample of recent estate sales to determine actual housing market values and then computing the percentage of market value the assessment of each home comprises. With out-of-date assessments like those in Delaware, you can expect that the most expensive homes will be taxed on the smallest percentage of their actual value. It's a predictable but worthy story. In short, we set out to illustrate something we believed we already knew. Nothing wrong in that, unless we had cruised past the best story. Because we were interested mainly in the assessment of residential properties, we nearly neglected to run a list of...","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-february-sdrs-fact-or-fiction","slug":"1995-february-sdrs-fact-or-fiction","url":"https://dwillis.github.io/uplink/article/1995-february-sdrs-fact-or-fiction.html","year":1995,"month":"February","monthNum":2,"headline":"SDRs: Fact or Fiction","kicker":"More problems uncovered with FAA data","author_name":"Andrew Lehren","author_title":"NICAR Staff","affiliation":"NICAR","summary":"The Cleveland Plain Dealer exposes issues with FAA's Service Difficulty Reports, including missing and inconsistent data regarding airplane maintenance.","keywords":["FAA","data analysis","aviation safety"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["FoxPro"],"full_text":"When a plane crashes - like the USAir jet outside Pittsburgh - journalists are quick to reveal the aircraft maintenance records maintained by the Federal Aviation Administration. But a huge piece of the puzzle is the FAA's own sloppy record keeping, and the way airlines compile information. The reporters went back to the FAA, back to airlines, back to overseers, mechanics, pilots, manufacturers and experts. What they found were problems everywhere. Perhaps there is no better way to dramatize the gaps than this: In July 1992, a TWA L-1011 was taking off at New York's JFK International Airport. The pilot said he saw a warning system telling him that the engines were stalling. He aborted the takeoff, veered off the runway and the plane erupted into flames. Fifty-one passengers suffered minor injuries. Investigators started checking on L-1011 warning systems. They found only two FAA reports of stall-warning system failures. But then they went further. Lockheed, the maker of the jet, had its own files. Those showed an additional 14 instances where warning systems went wrong. Investigators then looked at TWA. The airline had records of another 10 malfunctions. In other words, the FAA was missing records for 24 out of 26 incidents where warning systems went wrong a problem that appears to have contributed to a high-profile airplane accident. Cleveland's Plain Dealer has been sifting through FAA records since November and published its first investigation last month. The results show the TWA accident does not represent the exception. It embodies the way of life at an agency that's supposed to regulate airline safety. The newspaper has been exploring the FAA's service difficulty reports, often referred to by their initials, SDR. The records are not supposed to be some kind of bureaucratic wasteland. They're designed to play a vital role - helping the FAA troubleshoot airplanes by looking for trends. But bad data and a lack of staff mean the FAA can't do the job. The funny thing is, that isn't the story Plain Dealer set out to do. The newspaper sent two reporters - Elizabeth Marchak and Keith Epstein - to find which airlines are the safest and least safe. They were also told to do the same for types of aircraft. And, to drive the point home, they were told to concentrate on aircraft flying in and out of Cleveland's Hopkins International Airport. Marchak and Epstein used the FAA's SDR database as a starting point. But it was just that: a starting point. FoxPro proved a powerful tool. The reporters conducted a variety of searches, categorizing and ranking records by aircraft and maintenance problem. Here are some of the stumbling blocks, and how Marchak and Epstein worked to get things right. The SDR database tracks planes with tail numbers and serial numbers. But airlines over the years may repaint identification numbers. That rendered tail numbers almost meaningless as a way to rummage through the SDR database. Airlines also don't consistently report airplane serial numbers. In addition, the airlines are supposed to rank the seriousness of the maintenance problem on a scale of 1 to 5. However, the Plain Dealer found those rankings are often haphazard. Marchak spotted the shifting serial numbers while performing data analysis. Sometimes they were three digits. In later years, they were six digits. Marchak went into the data line by line to get serial numbers to match. Despite the problems encountered, serial numbers are still the easiest way to follow an aircraft's history. At the same time, the Plain Dealer asked those airlines flying in and out of Cleveland to send lists of the aircraft they were using, including tail numbers. And the newspaper sent a reporter to spend a day at Hopkins, jotting down all the tail numbers. Marchak later checked those numbers against the SDR databases. And a lot of numbers were not matching up. The problem wasn't obvious at first. The gap became apparent not by checking serial numbers, but the dates in the SDR database for when the planes underwent repairs. That's when Marchak noticed a disturbing mismatch: a lot of jets flying out of Hopkins showed no maintenance updates since 1992. The reporters went back to the FAA, back to airlines, back to overseers, mechanics, pilots, manufacturers and experts. What they found were problems everywhere. In some instances, the airlines were submitting the data to the FAA - sometimes doing the job electronically - yet the information couldn't always be found in SDR. Many times, the airlines were not reporting maintenance as many as nine of 10 repairs went unreported The problem was particularly acute among commuter airlines. And the FAA, left with weak provisions to enforce the rules, can do little to force airlines to change their ways. In addition, records were sloppy- Repairs were often mislabeled. The judgment was left to airline mechanics, who sometimes softpeddle the seriousness of a repair. Marchak and Epstein also looked for what wasn't tracked at all in the database, and found key numbers were missing - the number of hours flown, and number of takeoffs. The stuff that would better track the repairs versus wear and tear. Along with poor documentation and no enforcement, the Plain Dealer used the database as a springboard to show how the FAA poorly analyzes its own records. Agency officials admitted that, in the wake of budget cutbacks, spotting dangerous trends with SDR doesn't get done. The kicker: FAA employees often don't use the agency's database. A private consultant is among those who buys SDR data. He repackages the information and resells the data. Among his biggest customers is - that's right - the FAA. Agency workers find its easier to sift through his version, instead of the FAA's own compilation. For journalists using FAA data, the Plain Dealer's analysis offers another lesson. Learning an aircraft's maintenance record sometimes reveals less than meets the eye. The Plain Dealer demonstrated that by profiling a MidWest Express aircraft with a raft of repairs. The newspaper compared that with the USAir jet that crashed outside Pittsburgh. The MidWest Express passengers were on an airplane that underwent more than 144 repairs, but there appears nothing that should have raised concerns. The same went for the USAir flight's 37 repairs. 'Trivial' was the way one expert described the maintenance record. In the article Marchak and Epstein wrote: 'The two planes' histories illustrate the perils - for regulators, inspectors, reporters and the public - of drawing conclusions about relative safety from information in federal government computers. Even though that's why the computerized information exists in the first place.' For more information, the reporters can be reached by phone at 202-638-1366. To send e-mail, reach Epstein at 70404.247@compuserve.com and Marchak at Marchak@digex.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-february-sdr-reports-still-useful","slug":"1995-february-sdr-reports-still-useful","url":"https://dwillis.github.io/uplink/article/1995-february-sdr-reports-still-useful.html","year":1995,"month":"February","monthNum":2,"headline":"SDR reports still useful","kicker":"Seattle Times overcomes problems with FAA database","author_name":"Andrew Lehren","author_title":"Staff","affiliation":"NICAR","summary":"The Seattle Times analyzed two decades of FAA service difficulty reports to document a pattern of rudder malfunctions in Boeing 737s, finding connections to unsolved crashes and prompting regulatory action. The investigation revealed that rudder problems may have contributed to multiple incidents, including the USAir crash near Pittsburgh and a 1991 Colorado Springs disaster.","keywords":["rudder","safety","Boeing"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["dBase","Paradox"],"full_text":"The Federal Aviation Administration's database of service difficulty reports, despite the gaps found by the Cleveland Plain Dealer, is still good for spotting safety problems. The Seattle Times drove that point home in a front-page story documenting a raft of rudder problems in Boeing 737s.\n\nThe newspaper rummaged through two decades' worth of service difficulty reports, and found a pattern of malfunctions linked to the jet's rudder. In two dozen cases, pilots were forced to make unscheduled landings, emergency descents or return to the terminal gate.\n\nConcerns ran deep that the rudder could sometimes reverse itself. The FAA ordered airlines in March 1994 to replace the rudder system's central hydraulic mechanism within five years.\n\nThe newspaper reminded readers that investigators considered rudder problems a possible cause of the USAir crash near Pittsburgh, but had yet to draw a conclusion.\n\nReporter Byron Acohido checked with USAir, which reported the jet that crashed did not have the improved rudder system. And the newspaper pulled an unsolved crash out of the National Transportation Safety Board's files - a 1991 disaster outside Colorado Springs; the last jetliner to stump NTSB investigators. It, too, was a 737. Investigators had focused on two theories, including a malfunctioning rudder.\n\nIn fact, Acohido said, as soon as he learned of the Pittsburgh crash, he suspected rudder problems may have played a role because of what he learned from the Colorado Springs disaster. \"I had kept a file on the rudder problem since Colorado Springs,\" he said.\n\nA private company supplied its catalog of FAA data to the Seattle Times. The firm provided 737 in dBase to the Seattle Times. Editorial systems editor Bob Higgins helped Acohido examine the data using Paradox.\n\nThe story included a chart depicting the rudder hydraulic system. And it highlighted, in a sidebar, 11 cases where flights went wrong because of rudder problems. Later, during hearings, Boeing officials said they knew of even more 737 flights disrupted by rudder problems than what was listed in the FAA database.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-february-car-routine-in-st-louis","slug":"1995-february-car-routine-in-st-louis","url":"https://dwillis.github.io/uplink/article/1995-february-car-routine-in-st-louis.html","year":1995,"month":"February","monthNum":2,"headline":"CAR routine in St. Louis","kicker":"The CAR Lot","author_name":"Jim Mosley","author_title":"Reporter","affiliation":"St. Louis Post-Dispatch","summary":"The St. Louis Post-Dispatch integrates computer-assisted reporting into everyday journalism practice, involving all reporters and editors in data usage.","keywords":["CAR","journalism","St. Louis"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["FoxPro","Excel","Atlas GIS","9-track tape","magneto-optical drive","Windows","Internet"],"full_text":"Computer-assisted reporting at the St Louis Post-Dispatch has gone from a highly specialized skill to just part of the job. We were among the first papers in the country to dedicate a full-time reporter to computer-related projects. Now, we are working to mainstream CAR into the newsroom, and are encouraging beat and specialty reporters to incorporate the power of personal computers onto their beats. Management - from the managing editor to assistant city editors is pushing reporters and editors to embrace the gospel of spreadsheets and databases. 'I want to get computers out of the temple and into the professional kit of everyone in our newsroom,' says Managing Editor Foster Davis. 'The computer does ordinary tasks so rapidly it seems like a genius instead of a stupid workaholic. The more that people understand what tasks the computer can do, the more they are freed to do the work that a computer cannot do. What drives me nuts is seeing bright people tied up doing stuff a computer could do for them, if we equipped them and trained them.' The Post-Dispatch also has specialized hardware, such as a nine-track drive and a 1-gig magneto-optical drive. This spring, we will buy a large NT server (with 16 GB of storage space) to provide true client-server database services. Spending money and time training people. We have sent nine staff members so far to the NICAR boot camp. In February 1995, we are scheduled to begin an in-house computer-assisted training program that will cover everything from data acquisition to index tags. Getting everyone on the Internet. The publisher has approved an Internet node for high-speed access from all office PCs. In addition, we have contracted to provide full Internet access from home for any reporter or editor who wants it. Several reporters and editors already have e-mail addresses, but we want to make knowledge of the Internet part of every reporter's job. We also are maintaining our commitment to bigger computer-assisted projects. Our practice has been to pair a beat reporter or reporters with the CAR specialist, which has allowed us to draw on the expertise of both sides, to get stories into the paper faster and to show more reporters the power of the PC. Tim Novak and I analyzed 130,000 Illinois pension records to show that legislators had written state pension law to favor themselves and state judges. Joan Little and I found that black students overwhelmingly make up the waiting list to get into magnet schools, considered among the best in St. Louis. Carolyn Tuft, Joe Holleman and I (all NICAR bootcamp grads) matched records of people licensed to drive ambulances in Missouri with the database of moving violations. We found that the state was not doing background checks on applicants, and that people with poor driving records and multiple drunken-driving convictions were licensed to drive ambulances. Our story led to other stories about the state granting ambulance licenses to people who had stolen drugs and money from ambulance districts. The stories prompted the state to remove the head of ambulance licensing and to change the policy for granting licenses. Post-Dispatch beat reporters are beginning to do their own data analysis for investigative and enterprise stories - in addition to the work done by the two people who work full time on computer-related projects. Washington correspondent Kathy Best (a NICAR bootcamp grad) used Microsoft FoxPro to track spending in a U.S. Senate election in Missouri to find out what areas of the state and country were giving the most. Business reporter Jerri Stroud used Excel and Atlas GIS to analyze and map farmers' participation in the federal Conservation Reserve Program. Here's how we are involving our newsroom more deeply in computer-assisted efforts: Giving people the hardware and software they need. We have about 80 486 PCs in our newsroom and bureaus. We hope early this year to hit our goal of providing a PC to every reporter who wants one. The machines have enough disk space, memory and horsepower to allow a beat reporter to analyze just about any chunk of data. We equip our machines with Excel, FoxPro and Microsoft Access. We have several reporters proficient in FoxPro, but we are starting to make the transition to Access as our primary database manager. We also have other software, such as Atlas GIS and SPSS.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-february-from-the-nicar-library","slug":"1995-february-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1995-february-from-the-nicar-library.html","year":1995,"month":"February","monthNum":2,"headline":"From the NICAR library","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR library pricing guide for various government and public databases, including federal firearms records, FEC campaign contributions, crime reports, and other datasets, with costs varying by circulation size and geographic scope.","keywords":["databases","pricing","government"],"topics":["Data & Databases"],"technologies":[],"full_text":"From the NICAR library.\n\nCirculation\n50K\t50-100K\t>100K\n1994 ATF Federal Firearms\nU.S.\t$250\t$250\t$250\nSpecific State\t$40\t$60\t$80\nFAA Service Difficulty Reports\t$125\t$125\t$125\n(includes Airmen Directory)\nFEC Campaign Contributions\nIndividuals U.S.\t$150\t$150\t$150\nIndividuals - State\t$50\t$50\t$50\nPAC Contributions\nUnited States\t$75\t$75\t$75\nState\t$35\t$35\t$35\nFederal Contracts Data\nUnited States\t$150\t$175\t$200\nState\t$50\t$75\t$100\nNational Bridge Inventory Survey\nUnited States\t$150\t$175\t$200\nState\t$50\t$75\t$100\nOSHA\nUnited States\t$150\t$150\t$150\nState\t$40\t$60\t$80\nTruck Census or Truck Accident Data\nUnited States\t$125\t$125\t$125\nState\t$40\t$60\t$80\nSSA Master Death Records\nUnited States\t$400\t$450\t$500\nState\t$200\t$200\t$200\nHMDA\nUnited States\t$150\t$150\t$150\nState\t$40\t$40\t$40\nFBI Uniform Crime Reports\nUnited States\t$400\t$450\t$500\nState (Per Database)\t$40\t$60\t$80\nState (All Databases)\t$200\t$250\t$300\n\nIf you request information for a particular state or states, please specify the state(s) you are interested in. Prices are subject to change. For up-to-the-minute information on all data, please call NICAR at (314) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-february-road-seminars-open-to-all","slug":"1995-february-road-seminars-open-to-all","url":"https://dwillis.github.io/uplink/article/1995-february-road-seminars-open-to-all.html","year":1995,"month":"February","monthNum":2,"headline":"Road seminars open to all","kicker":"","author_name":"Jennifer LaFleur and Brant Houston","author_title":"NICAR training staff","affiliation":"NICAR","summary":"NICAR is conducting computer-assisted reporting training sessions across the United States in February through April 1995, with most seminars open to all journalists at various locations including Washington D.C., Springfield, Boston, Indianapolis, Stamford, Chicago, and Albany.","keywords":["training","journalism","computer"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":[],"full_text":"NICAR's U.S. tour continues in 1995 with on-site training around the country in March and April. Many are open to any journalist wishing to participate. Here's a partial list:\n\nWashington D.C.\n\nNICAR will be bringing our in-depth, hands-on boot camp to Medill News Service. This seminar, cosponsored by Medill News Service, will be held over two weekends - Feb. 25-26 and March 4-5 at the Medill News Service. It limited to 20 participants. For more information call Ellen Shearer at Medill News Service at (202) 662-1801. Open to all journalists.\n\nSprigfield, IL NICAR and the Illinois Press Association will hold a two-day training in computer-assisted reporting in Springfield, March 1 - 2. Call NICAR at 314-882-0684 for more information. Open to all journalists.\n\nBoston, MA\n\nNICAR will conduct a three-day boot camp at the Boston Globe, Feb. 27-March 1. Private seminar.\n\nIndianapolis, IN\n\nNICAR will participate in the annual computer-assisted reporting conference at the Indiana University/Purdue University campus, March 17-19. Call James Brown at 317-274-2773 for more information. Open to all journalists.\n\nStamford, CT\n\nNICAR will participate in a regional SPJ conference on Saturday, March 25, and conduct hands-on training on March 26. Enrollment for hands-on training is limited to 20 people and costs $50 per attendee. Call NICAR for more information. Open to all journalists.\n\nChicago, IL\n\nNICAR will participate in the IRE regional conference, March 31-April 2. Hands-on training may be added. Call NICAR if you are interested in training. Open to all journalists.\n\nAlbany, NY\n\nNICAR will conduct an overview session at the New York State Press Association conference on April 7. Hands-on training sessions may be added. Call NICAR if you are interested. Open to all journalists.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-february-bits-bytes-and-barks","slug":"1995-february-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1995-february-bits-bytes-and-barks.html","year":1995,"month":"February","monthNum":2,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This journalism newsletter covers several technology and reporting developments, including a bug discovery in OS/2 Warp, promotions in online journalism, new books for digital journalists, and innovations in computer-assisted reporting at news organizations.","keywords":["software","journalism","database"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["OS/2","Internet","World Wide Web"],"full_text":"Gillmor spots OS/2 Warp bug\n\nDan Gillmor of the San Jose Mercury News received mention in the December issue of Bug Net after he identified an installation problem in IBM's new OS/2 Warp software.\n\nBug Net, a newsletter devoted to \"computer bugs, glitches, incompatibles and their fixes,\" noted that Gillmor's discovery prompted IBM to recall a number of its Warp packages in order to correct the problem.\n\nBuchanan heads up online\n\nFreedom Forum's Brian Buchanan, a great supporter of computer-assisted reporting and NICAR, has received a promotion. He's now director of online journalism programs. He remains in charge of fellowship programs for the Arlington, Va.-based journalism foundation.\n\nOverview delayed\n\nUplink was about to do a review of last year's computer-assisted stories when a ton (that may be an accurate figure) of newspaper clippings submitted for the IRE awards dropped in from the mailroom. There was too much good stuff to digest in a few days so we're delaying the review until next month.\n\nNew books\n\nTwo terrific books for online journalists have just been released.\n\n\"The Online Journalist\" by Randy Reddick and Elliot King is a solid introduction to using online services in a practical way. It covers the basics of getting connected to the complexities of using the Internet on deadline. It's published by Harcourt Brace.\n\n\"Washington Online\" by Bruce Maxwell is a user-friendly look at significant government bulletin boards including defense, health and medicine, and almost any other subject you can think of. It's publish by Congressional Quarterly Inc.\n\nBillboards private in California\n\nIs your name up in lights? If it is in California we may never know it.\n\nNICAR recently called to obtain the California Highway Department database from its Adopt-a-Highway program. This always makes for a festive, easy story with quick turnaround. On the 110 in LA, for example, there is a sign reading: \"the next 2 miles sponsored by Bette Midler.\" Because of privacy laws, however, Bette's name never appears in the database.\n\nThe database, which is difficult to obtain electronically, does not contain the name of any individual who sponsors sections of California's highways, so the only way to get that information is to drive around and find the billboards bearing their names.\n\nWDIV launches Web page\n\nWDIV-TV in Detroit is combining its online presence with its CAR work. The station's World Wide Web home page at http:/\\www.wdiv.com\\ recently featured the results of reporter Mike Wendland's CAR analysis of FBI Uniform Crime Reports. Interested citizens could find out crime statistics for their city by clicking on its name.\n\nNew SSA database available\n\nThe Social Security Adminstration is making available new database that tracks changes in who receives benefits among the elderly and the disabled, according to a recent issue of the newsletter of the Association of Public Data Users.\n\nThe database contains demographic, employment, marital and income information. For the elderly, it includes data on efforts to get post-retirement jobs; for the disabled, it includes their efforts to return to work. It also tracks those surveyed during several years.\n\nThe agency reportedly plans to make the information available on the Internet. For more information call Dr. Hoard Iams at (202) 282-7092.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-uplink-update","slug":"1995-march-uplink-update","url":"https://dwillis.github.io/uplink/article/1995-march-uplink-update.html","year":1995,"month":"March","monthNum":3,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Uplink's new monthly column features guidance from journalism experts on using internet resources and public records for investigative reporting, including examples of award-winning investigations into natural disasters, crime statistics, elevator safety, and pesticide enforcement.","keywords":["investigation","records","reporting"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Internet"],"full_text":"This month's Uplink features a new monthly column that will help reporters use the Internet. Nora Paul, a former Miami Herald librarian now at the Poynter Institute, tells where to go online when your community is hit by a natural disaster.\n\nNICAR's Gwen Carleton and David Royse tell how to mine crime statistics, and warn about the dangers in the data. David Armstrong of the Boston Globe recounts how he explored state inspection records for elevators and escalators. He worked with a reporting team that found lax enforcement, loafing inspectors and shoddy machinery. The series brought the demotion of state officials, and won a George Polk award this year for metro reporting. Michael Fabey tells how he examined state pesticide enforcement records and uncovered a regulatory system gone wrong. His work recently won a Scripps-Howard Meeman bronze medallion for environmental journalism.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-when-crime-doesnt-add-up","slug":"1995-march-when-crime-doesnt-add-up","url":"https://dwillis.github.io/uplink/article/1995-march-when-crime-doesnt-add-up.html","year":1995,"month":"March","monthNum":3,"headline":"When crime doesn't add up","kicker":"Beware examining FBI data","author_name":"Gwen Carleton","author_title":"NICAR staff","affiliation":"NICAR","summary":"Journalists increasingly analyze crime statistics themselves, but face significant challenges with data accuracy, as demonstrated by errors in FBI crime databases including missing data, definitional inconsistencies, and occasional deliberate misreporting by police agencies.","keywords":["statistics","crime","data"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["Uniform Crime Reports"],"full_text":"Crime statistics are full of potential disasters. Just ask Shawn Macintosh of the Dallas Morning News. \"Several years ago, I found a ring of 9-year-old kiddie crack dealers in Macon, Georgia,\" she recalled. But Macon police said there was no problem. And it turned out they were right. The kids were fabrications, the result of data entry mistakes. In the past, many reporters left statistical analysis up to the experts. Today, more journalists are analyzing the data themselves, and are producing sophisticated stories about crime rates and trends as a result. But they also are finding just how difficult it can be to get the story right. The FBI's Uniform Crime Reports are among the nation's largest and most popular crime databases. The collection of six databases, compiled and released annually, provides data on the victims, offenders and circumstances of millions of crimes committed nationwide. The FBI publishes \"Crime in America,\" a book-length analysis of the year's statistics, shortly before it releases the raw data each year.\n\nThe Uniform Crime Reports have a number of strengths, among them size, detail and a 66-year history. But there are weaknesses as well. The FBI gathers its data from local police, then checks the numbers for mathematical errors and unusual fluctuations. But it doesn't catch every mistake. In addition, participation in the program is voluntary - and not every town volunteers.\n\n\"It's a 'we take your word for it' situation,\" said Carol Napolitano, computer-assisted journalism coordinator for the Munster (Ind.) Times. \"Often, there were significant discrepancies between what we got from the FBI and individual agencies.\"\n\nGarbage in\n\nThe discrepancies usually happen when federal and state agencies define crimes differently, or when police change reporting procedures. But cheating happens, said Jim Omohundro, writer/editor of the Uniform Crime Reports Program. \"There was one major city - I won't say which one - that we sanctioned because we found they were deliberately underreporting homicides,\" he said.\n\nThe FBI excludes any data it deems unreliable from its reports. Some omissions, such as rape statistics for Michigan, Illinois and Minnesota, are consistent: those states define sex crimes more broadly than the rest of the nation. Others come as surprises. Montana, for example, is missing from the 1993 Supplemental Homicide Report, and Kansas and Illinois are missing from several 1993 databases.\n\nThe new statistics have provided reporters with some more subtle surprises as well. \"Crime in the United States, 1993\" stated 70 police officers were killed in 1993, for example, while the electronic version of the data showed 51. The difference? The FBI left 19 state, federal and Puerto Rican officers out of the electronic aggregates.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-the-case-of-the-unknown-stranger","slug":"1995-march-the-case-of-the-unknown-stranger","url":"https://dwillis.github.io/uplink/article/1995-march-the-case-of-the-unknown-stranger.html","year":1995,"month":"March","monthNum":3,"headline":"The case of the unknown stranger","kicker":"","author_name":"David Royse","author_title":"","affiliation":"University of Missouri","summary":"Armstrong reveals safety failures in elevators and escalators through data analysis, leading to significant state-level repercussions.","keywords":["inspection","safety","elevators"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Run for your lives! Everyone in the United States has a realistic chance of being murdered!\n\nAfter all, the FBI said so in December. In Crime in the United States, 1993, a summary of the Uniform Crime Reports, the agency said the statement \"is somewhat supported by the fact that a majority of the nation's murder victims are now\n\nkilled by strangers or unknown persons.\" In other words, random murders are soaring.\n\nScary stuff. Numerous newspapers and wire services ran with the FBI's assertion. But the claim is not backed by the numbers. That is because there is a big difference between a stranger and an unknown person.\n\nThe distinction brings up a crucial rule, regardless of whether a reporter is running with a press release on a tight deadline or sifting through massive amounts of data: Know your definitions.\n\nThe difference between a stranger and an unknown person is enormous. A stranger, according to police definitions, is someone the victim did not know. An unknown offender, on the other hand, means the case is unsolved. They don't know who did it. It could have been the victim's mother. Or it could have been a friend. Or a stranger.\n\nThe FBI view\n\n\"There are still criminologists asserting that (murder) is a crime of passion, that it is a crime between family and friends, and something law enforcement has no control over,\" said Gil Gee, the principle author of the FBI study. \"We are seeing a trend away from that. It should be noted that there is a shift toward more ambiguous relationships between homicide victims and offenders.\"\n\nAmbiguous is the key word. That's because, in recent years, the biggest increase is not where the criminal is classified by stranger. It is where the criminal is unknown - and the case is unsolved.\n\n\"The essence of this study is that we are seeing a significant change in the circumstances surrounding murders,\" Gee said. \"When you look at murder, you find that going back to 1965, a third of murders occurred within the family or were committed by an acquaintance. In 1992 that's 12 percent.\"\n\nMurders by strangers are more difficult to solve than those by acquaintances and family members, Gee said. Thus, it is reasonable to assume that many of the unsolved crimes are committed by strangers.\n\n\"These cases would not be unsolved if they were that clear cut,\" he said.\n\nLook again\n\nThere are a number of police officials and other experts, however, who disagree with the FBI's interpretation of the numbers.\n\nThe FBI's announcement \"is not true,\" said Bill Burns, a spokesman for the Gary, Ind. police department. Located right outside of Chicago, Gary had the highest per capita murder rate in the nation last year. \"You're most likely to get killed by someone you know,\" Burns said.\n\nTim Carroll, a police homicide detective Chattanooga, Tenn., cautioned against speculating about the relationship between a killer and his victim when the crime is unsolved. \"Until we get a suspect, we don't know what the relationship is,\" he said.\n\nJames Fox, dean of the Northeastern College of Criminal Justice, said this year's FBI analysis focused on murders by strangers at the expense of some more important trends.\n\n\"There are two trends going different directions, and they miss it entirely,\" he said. \"The (murder) rate among is skyrocketing, and the rate among adults is plummeting.\"\n\nOne newspaper that did not lead with the FBI's contention was the New York Times. In his story on the 1993 data, reporter Keith Bradsher noted, \"a declining percentage of murder victims were clearly killed by family members or people with whom they were romantically involved. But because the relationship between murderer and the victim could not be determined in two of five cases, the figures on romantic and family killings may not be representative of national trends in actual killings.\"\n\nBradsher is not a reporter who covers crime. In fact, it's the only crime piece he recalled writing.\n\n\"It was a case of reading carefully,\" he said. \"I did my master's in public policy and economics. They taught us you don't just look at the statistics, you look at how they were defined. You don't just take numbers and run with it.\"\n\nkilled by strangers or unknown persons.\" In other words, random murders are soaring.\n\nScary stuff. Numerous newspapers and wire services ran with the FBI's assertion. But the claim is not backed by the numbers. That is because there is a big difference between a stranger and an unknown person.\n\nThe distinction brings up a crucial rule, regardless of whether a reporter is running with a press release on a tight deadline or sifting through massive amounts of data: Know your definitions.\n\nThe difference between a stranger and an unknown person is enormous. A stranger, according to police definitions, is someone the victim did not know. An unknown offender, on the other hand, means the case is unsolved. They don't know who did it. It could have been the victim's mother. Or it could have been a friend. Or a stranger.\n\nThe FBI view\n\n\"There are still criminologists asserting that (murder) is a crime of passion, that it is a crime between family and friends, and something law enforcement has no control over,\" said Gil Gee, the principle author of the FBI study. \"We are seeing a trend away from that. It should be noted that there is a shift toward more ambiguous relationships between homicide victims and offenders.\"\n\nAmbiguous is the key word. That's because, in recent years, the biggest increase is not where the criminal is classified by stranger. It is where the criminal is unknown - and the case is unsolved.\n\n\"The essence of this study is that we are seeing a significant change in the circumstances surrounding murders,\" Gee said. \"When you look at murder, you find that going back to 1965, a third of murders occurred within the family or were committed by an acquaintance. In 1992 that's 12 percent.\"\n\nMurders by strangers are more difficult to solve than those by acquaintances and family members, Gee said. Thus, it is reasonable to assume that many of the unsolved crimes are committed by strangers.\n\n\"These cases would not be unsolved if they were that clear cut,\" he said.\n\nLook again\n\nThere are a number of police officials and other experts, however, who disagree with the FBI's interpretation of the numbers.\n\nThe FBI's announcement \"is not true,\" said Bill Burns, a spokesman for the Gary, Ind. police department. Located right outside of Chicago, Gary had the highest per capita murder rate in the nation last year. \"You're most likely to get killed by someone you know,\" Burns said.\n\nTim Carroll, a police homicide detective Chattanooga, Tenn., cautioned against speculating about the relationship between a killer and his victim when the crime is unsolved. \"Until we get a suspect, we don't know what the relationship is,\" he said.\n\nJames Fox, dean of the Northeastern College of Criminal Justice, said this year's FBI analysis focused on murders by strangers at the expense of some more important trends.\n\n\"There are two trends going different directions, and they miss it entirely,\" he said. \"The (murder) rate among is skyrocketing, and the rate among adults is plummeting.\"\n\nOne newspaper that did not lead with the FBI's contention was the New York Times. In his story on the 1993 data, reporter Keith Bradsher noted, \"a declining percentage of murder victims were clearly killed by family members or people with whom they were romantically involved. But because the relationship between murderer and the victim could not be determined in two of five cases, the figures on romantic and family killings may not be representative of national trends in actual killings.\"\n\nBradsher is not a reporter who covers crime. In fact, it's the only crime piece he recalled writing.\n\n\"It was a case of reading carefully,\" he said. \"I did my master's in public policy and economics. They taught us you don't just look at the statistics, you look at how they were defined. You don't just take numbers and run with it.\"\n\nkilled by strangers or unknown persons. In other words, random murders are soaring.\n\nScary stuff. Numerous newspapers and wire services ran with the FBI's assertion. But the claim is not backed by the numbers. That is because there is a big difference between a stranger and an unknown person.\n\nThe distinction brings up a crucial rule, regardless of whether a reporter is running with a press release on a tight deadline or sifting through massive amounts of data: Know your definitions.\n\nThe difference between a stranger and an unknown person is enormous. A stranger, according to police definitions, is someone the victim did not know. An unknown offender, on the other hand, means the case is unsolved. They don't know who did it. It could have been the victim's mother. Or it could have been a friend. Or a stranger.\n\nThe FBI view\n\n\"There are still criminologists asserting that (murder) is a crime of passion, that it is a crime between family and friends, and something law enforcement has no control over,\" said Gil Gee, the principle author of the FBI study. \"We are seeing a trend away from that. It should be noted that there is a shift toward more ambiguous relationships between homicide victims and offenders.\"\n\nAmbiguous is the key word. That's because, in recent years, the biggest increase is not where the criminal is classified by stranger. It is where the criminal is unknown - and the case is unsolved.\n\n\"The essence of this study is that we are seeing a significant change in the circumstances surrounding murders,\" Gee said. \"When you look at murder, you find that going back to 1965, a third of murders occurred within the family or were committed by an acquaintance. In 1992 that's 12 percent.\"\n\nMurders by strangers are more difficult to solve than those by acquaintances and family members, Gee said. Thus, it is reasonable to assume that many of the unsolved crimes are committed by strangers.\n\n\"These cases would not be unsolved if they were that clear cut,\" he said.\n\nLook again\n\nThere are a number of police officials and other experts, however, who disagree with the FBI's interpretation of the numbers.\n\nThe FBI's announcement \"is not true,\" said Bill Burns, a spokesman for the Gary, Ind. police department. Located right outside of Chicago, Gary had the highest per capita murder rate in the nation last year. \"You're most likely to get killed by someone you know,\" Burns said.\n\nTim Carroll, a police homicide detective Chattanooga, Tenn., cautioned against speculating about the relationship between a killer and his victim when the crime is unsolved. \"Until we get a suspect, we don't know what the relationship is,\" he said.\n\nJames Fox, dean of the Northeastern College of Criminal Justice, said this year's FBI analysis focused on murders by strangers at the expense of some more important trends.\n\n\"There are two trends going different directions, and they miss it entirely,\" he said. \"The (murder) rate among is skyrocketing, and the rate among adults is plummeting.\"\n\nOne newspaper that did not lead with the FBI's contention was the New York Times. In his story on the 1993 data, reporter Keith Bradsher noted, \"a declining percentage of murder victims were clearly killed by family members or people with whom they were romantically involved. But because the relationship between murderer and the victim could not be determined in two of five cases, the figures on romantic and family killings may not be representative of national trends in actual killings.\"\n\nBradsher is not a reporter who covers crime. In fact, it's the only crime piece he recalled writing.\n\n\"It was a case of reading carefully,\" he said. \"I did my master's in public policy and economics. They taught us you don't just look at the statistics, you look at how they were defined. You don't just take numbers and run with it.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-dusting-pesticide-records","slug":"1995-march-dusting-pesticide-records","url":"https://dwillis.github.io/uplink/article/1995-march-dusting-pesticide-records.html","year":1995,"month":"March","monthNum":3,"headline":"Dusting pesticide records","kicker":"Computer records reveal environmental havoc","author_name":"Michael Fabey","author_title":"Reporter","affiliation":"Fayetteville Observer-Times","summary":"A newspaper investigation into pesticide spray incidents revealed systematic regulatory failures by discovering that government agencies were withholding or underreporting violation data, uncovered through computer database analysis and open records requests.","keywords":["pesticides","investigation","regulators"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Paradox","PC","floppy disk","ProfNet","modem/BBS"],"full_text":"When local resident Jim Pickron contacted the Fayetteville (N.C.) Observer-Times, we figured the most that would come about was a human-interest piece about the dangers of careless crop dusting.\n\nInstead, using computers, we found story was about a system gone wrong - that government regulators don't do their job when it comes to watching those who spray pesticides.\n\nPickron was putting up a fence in his backyard when he was accidently sprayed with chemicals. The state fined the pilot, who admitted his error. We wanted to find out whether Pickron was the victim in an isolated incident. We first asked the state for similar cases. They gave us a printed list of about 200 names of those who misused pesticides in the last two years, plus the victims, the fine, the violation code, and a little more.\n\nI was beginning to learn about computers, and wanted to compile the information into a spreadsheet. The newspaper installed Paradox into the IBM 386 on my desk. I typed for a day and a half. And I noticed some information was missing, such as the full name of complainants and violators.\n\nI went to the state Pesticide Division in Raleigh, and the clerks let me check some of the records in their computer. Their database included more than 50 fields of pertinent information, from types of chemicals to the extent of injuries. In addition, the database included twice as many records as the printout they first gave me. It turned out the printed version they gave me did not include cases where they had taken little action.\n\nI asked for a copy of the database on diskette. The agency balked, claiming they did not have authority to release it since some of the fields were not considered public record. I cited state open records law, and told them required to turn over a version of the database with all fields open for public inspection. They refused. I called the Pesticide Division administrator, and said I was working on a story about how the agency was violating state law by withholding information. I got the diskettes the next day.\n\nThe database was invaluable. We used Paradox and found repeat offender, the most common place for violations, and crop dusting that resulted in injuries. Then we used Paradox to print mailing labels - and we sent surveys to every person who filed a complaint in the last two years. The survey showed people believed the agency did not treat them well. We also got victims to tell their stories, vital for the project.\n\nWe also imported data to map where violations happened. I used ProfNet to find experts on pesticides. I found sources on several electronic bulletin boards. And I tracked down more information from the U.S. Environmental Protection Agency. EPA first wanted to charge me $3,000 for printed records on pesticide violations. When I asked for the information on diskette, the agency said that was no problem. And the price? Free.\n\nThe result was a five-day series that examined how local, state and federal regulators often failed to ensure the safety of the likes of Pickron. The articles also showed that those endangered people and the environment often went unpunished.\n\nMichael Fabey recently left the Fayetteville Observer-Times for the Journal of Commerce's New York bureau. He can be reached at 1-800-223-0243, or send email to mjf625775@aol.com. His series on pesticides is available from the Investigative Reporters and Editors' morgue. Call 314-882-2042, and ask for morgue number 10179.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-join-us-on-the-internet","slug":"1995-march-join-us-on-the-internet","url":"https://dwillis.github.io/uplink/article/1995-march-join-us-on-the-internet.html","year":1995,"month":"March","monthNum":3,"headline":"Join us on the Internet","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"Instructions for subscribing to NICAR and IRE listservs where journalists discuss investigative reporting techniques and professional development.","keywords":["listserve","journalism","investigative"],"topics":["Journalism & Newsroom"],"technologies":["Internet","Email"],"full_text":"Don't forget to keep up with NICAR on the Internet. Subscribe to our listserve, and join as reporters talk about how to do the job better.\n\nEmail to listserve@mizzoul.missouri.edu\n\nIn the message, on the first line, write: subscribe nicar-l your name. Join the Investigative Reporters and Editors' listserve the same way, except type on the first line of the message: subscribe ire-l your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-go-up-the-database-staircase","slug":"1995-march-go-up-the-database-staircase","url":"https://dwillis.github.io/uplink/article/1995-march-go-up-the-database-staircase.html","year":1995,"month":"March","monthNum":3,"headline":"Go up the database staircase","kicker":"Elevator inspections show bad rides","author_name":"David Armstrong","author_title":"Reporter","affiliation":"The Boston Globe","summary":"The Boston Globe's investigation into elevator and escalator safety, enabled by analysis of a state inspection database, revealed that nearly 40 percent of passenger elevators were not inspected as required and many failed elevators were not shut down, leading to disciplinary actions against inspectors and officials.","keywords":["safety","inspections","database"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["dBase","FoxPro","MapInfo"],"full_text":"Every day, millions of people use elevators and escalators to travel to work, shop or attend a sporting event or concert. Most people, however, give little thought to the safety of these machines. The Boston Globe recently completed a three-part series that examined the elevator and escalator industry's inattention to safety, and the frightening ramifications for the riding public.\n\nOne of the first things we requested was a copy of the state's database of elevator and escalator inspections. The database was created only two years earlier, replacing a card catalog system that had been in place since Mr. Otis invented the first elevator years ago. The card catalog system was a reporter's nightmare: a paper search that would have required perhaps hundreds of hours of work. The computerized records, however, allowed us to quickly, and conclusively, evaluate the state's ability to inspect the 30,000 elevators and escalators.\n\nPerhaps unaware of the potential manipulation of its own data, the state graciously agreed to provide us with a copy in dBase format of its inspection file only several days after it was requested.\n\nUsing FoxPro, we determined that nearly 40 percent of the state's passenger elevators had not been inspected as required. Even more revealing was the fact that of the elevators inspected, one-third flunked the state's safety test. And of those flunking the safety test, only two percent were shut down until mandated repairs were made.\n\nBy using the \"group by\" command, we could also see there was a wide disparity in inspection rates from city to city. In Worcester, the state's second largest city, nearly 75 percent of the elevators and escalators had not been inspected as required. We created a chart of the inspection rate by community and imported it to MapInfo to produce a map. This ran on the second day of the series and allowed readers to pinpoint the inspection rate in their community.\n\nThe database also helped us identify inspectors who were not doing their job. The database provided the address and date of each inspection performed by inspectors. There was a wide disparity in the workloads. This information was helpful in identifying inspectors who would later be observed by Globe reporters. The Globe found numerous inspectors goofing off on the job, spending their work days at home or running errands. Others worked two jobs.\n\nSeven inspectors and supervisors were suspended without pay after the Globe series was published. The two top officials in the elevator inspection office were demoted. And the state public safety official ultimately responsible for elevator inspections resigned from office two days before the series first appeared. The database also allowed us to say many inspectors appeared to be doing their job, judging by the number of inspections they completed.\n\nThe database of inspections was used as a continuing resource during research for the series. We could quickly check the status of particular buildings or accident sites. We even checked the status of elevators and escalators at the Globe. We found out two of the elevators had not been inspected. This information was reported in a sidebar to one of the many follow-up stories.\n\nWhile the database was an important resource for reporters, it was only one of many tools used. Court records, accident investigations, and elevator industry records were almost entirely on paper, and resulted in numerous cuts to computer-softened hands.\n\nDavid Armstrong can be reached at 617-929-2539, or send email armstr@news.glabe.com. Copies of the series is available from Investigative Reporters and Editors. Call 314-882-2042, and ask for morgue number 11001.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-data-available-from-the-nicar-library","slug":"1995-march-data-available-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1995-march-data-available-from-the-nicar-library.html","year":1995,"month":"March","monthNum":3,"headline":"Data available from the NICAR library","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides journalists access to numerous government databases including election contributions, aviation records, crime statistics, and occupational safety data, along with inexpensive data transfer services.","keywords":["databases","government","journalists"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"NICAR offers a number of government databases including:\n\nFederal Elections Commission contributions data\nFederal Aviation Administration data\nHome Mortgage Disclosure Act records\nFederal procurement data\nAlcohol, Tobacco and Firearms gun dealer records\nNational Bridge Inventory System data\nFBI Uniform Crime Reports\nSocial Security death records\nOccupational Safety and Health Administration violation data\nTruck Accidents and census data\n\nNICAR also offers inexpensive data transfer for journalists. Call NICAR for up-to-date prices and more information at (314) 882-0684, or send email to nicar@muccmail.missouri.edu.\n\nunderreporting homicides,","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":false}},{"id":"1995-march-which-assaults-matter","slug":"1995-march-which-assaults-matter","url":"https://dwillis.github.io/uplink/article/1995-march-which-assaults-matter.html","year":1995,"month":"March","monthNum":3,"headline":"Which assaults matter?","kicker":"","author_name":"Gwen Carleton","author_title":"Staff","affiliation":"NICAR","summary":"Journalists debate how to properly interpret FBI crime data, particularly regarding which assault categories to include in local crime analyses, with experts emphasizing the importance of clarity and accuracy in reporting statistics to the public.","keywords":["assaults","data","journalism"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Which assaults matter?\n\nAnother important question facing journalists is how to interpret the data. The epicenter of a recent debate was in Detroit, where two television news broadcasts used the FBI data and came up with different numbers.\n\nA key concern is assaults. While Detroit's WDIV-TV counted both aggravated and simple assaults in its local crime analysis, rival WJBK-TV used only aggravated assaults, resulting in much smaller totals. WDIV's approach, created with NICAR's help, mirrored the FBI's format in its electronic data. WJBK's, on the other hand, corresponded with the FBI \"crime index\" categories presented in the text version of the data and in Bureau press releases.\n\n\"In a big city environment, I wouldn't pay much attention to simple assaults,\" said Josh Barbanel, a reporter for the New York Times. \"Peoples' idea of danger concerns moving in a public space.\"\n\nBut in some areas, such as the Detroit suburbs, simple assault statistics might be more relevant, Barbanel said.\n\nThe FBI has no official position on how the statistics are used, as long as they are used accurately, Omohundro said. \"Generally, aggravated assault is reported exclusive of simple assault,\" he said. \"(But) if someone wants to use the two categories, we just ask they make the distinction clear.\"\n\nThere are many aspects of FBI data that journalists should clarify for their readers. The database represents only reported crime; actual crime is higher. Statistics on violent crimes are more reliable and uniform than those for lesser crimes. Some crimes, such as spousal abuse and child abuse, are not included at all. Others are excluded because the FBI lists only the most serious crime to occur during a single incident.\n\nFBI will revamp system\n\nThe FBI, aware of the database's weaknesses, is well on its way to implementing a new system. The National Incident Based Reporting System promises to provide data free of the above problems, possibly by the year 2000. Omohundro just hopes journalists will be ready. \"It will give a clearer picture of crime in this country,\" he said. But the FBI analyst is steeling himself for journalists who misread the numbers. \"The disadvantage is, the media will go crazy. They'll say crime increased 300 percent.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-doing-time-in-miami","slug":"1995-march-doing-time-in-miami","url":"https://dwillis.github.io/uplink/article/1995-march-doing-time-in-miami.html","year":1995,"month":"March","monthNum":3,"headline":"Doing Time In Miami","kicker":"Combine crime databases for the big story","author_name":"Gwen Carleton","author_title":"NICAR Staff","affiliation":"NICAR","summary":"The Miami Herald's investigation of Dade County's criminal justice system used 10 different criminal databases to reveal that convicted felons were significantly less likely to go to prison compared to the national average. The project uncovered data inconsistencies and led to reforms including changes to the drug court program and appointment of special judges.","keywords":["databases","sentencing","investigation"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["SAS","Uniform Crime Reports"],"full_text":"The Miami Herald's \"Crime and No Punishment\" investigation of Dade County's criminal justice system shows how different databases, used together, become more than the sum of their parts.\n\nTo create the 8-part series, Herald reporters used 10 different criminal databases from local, state and federal agencies, including 14 years' worth of FBI Uniform Crime Reports. The statistics proved what many long suspected: in south Florida, crime pays.\n\nThe reporters found that Dade County, with the highest crime rate in the nation in 1992, sent only 15 of every 100 convicted felons to state prison, three times below the national average. The odds of a Dade robber getting arrested and going to prison were 1 in 40, compared to 1 in 20 elsewhere in America. And when Dade felons did go to state prison, they got significantly less time.\n\nThe Herald's key sources were the Uniform Crime Reports (both electronic and text versions); criminal justice databases from state and federal agencies; reports on various criminal justice topics from federal agencies; and original research, used to obtain conviction rates and prison data for 24 big cities in 13 states.\n\nMixing many different databases added to the project's complexity, according to Steve Doig, the Herald's associate editor/research. Doig discovered that Hialeah, the second largest city in Dade County, has not reported its crimes and arrests for several years. As a result, Dade's real crime rate in 1993 was at least 10 percent higher than that recorded in both the Florida and FBI databases.\n\nDoig also discovered the prosecutor's office had seriously overcounted convictions.\n\n\"Fifty percent fewer going to prison than they were saying,\" Doig said. \"Their computer system was counting cases instead of people, throwing their math off.\" He used SAS statistical software to correct the problem.\n\nHow to report on sentencing raised more problems. How do you compare different judicial circuits? Should a comparison include all criminals, or just some of them? Doig decided to calculate crimes indexes in all Florida circuits, then compare the circuits with the most violent crimes with Dade. That helped screened out the rural circuits with lighter caseloads and notoriously tougher sentences.\n\nThe Herald published \"Crime and No Punishment\" in August, 1994. \"We uncovered real abuses with the drug court program our story shut off the use of for ineligible people,\" Doig said. \"A number of special judges were appointed.\"\n\n\"The most gratifying thing,\" Doig said, \"was, after the initial squeals of dismay from judges, came choruses of 'We didn't know it was that bad - we need to do something.'\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-portables-can-do-big-jobs","slug":"1995-march-portables-can-do-big-jobs","url":"https://dwillis.github.io/uplink/article/1995-march-portables-can-do-big-jobs.html","year":1995,"month":"March","monthNum":3,"headline":"Portables can do big jobs","kicker":"Tech Tip","author_name":"Drew Sullivan","author_title":"Systems Director","affiliation":"NICAR","summary":"Notebook computers have significantly improved in speed, storage capacity, and expandability, making them viable tools for reporters doing data-intensive work. Modern notebooks now offer large hard drives, upgradeable RAM, PCMCIA slots for peripherals, and docking stations, though they remain expensive compared to desktop computers.","keywords":["notebooks","storage","reporting"],"topics":["Investigative Methods","Technology"],"technologies":["PC","floppy disk","magnetic tape","CD-ROM","PCMCIA","modem/BBS","SCSI","Ethernet","Pentium"],"full_text":"Your next big number-crunching computer may not be so big. In fact, you may be able to carry it with you to your local government agency.\n\nNotebook computers are steadily gaining in speed, capacity and adaptability. It might be time to take another look.\n\nIn the past, notebooks could only handle the smallest databases. Hard drives were small, memory limited and the only input devices were floppy disc drives. There was little standardization in the industry - so you could only buy extra memory or a larger hard drive from the manufacturer. This meant higher prices and configuration restrictions.\n\nBut a new group of notebooks is hitting the market that meet the reporter's need for large RAM and storage memory, as well as the flexibility to attach peripherals such as tape drives and take advantage of upgrades in bus speeds.\n\nMost notebooks come with at least 250MB hard drives with some upgrades reaching 1GB. Notebooks with removable hard drives allow for an almost infinite storage capacity as long as your analysis programs are on each drive. 4MB is still the standard for RAM but many vendors now offer cheap upgrades to 16MB.\n\nA number of notebooks are available with built in CD-ROM. IBM offers a notebook with a \"hot swap\" feature that allows you to pull out your floppy and stick in a CD-ROM drive without turning off your system. External CD-ROMS can be added on notebooks with PCMCIA slots.\n\nFor on-line users, PCMCIA also allow you to add 14,400 or 28,800 bps modems and guarantee you can upgrade to future speeds. SCSI cards, Ethernet adapters, hard drives and sound cards can all be added using the PCMCIA ports.\n\nThe real news for reporters are the new docking stations offered by many vendors. The station acts as a expansion device allowing anything from extra hard drives, extra batteries, CD-ROM drives or peripheral cards.\n\nWhile many of these new tools are finally available, they are still expensive. Prices have been steadily falling though and expect that to continue. Overall, expect to pay a little less than twice the cost of a desktop. RAM runs $45-$60/MB for notebooks. Hard drive prices are still somewhat expensive. For example, the EPS Pentium-90 Notebook with 16MB RAM and a 810MB hard drive can be bought for HD, $4588 delivered. Not bad for portable computer assisted reporting.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-cool-internet-site","slug":"1995-march-cool-internet-site","url":"https://dwillis.github.io/uplink/article/1995-march-cool-internet-site.html","year":1995,"month":"March","monthNum":3,"headline":"Cool Internet Site","kicker":"","author_name":"","author_title":"","affiliation":"National Press Club","summary":"The National Press Club provides journalists with tips on trending stories and comprehensive resources to support their reporting work.","keywords":["journalists","stories","resources"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"The National Press Club offers tips on hot stories and a variety of information for journalists. http://town.hall.org/places/npc","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-get-computer-training","slug":"1995-march-get-computer-training","url":"https://dwillis.github.io/uplink/article/1995-march-get-computer-training.html","year":1995,"month":"March","monthNum":3,"headline":"Get computer training","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR is offering computer-assisted reporting training sessions across the United States in March and April, with locations in Indianapolis, Stamford, Chicago, and Albany, plus an intensive weeklong bootcamp in May.","keywords":["training","journalism","conference"],"topics":["Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Get computer training\n\nNICAR's U.S. tour continues training around the country in March and April. These are open to all journalists:\n\nIndianapolis, IN\n\nNICAR will participate in the annual computer-assisted reporting conference at the Indiana University/Purdue University campus, March 17-19. Call James Brown at 317-274-2773 for more information.\n\nStamford, CT\n\nNICAR will participate in a regional SPJ conference, March 25, and conduct hands-on training on March 26. Enrollment for hands-on training is limited. Call NICAR for more information.\n\nChicago, IL\n\nNICAR will participate in the IRE regional conference, March 31-April 2. Hands-on training may be added. Call NICAR if you want hands-on training.\n\nAlbany, NY\n\nNICAR will conduct an overview session at the New York State Press Association conference, April 7. Hands-on training may be added. Call NICAR for more information.\n\nNICAR Bootcamp\n\nNICAR's weeklong intensive training seminar, May 14-19, Columbia, Mo. Call NICAR for more information.\n\nCall NICAR at 314-882-0684, or send email to nicar@muccmailmissouri.edus","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-disaster-sites-to-tap","slug":"1995-march-disaster-sites-to-tap","url":"https://dwillis.github.io/uplink/article/1995-march-disaster-sites-to-tap.html","year":1995,"month":"March","monthNum":3,"headline":"Disaster sites to tap","kicker":"On the Internet","author_name":"Nora Paul","author_title":"librarian/researcher","affiliation":"Poynter Institute","summary":"This article provides journalists with a curated list of Internet resources for covering natural disasters, including government agencies, relief organizations, and specialized sites for hurricanes, earthquakes, and floods.","keywords":["disasters","resources","journalism"],"topics":["Journalism & Newsroom","Technology","Health & Environment"],"technologies":["World Wide Web","Mosaic","Netscape","gopher","FTP","Internet"],"full_text":"Disaster sites to tap\n\nBy Nora Paul\n\nThis year began with dramatic proof of man's inability to control nature. When you cover the latest natural disaster, you need current information, historical references, comments from people who understand these disasters. Here are some Internet sites that can help you the next time mother nature gets angry. Remember, any address that begins with http is available on the World Wide Web, so use search programs like Mosaic and Netscape. Any address that begins with gopher is available both on the World Wide Web and through gopher programs.\n\nGeneral information\n\n- The Federal Emergency Management Agency offers publications, press releases, fact sheets and guides on how to be prepared for disasters. It also features photos of disaster scenes from the Mississippi floods and the California earthquake. http://www.fema.gov/fema/index.html\n\n-The American Red Cross includes its activities and net locations. A section for updating world crises is under construction. http://www.crossnet.org/\n\n-Emergency Preparedness Information Exchange (EPIX) provides documents compiled by the Centre for Policy Research on Science and Technology at Simon Fraser University in Vancouver, B.C. It includes material on emergency management, and connections to other sites. gopher://hoshi.cic.sfu.ca:5555/11/epix\n\nDisaster relief groups\n\n- International Service Agencies lists groups concerned with refugee and disaster support. http://www.charity.org/\n\nVolunteers in Technical Assistance provides information on more than 30 members of National Voluntary Agencies Active in Disaster. gopher://vita.org/11/disaster\n\nDisaster updates\n\n-EPIX's disaster reports include those from the United Nations and from InterAction. gopher://hoshi.cic.sfu.ca:5555/11/epix/curevents/intsitreps\n\nHurricanes and cyclones\n\n- The National Hurricane Center includes links to other meterological sites. Maps show past tracks of tropical cyclones from 1961 to 1993. http://nhc-hp3.nhc.noaa.gov/index.html\n\n-The National Hurricane Center's Tropical Cyclone page maps the paths of destruction from Andrew and Hugo, and a publication listing the deadliest, costliest and most intense U.S. hurricanes this century. http://nhc-hp3.nhc.noaa.gov/hurricane_info.html\n\nEarthquakes\n\n-U.S. National Earthquake Info Service Earthquake Lists includes key background information, like the deadliest and costliest quakes. gopher://wealaka.okgeosurveyl.g 11/other_bulletins/neis/neislists\n\n- The Kobe Earthquake Reference Page points to a variety of Japanese and American sources. http://www.msen.com/~emv/kobe.html\n\nKobe earthquake news group. alt.current.events.kobe-quake\n\n-A news group on scientific aspects of earthquakes. sci.geo.earthquakes\n\n-General discussion of earthquakes. alt.disasters.earthquake\n\nFloods\n\nA Dutch site put together within days of the flooding. Maps, a chronology of events and links to other sites (most in Dutch). http://www.hvu.nl/flood/\n\n- California Data Exchange Center Division of Flood Management, with links to flood watch sites, weather information and updates of flood conditions. http://resources.agency.ca.gov/flood2.html\n\nVolcanoes\n\n-Michigan Technological University Volcanoes Page provides information on most of the world's active volcanoes. http://www.geo.mtu.edu/volcanoes/\n\n-NASA's great fact sheet of information about volcanoes includes photos. http://spso2.gsfc.nasa.gov/nasa_facts/volcanoes/volcano.html\n\n- Volcano listserv provides monthly reports from the Smithsonian Institution's Global Volcanism Network before they appear in print. To subscribe, send e-mail with the one line message \"SUBSCRIBE VOLCANO your full name\". listserv@asuacad.bitnet\n\nIf you have suggestions for this column, e-mail Nora Paul at npaul@poynter.org. Visit the Poynter Institute's website at http://www.nando.net/proff poynter/home.html. She also a feature called Hot News/Hot Research in the Poynter Institute's library section. It takes news stories and links you to some good sources for information to cover that story. To go directly to the page: http://www.nando.net/profi poynter/hrintro.html\n\nNora Paul is a former Miami Herald librarian now at the Poynter Institute. She is the author of Computer Assisted Research, available from the Poynter Institute and NICAR.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-bits-bytes-and-barks","slug":"1995-march-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1995-march-bits-bytes-and-barks.html","year":1995,"month":"March","monthNum":3,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"Jennifer LaFleur","author_title":"NICAR training director","affiliation":"NICAR","summary":"A survey by University of Miami Professor Bruce Garrison tracks the growing adoption of computer-assisted reporting in U.S. newspapers, while the journalism field experiences personnel changes and innovative uses of data analysis in newsrooms.","keywords":["reporting","computers","newspapers"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["spreadsheet"],"full_text":"Bits, Bytes and Barks\n\nThe survey says\n\nMore news organizations are doing computer-assisted reporting. University of Miami Professor Bruce Garrison is tracking that growth, and wants journalists to respond to his annual survey. The eight-page questionnaire was sent to the managing editors of all U.S. newspapers with a daily circulation exceeding 20,000. If you have not seen the survey form, ask your managing editor, or contact Garrison at 305-284-2265, or bgarriso@umiamivm.ir.miami.edu.\n\nThe 1994 results showed 29 percent said they had computer-assisted reporting desks, and 30 percent said they plan to add them. Few local broadcast outlets are trying computer-assisted reporting, and, at those that do, the effort comes from the reporter.\n\nAP hires Port; St. Pete job opens\n\nBob Port is leaving the St. Petersburg Times in Florida to join the Associated Press in New York. He will be a special assignment editor to work on investigative reporting projects.\n\nThe move means the St. Petersburg Times is seeking a computer-assisted reporting editor familiar with a wide variety of equipment. If interested, call Susan Taylor-Martin, Deputy Managing Editor, at 813-893-8869 or e-mail her at susanskate@aol.com Send a resume and clips to her at St. Petersburg Times, 490 First Avenue South, St. Petersburg, FL 33701.\n\nMiami Herald promotes Gordon\n\nThe Miami Herald recently named Rich Gordon as the newspaper's newsroom technology coordinator. The two-year assignment includes establishing computer-assisted reporting training programs, and helping design technological improvements that will benefit reporters.\n\nComputers shadow groundhog\n\nEven an old story can be given new life with a little computer analysis. That's just what some of the IRE and NICAR staff did Groundhog Day. In trying to come up with new training curriculum, Jennifer LaFleur, NICAR training director and Rosemary Armao, IRE executive director, thought getting the database of Punxsutawney Phil's predictions would be a unique idea.\n\nUnfortunately no one in the marmot's Pennsylvania hometown had that information, so with the assistance of their mothers, LaFleur and Armao dug up the history of this meteorological mole by researching 109 years of Feb. 3 issues of the Punxsutawney Spirit and the Puncsutawney Plaindealer. They input the information into a spreadsheet and ran some quick statistics: The groundhog sees his shadow 91 percent of the time, based on 97 years for which data were available. For nine years, no data were available and in 1943, the results from the Quarryville, Pa., groundhog were used. The average time of day Phil looks for his shadow is 8:19 a.m. By graphing the prognostication times, we also were able to see this event appears to have more planning. While times throughout the past 110 years have varied from 6 a.m. to 4:30 p.m., during the last 15 years, those time have leveled to between 7:20 and 7:30 a.m.\n\nThe next step is to test Phil's accuracy with a more sophisticated analysis using National Weather Service data.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-the-survey-says","slug":"1995-march-the-survey-says","url":"https://dwillis.github.io/uplink/article/1995-march-the-survey-says.html","year":1995,"month":"March","monthNum":3,"headline":"The survey says","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"University of Miami Professor Bruce Garrison is conducting an annual survey to track the growing adoption of computer-assisted reporting in U.S. newsrooms, with 1994 data showing 29% of major newspapers already have such desks and 30% planning to add them.","keywords":["survey","reporting","newspapers"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"More news organizations are doing computer-assisted reporting. University of Miami Professor Bruce Garrison is tracking that growth, and wants journalists to respond to his annual survey. The eight-page questionnaire was sent to the managing editors of all U.S. newspapers with a daily circulation exceeding 20,000. If you have not seen the survey form, ask your managing editor, or contact Garrison at 305-284-2265, or bgarriso@umiamivm.ir.miami.edu.\n\nThe 1994 results showed 29 percent said they had computer-assisted reporting desks, and 30 percent said they plan to add them. Few local broadcast outlets are trying computer-assisted reporting, and, at those that do, the effort comes from the reporter.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-ap-hires-port-st-pete-job-opens","slug":"1995-march-ap-hires-port-st-pete-job-opens","url":"https://dwillis.github.io/uplink/article/1995-march-ap-hires-port-st-pete-job-opens.html","year":1995,"month":"March","monthNum":3,"headline":"AP hires Port; St. Pete job opens","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Bob Port is leaving the St. Petersburg Times to join the Associated Press in New York as a special assignment editor for investigative reporting. The St. Petersburg Times is now seeking a computer-assisted reporting editor to fill the vacancy.","keywords":["journalism","hiring","investigation"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Bob Port is leaving the St. Petersburg Times in Florida to join the Associated Press in New York. He will be a special assignment editor to work on investigative reporting projects.\n\nThe move means the St. Petersburg Times is seeking a computer-assisted reporting editor familiar with a wide variety of equipment. If interested, call Susan Taylor-Martin, Deputy Managing Editor, at 813-893-8869 or e-mail her at susanskate@aol.com Send a resume and clips to her at St. Petersburg Times, 490 First Avenue South, St. Petersburg, FL 33701.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-miami-herald-promotes-gordon","slug":"1995-march-miami-herald-promotes-gordon","url":"https://dwillis.github.io/uplink/article/1995-march-miami-herald-promotes-gordon.html","year":1995,"month":"March","monthNum":3,"headline":"Miami Herald promotes Gordon","kicker":"","author_name":"","author_title":"","affiliation":"Miami Herald","summary":"The Miami Herald has appointed Rich Gordon as newsroom technology coordinator for a two-year term to develop computer-assisted reporting training and implement technological improvements for reporters.","keywords":["technology","journalism","training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":[],"full_text":"The Miami Herald recently named Rich Gordon as the newspaper's newsroom technology coordinator. The two-year assignment includes establishing computer-assisted reporting training programs, and helping design technological improvements that will benefit reporters.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-march-computers-shadow-groundhog","slug":"1995-march-computers-shadow-groundhog","url":"https://dwillis.github.io/uplink/article/1995-march-computers-shadow-groundhog.html","year":1995,"month":"March","monthNum":3,"headline":"Computers shadow groundhog","kicker":"","author_name":"Jennifer LaFleur","author_title":"NICAR training director","affiliation":"IRE and NICAR","summary":"IRE and NICAR staff analyzed 109 years of Punxsutawney Phil's Groundhog Day predictions by digitizing historical newspaper records and discovered that the groundhog sees his shadow 91% of the time, with prediction times becoming increasingly standardized in recent years.","keywords":["analysis","groundhog","data"],"topics":["Investigative Methods","Data & Databases"],"technologies":["spreadsheet"],"full_text":"Even an old story can be given new life with a little computer analysis. That's just what some of the IRE and NICAR staff did Groundhog Day. In trying to come up with new training curriculum, Jennifer LaFleur, NICAR training director and Rosemary Armao, IRE executive director, thought getting the database of Punxsutawney Phil's predictions would be a unique idea.\n\nUnfortunately no one in the marmot's Pennsylvania hometown had that information, so with the assistance of their mothers, LaFleur and Armao dug up the history of this meteorological mole by researching 109 years of Feb. 3 issues of the Punxsutawney Spirit and the Puncsutawney Plaindealer. They input the information into a spreadsheet and ran some quick statistics: The groundhog sees his shadow 91 percent of the time, based on 97 years for which data were available. For nine years, no data were available and in 1943, the results from the Quarryville, Pa., groundhog were used. The average time of day Phil looks for his shadow is 8:19 a.m. By graphing the prognostication times, we also were able to see this event appears to have more planning. While times throughout the past 110 years have varied from 6 a.m. to 4:30 p.m., during the last 15 years, those time have leveled to between 7:20 and 7:30 a.m.\n\nThe next step is to test Phil's accuracy with a more sophisticated analysis using National Weather Service data.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-april-uplink-update","slug":"1995-april-uplink-update","url":"https://dwillis.github.io/uplink/article/1995-april-uplink-update.html","year":1995,"month":"April","monthNum":4,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This journalism newsletter focuses on computer-assisted reporting techniques for education beat coverage, featuring expert journalists discussing data analysis methods for stories on segregation and test scores. It also includes guidance on using data to investigate regulatory enforcement issues and accessing federal information online.","keywords":["education","data","reporting"],"topics":["Education","Investigative Methods","Public Safety"],"technologies":["Internet"],"full_text":"School is in. The education beat is made for computer-assisted reporting. In this issue, top journalists from Florida, New Jersey and Washington, D.C, tell you everything from analyzing segregation to SAT scores. These stories underscore how reporters can improve their coverage day in and day out by knowing how to handle electronic information.\n\nIn addition, Dateline NBC's David Hinchman recounts how he battled for data to show how the FAA doesn't enforce its rules, and how commuter airline safety suffers.\n\nAnd we help you keep abreast of what's happening on the Internet, including tips on how to check the federal budget on line.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-april-the-school-of-hard-data","slug":"1995-april-the-school-of-hard-data","url":"https://dwillis.github.io/uplink/article/1995-april-the-school-of-hard-data.html","year":1995,"month":"April","monthNum":4,"headline":"The school of hard data","kicker":"Use computers for good education stories","author_name":"Penny Loeb","author_title":"","affiliation":"U.S. News & World Report","summary":"This newsletter edition of Uplink explores the potential of computer-assisted reporting on educational issues like school segregation and SAT scores, and highlights reporters’ utilization of electronic data.","keywords":["education","reporting","data"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"If you want to do a computer-assisted story about schools, you have hit the jackpot. Probably no other beat offers such a wealth of data at so little cost. Schools and colleges use data to track spending, and meet state and federal reporting requirements. At U.S. News, we have documented school segregation, inequities in a key federal grant program, and the run-down condition of school buildings. Computerized records were vital to our Dec. 13, 1993 cover story \"Separate and Unequal,\" documenting how special education programs are segregated, rife with misspending, and harming children.\n\nDoing \"Separate and Unequal\"\n\nThe basic data for \"Separate and Unequal\" was the survey of racial makeup of schools, done every two years by the U.S. Department of Education's Office of Civil Rights. It encompasses districts with more than 25,000 students, and a random sample of about a third of the smaller districts. The most recent data is for 1992. Historical data goes back to 1968.\n\nThe survey includes the number of students by race and gender by school. It also has a racial breakdown for those in special education, gifted programs, and those receiving corporal punishment. Journalists should get the additional documentation for the survey, which shows what data is missing. There are some major omissions and quirks.\n\nWe supplemented this data with state records. Connecticut, for instance, includes many more details.\n\nWe used this data to show that black kids are more likely to be in special education than white kids. It also showed that black kids are more likely to be labeled mentally retarded, while white kids get the less stigmatizing label of learning disabled. We found tremendous variation in the type of education for these students. Some are included in regular classrooms. Others spend all day in separate classrooms, which makes them feel inferior.\n\nWe also examined the high cost. Early research shows reimbursement formulas often influence the number of children in special education.\n\nChapter One inequities\n\nThe Johnson administration established Chapter One as the cornerstone of federal education policy. It is supposed to provide money to schools to enhance opportunities for impoverished students. At $6.2 billion a year, it is now the largest single federal school aid program.\n\nThe money is distributed using a convoluted formula inequity. In the end, schools can use the money as they please. Said one Connecticut official: \"Chapter One, that doesn't have anything to do with poverty. It's for reading scores.\"\n\nThe federal government has no central database for where Chapter One money goes. So we asked 42 states to supply us with their data. They all complied amazingly fast - within a week. We then hired a temp to input the numbers, and we check each of the 12,538 entries. We compared the results with three federal databases that show poverty and per pupil spending per district. We found Chapter One money going to schools that were spending as much as $20,000 a student. We found places like Beverly Hills and Scarsdale, N.Y., still get Chapter One funds, though they are wealthy communities with few poor students.\n\nSchool segregation\n\nFor the 40th anniversary of Brown V. Board of Education, we looked at school segregation. We profiled the four districts whose cases led to the landmark Supreme Court decision. We did two kinds of analyses. We produced detailed statistics, and looked at the racial makeup for both the school systems and their communities.\n\nWe also looked at school segregation nationwide, using data that goes back to 1968. Then we followed the methodology of Harvard University Professor Gary Orfield, who recently found that schools were becoming more segregated for the first time since the 1960s. We measured segregation by the percent of blacks in schools that are more than 50 percent minority.\n\nRundown school buildings\n\nWe used three years' worth of the Census Survey of Governments, Education Finance Record, to examine spending for school construction. We combined this with racial and poverty data to show that minority inner-city schools spend less than white suburban districts.\n\nPenny Loeb can be reached at (202) 955-2640, or send email to ploeb@capcon.net.\n\nWhere to get data\n\nFor the Office of Civil Rights racial makeup survey, call (202) 205-9566.\n\nFor reports on special education spending, call the Center for Special Education Finance (415) 493-3550. For poverty, get the 1990 School District Census. Call (202) 219-1731.\n\nFor more on poverty and racial makeup, call for Common Core, (202) 219-1522\n\nFor spending, get the Census Survey of Governments, Education. Finance Record. Call (301) 763-4502.\n\nFor historical racial data, call the National Archives, (301) 713-6630.\n\nThere are countless other databases available. Check the Abstracts of the Summer Data Conference of the National Center for Educational Statistics. Call (202) 219-1522.\n\nThe U.S. Office of Educational Research has a directory of dozens of databases compiled during the last two decades. Call (800) 424-1616.\n\nMany of these databases are in the NICAR library. Call NICAR at (314) 882-0684.\n\nWe used FoxPro to examine the data.\n\nAlso, be sure to get your local school budgets on a computer diskette, or create your own spreadsheet. Look where spending goes up or down. Compare spending on art versus football, or girls' sports versus boys' sports. Get test scores. And collect data on crimes committed in schools - several places have started to gather this information, including Florida and New York City.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-april-whats-affecting-test-scores","slug":"1995-april-whats-affecting-test-scores","url":"https://dwillis.github.io/uplink/article/1995-april-whats-affecting-test-scores.html","year":1995,"month":"April","monthNum":4,"headline":"What's affecting test scores?","kicker":"","author_name":"Rick Linsk","author_title":"Asbury Park (NJ) Press","affiliation":"Asbury Park Press","summary":"A New Jersey analysis of over 2,000 public high schools reveals that poverty is the strongest factor correlating with test scores, prompting investigation after the state stopped producing annual school report cards.","keywords":["poverty","schools","achievement"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"New Jersey encompasses more than 2,000 public high schools, but one factor divides them more than any other: poverty. Nothing else correlated as closely in our school-by-school analysis of test scores. Newspapers from the New York Times to the Philadelphia Inquirer have done similar stories. We took on the task after the state decided it would no longer produce its own yearly \"school report card,\" and lawmakers debated how to meet a court mandate to help poor districts.\n\nGetting state data\n\nThe state still collects much of the data for the report cards: test scores, drop-out rates, enrollment.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-april-go-to-ires-annual-conference","slug":"1995-april-go-to-ires-annual-conference","url":"https://dwillis.github.io/uplink/article/1995-april-go-to-ires-annual-conference.html","year":1995,"month":"April","monthNum":4,"headline":"Go to IRE's annual conference","kicker":"","author_name":"Rosemary Armao","author_title":"executive director","affiliation":"IRE","summary":"The Investigative Reporters and Editors annual conference in Miami (June 8-11) offers intensive computer-assisted reporting training across three tracks covering introductory skills, beat-specific techniques, and specialized skills like data visualization and internet research.","keywords":["conference","training","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","spreadsheet","modem/BBS"],"full_text":"Four days in Florida? Sounds like a junket. But in June? Investigative Reporters and Editors' annual conference at the Intercontinental Hotel in Miami June 8-11 will include panels, discussions and hands-on computer-assisted reporting training at all levels. There won the time for little umbrella drinks. We're offering intense training on the conference's Optional Day, Thursday, June 8: The program is divided into three tracks that will run simultaneously. Each track contains five sessions. The rundown:\n\nAn introduction to computer-assisted reporting\n\n1. Quick-hit story ideas for every section of the paper.\n2. Getting started: Where to get training, what equipment to buy, what software to consider, getting editors interested.\n3. Power Up and Plug In: On-line bulletin boards, government databases not covered in beat sessions.\n4. Introduction to Spreadsheets.\n5. Introduction to Databases.\n\nComputer-assisted reporting on the beat\n\n1. Crime: Cyber sources for crime reporters, plus how to track gun violence in your community and how to find out how bad crime really is.\n2. Education: Cyber sources for school reporters, plus how to rate your school district's performance.\n3. City Hall: Cyber sources for city hall reporters, plus how to ferret out zoning abuses.\n4. Courts: Cyber sources for legal reporters, plus how to rate the performance of judges.\n5. Politics: Cyber sources for political reporters, plus how to use the Federal Election Commission database to track campaign contributions.\n\nSpecial skills\n\n1. Introduction to Internet.\n2. Florida's open government: Electronic records available to journalists.\n3. Polling: How to write survey questions and build on what pollsters provide.\n4. Visualizing information: Mapping, charting, investigative graphics.\n5. Access: Electronic data - What you're entitled to and how to get it.\n\nIn addition, Philip Meyer, author of The New Precision Journalism, will teach a session for those with advanced skills.\n\nInternet demo room\n\nMultiple hookups to the Internet in a room staffed with experts. Find out how to better use this exciting research and communications tool. For information and to register, call IRE at (314) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-april-get-computer-training","slug":"1995-april-get-computer-training","url":"https://dwillis.github.io/uplink/article/1995-april-get-computer-training.html","year":1995,"month":"April","monthNum":4,"headline":"Get computer training","kicker":"","author_name":"Jennifer LaFleur and Brant Houston","author_title":"","affiliation":"NICAR","summary":"NICAR is offering multiple computer-assisted reporting training sessions across the United States in spring and summer, open to all journalists at various skill levels.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"NICAR's U.S. tour continues training around the country. These are open to all journalists:\n\nAdvanced Seminar on Computer-Assisted Reporting\n\nVirginia Press Association\n\nNICAR will conduct training June 1-3, in Richmond, Va. Call Deb Roberts at 804-550-2361.\n\nNICAR and the University of North Carolina School of Journalism will conduct this session at Chapel Hill, N.C., for those who already have some basic computer skills, May 7-12. Call NICAR for more information.\n\nCalifornia AP\n\nNICAR and the Associated Press will conduct training June 19-21 in the Los Angeles area and Sacramento. Call the AP's Daniel Da at (415) 621-7432.\n\nNICAR Bootcamp\n\nNICAR's weeklong intensive training seminar, Aug. 6-11, Columbia, Mo. Call NICAR for more information.\n\nCall NICAR at (314) 882-0684, or send email to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-april-finding-the-toothless-tiger","slug":"1995-april-finding-the-toothless-tiger","url":"https://dwillis.github.io/uplink/article/1995-april-finding-the-toothless-tiger.html","year":1995,"month":"April","monthNum":4,"headline":"Finding the toothless tiger","kicker":"Data shows lax FAA enforcement","author_name":"David Hinchman","author_title":"reporter","affiliation":"Dateline NBC","summary":"An NBC journalist describes a months-long investigation into commuter airline safety using FAA databases, discovering that Mesa Airlines had a decade-long pattern of safety violations, falsified records, and use of substandard parts despite minimal enforcement consequences.","keywords":["safety","database","airlines"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["Enforcement Information System (EIS)","Aviation Safety Reporting System (ASRS)"],"full_text":"The FAA official paused for a long time, especially for prime time television, as he groped for a response.\n\nDateline NBC's David Bloom had just told him, \"Our research shows that for every $100 that the FAA fines a commuter airline they actually collect $12. That doesn't seem to have much teeth.\"\n\nThe moment culminated a long and exhausting data dance with the FAA.\n\nThe story started six months earlier, when we decided to probe commuter airline safety. I knew the FAA's Oklahoma City office kept an inspections database, so I asked for it. They seemed cooperative. I didn't need to file a FOIA. After waiting several weeks, I received an inspections database.\n\nBut I discovered there was no information on violations or fines. I called Jack Price, the FAA point man in Oklahoma City. He said \"Oh, you want to know what they found when they inspected.\" I groaned.\n\nAfter several more weeks, I received a version of the Enforcement Information System (EIS) database. It tracks every fine and violation the FAA dishes out, including the proposed fine and final settlement. There was only one problem: many of the names of the carriers were missing. I called Price again. He told me to call someone in Washington. About 50 phone calls later I discovered Jack had left out the \"violator\" field.\n\nSeveral months went by. But in the meantime, I found valuable information in the EIS database. It included case numbers, dates, locations and fines. I used that to send a number of FOIA letters to get the case files of those fined for the biggest amount. The most important information comes from the FAA field office handling a case.\n\nI soon found something remarkable about Mesa airlines, the largest independent commuter in the country. It seemed the airline went nine months using substandard parts in many of its planes. To make sure no one found out, Mesa falsified records. The coverup unravelled when an engine failed in flight, forcing an emergency landing.\n\nThat wasn't the only time the FAA accused Mesa of flying \"unairworthy\" planes. Cracks, dents, missing rivets and numerous other incidents stretched back 10 years. The database revealed a wide array of problems with maintenance, flight operations and security.\n\nWhen we started talking to pilots, we found near unanimity both at the airline and within the industry that Mesa had serious safety problems. We couldn't convince former Mesa managers to go on camera. Several Mesa pilots did agree to talk if they were disguised. They told viewers about safety problems and predicted a crash if things didn't change.\n\nWe used the Mesa story as part of a larger probe of problems in the commuter airline industry.\n\nWe covered a lot of ground using standard paper trail reporting, but the databases set the story apart. NASA's Aviation Safety Reporting System (ASRS) revealed pilots who fell asleep at the controls. The National Transportation Safety Board's investigations database showed every crash during the past 20 years, including several at Mesa. The FAA Service Difficulty Report (SDR), showed 13 in-flight engine failures at Mesa during a recent two-year period.\n\nWhen I finally got a complete and updated version of the EIS from Price, I compared the proposed fines with the final settlements, and discovered that during the past five years the FAA reduced fines against commuters by 88 percent.\n\nWhen the camera rolled, the FAA representative in charge of commuter airline safety fumbled for a response. After his moment of silence, he said, \"I think the FAA does have teeth. We do surveillance. If we find something that shows us that there isn't full compliance with the regulations or company procedures we take action. I really can't comment on the terms of the fines that are suggested and what the final settlement...\"\n\nWhere to get the data\n\nEnforcement Information System (EIS): Call Jack Price at (405) 954-7156, Federal Aviation Administration Data Processing Division in Oklahoma City, OK\n\nNASA Aviation Safety Reporting System (ASRS): Call Bill Reynard at (415) 969-3969, National Aeronautics and Space Administration, near Menlo Park, CA\n\nNTSB Accident and Incident Investigations Database: call Stan Smith at (202) 382-6538, National Transportation Safety Board in Washington.\n\nDavid Hinchman can be reached at (202) 885-4405.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-april-all-in-the-family","slug":"1995-april-all-in-the-family","url":"https://dwillis.github.io/uplink/article/1995-april-all-in-the-family.html","year":1995,"month":"April","monthNum":4,"headline":"All in the family","kicker":"Finding nepotism in school districts","author_name":"Scott Anderson","author_title":"Reporter","affiliation":"Fort Lauderdale Sun-Sentinel","summary":"An investigation into Broward County's seventh-largest school district revealed millions spent on questionable personnel practices, including widespread nepotism, excessive overtime, and six-figure retirement payoffs, with the series prompting policy changes before publication.","keywords":["nepotism","oversight","database"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["9-track tape"],"full_text":"The Broward County school district - the seventh largest in the nation - has spent millions of dollars in questionable personnel practices. It's happening in a system where nearly one in five employees are related thanks to one of the weakest nepotism policies in the state.\n\nIn addition, substantial overtime was paid to employees and high-ranking administrators were retiring with six-digit golden handshakes because of policies that paid off an unlimited amount of unused vacation and sick leave.\n\nThe series ran May 22, 1994. A preview ran May 13 when district officials made policy changes - primarily because they knew we were about to publish a series about the problems.\n\nWe used the school district's nine-track tapes of all personnel information available to the public. This being Florida - blessed with one of the best open records law in the country - that translated into virtually everything the school district collected about employees except their most recent evaluations and medical history. Then we built our own database and did numerous sorts. That included who were the highest paid employees, who had the most vacation and sick leave saved up and a dozen other categories. The most interesting and productive sort matched everyone who had the same home telephone number, giving us a list of employees (most of whom had the same last name) who were probably related. That gave us the lead on our nepotism story, even though we knew that it did not catch most sibling and parental relationships.\n\nWe've also used computers for other education stories, including an analysis that found teachers who were teaching subjects they were not certified to teach. The data was obtained from the local school board.\n\nScott Anderson can be reached at (305) 356-4590, or send email to flscotta@sunsent.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-april-from-the-nicar-library","slug":"1995-april-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1995-april-from-the-nicar-library.html","year":1995,"month":"April","monthNum":4,"headline":"From the NICAR library","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides journalists access to a comprehensive collection of federal government databases including election contributions, aviation records, mortgage disclosure data, and crime statistics, along with affordable data transfer services and an expanding library of state databases.","keywords":["databases","government","journalists"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n- Federal Elections Commission contributions data, including donations by individuals and political action committees.\n- Federal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\n- Home Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\n- Federal procurement data, 1992-1994, includes breakdowns by agency.\n- Alcohol, Tobacco and Firearms gun dealer records.\n- National Bridge Inventory System data, includes inspection grades.\n- FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders.\n- Social Security death records, by name and social security number, going back to 1937.\n- Occupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\n- Truck Accidents and census data, includes breakdowns by company and road.\nNICAR also offers inexpensive data transfer for journalists, and houses an expanding library of state databases. NICAR is also constantly adding up-to-date data. Call NICAR for up-to-date prices and more information at (314) 882-0684, or send email to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-april-tracking-hospitals-that-mistreat-babies","slug":"1995-april-tracking-hospitals-that-mistreat-babies","url":"https://dwillis.github.io/uplink/article/1995-april-tracking-hospitals-that-mistreat-babies.html","year":1995,"month":"April","monthNum":4,"headline":"Tracking hospitals that mistreat babies","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"","summary":"Investigative journalists at two major newspapers analyzed birth records to expose how hospitals inadequately care for sick and premature newborns, with Ohio hospitals keeping high-risk babies they weren't equipped to treat and New York public hospitals showing significantly higher death and injury rates compared to private hospitals.","keywords":["hospitals","newborns","investigation"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Tracking hospitals that mistreat babies\n\nReporters at two major newspapers recently analyzed birth records to probe how hospitals mistreat newborn infants.\n\nIn their December series \"Babies at Risk,\" Cleveland Plain Dealer reporters Dave Davis and Joan Mazzolini used birth records to show that three-quarters of Ohio's 127 community hospitals have been trying to care for sick and extremely premature babies they weren't qualified to treat.\n\nAccording to regulators, such babies should be transferred to one of the state's 13 highly sophisticated regional medical centers. This was often not happening. According to critics, the blame falls largely on a system where hospitals compete too aggressively for patients.\n\nShortly after the series ran, Ohio's top health official called on lawmakers to give him more power to prevent such practices in the future.\n\nThe New York Times' Dean Baquet and Jane Fritsch, in \"Mismanaged Care: how public hospitals fail,\" compared the city's public and private hospitals when it comes to delivering babies.\n\nA computer analysis of 1993 city and state birth records produced some dramatic numbers. For babies weighing more than 5.5 pounds - the cutoff doctors use as an indication of good health - the death rate for babies born at the public hospitals was 80 percent more than the rate at the private hospitals. Public hospitals also had higher rates of serious birth injuries. The differences remained significant even after Baquet and Fritsch took into account the differences in the health of mothers at the various hospitals.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-april-check-clintons-budget-on-line","slug":"1995-april-check-clintons-budget-on-line","url":"https://dwillis.github.io/uplink/article/1995-april-check-clintons-budget-on-line.html","year":1995,"month":"April","monthNum":4,"headline":"Check Clinton's budget on-line","kicker":"Bits, Bytes and Barks","author_name":"Beth Marchak","author_title":"","affiliation":"Cleveland Plain Dealer","summary":"The 1996 federal budget is now available online through the U.S. Commerce Department's STAT-USA service in searchable text format, accessible via FTP, gopher, and the World Wide Web.","keywords":["budget","online","government"],"topics":["Technology","Government & Politics"],"technologies":["Internet","World Wide Web","FTP","gopher","Adobe Acrobat","PKUNZIP"],"full_text":"The federal budget comes in four cumbersome books or four searchable text files. The choice is yours.\n\nThe 1996 proposed budget and free full-text searching software may be obtained on-line through the U.S. Commerce Department's STAT-USA information service. The documents are available through STAT-USA's Internet service using file transfer protocol (ftp), gopher and the World Wide Web (www) at the following addresses:\n\nThe ftp site is ftp.stat-usa.gov/pub/budgetfy96\nThe gopher site is gopher.stat-usa.gov/budgetfy96\nThe web site is http://www.doc.gov/inquery/budgetfy96\n\nAt the sites, look first for information about Adobe Acrobat, the software that lets you search the text. The accompanying readme file tell how to load the software in Windows. The budget text files require PKUNZIP, and that's provided, too.\n\nFor more information, call STAT-USA at (202) 482-1986.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-april-join-nicar-on-the-internet","slug":"1995-april-join-nicar-on-the-internet","url":"https://dwillis.github.io/uplink/article/1995-april-join-nicar-on-the-internet.html","year":1995,"month":"April","monthNum":4,"headline":"Join NICAR on the Internet","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"Instructions for subscribing to NICAR and Investigative Reporters and Editors listservs to connect with journalists and improve reporting practices.","keywords":["listserve","journalists","subscribe"],"topics":["Journalism & Newsroom"],"technologies":["Internet","Listserv"],"full_text":"Don't forget to keep up with NICAR on the Internet. Subscribe to our listserve, and join as reporters talk about how to do the job better. Email to listserv@mizzoul.missouri.edu In the message, on the first line, write: subscribe nicar-1 your name.\n\nTo join Investigative Reporters and Editors on the Internet, the instructions are the same except, on first line, write: subscribe ire-] your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-crunch-those-numbers","slug":"1995-may-crunch-those-numbers","url":"https://dwillis.github.io/uplink/article/1995-may-crunch-those-numbers.html","year":1995,"month":"May","monthNum":5,"headline":"Crunch Those Numbers","kicker":"Reporters tell how statistics nailed big stories","author_name":"Gwen Carleton","author_title":"staff","affiliation":"NICAR","summary":"Journalists increasingly use statistical analysis and regression techniques to investigate stories and verify claims, moving beyond anecdotes to provide more rigorous and convincing evidence. Examples include New York Times reporting on infant mortality in public hospitals, WCCO-TV's analysis of unwed mother statistics, and other news organizations using data analysis to uncover meaningful patterns.","keywords":["statistics","journalism","analysis"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["SPSS","SAS"],"full_text":"When hospital officials told The New York Times that babies were dying because of unusually sick mothers, reporter Josh Barbanel knew what to do. He ran a regression. And like many reporters who work with statistics, he found story: babies born in New York's public hospitals are significantly more likely to die than those born in private hospitals - even when the babies are born healthy. Statistics have always been a powerful tools in the hands of knowledgeable journalists. Now, as more news organizations invest in computers and statistical software, number crunching is more important. Statistics-savvy journalists are proving that statistics can mean more accurate, incisive reporting. In recent months, Minneapolis' WCCO-TV found mistakes in a state report on unwed mothers. The Raleigh News & Observer showed the link between ex-convicts who keep commiting crimes and drug or alcohol abuse. In December 1993, U.S. News & World Report proved that black students were more likely to be relegated to special education programs.\n\nProving the point\n\nThe New York Times published \"Mismanaged Care: How Public Hospitals Fail\" on March 5-7. The story did not need statistics to get into the paper, Barbanel said. Reporters Dean Baquet and Jane Fritsch had already uncovered scores of horror stories.\n\n\"But (the story had been) argued through anecdotes. Someone who didn't believe unnamed sources could dispute it,\" he said. The numbers made it harder to dismiss.\n\nBarbanel's analysis showed the infant mortality rates were significantly different even after accounting for such variables as prenatal care, the mothers' health and the babies' health. The numbers also allowed the newspaper to question the causes.\n\nBarbanel others have discovered that a careful analysis can quiet controversy and convince skeptics. It could also prevent costly missteps.\n\nWhen WCCO-TV in Minneapolis got a state report on unwed mothers, reporter Alan Cox double checked the numbers with regression analyses.\n\nThe state suggested poor mothers have more out-of-wedlock births. Cox said, \"I couldn't find the correlation, and chose to weed that out of my story.\"\n\nCox said he often runs rudimentary regression analyses to decide how or whether- to report a story. The CBS affiliate also uses them for election returns and exit polls.\n\n\"It's very helpful to have some understanding of sampling techniques,\" he said. \"I think by having statistical training in that area, you avoid going out on a limb on election night.\"\n\nIn television, it's hard to convey numbers. Cox recommends using graphs. And, he said, viewers have been enthusiastic.\n\n\"We get e-mails: Where did you get those numbers?\" he said. \"I think there's a hunger out there, though maybe not an overwhelming one.\"\n\nGoing back in jail\n\nSampling techniques are good for more than just interpreting polls. For a recent story about why ex-convicts keep going back to prison, Jon Schmid, assistant database editor with the Raleigh News & Observer, picked a random sample.\n\n\"We found who had come back to prison about 200 people and we wanted to find out why,\" he said. Schmid worked with a reporter who believed people got back into trouble because of severe drinking or drug problems.\n\nBecause he did not have time to background 200 prisoners, Schmid selected a small sample. He and the reporter checked those prisoners thoroughly. They found many were indeed sent back to prison for drug and alcohol-related crimes.\n\n\"I buy into the statistical theory that using a random sample, you can draw an inference,\" Schmid said. \"I dont think we could have done that story based on anecdotal evidence.\"\n\nSchmid rarely works with correlations and high-end statistics because he works mostly with data based on observations. That makes it difficult to establish cause and effect. But he frequently uses what he calls \"exploratory statistics\" - like cross tabs and graphs to look for trends and unusual occurrences.\n\n\"They help you to look at data, to go beyond making a list,\" he said. \"The tools are a compass that lets me see through the fog of data.\"\n\nCross-tabs at work\n\nSarah Cohen, a St. Petersburg Times business reporter, also is a fan of cross tabs: those tables that test two or more variables at once.\n\n\"They work out well,\" she said. \"They are easier to show to editors because you can see immediately how to tell the story to people.\"\n\nCohen, a former government economist, cautioned that some statistical methods may be better than others.\n\n\"My sense is that people out there can do (high-end statistics) better than we can,\" she said. \"I've used these tools for a long time, and I never saw an economics study you couldn't fight with. Why should we get in on that in newspapers?\"\n\nOn the other hand, professional statisticians are not interested in countless topics, Cohen said. These areas are full of opportunities for journalists, and often require only simpler statistical processes.\n\nInstead of ranking schools based on just one measure, for example, education writers can create an index. Then they can take test scores, spending per pupil, teacher salaries and drop-out rates into account. Journalists can do this quickly, and improve stories significantly.\n\n\"These are tools that help you sort through a huge number of different combinations of things,\" she said. \"Indexes, cross (tools) where its clear what you're talking about when you're finished.\"\n\nSchmid agrees that some analyses are better done on deadline than others.\n\n\"Cross tabs are really straight forward,\" he said. \"(But) when you get into real stats, like taking a sample and so forth, you have to be really careful. It's based on some really strict assumptions.\"\n\nCohen recommends that journalists get comfortable with statistics through skepticism and practice.\n\n\"One thing we're not doing well is being as skeptical of numbers as we are of words,\" she said. \"We need to remember we're not very skeptical and think some more.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-uplink-update","slug":"1995-may-uplink-update","url":"https://dwillis.github.io/uplink/article/1995-may-uplink-update.html","year":1995,"month":"May","monthNum":5,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"Uplink","summary":"This article explores how journalists use statistical analysis and computer-assisted reporting to improve accuracy in covering stories, featuring an example of a young researcher analyzing basketball statistics and discussing how journalists can reduce errors through proper statistical methods.","keywords":["statistics","journalism","reporting"],"topics":["Investigative Methods","Data & Databases"],"technologies":["Internet"],"full_text":"It's like the old Groucho Marx joke: Math is so easy a 12 year old can do it. Quick. Get me a 12 year old. The St. Louis Post-Dispatch did just that. In February, the newspaper recounted how Carrie Coughlin used statistics to figure out the home court advantage in Big Eight basketball. And she checked for significance. For journalists, statistics are one more way to better cover stories. \"Computer-assisted reporting gives greater access to greater volumes of information,\" Philip Meyer said. \"That means there is a greater capacity for making mistakes. By learning how to use statistics, we reduce the risk of getting it wrong.\" In this issue, Uplink offers stories so you get it right. Meanwhile, in the wake of the Oklahoma City bombing, Nora Paul trolls the Internet for militia groups. Uplink also updates government access to electronic records.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-access-tight-for-electronic-records","slug":"1995-may-access-tight-for-electronic-records","url":"https://dwillis.github.io/uplink/article/1995-may-access-tight-for-electronic-records.html","year":1995,"month":"May","monthNum":5,"headline":"Access tight for electronic records","kicker":"","author_name":"David H. Morrissey","author_title":"Professor, Department of Technical Journalism","affiliation":"Colorado State University","summary":"President Clinton's FOIA guidelines and congressional efforts have begun to expand public access to federal electronic records, but significant gaps remain as most government documents will soon exist only in digital form.","keywords":["FOIA","electronic","access"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Access tight for electronic records\n\nBy David H. Morrissey\nColorado State University\n\nPresident Clinton issued Freedom of Information Act guidelines in 1993 that he said would expand public access to federal computerized data.\n\nA year later, Congress initiated on-line access to thousands of federal documents by passing the General Printing Office Electronic Information Access Enhancement Act.\n\nBoth actions were responses to the fact that- within five years - 75 percent of Washington's vast storehouse of documents will be created, stored and disseminated electronically.\n\nWithout electronic access, the public will have little ability to check what Washington does. A key question now is: How much access do we have?\n\nThe answer halfway through the Clinton administration is clearly \"not enough.\"\n\nClinton policies\n\nThe Clinton FOIA policy is the first by a president to specifically mention the public's right to electronic records. The president encouraged agencies \"to enhance public access through the use of electronic information systems.\"\n\nClinton's policy also limits when the Justice Department can defend agencies withholding documents requested under FOIA. Prior to this, Reagan/Bush FOIA guidelines established in 1981 required the Justice Department to defend agencies if there was \"a substantial legal basis\" for secrecy. In effect, agencies were told \"when in doubt, deny requests.\"\n\nClinton changed that. He requires agencies to cite specific \"foreseeable harm\" before the Justice Department will defend them, a policy often called, \"When in doubt, release.\"\n\nUnless this new standard is overturned by a new president, Clinton's policy should eventually increase public access to electronic records.\n\nIn addition, the president ordered in 1993 a review of the security classification system. A draft executive order from this review expands national security information releasable under the FOIA - expanding electronic access in the process - and overturns a Reagan-era executive order encouraging secrecy.\n\nOn April 17, the order was issued, and it expands the amount of information available.\n\nCongress, meanwhile, has twice recently attempted to address electronic records. In 1994, the House killed a Senate bill that would define FOIA-releasable records to include \"computer programs, machine readable materials, and computerized, digitized, and electronic information...\"\n\nThe bill, which may be reintroduced, would require agencies to make reasonable efforts to create or modify computer programs to retrieve records; create computer programs to separate disclosable from non-disclosable information; and provide information in formats specified by requesters.\n\nMeanwhile, in 1994, Congress did pass the GPO Access Act. The GPO now provides on-line access to several thousand government documents.\n\nWhile useful, the act remains fundamentally flawed. Congress left agencies in control of what records will be available electronically - not the requesters. The measure covers only some of Washington's electronic information. No agency is required to release electronic records; their compliance is voluntary.\n\nFew real changes\n\nDespite the president's call for openness, and congressional encouragement, few agencies are rushing into the electronic future. I surveyed 70 federal agencies after the Clinton FOIA policy was announced, and found not one had changed policies to make access to electronic records easier. Some called the president's policy too vague. Others claimed they already practiced maximum disclosure and saw no need to change policies.\n\nDetailed presidential or congressional action may increase public access to Washington's information. Passing the FOIA demonstrated this change. But as responses from the 70 agencies suggest, broad policy statements produce only minimal agency change. Many electronic records remain difficult to obtain. The agencies — not the public — still set most policies guiding release of electronic federal records.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-you-may-have-a-friend-in-pa","slug":"1995-may-you-may-have-a-friend-in-pa","url":"https://dwillis.github.io/uplink/article/1995-may-you-may-have-a-friend-in-pa.html","year":1995,"month":"May","monthNum":5,"headline":"You may have a friend in Pa.","kicker":"The long fight for electronic records","author_name":"Bob Warner","author_title":"Reporter","affiliation":"Philadelphia Daily News","summary":"Pennsylvania's outdated 1957 public records law fails to address electronic access to government documents, as demonstrated by the Philadelphia Daily News's 18-month legal battle to obtain an electronic employee payroll list. Governor Tom Ridge has signaled support for modernizing the law to ensure electronic documents are as accessible as paper records, but the state Senate has resisted such transparency reforms.","keywords":["Pennsylvania","records","transparency"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Whatever's happening in Washington, the information revolution is slower in some state capitals.\n\nIt took 18 months, a lawsuit, and a new Pennsylvania governor for the Philadelphia Daily News to secure an electronic list of 82,034 state employees -- the same information that's been available to the public for years on microfiche at the state library in Harrisburg.\n\nFormer Gov. Robert P. Casey refused to provide a computer tape of the state payroll, with no explanation beyond saying he was satisfied with the old arrangements. The Daily News sued in Commonwealth Court and Casey stalled. He left office January. His successor, Gov. Tom Ridge, settled the case by turning over the computer tape three weeks after taking office.\n\nPennsylvania reporters remain hampered by a 1957 public records law that doesn't mention computers, requiring only that public officials establish \"reasonable rules\" for providing copies of public documents. The Allentown Morning Call's Jack Kraft and David Washburn recently reported on the trouble this has meant for local governments, citizen groups and journalists throughout the state.\n\nRidge said during his 1994 campaign that if a government document is considered public when it's on paper, then the same information should be accessible to the public in electronic formats, technology permitting. State legislator Stewart Greenleaf has sponsored legislation to that effect. But to become law, it needs approval from a state Senate that still denies the public even paper copies of Senate payroll and expense records.\n\nBob Warner can be reached at (215) 854-5885 or send e-mail to bwarner@omni.voicenet.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-from-the-nicar-library","slug":"1995-may-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1995-may-from-the-nicar-library.html","year":1995,"month":"May","monthNum":5,"headline":"From the NICAR library","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides journalists with access to numerous federal government databases covering elections, aviation, mortgages, crime, and other regulatory data for investigative reporting, along with affordable data transfer services.","keywords":["databases","journalism","federal"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nFederal Elections Commission contributions data, including donations by individuals and political action committees.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1994, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records.\nNational Bridge Inventory System data, includes inspection grades.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1994. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nNICAR also offers inexpensive data transfer for journalists, and houses an expanding library of state databases. NICAR is also constantly adding up-to-date data.\n\nCall NICAR for up-to-date prices and more information at (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-get-computer-training","slug":"1995-may-get-computer-training","url":"https://dwillis.github.io/uplink/article/1995-may-get-computer-training.html","year":1995,"month":"May","monthNum":5,"headline":"Get Computer training","kicker":"","author_name":"","author_title":"","affiliation":"Virginia Press Association","summary":"NICAR offers computer training seminars in Richmond, Los Angeles, Sacramento, and Columbia during June and August, with contact information provided for registration.","keywords":["training","computer","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Get Computer training\n\nVirginia Press Association\n\nNICAR will conduct training June 1-3 in Richmond, Va. Call Deb Roberts, (804) 550-2361. California AP NICAR and the Associated Press will conduct training June 19-21 in the Los Angeles area and Sacramento. Call the AP's Daniel Day, (415) 621-7432. NICAR Bootcamp NICAR's weeklong intensive training seminar, Aug. 6-11, Columbia, Mo. Call NICAR, (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-slice-percents-right","slug":"1995-may-slice-percents-right","url":"https://dwillis.github.io/uplink/article/1995-may-slice-percents-right.html","year":1995,"month":"May","monthNum":5,"headline":"Slice percents right","kicker":"Yogi Berra might say 90 percent of journalism is half mental","author_name":"Jennifer LaFleur","author_title":"Staff","affiliation":"NICAR","summary":"This article explains three essential mathematical formulas for computer-assisted reporting in journalism: percent change, percent of total, and per capita, with practical examples to help reporters analyze data effectively.","keywords":["formulas","reporting","data"],"topics":["Investigative Methods","Data & Databases"],"technologies":["spreadsheet"],"full_text":"So you went into journalism to avoid math? For the more nerdy, mathematical formulas may come as naturally as slipping on a pocket protector each morning; however, to those just starting in computer-assisted reporting, math can be intimidating.\n\nI'm not suggesting that to do computer-assisted reporting you have to have an immediate grasp of differential equations, but whether you're looking at hard copy reports from a government agency or analyzing data in a spreadsheet, you should be familiar with three key formulas: percent change, percent of total and per capita.\n\nPercent of total\n\nThis one is easy to get turned around. The rule is: the amount divided by the total (amount/total). For example, if Bob owns 10 animals and two of those animals are cats, 2 divided by 10 (2/10) or 20 percent of his animals are cats.\n\nPercent change\n\nYou're dealing with two years' worth of data and want to figure percent change from one year to the next. Take the difference of the two years divided by the amount from the earlier year. Say we're looking at 1993-1994 data. The difference: 2,567 - 2,345 = 222. Now: 222/2,345 = 0.09.\n\nPer capita\n\nLooking at occurrences by city or county or state can be interesting. But remember to keep things on common ground. You can't compare raw numbers if the \"bases\" are different. That means if I'm looking at the number of murders by city in the United States, Los Angeles and New York probably will come out on top, but only because they have more people. A more useful measure would be per capita murders. In this case, per capita murders would be the number of murders divided by population.\n\nSometimes these measures will be adjusted a little if the number of occurrences is low. Crime statistics, for example often are reported per 100,000 or per 10,000 people. These measures are used because (hopefully) the number of murders in a city is fairly low compared to the number of people. The per capita measure might be 0.0005 - yikes! By looking at the figure per 100,000 people, it becomes easier to read: 50 murders per 100,000 people.\n\nWhen you're trying to figure if persons per household is households divided by population or population divided by households. Do one simple thing: turn your per into a division sign. This means that persons PER households becomes persons/households or population/households. If that doesn't work, trying narrowing the field to a few items: for example if I have two houses and 10 people, how many persons per household is that? Of course, it's five a house.\n\nJennifer LaFleur can be reached at (314) 882-9491, or send e-mail to jenster@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-books-to-help-reporters-work-better-with-numbers","slug":"1995-may-books-to-help-reporters-work-better-with-numbers","url":"https://dwillis.github.io/uplink/article/1995-may-books-to-help-reporters-work-better-with-numbers.html","year":1995,"month":"May","monthNum":5,"headline":"Books to help reporters work better with numbers","kicker":"","author_name":"Jennifer LaFleur","author_title":"Staff","affiliation":"NICAR","summary":"A curated list of recommended books for journalists to improve their numerical literacy and data analysis skills, covering topics from news statistics to mathematical understanding and fact verification.","keywords":["journalism","numbers","literacy"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Here are some books to help reporters work better with numbers:\n\nCohn, Victor. News & Numbers. 1989. Iowa State University Press.\n\nCrossen, Cynthia. Tainted Truth: The Manipulation of Fact in America. 1994. Simon & Schuster.\n\nMeyer, Philip. The New Precision Journalism. 1991. Indiana University Press.\n\nThe Economist Numbers Guide: The Essentials of Business Numeracy. 1993. Penguin Books.\n\nPaulos, John Allen. Innumeracy: Mathematical Illiteracy And Its Consequences. 1988. Hill and Wang.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-when-an-apple-isnt-an-orange","slug":"1995-may-when-an-apple-isnt-an-orange","url":"https://dwillis.github.io/uplink/article/1995-may-when-an-apple-isnt-an-orange.html","year":1995,"month":"May","monthNum":5,"headline":"When an apple isn't an orange","kicker":"","author_name":"Andrew Lehren","author_title":"NICAR staff","affiliation":"NICAR","summary":"Philip Meyer, a Pulitzer Prize-winning pioneer of computer-assisted reporting, shares five statistical techniques journalists can use to strengthen their stories, including adjusting for inflation, controlling for lurking variables, using scatterplots, and applying the chi-square test.","keywords":["statistics","journalism","data"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"Philip Meyer, a founder of computer-assisted reporting and author of The New Precision Journalism, believes there is much more reporters can do with numbers to improve their stories. In a recent Uplink interview, the Pulitzer Prize winner listed several tips:\n\nAvoid apples-and-oranges comparisons. Among the most common, he said, \"is any story where there is a dollar comparison over time.\" In other words, a newspaper may have cost only a few cents at the turn of the century. But before you get misty eyed about the good old days, don't forget to figure for inflation. Reporters can do that getting the Consumer Price Index issued by the U.S. Bureau of Labor Statistics.\n\nWatch out for \"lurking variables.\" Journalists often write about how one thing correlates with another. Meyer believes they should always check for other variables, and see what happens when those other variables are held constant.\n\nFor example, he cited stories that rank states based on Standard Achievement Test scores. The problem is that, in some places, a smaller percentage of students take the SAT scores the college-bound who, overall, post better scores. Elsewhere, a larger percentage of students take the test. By using a simple regression, reporters could adjust for differences.\n\nShow a scatterplot. No math here. This kind of graphic is good for showing relationships. Say you are reporting about how a natural disaster reaped heavier damage in neighborhoods built during a certain period. Peg the vertical (y) axis to the year constructed; the horizontal (x) axis to percent destroyed. Then plot each block on the graph. The scatterplot marks each bit of data effectively. Newspapers don't use scatter blocks enough, Meyer said.\n\nThe chi-square test. What are the odds that something happened by coincidence? The chi-square test is a quick way to find out. A variety of books describe the chi-square test, including Meyer's The New Precision Journalism.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-may-cool-math-tip","slug":"1995-may-cool-math-tip","url":"https://dwillis.github.io/uplink/article/1995-may-cool-math-tip.html","year":1995,"month":"May","monthNum":5,"headline":"Cool math tip","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A guide to adjusting historical dollar amounts for inflation using the Consumer Price Index (CPI), with a simple multiplication method to convert older figures into current dollars while accounting for regional variations.","keywords":["inflation","dollars","CPI"],"topics":["Data & Databases"],"technologies":["gopher"],"full_text":"Cool math tip: Each year a dollar loses value. Here's how to adjust for inflation. Just look at the percent change in the Consumer Price Index (CPI) from one year to the next. Adjust the earlier year's sum \"up\" to reflect them in current dollars. It's a simple multiplication: percent change (expressed in decimal form, not percent) times the older dollar amount. Add that to the older figure. Keep in mind that CPI varies regionally, so use the CPI for your area. The CPI for most areas is available on the Internet at gopher://hopi.bls.gov:70/00hopiftp.dev/special.requests/cpi","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-ires-annual-conference","slug":"1995-may-ires-annual-conference","url":"https://dwillis.github.io/uplink/article/1995-may-ires-annual-conference.html","year":1995,"month":"May","monthNum":5,"headline":"IRE's Annual Conference","kicker":"Find cyber sources on the beat","author_name":"Rosemary Armao","author_title":"Executive Director","affiliation":"IRE","summary":"IRE's 20th annual National Conference in Miami (June 8-11) is offering an Optional Day on June 8 focused on computer-assisted reporting (CAR) across various journalism beats, featuring panels, seminars, and demonstrations led by journalism experts.","keywords":["conference","journalism","computer"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","spreadsheet"],"full_text":"The program is firming up for IRE's 20th annual National Conference set for the Inter-continental Hotel in Miami June 8-11. Here's the program for the Optional Day, June 8, which is dedicated to \"CAR on the Beat.\"\n\nIn addition to the panels listed below, we'll be offering an advanced seminar on statistics with Precision Journalism guru Philip Meyer and Steve Doig of the Miami Herald instructing, a souped-up computer demo room and an Internet Lab.\n\nE-mail us jourire@muccmail.missouri.edu if you'd like to see the entire preliminary schedule. Be sure to sign up for this major journalism event.\n\nTHURSDAY, JUNE 8\n\n9 a.m.-noon Statistics Seminar, Philip Meyer, The New Precision Journalism, and Steve Doig, Miami Herald\n\n9-10:25 a.m.\n\nCar on the City Hall Beat Moderator, Rose Ciotta, Buffalo News David Migoya, Detroit Free Press\nCAR, Quick and Easy NICAR Staff\nAccess: What You're Entitled To and How to Get It\nModerator, Shawn McIntosh, Dallas Morning News Barbara Peterson, First Amendment Foundation Bob Warner, Philadelphia Daily News\n\n10:30-11:55 a.m.\n\nCar on the Education Beat Moderator, Nora Paul, Poynter Institute George Jordan, Newsday Neil Mara, Charlotte Observer\nGetting Started David Milliron, Gannett News Service\nIntroduction to the Internet Beth Marchak, Cleveland Plain Dealer Dan Keating, Miami Herald\n\n1:45-3:10 p.m.\n\nCar on the Courts Beat Moderator, Steve Doig, Miami Herald Nancy Stancill, Charlotte Observer Michelle Quigley, Palm Beach Post\nPowering Up, Plugging In Nora Paul, Poynter Institute\nPolling: How to Write Survey Questions and Build on what Pollsters Provide\nJennifer LaFleur, NICAR John Brennan, Los Angeles Times\n\n3:15-4:40 p.m.\n\nCar on the Crime Beat Jeff Kummer, St. Paul Pioneer Press Mike Berens, Columbus Dispatch Nora Paul, Poynter Institute\nIntroduction to Spreadsheets Rich Gordon, Miami Herald Sarah Cohen, St. Petersburg Times\nFlorida Government: Electronic records Bob Port, Associated Press Scott Anderson, Fort Lauderdale Sun-Sentinel Barbara Peterson, First Amendment Foundation\n\n4:45-6 p.m.\n\nCar on the Politics Beat Bob Warner, Philadelphia Daily News Jackie Duobinis, National Library on Money & Politics\nIntroduction to Databases Shawn McIntosh, Dallas Morning News Richard Mullins, University of Missouri\nVisualizing Information: Mapping, Charts and Graphics\nTed Mellnik, Charlotte Observer Jennifer LaFleur, NICAR\n\nInternet demo room\n\nIn addition, look for the room with multiple hookups to the Internet staffed with experts. Find out how to better use this exciting research and communications tool. For more information and to register, call IRE at (4) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-on-the-internet-when-hate-builds-a-bomb","slug":"1995-may-on-the-internet-when-hate-builds-a-bomb","url":"https://dwillis.github.io/uplink/article/1995-may-on-the-internet-when-hate-builds-a-bomb.html","year":1995,"month":"May","monthNum":5,"headline":"On the Internet When hate builds a bomb","kicker":"","author_name":"Nora Paul","author_title":"","affiliation":"","summary":"The article describes how media organizations, government agencies, and online resources provided coverage and background information about the Oklahoma City bombing, including details on news sites, emergency services, and militia group research materials available on the Internet.","keywords":["bombing","Internet","militia"],"topics":["Technology","Investigative Methods","Crime & Justice"],"technologies":["Internet","World Wide Web","FTP","modem/BBS"],"full_text":"When breaking news hits - and rarely does a story hit harder than the bombing in Oklahoma - the Internet can be your link to the latest news, and the most esoteric background materials.\n\nAlmost immediately, newspapers, radio and television stations around the country created sites with information about the bombing, the survivors, and the victims. A sampling:\n\nMedia organizations\n\nThe Edmond (Okla.) Evening Sun had just started a web site. Soon after the bombing, the paper posted letters to the editor and a photo album. http://www.icon.net/sun/index.html\n\nKBIA Radio News (from University of Missouri) started a special report about the bombing containing audio files of Clinton's and Reno's statements. It offers a comprehensive list of other sources on the bombing and its aftermath. http://www.missouri.edu/-jschool/kbia/bombing/index.html\n\nKOCO-TV in Oklahoma City posted pictures from the blast site, and has been keeping the community informed with food bank, and Red Cross services available for victims. http://www.ionet.net/koco/index.html\n\nThe Raleigh News & Observer has been constantly updating its Oklahoma City bombing report with photos and news stories. Its list of links is invaluable. http://www.nando.net/newsroom/nt/bbextra.html\n\nTime Warner's Pathfinder service leads the pack of news magazines. http://www.pathfinder.com/\n\nGovernment agencies\n\nWhen it comes to covering what should be done in emergencies, several agencies offer useful information:\n\nThe Federal Bureau of Investigation web site describes the government's reward for information about the bombing. It is also the best site for background information on the continuing Unabomber attacks. http://naic.nasa.gov/fbi/\n\nAlso, the Oklahoma State Government Information Server for official state and local reports. http://www.oklaosf.state.ok.us/\n\n(For more information on Internet sources for disaster coverage, including the American Red Cross, Federal Emergency Management Agency and Emergency Preparedness Information eXchange see Nora Paul's Uplink article in the March 1995 edition.)\n\nStalking militia groups\n\nWhen the suspect was caught, and his links to radical anti-government militias were discovered, finding background and information on the militia groups became research priority. Here's where the Internet could help:\n\nMilitia Groups by State: State listing of militia groups, with contact names, addresses, e-mail, and bbs numbers when available. http://www.io.com/user/sdragon/docs/dir3.1\n\nThe Stormfront White Nationalist Resource page includes various tracts (including comments on the Oklahoma City bombing, how the Bible fits with ideology, and works by David Duke), plus links to the Aryan News Agency and a list of like-minded radio shows, including those on AM, FM and shortwave. http://204.181.176.4/stormfront/\n\nPatriot Web's Militia Files: This ftp site includes commentary, directories and profiles on prominent members like Linda Thompson. http://www.tezcat.com/patriot/Militia/\n\nThe ftp site (http://www.tezcat.com/patriot/) \"Patriot\" includes directories of files on topics such as Waco Weaver, Patriotic Justice journals, historical files, and a timeline of individual rights \"violations\" throughout American History. This site provides great (although, obviously, biased) background on the issues that militia groups feel they are dealing with. Another collection of links is at http://www.primenet.com/~lion/data01.html.\n\nThey Fight the Right lists groups that provide research about right-wing organizations, cults, and hate groups. It's compiled by Political Research Associates. There are some organizations which may be useful in locating background information on militias. This is a wonderful resource with contact information and other details. The Terrorism Handbook (which includes a recipe for ammonium nitrite fuel oil bombs) is often pulled from lists almost as soon as its posted. (It is, however, still available in the newsgroup zer.t-netz.pyrotechnik) http://www.qrd.org/qrd/.html/ftr/theyfigh.html\n\nNews Groups to keep an eye on: misc.activism.militia us.events.ok-explosion talk.politics.guns alt.politics.org.batf\n\nNora Paul is a former Miami Herald librarian now at the Poynter Institute. She is the author of Computer Assisted Research, available from the Poynter Institute and NICAR.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-may-bits-bytes-and-barks","slug":"1995-may-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1995-may-bits-bytes-and-barks.html","year":1995,"month":"May","monthNum":5,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Many university journalism programs have not adopted computer-assisted reporting (CAR) training despite its proven value, with barriers including equipment costs, institutional resistance, and teacher shortages. The article highlights how CAR skills benefit journalists and provides examples of successful reporting using online databases and internet research.","keywords":["journalism","reporting","computer"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","Email"],"full_text":"Not every college offers CAR knowledge\n\nDespite advances in newsrooms, many university journalism programs have not yet introduced computer-assisted reporting. In a 1993 survey, less than a third taught on-line searching, and even fewer dealt with databases.\n\nKevin Lee, an assistant communications professor at West Virginia Wesleyan College, conducted a blind survey of more than a dozen experts, asking how to change the situation. The participants included Elliot Jaspin at Cox Newspapers; Penny Loeb at U.S. News & World Report; Steve Ross at Columbia University; and Shawn McIntosh, Dallas Morning News.\n\nThe results are that schools would enhance their reputations, and students would be better journalists, if more universities taught computer-assisted reporting. The hurdles include getting and maintaining good equipment, resistance from school officials, and a lack of qualified teachers.\n\nInternet in a beat\n\nPreston P. Forman, a reporter for the Standard-Times of New Bedford, Mass., recounted two instances where the Internet helped his small newspaper get the story in a hurry.\n\nWhen a television station license was sold, he needed to find someone at the Federal Communications Commission - an agency he had never covered. He went to http://www.fcc.gov, found the agency's internal phone directory, and got the right official quickly.\n\nAnother time, there was a dispute over the lack of accessibility at town buildings for the disabled. He went to http://www.usdoj.gov, and, a few clicks later, got details on what the Americans with Disabilities Act means for municipalities.\n\nArizona TV station seeks CAR reporter\n\nKPNX-TV of Phoenix, Ariz., is seeking a reporter with at least three years experience, including work in computer-assisted reporting.\n\nMore information is available from Jeanine L'Ecuyer, assistant news director, KPNX-TV, P.O. Box 711, Phoenix, Az. 85001; (602) 257-6630.\n\nJoin NICAR on the Internet\n\nDon't forget to keep up with NICAR on the Internet. Subscribe to our listserve, and join as reporters talk about how to do the job better. E-mail to listserv@mizzoul.missouri.edu. In the message, on the first line, write: subscribe nicar-1 your name.\n\nTo join Investigative Reporters and Editors on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.\n\nDon't bank on it\n\nThe Wall Street Journal recently used computer-assisted reporting to show that big banks - despite ads to the contrary - don't lend that much to small businesses.\n\nThe newspaper studied bank filings (known as call reports) to the Federal Deposit Insurance Corp. The computer helped the paper sift through almost 12,000 reports, and help compare small business deals versus overall company loans.\n\nThe newspaper contrasted the numbers with the banks' advertising claims. It also described how small business owners face a tough time getting loans, and how analysts believe the decline of local banking means tougher times ahead.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-may-not-every-college-offers-car-knowledge","slug":"1995-may-not-every-college-offers-car-knowledge","url":"https://dwillis.github.io/uplink/article/1995-may-not-every-college-offers-car-knowledge.html","year":1995,"month":"May","monthNum":5,"headline":"Not every college offers CAR knowledge","kicker":"","author_name":"","author_title":"assistant communications professor","affiliation":"West Virginia Wesleyan College","summary":"Many university journalism programs have not adopted computer-assisted reporting education despite its proven benefits, facing obstacles like equipment costs, institutional resistance, and a shortage of qualified instructors.","keywords":["journalism","technology","education"],"topics":["Journalism & Newsroom","Investigative Methods","Education"],"technologies":[],"full_text":"Not every college offers CAR knowledge\n\nDespite advances in newsrooms, many university journalism programs have not yet introduced computer-assisted reporting. In a 1993 survey, less than a third taught on-line searching, and even fewer dealt with databases.\n\nKevin Lee, an assistant communications professor at West Virginia Wesleyan College, conducted a blind survey of more than a dozen experts, asking how to change the situation. The participants included Elliot Jaspin at Cox Newspapers; Penny Loeb at U.S. News & World Report; Steve Ross at Columbia University; and Shawn McIntosh, Dallas Morning News.\n\nThe results are that schools would enhance their reputations, and students would be better journalists, if more universities taught computer-assisted reporting. The hurdles include getting and maintaining good equipment, resistance from school officials, and a lack of qualified teachers.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-internet-in-a-beat","slug":"1995-may-internet-in-a-beat","url":"https://dwillis.github.io/uplink/article/1995-may-internet-in-a-beat.html","year":1995,"month":"May","monthNum":5,"headline":"Internet in a beat","kicker":"","author_name":"Preston P. Forman","author_title":"reporter","affiliation":"Standard-Times of New Bedford","summary":"A New Bedford newspaper reporter describes how using government websites helped him quickly find sources and information for stories about FCC licensing and disability access compliance.","keywords":["Internet","sources","government"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Preston P. Forman, a reporter for the Standard-Times of New Bedford, Mass., recounted two instances where the Internet helped his small newspaper get the story in a hurry.\n\nWhen a television station license was sold, he needed to find someone at the Federal Communications Commission - an agency he had never covered. He went to http://www.fcc.gov, found the agency's internal phone directory, and got the right official quickly.\n\nAnother time, there was a dispute over the lack of accessibility at town buildings for the disabled. He went to http://www.usdoj.gov, and, a few clicks later, got details on what the Americans with Disabilities Act means for municipalities.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-arizona-tv-station-seeks-car-reporter","slug":"1995-may-arizona-tv-station-seeks-car-reporter","url":"https://dwillis.github.io/uplink/article/1995-may-arizona-tv-station-seeks-car-reporter.html","year":1995,"month":"May","monthNum":5,"headline":"Arizona TV station seeks CAR reporter","kicker":"","author_name":"","author_title":"","affiliation":"KPNX-TV","summary":"KPNX-TV in Phoenix is hiring an experienced reporter with at least three years of journalism experience, including expertise in computer-assisted reporting.","keywords":["reporter","Phoenix","journalism"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"KPNX-TV of Phoenix, Ariz., is seeking a reporter with at least three years experience, including work in computer-assisted reporting.\n\nMore information is available from Jeanine L'Ecuyer, assistant news director, KPNX-TV, P.O. Box 711, Phoenix, Az. 85001; (602) 257-6630.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-join-nicar-on-the-internet","slug":"1995-may-join-nicar-on-the-internet","url":"https://dwillis.github.io/uplink/article/1995-may-join-nicar-on-the-internet.html","year":1995,"month":"May","monthNum":5,"headline":"Join NICAR on the Internet","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR and IRE offer listserve subscriptions where journalists can discuss improving their reporting skills and techniques through online communication.","keywords":["journalists","listserve","networking"],"topics":["Journalism & Newsroom"],"technologies":["Internet","Listserv"],"full_text":"Join NICAR on the Internet\n\nDon't forget to keep up with NICAR on the Internet. Subscribe to our listserve, and join as reporters talk about how to do the job better. E-mail to listserv@mizzoul.missouri.edu. In the message, on the first line, write: subscribe nicar-1 your name.\n\nTo join Investigative Reporters and Editors on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-may-dont-bank-on-it","slug":"1995-may-dont-bank-on-it","url":"https://dwillis.github.io/uplink/article/1995-may-dont-bank-on-it.html","year":1995,"month":"May","monthNum":5,"headline":"Don't bank on it","kicker":"","author_name":"","author_title":"","affiliation":"The Wall Street Journal","summary":"The Wall Street Journal used computer-assisted reporting to analyze FDIC bank filings and reveal that major banks lend significantly less to small businesses than their advertising claims suggest, highlighting growing challenges for entrepreneurs seeking capital.","keywords":["banks","lending","analysis"],"topics":["Investigative Methods","Business & Economy","Data & Databases"],"technologies":[],"full_text":"The Wall Street Journal recently used computer-assisted reporting to show that big banks - despite ads to the contrary - don't lend that much to small businesses.\n\nThe newspaper studied bank filings (known as call reports) to the Federal Deposit Insurance Corp. The computer helped the paper sift through almost 12,000 reports, and help compare small business deals versus overall company loans.\n\nThe newspaper contrasted the numbers with the banks' advertising claims. It also described how small business owners face a tough time getting loans, and how analysts believe the decline of local banking means tougher times ahead.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-june-untapping-dirty-tap-water","slug":"1995-june-untapping-dirty-tap-water","url":"https://dwillis.github.io/uplink/article/1995-june-untapping-dirty-tap-water.html","year":1995,"month":"June","monthNum":6,"headline":"Untapping dirty tap water","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"An investigative series by The Hartford Courant explores the state of drinking water in Connecticut, revealing that a significant portion of the population is supplied by systems with severe safety violations. The story uncovers the failure of small utilities to comply with safety standards, leading to potentially dangerous drinking water.","keywords":["water","safety","violations"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Internet"],"full_text":"Check-up time. Reporters are using computers to probe the medical industry. From the business of health care to dying babies, journalists are using databases for better stories. This month, Uplink focuses on those efforts. The coverage begins on page 4. The Poynter Institute's Nora Paul tells you where to go on the Internet for information on health-care finances. Reporters from Newsday, U.S. News & World Report, Hartford Courant and Tacoma (Wash.) News Tribune offer tips from their work. Those range from examining transplant survival rates to uncovering hospitals that infect patients. In addition, government agencies are beginning to offer certain reports only on-line. Check the back page for two that have started to release information only that way. More parts of the federal government are considering such moves. It's one more sign that computer-assisted reporting is not optional for newsrooms.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-june-missed-you-in-miami","slug":"1995-june-missed-you-in-miami","url":"https://dwillis.github.io/uplink/article/1995-june-missed-you-in-miami.html","year":1995,"month":"June","monthNum":6,"headline":"Missed you in Miami","kicker":"How to order convention tapes handouts","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE conference attendees can purchase cassette tapes of presentations and handouts from the Miami national conference through Sound Images and the IRE resource center. A list of computer-assisted reporting panels available for ordering is provided.","keywords":["conference","cassettes","journalism"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Access","spreadsheet"],"full_text":"If you missed IRE's national conference in Miami, you can still hear what the speakers said. Cassette tapes are available from Sound Images. Each cassette costs $9, and Visa and MasterCard are accepted. Call (303) 693-5511.\n\nIn addition, you can get any handouts distributed at the panels by calling the IRE resource center at (314) 882-2042. Prices vary.\n\nHere is a list of the computer-assisted reporting panels, which were held on June 8, to choose from when ordering:\n\nStatistics Seminar: An advanced three-hour seminar led by Philip Meyer, author of The New Precision Journalism; and Steve Doig, Miami Herald.\n\nCAR on the City Hall Beat.\nCAR, Quick and Easy.\nCar on the Education Beat.\nGetting Started.\nIntroduction to the Internet.\nAccess.\nCAR on the Courts Beat.\nPowering Up, Plugging In.\nPolling: How to Write Survey Questions and Build on what Pollsters Provide.\nCAR on the Crime Beat.\nIntroduction to Spreadsheets.\nFlorida government: Electronic records.\nCAR on the Politics Beat.\nIntroduction to Databases.\nVisualizing Information: Mapping, Charts and Graphics.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-june-from-the-nicar-library","slug":"1995-june-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1995-june-from-the-nicar-library.html","year":1995,"month":"June","monthNum":6,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides journalists with access to a comprehensive collection of federal government databases covering elections, aviation, lending, crime, workplace safety, transportation, and education, along with affordable data transfer services and an expanding library of state databases.","keywords":["databases","journalism","government"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nFederal Election Commission contributions data, including donations by individuals and political action committees.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1994, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records.\nNational Bridge Inventory System data, includes inspection grades.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1994. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Department of Education's Common Core of Data runs from the 1987-88 to the 1992-93 school year. It includes data on school enrollments, finances, staffing and dropouts.\n\nNICAR also offers inexpensive data transfer for journalists, and houses an expanding library of state databases. NICAR is also constantly adding up-to-date data.\n\nCall NICAR for up-to-date prices and more information at (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-june-when-babies-die","slug":"1995-june-when-babies-die","url":"https://dwillis.github.io/uplink/article/1995-june-when-babies-die.html","year":1995,"month":"June","monthNum":6,"headline":"When babies die","kicker":"State and local databases trace a grim trail","author_name":"Sarah Christian","author_title":"NICAR staff","affiliation":"NICAR","summary":"Cleveland Plain Dealer reporters used state birth and death certificate databases to uncover failures in Ohio's system for routing high-risk pregnancies and premature infants to appropriate hospitals, while also revealing disparities in underweight births across low-income communities.","keywords":["databases","hospitals","infants"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["GIS"],"full_text":"Reporters at the Cleveland Plain Dealer and The New York Times, using very different methods and approaches, found infants and pregnant women are at risk of critical injury and death.\n\nThe Plain Dealer used computer databases from the beginning. The reporters said the story would have been impossible to do without them. At The New York Times, computer-assisted methods were the final phase of the reporting.\n\nCleveland reporters Joan Mazzolini and Dave Davis looked at Ohio's system for treating at-risk pregnancies and infants. A state formula gives hospitals designations to ensure that the sickest babies and most at-risk mothers are routed to the best medical care. But the reporters found that the system has broken down and that every year hundreds of extremely ill and premature babies are sent to hospitals that may not have the equipment and expertise to care for them.\n\nUsing birth certificate databases purchased from the state Department of Health, Mazzolini and Davis were able to identify how many premature babies were born in Ohio in 1992 and 1993, where they were born and where they and their mothers were transferred.\n\n\"This really allowed us to question hospitals about treating babies not suited to their designation,\" Davis said. \"The thing we really tried to focus on was looking at hospitals treating premature babies they were not equipped for.\"\n\nDeath certificates were integral in identifying the babies. Ohio law requires that the names on birth certificates be removed before being released to the public, but names are included in death certificates.\n\n\"There was enough matching information to link (the two databases) and figure out names,\" Davis said. \"In matching the birth and death records we identified the babies who had died and were able to find the families. We got some very moving stories. Without the computer we couldn't have done that. Matching the birth and death records allowed us to find real people. Some of the stories were amazing.\"\n\nThe series also showed that low-income communities are prone to underweight babies. \"It's something we always kind of knew,\" Mazzolini said. \"But it's nice to see it on paper.\" Using census information and zip codes from the birth certificates, along with mapping software, the reporters were able to plot where underweight babies are born. The three-month investigation published on Nov. 27, 1994, was \"not as difficult as it looks,\" Mazzolini said. \"Reporters need to find out what is in their state's birth certificates. Ohio's turned out to be a wealth of information. You can get great snapshots of your city, county or your state.\"\n\nDavis advised reporters thinking about tackling a similar story to be ready for a fight. \"Hospitals are very powerful institutions. I've taken on a lot of different types of institutions and people, but I think doctors and hospitals are the most powerful seen,\" he said. \"We had letters coming in threatening to sue us. And they weren't going to us. They were going to the publisher. That is a sinking feeling. You just have to be prepared for it.\"\n\nIn New York, reporters Jane Fritsch and Dean Baquet had already uncovered the horror stories of the city's public hospitals before computer-assisted reporting was employed to analyze mortality rates. In three-part series, \"Mismanaged Care: How Public Hospitals Fail,\" beginning on March 5, 1995, the reporters showed that every year dozens of newborns die or are left to struggle with life-long injuries because they are treated by inexperienced doctors or poorly supervised midwives and nurses in New York City's overcrowded delivery rooms. The story got started when The Times decided to look at how much litigation was taken against the New York Health and Hospitals Corp., a city agency that runs the public hospitals. \"It quickly became clear that there was something going on in the delivery rooms,\" Fritsch said. Court records that allowed the reporters to take a peek at usually-closed hospital records, was central to the investigation, Fritsch said, adding that they used the whole spectrum of document-gathering, from public records to FOIA requests.\n\n\"The computer stuff really came at the end,\" she said. \"I resist depending on the computer and I think some other people resist it because you can end up with stories that report the obvious.\"\n\nFritsch added, however, that the statistical analysis for Mismanaged Care, gave the reporters more information to work with, confirmed hypotheses and \"rounded things out.\"\n\n\"The overall point of the computer study was our ability to put into perspective the claims of the city people and the hospital people who said they had more problems because they have a vastly sicker population,\" Fritsch said. \"It turned out the computer study brought that assertion into question.\"\n\nThe reporters had much anecdotal evidence to back up the story and statistical analysis of infant mortality rates done by Josh Barbanel made the narratives difficult to dismiss. Even accounting for such variables as poor prenatal care and the mother's health, the statistics showed infant mortality to be significantly higher at public hospitals in the city compared to the private institutions. While the computer-generated information gave the reporters more ammunition for their story, Fritsch advised reporters wanting to do similar work to avoid the computer at the outset of the investigation. \"Start with sources, with documents, with real people. Otherwise you could end up with something that is theoretical and distant from the subject,\" she said. \"Use the computer to test hypothesis - but develop the hypothesis first.\"\n\nThousands of patients get sick every year from infections spread inside hospitals mainly from health workers' hands. The Tacoma (Wash.) News Tribune examined 1.7 million records from the state Health Department's Comprehensive Hospital Abstract Reporting System. Thirty-five other states have similar databases. Reporters Adam Berliant and Elaine Porterfield cited several instances where patients died from hospital-borne infections. Even though the health department database did not include patient names, the journalists were able to find those cases by linking the database with a state death certificates. They found likely cases by joining the day, place and cause of death in the two sets of records. The series ran May 28-29, 1995.\n\nAfter finding gaps in a Centers for Disease Control database that extrapolated cancer rates, Newsday built its own database. The newspaper used U.S. Census records to find 75 similar counties - places with large populations and high development. Bob Tiernan, senior editor for projects, said the newspaper surveyed the health departments for those counties. The results were a key part of reporter Ford Fessenden's story on June 27, 1993, which showed Long Island posted one of the biggest cancer rates.\n\nNewsday also used a New York State Health Department database to examine how area hospitals treated breast cancer with surgery. The Sept. 11, 1994, story revealed a vast difference in the rate of lumpectomies versus mastectomies, including one controversial hospital that performed twice as many mastectomies as other facilities in the region.\n\nU.S. News & World Report used the Federal Drug Administration's MedWatch database for a Jan. 9, 1995, cover story that examined drugs with dangerous side effects, including the Norplant birth control implant. U.S. News' Penny Loeb said the database is \"tough to work with,\" but helped yield a powerful story. The magazine also discovered many drugs banned by the FDA are still sold legally over the counter in pharmacies across the country. That, according to the May 15, 1995, story, includes drugs sold at a Rite Aid located inside FDA headquarters.\n\nUsing data from a U.S. Department of Health and Human Services, Hartford Courant reporter Robert S. Capers Jr. tracked survival rates of transplant recipients at area hospitals. He downloaded the data, which was compiled by the United Network for Organ Sharing, into an Excel spreadsheet. The organ-by-organ, hospital-by-hospital analysis ran March 26, 1995.\n\nSarah Christian can be reached at (314) 882-0684, or send e-mail to c636013@showme.missouri.edu","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-june-database-uncovers-abuse","slug":"1995-june-database-uncovers-abuse","url":"https://dwillis.github.io/uplink/article/1995-june-database-uncovers-abuse.html","year":1995,"month":"June","monthNum":6,"headline":"Database uncovers abuse","kicker":"Following the money in medicine","author_name":"Seth Hamblin","author_title":"","affiliation":"University of Missouri","summary":"Journalists in Florida used databases to expose serious problems in Medicare and Medicaid HMOs, including poor access to care and high patient complaint rates, leading to state investigations and potential legislative reforms.","keywords":["healthcare","databases","investigation"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["dBase","SPSS"],"full_text":"Health care is big business, but that doesn't mean it's a well-run business. In Georgia, reporters found medical services were done haphazardly. In Florida, journalists uncovered a Medicare system with severe problems. And in both cases, databases were vital to the story.\n\nThe Fort Lauderdale Sun-Sentinel examined Medicare HMOs, which are supposed to save patients' money. While these HMOs cover all medical expenses, as opposed to the standard Medicare rate of 80 percent, Medicare HMOs can still be a gamble.\n\nThe newspaper found some South Florida Medicare HMOs are half as likely as others to provide access to doctors and hospital services. Seven of these HMOs posted high rates of patients quitting their programs. In some cases, twice as many patients were leaving as were joining. More than 10,800 patients and their families complained about the care to state and federal officials - about the HMOs' not paying their medical bills.\n\nReporters Fred Schulte and Larry Keller requested the Beneficiary Information Tracking System from the U.S. Health Care Financing Administration, but received little cooperation.\n\n\"We asked for a tape, but they said, 'Sorry, we can't send it to you that way. We'll send you a hard copy,'\" Schulte recalled. The reporters took the paper records and entered them into a dBase database by hand before analyzing the information with SPSS statistical software.\n\nSchulte said problems with Florida's access laws often leave him building his own databases. \"Even when I find a database, they won't give it to me,\" he said.\n\nThe work led to a series on Nov. 7-11, 1993, \"The HMO Maze: How Medicare Fails Seniors.\"\n\nA year after the stories, Schulte teamed with Jenni Bergal for another investigation, \"Profits From Pain,\" on Medicaid HMOs, which ran Dec. 11-15, 1994. Again, the reporters created their own database. This time the records came from the state's HMO peer review board for all hospitals.\n\n\"They just collected the data, put it in a box and left it sitting there gathering dust,\" Schulte said. The reporters became the first to analyze what was in those boxes.\n\nThe results showed Medicaid HMOs had dangerously lower standards than private HMOs. At the same time, many Medicaid HMOs were spending much more on administrative and non-medical costs than private HMOs.\n\nThe series also showed that 12 of the 15 Medicaid HMOs got \"poor\" and \"very poor\" reviews when it came to caring for children and pregnant women. These HMOs covered almost all of the 25,000 patients enrolled in Florida's Medicaid system.\n\nIn addition, the journalists found nearly 200 of the 300 Medicaid HMO doctors and medical centers failed to meet standards for treatment or proper documentation.\n\nWhile building the database took the reporters two to three weeks, the drudgery was worth it, Schulte said. After the series, a state investigation ensued, and Florida is considering legislation to crack down on health-care abuses.\n\n'In Sickness And In Wealth'\n\nIn Georgia, meanwhile, The Atlanta Journal-Constitution found that health care doesn't run by the laws of supply and demand. Instead of medical need, costs are rising because of the number of doctors and hospitals beds.\n\nThe newspaper analyzed more than 600,000 health insurance claims and found a wide fluctuation in the amount of surgery and hospital care doled out from area to area. It found a seemingly haphazard delivery of medical services and published the results Sept. 30-Oct. 4, 1990.\n\nIn east Georgia, for instance, policy holders were twice as likely to undergo coronary angioplasty, where a balloon-like device is used to widen arteries. There happen to be more cardiologists in east Georgia than in the rest of the state.\n\nThe variation in treatment by itself doesn't prove that care is rationed in some places or wasted in others, the Journal-Constitution contended. But the newspaper cited experts who said the gaps represent billions of dollars in unnecessary procedures.\n\nHal Straus, who wrote the stories with Mike King in late 1990 as part of the series \"In Sickness and In Wealth,\" said the idea came from earlier academic studies on prostate surgery rates in Connecticut.\n\n\"The studies showed a lot of variation, and we wanted to see if the same thing was going on in Georgia,\" Straus said.\n\nTo test the theory, Straus and King needed data from a handful of different sources. They obtained demographic data on Medicare enrollees, as well as data on the procedures and hospitalization days for these enrollees, from the Georgia Medical Care Foundation. The foundation reviews Medicare spending in Georgia under a contract with HCFA.\n\nFor similar data on privately insured patients, Straus and King turned to Blue Cross and Blue Shield, the state's largest insurer.\n\nThey fed nine-track tapes from both sources into the newspaper's mainframe computer and, using SAS statistical software, compared it with data on physician and hospital bed supply for various parts of the state. That data came from the U.S. Health Resources and Services Administration and the Georgia Hospital Association.\n\nThe resulting story shows gaping holes in the state's system of determining when to use medical procedures. Nevertheless, there has been little fallout from their study, Straus said. \"We did the story a couple of years too early,\" he said. \"It would have meant a lot more when there was a big push for health-care reform. But maybe people are more likely now to say to their doctor, 'I don't want this surgery unless it's absolutely necessary.'\"\n\nFor more information, call Hal Straus at (404) 526-5348, or send e-mail to nal_straus@ajc.com. Fred Schulte can be reached at (305) 356-4591, or send e-mail to ferds@aol.com.\n\nSeth Hamblin, a student at the University of Missouri, is interning this summer at The Washington Post.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-june-organ-transplants-tracked-on-line","slug":"1995-june-organ-transplants-tracked-on-line","url":"https://dwillis.github.io/uplink/article/1995-june-organ-transplants-tracked-on-line.html","year":1995,"month":"June","monthNum":6,"headline":"Organ transplants tracked on-line","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The United Network for Organ Sharing, a federally funded non-profit organization, maintains a comprehensive online database providing public access to organ transplant data including donation statistics, survival rates, and waiting list information.","keywords":["transplants","database","donations"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"The United Network for Organ Sharing, a federally funded non-profit that tracks transplant surgery, offers an Internet site filled with data. It includes information on organ donations, survival rates and waiting lists. The site is http://www.infi.net/~shreorg/unos.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-june-medical-coverage-sites","slug":"1995-june-medical-coverage-sites","url":"https://dwillis.github.io/uplink/article/1995-june-medical-coverage-sites.html","year":1995,"month":"June","monthNum":6,"headline":"Medical coverage sites","kicker":"On the Internet","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute","summary":"A curated directory of online resources for covering health-care insurance and Medicare policy, including government databases, statistical reports, historical documents, and relevant case studies.","keywords":["Medicare","insurance","healthcare"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"The health-care insurance debate continues. Here's an array of sites with background information, key historical documents, statistical reports, source lists, and database sellers, that can help you cover this complex and critical issue. The Health Care Financing Administration's site is full of useful background material. The text to the 1995 Medicare Handbook is there, covering topics such as \"What is Medicare?\" \"What Medicare Does not Pay For,\" \"How to Appeal Medicare Decisions,\" and \"Events That Can End Your Medicare Protection.\" There are also other pamphlets, and a helpful listing of commonly asked questions about Medicare (good for sidebars). There are statistical tables available by state and county, such as \"Standardized per capita rates of payment aged and disabled - parts A and B,\" which allows quick comparisons between regional Medicare payments. Speeches and press releases from the agency, and links to other sites with information related to health care, round out the service. Put this on your bookmark list. http://www.ssa.gov/hcfa/hcfahp2.html\n\nThe Census Department's Statistical Abstract of the United States lists percentages, by state, of people without health insurance for 1990-1992. http://www.census.gov/stat_abstract/hlthtb.pm\n\nThe document http://www.census.gov/stat_abstract/health.gif is a color coded map indicating levels of non-insured persons.\n\nThis very ugly address leads to a very pretty application, a searchable \"Guide to Sources\" of other data from the Census Bureau, other Federal agencies, and private organizations. Type \"medicare\" into the query box and get a leads list to places you should check for more information. http://www.census.gov/cgi-bin/htgrep file-source.txt&style=pre\n\nA press release from President Clinton, dated Oct. 27, 1993, discusses the Medicare plan for older Americans. http://docs.whitehouse.gov/white-house-publications/1993/10/1993-10-27-the-health-plan-appendix-i.text\n\nText of the Health Security Act of 1993 is a key background document. http://sunsite.unc.edu/nhs/executive/x-Summary-toc.html\n\nThe Bipartisan Commission on Entitlement and Tax Reform Interim Report to the President, August 1994: Finding No. 4 discusses the impact of increased health care costs. Links to a graphic showing a timeline of \"What our future looks like - if we fail to act,\" illustrates that by 2030, Medicare, Medicaid, Social Security, and federal employee retirement will consume all federal tax revenue. http://www.charm.net/~dcarolco/interim/interim.html\n\nMeanwhile, Arent Fox Kintner Plotkin & Kahn, a law firm in Washington D.C., has put together an interesting list of Medicare-related cases it has worked on. Although this is obviously a promotion for the firm, the types of cases might be an interesting read. http://www.arentfox.com/e1.4.html\n\nFor one interest group's interpretation of the health-care financing situation in America, check out the International Society For Individual Liberty pamphlet called, \"We Can Have Affordable Health Care.\" It's worth visiting just to see the graphic of the doctor coming at you with stethoscope. http://w3.ag.uiuc.edu/liberty/isil/healthcare.html\n\nHCIA Inc., a health-care information company, harbors an interesting inventory of health-care related databases, including order forms for data on 6,500 U.S. hospitals, 16,000 nursing homes, and more than 2.5 million detailed patient records. It also includes links to the Centers for Disease Control, which also sells data. http://www.hcia.com/index.html\n\nThe compilers at Global Network Navigator have put together a wonderful list of sources on a variety of health related topics. It links to the best sites for background on alternative medicine, disabilities, diseases and conditions, general health, mental health, nutrition, professional medicine, and health organizations. http://nearnet.gnn.com/wic/med.toc.html#policy\n\nFinally, for news groups, look for talk.politics.medicine\n\nLet me make a little pitch here for Infoseek. For $9.95 a month (first month free) you get access to a powerful free-text search service. You can search the text of thousands of messages groups by any word. A great way to find people who are talking about a particular topic. To find out more about Infoseek (no, I'm not a stockholder nor do any relatives work for them) go to http://www.infoseek.com.\n\nIf you have suggestions for this column, email Nora Paul at npaul@poynter.org. Visit the Poynter Institute's website at http://www.nando.net/prof/poynter/home.html. It features Paul's Hot News/Hot Research in the Poynter Institute's library section. It takes news stories and links you to some good sources for information to cover that story. To go directly to the page: http://www.nando.net/prof/poynter/hrintro.html\n\nNora Paul is a former Miami Herald librarian now at the Poynter Institute. She is the author of Computer Assisted Research, available from the Poynter Institute and NICAR.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-june-bits-bytes-and-barks","slug":"1995-june-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1995-june-bits-bytes-and-barks.html","year":1995,"month":"June","monthNum":6,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Government agencies including the Census Bureau and FDIC are increasingly making reports and databases available exclusively online to save money and distribute information faster, while journalism organizations like NICAR and IRE encourage reporters to use internet resources and collaborative listservs for computer-assisted reporting.","keywords":["Internet","data","reporting"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["Internet","World Wide Web","CD-ROM","CSV","Quattro Pro"],"full_text":"You're on the 'Net or you're nowhere\n\nThe U.S. Census Bureau recently started making some of its reports available only on the Internet. The agency's government division began this year by issuing government finance reports only on-line. That included surveys on state and local government finances, employment and retirement systems. Reporters were also sent short summary sheets. The key reason, according to the Association of Public Data Users, is to save money and get reports out faster. Census may issue some of that information later on CD. The Internet address is http://www.census.gov.\n\nIt's not the only agency to take that step. Among the others is the Federal Deposit Insurance Corp. Its quarterly Statistics on Banking is available only on the Internet. So is its Historical Statistics on Banking, 1934-1993. It is also in the process of putting massive databases on its site. The Internet address is http://www.fdic.gov.\n\nJoin NICAR on the Internet\n\nDon't forget to keep up with NICAR on the Internet. Subscribe to our listserve, and join as reporters talk about how to do the job better. E-mail to listserv@mizzoul.missouri.edu. In the message, on the first line, write: subscribe nicar-1 your name.\n\nTo join Investigative Reporters and Editors on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.\n\nSt. Pete hires new reporter\n\nThe St. Petersburg Times recently hired Brad Goldstein to become its lead computer-assisted reporter. Goldstein, a recent Neiman fellow, had worked at the Lawrence (Mass.) Eagle-Tribune.\n\nBack to school CD\n\nThe National Center for Education Statistics has made its data even more user-friendly with a new CD-ROM of the Common Core of Data, 1987-88 through 1992-93. This combines all previously separate data on school enrollment, finances, school district census, staffing and dropouts. It even has Chapter 1 funding, which hadn't previously been available from the agency.\n\nThe CD runs in Windows and DOS, and is simple for looking up individual schools and districts. Downloading data for a single district or state is also easy if done as a CSV file. Downloading the entire country might be more difficult.\n\nNeil Mara of the Charlotte Observer used the first version of this CD to do stories on school segregation last year. He cautioned against using the minority percentage calculated on the disk; the enrollment numbers by race seem to be accurate, though. The CD is free right now. Order by calling John Sietsema at (202) 219-1335.\n\nWorking overtime with a spreadsheet\n\nThe Waterbury (Conn.) Republican-American used a Quattro Pro spreadsheet to examine city payroll records. The analysis by reporter Stephanie Reitz showed dozens of police officers and firefighters nearly doubled their salaries with overtime pay; a legacy of staffing shortages.\n\nThe April 23, 1995, story also showed how a 1986 state law designed to attract top teachers - a program later stripped of state funding - meant some of the city's highest salaries went to teachers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-july-uplink-update","slug":"1995-july-uplink-update","url":"https://dwillis.github.io/uplink/article/1995-july-uplink-update.html","year":1995,"month":"July","monthNum":7,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This journalism guide addresses the challenges of covering campaign contributions through computer-assisted reporting, offering practical advice from experienced reporters on digitizing paper records, using the FEC database, and analyzing contribution data.","keywords":["contributions","reporting","campaigns"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["Internet"],"full_text":"Covering campaign contributions is a staple of computer-assisted reporting. But it's tricky.\n\nA lot of states do not offer computerized donation databases. The Federal Election Commission's online service is not perfect. And there is always the need to go beyond the numbers for hard-hitting stories. This issue helps you with these problems.\n\nTwo Associated Press reporters tell how they computerized state election records available only on paper. One used scanners. The other arranged a massive data entry team. Both got good stories. Padraic Cassidy of the Los Angeles Times (and a former NICAR staff member) tells how to use the FEC on-line database.\n\nChristopher Schmitt of the San Jose Mercury News and Jackie Duobinis from the National Library on Money & Politics offer tips on analyzing contribution records.\n\nAnd the Poynter Institute's Nora Paul offers tips on how to use the Internet to better cover campaigns.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-july-federal-campaign-reports-that-dont-add-up","slug":"1995-july-federal-campaign-reports-that-dont-add-up","url":"https://dwillis.github.io/uplink/article/1995-july-federal-campaign-reports-that-dont-add-up.html","year":1995,"month":"July","monthNum":7,"headline":"Federal campaign reports that don't add up","kicker":"","author_name":"Padraic Cassidy","author_title":"","affiliation":"Los Angeles Times","summary":"Dedicated to improving coverage of campaign contributions, this issue includes experiences from journalists using technology to handle non-digitized state election records, tips on navigating federal databases, and leveraging internet resources for political reporting.","keywords":["Campaign Contributions","Database","Reporting"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Direct Access Program"],"full_text":"From the campaign treasurer to your computer's hard drive, data from the Federal Election Commission follows a tortured route. The paper records are filed first with the Clerk of the House or the Senate, who photocopies them and send them to the FEC on microfilm. The FEC then makes paper copies from the microfilm, keeps a copy for public view and sends a copy out to be keypunched. That data then is checked and made available to the Direct Access Program users. The good news is that the FEC makes a mammoth effort to have data on thousands of committees available as soon as possible after the reporting period deadlines. The bad news is that the smallest keypunch errors can wreak havoc on a news story. \n\nRecently a problem cropped up in the amounts each candidate was supposedly contributing to their own campaign: For a while, the records showed Senate hopeful Michael Huffington giving his campaign $2 million more than he actually did, although that may have been hard to see in a campaign where he chipped in $28 million of his own money. \n\nBut take the case of Kyle McSlarrow in his losing campaign for Virginia's 8th district. Just two keypunch errors accounted for a $3 million mistake in a campaign where he spent less than $700,000. The FEC is working to correct the errors and the best advice is to sum all figures and check, check, check those totals. \n\nThe majority of the useful information and the type most easily accessible for stories on deadline-hangs out in two major areas: The \"5\" report section for PACs, party committees and other committees, and the \"1\" report section for individual donors. The FEC supplies totals for almost every search.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-july-best-votes-money-can-buy","slug":"1995-july-best-votes-money-can-buy","url":"https://dwillis.github.io/uplink/article/1995-july-best-votes-money-can-buy.html","year":1995,"month":"July","monthNum":7,"headline":"Best votes money can buy","kicker":"","author_name":"Christopher H. Schmitt","author_title":"","affiliation":"San Jose Mercury News","summary":"A regression analysis of California's 1993-94 legislative session demonstrates that campaign contributions are strong predictors of lawmakers' votes on individual bills, even though direct causation between money and votes is difficult to prove without explicit quid pro quo evidence.","keywords":["contributions","legislation","analysis"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"It's the ultimate question when journalists focus on campaign cash and elected officials' behavior: Can you prove that the money directly influences votes on particular bills?\n\nUltimately, no, unless officials are caught asking for money in exchange for votes. But short of that, we recently used a regression analysis to show campaign contributions are often a strong predictor not only of aggregate behavior, but of individual votes on individual bills.\n\nThe analysis, which focused on 10 major bills from California's 1993-94 legislative session, showed that the more money individual lawmakers took from interests pushing a par-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-july-scan-for-the-facts","slug":"1995-july-scan-for-the-facts","url":"https://dwillis.github.io/uplink/article/1995-july-scan-for-the-facts.html","year":1995,"month":"July","monthNum":7,"headline":"Scan for the facts","kicker":"Milking campaign funds","author_name":"Rich Eggleston","author_title":"","affiliation":"Associated Press","summary":"The Associated Press Wisconsin bureau used optical scanning technology and custom software to convert paper campaign finance reports into an electronic database, enabling computer-assisted reporting on political fundraising that would have been impractical to analyze manually. The successful project led to the creation of CampaignWatch, a company that sells the scanning and data entry package.","keywords":["scanning","fundraising","database"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["R:BASE","DataPen","FoxPro"],"full_text":"The Associated Press bureau in Wisconsin showed that the lack of an electronic database is no obstacle to good computer-assisted reporting.\n\nThe bureau wanted to track campaign fundraising by Gov. Tommy Thompson and other state politicians. Wisconsin, like some other states, does not store that electronically, it only requires candidates to file on paper. The longer reports - like Thompson's almost defy manual analysis.\n\nOur solution: a scanner.\n\nWe took the step last year, after Nick Schiavo, a former Air Force computer expert, asked how a computer could solve problems with government records. I told him about campaign reports, and how few news organizations spend the time or money inputting the information.\n\nSchiavo believes the latest scanning technology is faster and more accurate than punching it in. He wrote a program to accommodate the state forms. He customized optical character recognition and data entry systems. Microsoft and Microrim R:BASE software runs the programs. He scanned in the first reports using DataPen, a hand-held scanner, but later ones were scanned in more efficiently with a sheet-fed scanned. He dumped the information into FoxPro. The endeavor went so well that Schiavo created his own small company, CampaignWatch, which sells the package.\n\nEach computerized record included the page number from the paper report. We traced back anything that looked funny. The scanner did turn one $1,200 contribution into a $120,000 contribution, but we used the page number to catch the error. There were typos, but the product was remarkably accurate. When one male contributor was listed in the database as a housewife, we went back to the paper records and found nothing wrong - the campaign listed him that way.\n\nOur bureau produced stories on business executives backing the incumbent, and a challenger relying on support from one county. We also examined cross-giving to both candidates. I found contributions timed with a special legislative session that loosened state telecommunications regulation, and noted how accountants and builders said \"Thank You\" with their checkbooks for laws that benefited their professions.\n\nSchiavo plans to upgrade his package as optical scanning equipment improves. Some fonts are too difficult to scan in, and handwritten documents are still too tough. He is also starting to cross-reference the state contribution data with Federal Election Commission records. As our governor pushes legislation for the next four years, I'll keep checking his moves against the list of political I.O.U.'s in my hard drive. Send e-mail to Rich Eggleston at richeggl@ix.netcom.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-july-how-to-check-campaign-coffers","slug":"1995-july-how-to-check-campaign-coffers","url":"https://dwillis.github.io/uplink/article/1995-july-how-to-check-campaign-coffers.html","year":1995,"month":"July","monthNum":7,"headline":"How to check campaign coffers","kicker":"","author_name":"Jackie Duobinis","author_title":"","affiliation":"The National Library on Money & Politics","summary":"This article provides practical guidance for journalists investigating campaign contributions by examining patterns, timing, and connections between donors and candidates to uncover potential conflicts of interest or improper influence.","keywords":["contributions","campaign","investigation"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Here are some tips to remember when you examine campaign contributions.\nCheck campaign records for patterns. Sort contributions chronologically to spot fundraising events and bundling. Sort by employer to also check for contribution bundling. Sort by last names to find family members and multiple contributions from an individual.\nAs always, stop and ask why? Why are there 50 contributions on the same day? Why did every executive from Jones & Johnson give? Look for people at corporations who would not normally be giving, like secretaries and assistants. Were they reimbursed or pressured?\nLook for coincidences. Does the candidate own stock in a corporation whose PAC is contributing? Is there a bill sponsored by that legislator that affects the individual or corporation?\nTake a look at the paper trail. Even though looking at computerized campaign finance records is important, some things aren't computerized. Check the disbursements on Schedule B. Track payments to consultants and companies. Check for loans on Schedule C. Look at the detail of debt on Schedule D.\nRemember this is only a start. The contributions or expenditures point you in the right direction. Rarely are they the whole story.\nJackie Duobinis can be reached at (202) 857-0318.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-july-the-new-yorker-probed-how-media-and-telecommunication-conglomerates","slug":"1995-july-the-new-yorker-probed-how-media-and-telecommunication-conglomerates","url":"https://dwillis.github.io/uplink/article/1995-july-the-new-yorker-probed-how-media-and-telecommunication-conglomerates.html","year":1995,"month":"July","monthNum":7,"headline":"The New Yorker probed how media and telecommunication conglomerates","kicker":"","author_name":"","author_title":"","affiliation":"The New Yorker","summary":"The New Yorker investigated how media and telecommunications conglomerates use campaign contributions and lobbying to influence Congressional decisions on communications and tax policy. The Center for Responsive Politics is launching an internet site to track this influence and make campaign finance data publicly available.","keywords":["lobbying","Congress","influence"],"topics":["Government & Politics","Business & Economy","Investigative Methods"],"technologies":["gopher","Telnet","Internet"],"full_text":"The New Yorker probed how media and telecommunication conglomerates, using campaign contributions and lobbying efforts, are trying to sway Congress on federal communication and tax laws. The June 5 story by Ken Auletta, called \"Pay Per Views,\" examined cross-giving and other patterns as executives seek to sway policy.\n\nThe Center for Responsive Politics plans to launch an Internet site this month. The address will be gopher.crp.org. The center also plans to make data available via a telnet site. The center publishes a bimonthly newsletter, Capital Eye. Subscribe by calling (202) 857-0044.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-july-theyre-in-the-money","slug":"1995-july-theyre-in-the-money","url":"https://dwillis.github.io/uplink/article/1995-july-theyre-in-the-money.html","year":1995,"month":"July","monthNum":7,"headline":"They're in the money","kicker":"On the Internet","author_name":"Nora Paul","author_title":"librarian","affiliation":"Poynter Institute","summary":"A guide to Internet resources for covering money and politics, featuring websites that track campaign finance, PAC contributions, and political activism, along with tools for accessing White House communications and presidential campaign information.","keywords":["campaign","finance","politics"],"topics":["Government & Politics","Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","gopher"],"full_text":"There are numerous sources on the Internet that may help you cover money and politics. Here are some highlights.\n\nNewt's PAC Money, brought to you by NewtWatch, is a beautiful example of data collection and presentation on the web. Click on the colorful pie chart divided by special interest sectors and see the breakdown of contributions and then link to a listing of organizations and their exact amounts of contribution. If only every elected official had such a page created about them. http://www.cais.com/newtwatch/pac$indx.html\n\nCharlotte's Web Government Reform Issues Page is compiled by \"United We Stand,\" the political action group that grew out of Ross Perot's independent bid in the 1992 presidential race. The listing of documents and websites contains information about federal budgeting and campaign financing. There is potential material for background and sources. http://www.emf.net/-cr/govreform.html\n\nInteractive activists\n\nWe the People focuses on campaign finance reform, single-payer health care, and environmental issues incinerators). The site has articles on the group's platform, audio files from radio programs and other materials. http://hoek.cchem.berkeley.edu/\n\nWashington Citizen Action describes itself as a national federation of state citizen organizations with a combined membership of more than 2.5 million Americans. The group is an advocate in Washington, D.C., for medical reform, environmental clean-up, and campaign finance reform. There isn't very much on its pages right now, but this is a group that would be good to have on your rolodex. http://www.cyberquest.com/~wca/welcome.html\n\nJustin's Political Infopage links to activist groups on the Net and \"Nonpartisan Resources\". http://www.cs.cmu.edu/afs/cs/user/jab/mosaic/politics.html\n\nNetworked Government Resources isn't anything specifically about campaign finance, but it is a great page of links to presidential, congressional and other federal offices. http://www.ai.mit.edu/projects/iiip/government.html\n\nCongressional Quarterly offers a wide variety of information about Capital Hill. gopher://gopher.cqalert.com\n\nC-Span offers a simple list of PAC contributions to federal candidates. It is located in the Government Resource Center. Get to C-Span through the Library of Congress' gopher at marvel.loc.gov\n\nTake a run at the White House\n\nFind out what President Clinton and his aides have said in speeches, press briefings and interviews about campaign finance reform. http://docs.whitehouse.gov/white-house-publications?election-campaign\n\nProject Vote Smart offers a phone directory for all the presidential campaigns. http://www.peak.ork/vote-smart\n\nOn the Road to the White House says, \"In preparation for the 1996 Presidential Voting Season, Internet Publishing Technologies is providing a collection of candidate information and links to various government and party related sites throughout the Web.\" Put this one on your bookmark, folks. They've got all the spin doctors - I mean sites - covered. http://www.ipt.com/vote/\n\nIn California\n\nThe Great Money Chase page links to a detailed study of campaign financing in California with charts and statistics from the national and state races. The study by Kim Alexander, with research assistance provided by Brian Tanner and Tracey Norwood, was published by California Common Cause in April. (For paper copy of the report contact Common Cause, 926 J Street, Suite 910. Sacramento, Calif.; (916) 443-1792, comcause@netcom.com.) http://www.uwsa.com:8972/uwsa/issues/cfr/moneychase.html\n\nThe California Campaign Finance Reform Page claims it contains a general discussion of campaign finance reform topics. applicable to any level of government anywhere in the United States. It also contains campaign finance information and potential solutions which are specific to California.\" There are some great key readings and background materials here. http://www.uwsa.com:8972/uwsa/issues/cfr/\n\nIf you have suggestions for this column, e-mail Nora Paul at npaul@poynter.org. Visit the Poynter Institute's website at http://www.nando.net/prof/poynter/home.html. It features Paul's Hot News/Hot Research in the Poynter Institute's library section. It takes news stories and links you to some good sources for information to cover that story. To go directly to the page: http://www.nando.net/prof/poynter/hrintro.html\n\nNora Paul is a former Miami Herald librarian now at the Poynter Institute. She is the author of Computer Assisted Research, available from the Poynter Institute and NICAR","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-july-bits-bytes-and-barks","slug":"1995-july-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1995-july-bits-bytes-and-barks.html","year":1995,"month":"July","monthNum":7,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A journalism newsletter highlighting resources for reporters, including government bulletin boards for tracking piracy and organ transplant data, Commerce Department's consolidated STAT-USA website, and online communities for investigative journalists.","keywords":["piracy","data","reporting"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["modem/BBS","gopher","FTP","spreadsheet","World Wide Web","Internet"],"full_text":"Bits, Bytes and Barks\n\nTracking pirates\n\nA speedboat with armed bandits chased down a cargo ship in the South China Sea. Masked bandits stole from a container ship anchored off Rio de Janeiro. These incidents are part of the rise in piracy, according to the Journal of Commerce. Reporter Michael Fabey used a government electronic bulletin board service (BBS), Marlinespike, to help report the April 8 story.\n\nThe BBS is (202) 366-8505, with settings 8-1-N. The section to check is the ONI World Wide Threat to Shipping.\n\nCheck out transplant rates\n\nUsing data from the United Network for Organ Sharing, Hartford Courant reporter Robert S. Capers charted the survival rates of organ transplant patients at hospitals across the country and in the newspaper's backyard.\n\nThe bulletin board information is easily downloaded and analyzed with spreadsheet or database software, and can be tailored for any audience in the country.\n\nThe UNOS data includes patient survival rates at four time intervals for six procedures at every U.S. transplant center over a three-year period. It also contains demographics on donors and recipients and comments from transplant center officials.\n\nTo find the UNOS organ transplant data, dial into the HHS bulletin board at (301) 443-5913. Enter your name and address when prompted and you will be added to the department's database for future logons.\n\nChoose the file menu, then the download option. The file is survival.zip. For those who don't have it, decompression software is also there. UNOS also runs a World Wide Web site with numerous facts and statistics on organ transplants. Visit http://www.infi.net/-shreorg/unos.html\n\nCommerce changes on-line data site\n\nThe U.S. Commerce Department consolidated its vast STAT-USA Internet sites June 15. That means the STAT-USA gopher and ftp servers are being shut down. The agency now charges a subscriber fee of $24.95 for three months, or $100 for a year, for access through its web site.\n\nSTAT-USA includes more than 300,000 government reports, including the federal government's budget, the president's economic report, regional and national economic data, and foreign trade reports. The sources include U.S. Bureau of Economic Analysis and the National Trade Data Bank.\n\nAmong the more useful sections is the Economic Bulletin Board, which offers reports \"just minutes after its official release\" from the Federal Reserve Board, the Bureau of Labor Statistics, the Treasury Department and other federal agencies.\n\nFor more information or to subscribe, go to www.stat-usa.gov, or call STAT-USA at (202) 482-1986.\n\nJoin NICAR on the Internet\n\nDon't forget to keep up with NICAR on the Internet. Subscribe to our listserve and join in as reporters talk about how to do the job better. E-mail to listserv@mizzoul.missouri.edu. In the message, on the first line, write: subscribe nicar-1 your name. To join Investigative Reporters and Editors on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-september-uplink-update","slug":"1995-september-uplink-update","url":"https://dwillis.github.io/uplink/article/1995-september-uplink-update.html","year":1995,"month":"September","monthNum":9,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"An investigative series exposes the misuse of a court database, unveiling illegal practices such as the jailing of poor individuals and secret cash deals to avoid convictions in Ohio.","keywords":["justice","database","corruption"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","World Wide Web","Windows"],"full_text":"This is a big month for computer-assisted reporting. NICAR, IRE and the Cleveland Plain Dealer host the annual conference for computer-assisted reporting. The event will show journalists how to do everything from handling data to trolling the Internet to running complex analysis. For those who miss it, you can rebound by getting tapes and handouts of the sessions. Information on how to do that is on the back of this issue. In addition, NICAR is growing. It is expanding with a new minority training program. Plus it is running a World Wide site. Visit NICAR at http://www.nicar.org. This edition of Uplink shows how reporters are using computers to probe everything from court houses to roller coasters. Dan Keating of the Miami Herald reviews the much-hyped Windows 95. And there is a look at some of the expanding federal government on-line offerings.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-september-data-tracks-secretive-justice-system-courting-the-poor","slug":"1995-september-data-tracks-secretive-justice-system-courting-the-poor","url":"https://dwillis.github.io/uplink/article/1995-september-data-tracks-secretive-justice-system-courting-the-poor.html","year":1995,"month":"September","monthNum":9,"headline":"Data tracks secretive justice system Courting the poor","kicker":"","author_name":"Michael J. Berens","author_title":"","affiliation":"The Columbus Dispatch","summary":"The Columbus Dispatch's investigation into municipal court data exposed widespread injustices including illegal jailing of poor people and secret cash deals for drunk drivers, requiring a three-year battle to obtain court records and sophisticated data analysis to decode inconsistent clerk entries that revealed systematic deceptions.","keywords":["justice","data","corruption"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["xBase","Macintosh","9-track tape"],"full_text":"There is journalistic justice in using a court database to ferret out the illegal jailing of thousands of poor people, secret cash deals to avoid convictions by drunken drivers, fabricated caseloads, and dozens of other deceptions. Our series, \"Cash-Register Justice.\" last year exposed a secretive justice system where money weighs heavier than guilt. The Columbus Dispatch had its own trial by fire - beginning with a three-year fight to buy municipal court data. Originally, the court wanted $15,000 for a year of data containing one million-plus records and filling a dozen nine-track tapes. Stunned by the initial estimate, we asked for a breakdown of costs. It included $80 an hour pay for some city employees - supposedly to write custom software and so on. These same employees made less than $15 hour in city wages. We argued that the city cannot charge a rate higher than normal pay. We threatened legal action. The price was reduced to $500. Using xBase by Microsoft and our newest toy - a Macintosh Power PC (64 meg RAM) with a gig drive - we crafted relational sorts, such as how many homeless people served jail time before being found guilty of crimes that are not legally punishable by jail. We paid a heavy price in time, too, as we struggled with misplaced and variable length fields, inaccurate entries and case information spread among 12 related databases. Hard lesson: Court data was typed by clerks who adopted personalized codes. The 30 -character charge disposition field became a stream-of-consciousness exercise for some clerks. A defendant sentenced to three years in jail with 30 days jail time credit, a year of probation and $500 fine plus court costs might look like: XRPNC-BOFTS,500 -CC or 1YRPROBPYR/30C/500+C. There was no consistency. Sweet success: Those pesky codes broke the biggest stories. Clerks entered \"NO PW\" for no prosecuting witness - a euphemism for cops who failed to appear in court. Another field code showed when drunken drivers - some testing more than double the legal limit with three prior convictions paid $1,000 to the city to avoid convictions. No kidding. The court never thought outsiders would look at this data- or understand it, if they did. Hard lesson: To em is human - and court clerks are very human. Thousands of cases were not updated accurately on the computer. We pulled paper files to verify results. This was cumbersome but deflected criticism of our findings. The court inflated caseload totals, such as doubling the number of drunken driving cases from 6,000 to 12,000 defendants. The inaccurate numbers hid a bigger secret: caseloads had decreased each of the last three years. One trick used by the court: If a defendant was charged with 32 counts of burglary, the court counted them as 32 cases. Counting is not the only thing courts do badly. Conviction rates are suspect, too. Prosecutors claimed a 70 percent conviction rate involving domestic violence cases. That figure fell to 7 percent when we crunched the data. Prosecutors claimed many victims dropped charges. While true, the data showed that some judges dismissed cases if the victim did not show up at the first hearing - sometimes held 10 hours after the victim was severely beaten. The series ran Sept. 25-28, 1994. Here are some angles that can make for good court projects: Time served: Defendants too poor to make bond often serve jail sentences before being found guilty, then are released at the hearing because they' already served the sentence-sometimes longer than the maximum allowed by law. How many homeless people are jailed and why? We found that homeless people were repeatedly jailed for failing to pay jaywalking tickets - an offense not punishable by jail. In many courts, failure to pay a ticket is treated more harshly than the crime. Are mandatory fines collected? In Ohio, 100 percent of the fines from seat belt violations (among other charges) helped support state programs. Rebellious judges who saw no profit to the local system, routinely dismissed those charges That meant less money to educate children about safety and less training for emergency crews. One lifesaving practice is to provide you statistical analysis to the subjects of the stories before publication. There is nothing more satisfying than watching the stunned faces of prosecutors as you hand them your spreadsheets. From alternative sentencing to campaign financing, we adopted journalism's most basic axiom: Follow the money. In court projects, also follow those who don't have money.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-september-merry-go-round-injuries","slug":"1995-september-merry-go-round-injuries","url":"https://dwillis.github.io/uplink/article/1995-september-merry-go-round-injuries.html","year":1995,"month":"September","monthNum":9,"headline":"Merry-go-round injuries","kicker":"","author_name":"Rick Linsk","author_title":"","affiliation":"Asbury Park Press","summary":"The Asbury Park Press analyzed a database of amusement park injuries from New Jersey's Department of Labor, spending months obtaining records from an outdated computer system and manually supplementing the data with paper records to investigate ride safety oversight.","keywords":["injuries","database","amusement"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["FoxPro","PC","floppy disk","ASCII"],"full_text":"Every year, millions of people ride and walk away happy from the Jersey Shore's boardwalk amusement rides and traveling carnivals.\n\nBut hundreds limp away - or are carried away - with injuries.\n\nThe Asbury Park (N.J.) Press reported its share of piecemeal injury stories. This summer, for the first time, we used computers to take a hard look at the injuries and at state oversight of the ride industry.\n\nHoping to produce a story for the start of summer, I asked the state Department of Labor in April for its database of amusement park injuries. The department first said records were not computerized. A month later, after officials sent us computer printouts, we pressed the issue. They admitted the records were indeed in a database, but said they could not make a copy. After another month of requests and delays, they finally delivered a set of disks.\n\nBig old disks\n\nThere was a snag: the disks were 8.5 inches square, from a now-ancient IBM System 36 computer. For $123, a data-conversion service recommended by NICAR copied the data to 3.5-inch diskettes for us in fixed ASCII format.\n\nThe data files included a master table of all licensed rides, a table of all companies licensed to operate rides, and a table of their insurance companies. The mother lode was the table on accidents, containing 784 injuries from 1992 through the first week of June 1995. Details included injury dates, ride names, ride type, riders' names and age ranges, and what caused the injury.\n\nThe database was limited, though. For one thing, it did not specifically describe riders' injuries. There were fields identifying which part of the body was injured (head, neck, trunk, upper, lower, and multi). But that made a muscle strain indistinguishable from a broken neck. It did not explain how the accident happened, or tell what if any regulatory action was taken against the ride owner.\n\nThere was also a statistical limitation. Because parks do not report ridership figures to any agency, and many guard the figures jealously, it was impossible to calculate how many people were injured for every 1,000 who went on a ride.\n\nBack to paper records\n\nThat left us unable to definitively compare big and small amusement parks or carnivals. The data needed to be augmented and cleaned up a great deal. With a notebook PC, I spent 10 hours during two days in state offices reviewing paper records on each accident and typing in the actual injuries, what happened, and whether state inspectors took any action.\n\nBack in the newsroom, I added new fields and standardized the wildly varying ride names (such as Go Karts, Go-Karts, Go Carts, Go-Carts) in order to get meaningful queries. I could not anticipate all the names that a park might give go carts or certain other rides, though, so our injury totals for them were probably low. I analyzed the data in FoxPro for Windows.\n\nI fleshed out the data by interviewing injured riders, ride owners, lawyers for riders and for parks, engineering professors, state regulators, and insurance underwriters. That included an incident where one hospital worker dubbed the popular Action Park as \"Traction Park.\"\n\nTo find accidents that had not been reported to the state, I checked court records. I used corporate and court records to background a water slide designer whose creations are in theme parks all over the United States, and who has been sued repeatedly.\n\nA ride well worth it\n\nThe fight with state officials and the work needed to clean the database meant the story was not finished until the end of summer.\n\nAmong the conclusions:\n\nState officials for years have minimized injuries as minor bumps and bruises. Their database classified 774 of 784 injuries over three years \"not serious.\" But a closer look at accident records showed the \"not serious\" injuries included broken bones, concussions, and serious cuts.\n\nThere may have been more than the 1,000 accidents reported since 1991, since some ride owners, especially traveling carnivals, don't report everything to the state.\n\nAsked about accidents, ride owners routinely whine about rider horseplay. But state investigators attributed only 6 percent of injuries to riders' misbehavior.\n\nAn understaffed, underfunded team of state inspectors cut from 17 to nine over five years shrugged or struck out again and again trying to figure out why certain accidents happened. They rarely fined parks, and when they did, the fines were minimal.\n\nRick Linsk can be reached at (908) 922-6000, extension 4360, or send e-mail to linsk@usa.net.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-september-cracking-windows-95","slug":"1995-september-cracking-windows-95","url":"https://dwillis.github.io/uplink/article/1995-september-cracking-windows-95.html","year":1995,"month":"September","monthNum":9,"headline":"Cracking Windows 95","kicker":"","author_name":"Dan Keating","author_title":"","affiliation":"The Miami Herald","summary":"A detailed review of Windows 95 that praises its interface improvements and stability enhancements over Windows 3.1, while cautioning that full benefits require new 32-bit software and recommending users without powerful machines wait before upgrading.","keywords":["Windows","stability","software"],"topics":["Technology"],"technologies":["Windows","OS/2","Macintosh","CD-ROM"],"full_text":"Windows 95 is a huge step up from Windows 3.1. Unfortunately, that's not much of a compliment. Windows 3.1 is so unstable, so clumsy and so limited that it's no surprise the world is awaiting its successor. My advice? Wait! Unless you already have a high-horsepower machine, you have absolutely no reason to rush. Let's look piece-by-piece at what Windows 95 offers: Interface and cosmetic changes: An \"A.\" It's worlds better than old Windows. It's fast and easy to get a database manager, word processor or spreadsheet open, and you can quickly flip between them with the new task bar along the bottom. The \"Explorer\" utility for scanning directories, however, seems crippled by the troubled legacy of the old File Manager. The new look should be an easy switch for experienced Windows 3.1 users. Taking advantage of the new advanced features, however, is going to require training. The desktop is simple and uncluttered, but offers a more powerful configuration than the old Windows Program Manager. If there is a program you want to use all the time, you can drag the icon right to the desktop. There is also a shortcut wizard for just about everything. They work smoothly and easily. Another \"A.\" Another cosmetic change is long file names. They are nice, but Microsoft is only catching up to what Macintosh and OS/2 have had for years. Stability: The bane of old Windows is the GPF (General Protection Fault) error message, which trashes everything you have not saved and triggers the three-finger reboot salute. The next worse nightmare is the Out of Memory allegation, even when you know there is plenty left, but Windows does not know where to look to find it. Windows 95 is a big improvement. It uses a better memory management system to keep programs from tripping over each other and crashing the system. But, and it's a big but, the big improvement in stability only kicks in with new 32-bit software programs, which are just now being written. If you use Windows 95 to run your old 16-bit software, you will still be susceptible to the rogue GPF demon. While it's better than Windows 3.1, Windows 95 still did not adopt the level of crash protection in the Windows NT or OS/2. Both run sessions in the entirely separate \"virtual machines,\" so if one goes down, the others are protected. Windows 95 can still collapse altogether because of one errant program. Windows 95 can break a program into multiple \"threads\" that can be handled separately. So while a thread handles printing or dialing the phone, you can keep working on something else. Fast promises await. But, as with stability, these improvements will only emerge in full strength when new software is written to take advantage of multiple threads. Running old 16-bit software just drags the system down to the old level. To explain, a 32-bit pathway is simply twice as big as a 16-bit pathway used by Windows 3.1. If the processor can handle twice as much information at once, the system gets a spectacular improvement in performance. So Windows 95 should be twice as fast as old Windows. And someday it probably will be. But not right away. The reason is that all the software programs have to be rewritten in 32-bit form. When 16-bit software runs through the 32-bit Windows 95, the extra space is just wasted. In fact, it often slows things down. Rewriting won't be as easy as you might guess. Software can do with one big 32-bit word than it otherwise might have done with a few little 16-bit words. It's similar to spoken language: A witty speaker can elegantly use one big word to replace lots of little words. But a lot of people only sound silly when they try to show off with big words. Past history has shown that the first version of 32-bit software can be very clunky. The first 32-bit programs will be appearing this fall. The next generation may emerge by spring of summer of 1996. Plug and play: The new system can automatically recognize hardware such as sound cards, CD-ROM players, printers, scanners and modems. That means a lot less hassle setting them up. Great thing. Don't be surprised if it does not work with old hardware, but I had good success on the test machines I used. Internet and online: Windows 95 comes with a connection to Microsoft Network built right in. Patterned after commercial services like prodigy and America Online, it will offer a basket full of attractions. It's a for-pay service. System requirements: The downside of all the wonderful things Windows 95 offers is that you need more machine to make them happen. Windows 95 will run with four megabytes, but you won't like it. There's no point having a Ferrari that only runs in first gear. You will need eight megabytes to be happy and 12 to 16 to really see what Windows 95 can do. Adding eight meg of RAM will cost you about $370. You will also need more disk space to hold the bigger operating system and the fatter 32-bit programs that you will want to run on it. A 540-megabyte hard drive is enough to skimp by. Think about a gigabyte or more if you like to run lots of different software. You can run Windows 95 with any 386 processor or better. Once again, however, the real advantages do not emerge until you are pumped into the stronger 486 and Pentium machines. If you are thinking about upgrading and you need to kick up the RAM, storage and processor, you are almost definitely better off thinking about a new machine. Also, a CD-ROM (at least double-speed) and sound card with speakers allow you to take advantage of improved multimedia features. Microsoft should have a Windows 95 upgrade with bug fixes by winter. By then, there will be more 32-bit software available, anyway, so you will not have lost much by holding off.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-september-nicar-web-site-expands","slug":"1995-september-nicar-web-site-expands","url":"https://dwillis.github.io/uplink/article/1995-september-nicar-web-site-expands.html","year":1995,"month":"September","monthNum":9,"headline":"NICAR web site expands","kicker":"See you at www.nicar.org","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR's website has expanded its offerings to include new home pages for Asian American Journalists Association and National Association of Black Journalists, with plans to add more organization pages in coming months. The site provides comprehensive information about NICAR's activities, conferences, training seminars, and resources for investigative journalists.","keywords":["website","journalists","resources"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"The NICAR site on the World Wide Web has increased its offerings. In the last month it has added home pages for the Asian American Journalists Association and the National Association of Black Journalists. We hope to add pages for other groups, including the Education Writers Association, in the next few months. Administered by NICAR's web master Wallace Winfrey, the Web site at http://www.nicar.org also offers detailed information about NICAR's activities. That includes its national conference CAR Rock '95, other upcoming events, training seminars, the newsletter Uplink, and NICAR's database library. The Web site also includes a menu for Investigative Reporters and Editors (NICAR is a joint venture of the Missouri School of Journalism and IRE) and for the Hillman Project, which is developing a site for tips and resources for journalists around the world. Try out the site and send recommendations to Brant Houston at brant@nicar.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-september-growing-collection-of-federal-databases","slug":"1995-september-growing-collection-of-federal-databases","url":"https://dwillis.github.io/uplink/article/1995-september-growing-collection-of-federal-databases.html","year":1995,"month":"September","monthNum":9,"headline":"Growing collection of federal databases","kicker":"From the NICAR library","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR maintains an extensive collection of federal government databases spanning elections, aviation, finance, crime, and education that journalists can access for investigative reporting. The organization offers inexpensive data transfer services and continuously expands its library of state and federal records.","keywords":["databases","journalism","federal"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection: Federal Election Commission contributions data, including donations by individuals and political action committees. Federal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration. Home Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns. Federal procurement data, 1992-1994, includes breakdowns by agency. Alcohol, Tobacco and Firearms gun dealer records. National Bridge Inventory System data, includes inspection grades. FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. Social Security death records, by name and social security number, going back to 1937. Occupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies. U.S. Department of Transportation truck accident and census data. It includes accidents by company and road. U.S. Small Business Administration loan guarantees, 1989-1994. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad. U.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad. U.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number. U.S. Department of Education's Common Core of Data runs from the 1987-1988 to the 1992-1993 school year. It includes data on school enrollments, finances, staffing and dropouts. U.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information. NICAR also offers inexpensive data transfer for journalists, and houses an expanding library of state databases. NICAR is also constantly adding up-to-date data. For up-to-date prices and more information, call (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-september-get-computer-training","slug":"1995-september-get-computer-training","url":"https://dwillis.github.io/uplink/article/1995-september-get-computer-training.html","year":1995,"month":"September","monthNum":9,"headline":"Get computer training","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article lists upcoming computer training opportunities for journalists in fall 1995 and winter 1996, including CAR Rock, AP Regional training, and NICAR Bootcamp, with contact information provided.","keywords":["training","journalism","conferences"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Get computer training:\n\nCAR ROCK '95 Sept. 21-24, Cleveland, Ohio\nAP Regional training Oct. 9-10 Birmingham, Ala.\nGay and Lesbian Journalist Association Oct. 19-22 Washington, D.C.\nNICAR Bootcamp, NICAR's week-long intensive training seminar, Jan. 7-12, Columbia, Mo. These dates are open to all journalists. For more information, call NICAR, (314) 882-0684, or send e-mail to nicar@muccmailmissouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-september-nicar-expands-minority-programs","slug":"1995-september-nicar-expands-minority-programs","url":"https://dwillis.github.io/uplink/article/1995-september-nicar-expands-minority-programs.html","year":1995,"month":"September","monthNum":9,"headline":"NICAR expands minority programs","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR is organizing three regional seminars in 1996 to provide free computer-assisted reporting training for minority journalists, covering spreadsheets, databases, and online resources. The organization is seeking funding and partnerships to expand these programs and offering fellowships to help journalists cover travel and lodging expenses.","keywords":["training","journalists","seminars"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":["spreadsheet","World Wide Web"],"full_text":"Plans are underway for three regional seminars in computer-assisted reporting for minority journalists in 1996. The seminars will consist of two days of lectures, demonstrations and hands-on training in the three basic tools of CAR: spreadsheets, database managers and on-line resources. NICAR will work with associations of minority journalists to set up the seminars that will provide free training for at least 50 journalists at each session. NICAR will offer fellowships to these seminars and NICAR training sessions to help minority journalists cover expenses for travel and lodging. NICAR is also seeking more support and funding from media corporations and foundations to expand these programs. NICAR is providing space on its World Wide Web server for home pages for minority journalists' associations and other associations. You can find your way to these home pages by going to http://www.nicar.org If you are interested in participating or helping with the NICAR minority training seminars, please e-mail Brant Houston at brant@nicar.org, call NICAR at (314) 882-0684 or write to NICAR, 100 Neff Hall, Missouri School of Journalism, Columbia, Mo., 65211.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-september-indiana-goes-electric","slug":"1995-september-indiana-goes-electric","url":"https://dwillis.github.io/uplink/article/1995-september-indiana-goes-electric.html","year":1995,"month":"September","monthNum":9,"headline":"Indiana goes electric","kicker":"New public record law passes","author_name":"Carol Napolitano","author_title":"","affiliation":"The (Munster, Ind.) Times","summary":"Indiana passed a new public records law that expands access to electronic records held by government agencies and limits fees agencies can charge for providing them. The law, which takes effect January 1, requires agencies to make reasonable efforts to provide records in electronic format and prohibits storing records in proprietary systems that would impose unreasonable costs on the public.","keywords":["records","Indiana","electronic"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"The second time is a charm. After trying last year and failing, the Hoosier State Press Association (HSPA) and supporters succeeded this year in getting the Indiana General Assembly to approve a new public records law that specifically addresses electronic records. The new law, signed by Gov. Evan Bayh, takes effect Jan. 1.\n\nCurrent law hardly mentions electronic records, except to say agencies are not required to provide them. The new law redefines the term \"public record\" to include all forms of electronic information. It requires public agencies to \"make reasonable efforts\" to provide public records in an electronic format, if the medium requested is compatible with the data storage system.\n\nThe new law also makes it illegal to store public records in such a manner that it requires the public to pay license fees or copyright royalties that \"unreasonably impairs the right of the public to inspect and copy the records.\" Journalists hope this clause will prevent public agencies from storing records in proprietary systems where steep license fees and other charges are made for access to the information.\n\nThe law also permits state or local governments to enact laws that would deny electronic records to those wishing to use them for commercial reasons. News organizations, nonprofits and academics are not considered commercial entities under the law.\n\nAgencies cannot charge more than direct cost for the electronic records. The law defines direct cost as 105 percent of the cost of \"initial development of a (computer) program, if any, the labor required to retrieve electronically stored data and any medium used for the electronic output.\"\n\nCompromises necessary to get the bill through the legislature meant some of the language was softened and certain clauses were added that were not part of the original HSPA proposal. As a result, Indiana courts may be called upon to interpret some of the language or rule on the legality of portions of the statute.\n\nCarol Napolitano can be reached at (219) 933-4174, or (800) 837-3232, ext. 4174; or send e-mail to napoli@howpubs.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-september-fun-fed-sites","slug":"1995-september-fun-fed-sites","url":"https://dwillis.github.io/uplink/article/1995-september-fun-fed-sites.html","year":1995,"month":"September","monthNum":9,"headline":"Fun fed sites","kicker":"On the Internet","author_name":"Andrew W. Lehren","author_title":"NICAR staff","affiliation":"NICAR","summary":"Government agencies are increasingly publishing reports and information online rather than in print, making it easier for journalists to access data quickly. The article provides a directory of useful federal government websites for reporters to find speeches, bills, financial reports, census data, and budget information.","keywords":["government","internet","journalists"],"topics":["Journalism & Newsroom","Technology","Government & Politics"],"technologies":["Internet","World Wide Web","FTP","gopher","spreadsheet"],"full_text":"One thing about \"reinventing government\" it's putting a lot of information journalists can use on the Internet.\n\nThe concept of reducing government paperwork means many agencies opt to churn out reports and information in electronic versions. It's cheaper than printing bulky reports. Some agencies are trying to save money by no longer publishing paper versions of reports. For reporters, this expanding use of the Internet means they can get information faster.\n\nAgencies are finding it hard to turn back. The Securities and Exchange Commission thought about killing its project that offered free access to corporate financial reports. An uproar ensued. The agency changed its mind.\n\nHere are some federal government sites you may find useful.\n\nSearchable sites\n\nCheck on the White House, including speeches, press releases and a searchable index of remarks at press briefings. This site also lets you sign up for daily e-mail from the White House, so you'll know what's new. http://www.whitehouse.gov/White_House/Publications/html/Publications.html\n\nSearch the congressional record and proposed bills by using key words. This is a great way to background an issue and find legislators on different sides of an issue. Click off for either bills or the congressional record from http://thomas.loc.gov/\n\nFor those who want perspective on events, visit the National Archives and Records Administration. It offers a searchable guide to the Federal Register's table of contents, and recent moves like notices that certain files are declassified. It also links to 11 presidential libraries. http://www.nara.gov\n\nYou can also search the congressional record, the full Federal Digest, General Accounting Office reports and more issued by the Government Printing Office at http://ssdc.ucsd.edu/gpo/multi.html You can also telnet for GAO reports swais.access.gpo.gov Log in as \"gao.\" Search using key words. Press \"m\" to e-mail the reports to yourself.\n\nEDGAR lives. This free resource for SEC filings recently avoided a proposed shutdown. Remember it is not complete. It won't be until 1997 at the earliest. And, as part of avoided shutdown, filings will be delayed 24 hours. Go to a pay-for service or the company itself for same-day documents. Until Oct. 1, the site remains at http://www.town.hall.org\n\nNeed check a zip code? Go to the U.S. Post Office's searchable index. http://www.usps.gov/ncsc/\n\nSearch the 1990 Census at these ever-improving sites. The major improvements are with the detailed STF3 series of reports. http://cedr.lbl.gov/cdrom/lookup http://bigsur.lbl.gov/cdrom/lookup\n\nThe U.S. budget is available from several sites. This handy one breaks the budget down by topic, and can be downloaded and cleaned up in a spreadsheet. gopher://sunny.stat-usa.gov/11/BudgetFY96\n\nTo check out the details of \"reinventing government,\" visit FinanceNet. It offers information on how the government sells off assets, resources for examining state and local government finances, and even details on how much federal officials can get reimbursed for travel expenses. http://www.financenet.gov/\n\nFor social programs, go to the Department of Health and Human Services. It offers a way to use ftp and grab the 'green book' a compilation of data for entitlement programs. HHS oversees a series of key agencies, including the Health Care Financing Administration, the National Institutes of Health, the Food and Drug Administration, and the Social Security Administration. All offer press releases and data. http://www.os.dhhs.gov/\n\nThe Department of Interior runs a series of key agencies. From here, you can quickly click to the Bureau of Indian Affairs, the National Park Service, the Bureau of Mines, the U.S. Geological Survey, and more http://www.usgs.gov/doi/ Check out data, including information on the Mississippi River floods, at the Geological Survey's data center. http://sun1.cr.usgs.gov/eros-home.htm\n\nAndrew Lehren can be reached at (314) 882-0684, or send e-mail to andy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-september-the-falling-bridges-of-madison-county","slug":"1995-september-the-falling-bridges-of-madison-county","url":"https://dwillis.github.io/uplink/article/1995-september-the-falling-bridges-of-madison-county.html","year":1995,"month":"September","monthNum":9,"headline":"The falling bridges of Madison County","kicker":"Bits, Bytes and Barks","author_name":"Gwen Carleton","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"Madison County, Iowa's bridges are aging and in poor condition, with 85 requiring rehabilitation and 59 needing replacement at an estimated cost of $4.5 million, though the famous covered bridges are relatively well-maintained and mostly restricted to pedestrian use.","keywords":["bridges","infrastructure","Iowa"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["National Bridge Inventory data"],"full_text":"When I was processing 1994 National Bridge Inventory data recently, I began to wonder how the bridges of Madison County stack up. As it turned out, the county's trestles have seen better days. A few quick queries revealed that according to federal inspectors, 85 of the bridges in Madison County, Iowa, require rehabilitation, and 59 require outright replacement. Federal inspectors estimated the county's bridges could use $4.5 million in work. Iowa engineers said the covered bridges of Robert James Waller fame actually are in good shape. General decay and drivers' affinity for crashing into the bridge walls resulted in the renovation and closure of most bridges years ago. Today, all but one of the covered spans is open only to pedestrians; admirers must cross less romantic metal bridges to reach the sites. The data provided generous detail for the story, everything from the age of the oldest bridge (115 years) to the types of problems the bridges suffered from (slumping embankments, disintegrating culverts). The story ran in the St. Louis Post-Dispatch on Aug. 20.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-september-who-says-you-cant-surf-cleveland","slug":"1995-september-who-says-you-cant-surf-cleveland","url":"https://dwillis.github.io/uplink/article/1995-september-who-says-you-cant-surf-cleveland.html","year":1995,"month":"September","monthNum":9,"headline":"Who says you can't surf Cleveland?","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Car Rock '95 IRE/NICAR computer-assisted reporting conference has launched web pages featuring panel information, speaker biographies, and resources for both attendees and those interested in obtaining conference materials through tapes and handouts.","keywords":["conference","journalism","online"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web","Internet"],"full_text":"Who says you can't surf Cleveland?\n\nThe local planning committee for Car Rock '95 is running a home page for the third annual IRE/NICAR computer-assisted reporting conference. It's at http://www.multiverse.com:80/carrock Or you can jump off NICAR's home page to http://nicar.org/nicar/cleveland/\n\nThe sites offer a lists of panels and speakers, speakers' biographies, and more.\n\nFor those who miss the conference, the sites are a handy way to figure out which panels you would want to catch on the rebound - from tapes and tip sheets.\n\nTo order tape recordings of sessions, call Sound Images at (303) 649-1811. For handouts, call Investigative Reporters and Editors at (314) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-september-chicago-sun-times-seeks-car-reporter","slug":"1995-september-chicago-sun-times-seeks-car-reporter","url":"https://dwillis.github.io/uplink/article/1995-september-chicago-sun-times-seeks-car-reporter.html","year":1995,"month":"September","monthNum":9,"headline":"Chicago Sun-Times seeks CAR reporter","kicker":"","author_name":"","author_title":"","affiliation":"Chicago Sun-Times","summary":"The Chicago Sun-Times is hiring an experienced computer-assisted reporting journalist with at least three years of experience in data analysis and government database research. Interested candidates should submit their resume and clips to the assistant metro editor.","keywords":["reporting","databases","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":[],"full_text":"The Chicago Sun-Times is looking for an experienced reporter knowledgeable in gathering and analyzing computer data and doing complex stories using government databases. Applicants should have a minimum of three years experience in computer-assisted reporting and be proficient in the use of public records. This is not an entry-level position. Those interested should send resume and clips to John Erickson, assistant metro editor, Chicago Sun-Times, 401 N. Wabash, Chicago, Ill., 60611.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-september-join-nicar-on-the-internet","slug":"1995-september-join-nicar-on-the-internet","url":"https://dwillis.github.io/uplink/article/1995-september-join-nicar-on-the-internet.html","year":1995,"month":"September","monthNum":9,"headline":"Join NICAR on the Internet","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Instructions for subscribing to NICAR and IRE listservs where journalists can discuss best practices and professional development.","keywords":["listserv","journalists","subscribe"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","Email"],"full_text":"Don't forget to keep up with NICAR on the Internet. Subscribe to our listserve and join in as reporters talk about how to do the job better. E-mail to listserv@mizzoul.missouri.edu In the message, on the first line, write: subscribe nicar-1 your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-uplink-update","slug":"1995-october-uplink-update","url":"https://dwillis.github.io/uplink/article/1995-october-uplink-update.html","year":1995,"month":"October","monthNum":10,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"CAR Rock '95, a four-day computer-assisted reporting conference in Cleveland, attracted nearly 500 journalists for over 60 panels and 80 training sessions covering data analysis and digital tools. The event demonstrated advanced techniques for both experienced and novice reporters, with resources available for those who missed it.","keywords":["conference","journalism","technology"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"For those who rocked in Cleveland, we salute you. CAR Rock '95, held last month, further showed what computer-assisted reporting means for journalism.\n\nNearly 500 reporters from the United States and elsewhere attended the four-day event. The conference included more than 60 panels that ranged from crunching numbers to cleaning data. There were also more than 80 hands-on training sessions, showing the ins and outs of the Internet, spreadsheets and database programs.\n\nThe quality was higher and the subject more advanced than ever before. The pros learned, and so did beginners. The Cleveland Plain Dealer was a terrific host. And there was even a peek into the Rock and Roll Hall of Fame.\n\nFor those who missed the event, NICAR and Investigative Reporters and Editors offer handouts, story reprints, and audio tapes from the panels. Stay tuned for the next one.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-tracking-military-justice-prisoners-hit-pay-dirt","slug":"1995-october-tracking-military-justice-prisoners-hit-pay-dirt","url":"https://dwillis.github.io/uplink/article/1995-october-tracking-military-justice-prisoners-hit-pay-dirt.html","year":1995,"month":"October","monthNum":10,"headline":"Tracking military justice: Prisoners hit pay dirt","kicker":"","author_name":"Jo Craven","author_title":"NICAR staff","affiliation":"NICAR","summary":"Dayton Daily News reporters Russell Carollo and Cheryl Reed uncovered that hundreds of military convicts were collecting paychecks while imprisoned, totaling over $1 million monthly, while crime victims received no restitution. Their five-month investigation using multiple databases and interviews led Congress to pledge reforms to this century-old practice.","keywords":["military","prisoners","investigation"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Paradox","FoxPro","Excel","Lotus 1-2-3"],"full_text":"Crime may not pay, but military imprisonment often does. That's what Dayton Daily News reporters Russell Carollo and Cheryl Reed discovered when they found that hundreds of soldiers and sailors convicted of serious crimes collect paychecks while in prison. Using at least five computer databases, Carollo and Reed revealed that in June 1994 alone, 680 military convicts collected salaries totaling more than $1 million of taxpayers' money. Meanwhile, victims, who had been raped, beaten or murdered, and their families received nothing, not even the restitution generally awarded by courts.\n\nCarollo sniffed out the story after becoming intrigued with the military justice system. Then, serendipity and reporter's instinct intervened. While ordering a database of military prisoners, Carollo was asked if he wanted the field \"with the pay in it.\"\n\nAs a result, Carollo and Reed spent five months investigating.\n\nThe military provided requested records slowly but with little resistance. The reporters obtained four databases from the Defense Finance and Accounting Service, which pays military members. These were cross-referenced with databases from the Army and Air Force, which revealed omissions in the DFAS data.\n\nThey also used prisoner databases, court-martial files and other pay reports. With the help of researcher Veronica Szente, the reporting team combined the information into a database they built themselves, correcting faulty military data as they went along. \"It was dirty data right from the start,\" Szente said. \"I had to do a lot of manipulation.\" Extraneous characters, flashing symbols and incorrectly placed fields were among the problems.\n\n\"If (data) doesn't work in one program, try something else,\" Szente recommended. \"If you can't upload it into your main database, try something that's a lower end Excel or Lotus; sometimes it's easier for them to pull it in.\"\n\nThe Dayton team analyzed its data using Paradox and FoxPro. But, Carollo and Reed did not stop there. They interviewed more than 100 inmates, their victims and families, prison officials, military legal experts, police officers and members of Congress.\n\nThe practice of paying convicted criminals in the military dated back more than 100 years, Carollo and Reed found, but many members of the military and Congress were unaware of it. After publishing six stories in a two-day series in December 1994, Congress and the military tackled the issue and pledged to reform the system.\n\nCarollo offered several pieces of advice to reporters who may tackle similar projects:\n\nAvoid flacks in favor of \"Joe in the basement\" - the data manager. \"The flack doesn't know anything about computer databases,\" Carollo said. \"Do your own homework and insist on talking to the computer person.\"\n\nMeet the data manager in person. This was particularly valuable in Carollo's dealings with the military. \"There are lots of acronyms, and they love jargon,\" Carollo said. \"You are going to have to keep calling this person a lot. You've got to have a face-to-face contact.\"\n\nDon't limit yourself. \"Don't ever get on the phone and say these are the only fields I need,\" Carollo said. \"This is an absolutely perfect example. There was this field that nobody in their right mind ever would have thought existed.\"\n\nAlways look for the unexpected. \"The Army, we discovered, wasn't being on the up and up with us,\" Carollo said. \"Their numbers did not include prisoners at Leavenworth.\" The reporters discovered the error by looking at the Leavenworth database and confronted the Army. \"They said, 'Hey you're right; we left them out, ha, ha, ha.'\"\n\nFinally, Carollo said, once the desired data is identified, fire off the FOIAs fast. \"The No. 1 advice is always file early,\" Carollo said. \"File FOIAs kind of like an artillery barrage, long before you ever land on the beach.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-mapping-branches-look-where-banks-arent","slug":"1995-october-mapping-branches-look-where-banks-arent","url":"https://dwillis.github.io/uplink/article/1995-october-mapping-branches-look-where-banks-arent.html","year":1995,"month":"October","monthNum":10,"headline":"Mapping branches: Look where banks aren't","kicker":"","author_name":"Paul D'Ambrosio","author_title":"","affiliation":"Asbury Park Press","summary":"The Asbury Park Press investigated banking discrimination in New Jersey by analyzing branch locations against commercial business data, finding that minority communities had significantly fewer bank branches relative to businesses compared to white communities, even after controlling for income and commercial activity.","keywords":["banking","discrimination","redlining"],"topics":["Investigative Methods","Data & Databases","Demographics & Census"],"technologies":["Atlas GIS","ArcInfo","Nexis"],"full_text":"To understand bankers, you have to first think like one. That is what the Asbury Park Press tried to do when it set out to study banking behavior in New Jersey. Bankers are fond of telling us that they build branches where there is the most business. Race and other basic demographics don't matter at all, they say, as long as there is money to be made. That isn't what we found. In order to compare apples with apples, I overlaid bank location data with commercial business data borrowed from our marketing department. We used commercial business data because it is a closer proxy for banking business than population or income data. That data came in a geographic layer that could be imported into mapping software. I used Atlas GIS for Windows in combination with ArcInfo, a more powerful mapping program, for some of the analysis.\n\nWe then overlaid the business data with bank branch locations for all commercial banks. We had to reconcile Federal Reserve data with our own survey because neither alone gave us a complete picture.\n\nWhat we found was startling. When we divided the state into census tracts and analyzed branch-to-business ratios, minority communities had significantly fewer bank branches relative to the number of businesses than white communities. Race turned out to be a significant predictor of banking service — even after controlling for income, commercial activity and population density.\n\nThis was accomplished by creating a regression model with the help of a Rutgers University professor who teaches statistical analysis. The model showed that race was independently associated with branch location, meaning it wasn't just a proxy for income or the density of commercial activity.\n\nWe also took a more traditional approach to redlining, overlaying branch data with income maps and looking at lending data from the Home Mortgage Disclosure Act. Again, the picture that emerged was one of underservice in minority communities.\n\nWe built all of our data into a geographic information system so readers could see these disparities on maps published with the stories. The maps made the numbers vivid in a way that tables and charts could not.\n\nThe whole package, titled, \"No checks or balances,\" ran Sept. 10-13. It can also be found on Nexis.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-getting-access-for-stories","slug":"1995-october-getting-access-for-stories","url":"https://dwillis.github.io/uplink/article/1995-october-getting-access-for-stories.html","year":1995,"month":"October","monthNum":10,"headline":"Getting Access for stories","kicker":"","author_name":"David Herzog","author_title":"database journalist","affiliation":"Allentown (Pa.) Morning Call","summary":"Microsoft Access is gaining popularity in newsrooms as a database management tool for computer-assisted reporting, offering a simpler query interface than competitors like FoxPro and Paradox, though it has performance limitations with larger databases.","keywords":["Access","database","queries"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["FoxPro","Paradox","Access","Excel","dBase","SQL"],"full_text":"For years, the debate over which database manager is best for computer-assisted reporting was a tug-of-war between FoxPro and Paradox. Now, Microsoft Access has joined the battle in newsrooms.\n\nLured by its simple query interface, Access is winning converts. I'm using Access for stories about gun dealers, unsafe trucks and crime. At the Los Angeles Times, Ralph Frammolino used it to show how jurors were picked for the O.J. Simpson trial (Dec. 1994 Uplink).\n\nAccess does not use layers of dialog boxes to perform advanced functions, unlike FoxPro. Using a graphical query-by-example window, you design sophisticated queries on a single screen. In the upper half, you add the tables.\n\nTo join tables, you drag the name of the key field in one to the matching field in the second. Then, drag the field names you want into a grid on the bottom half of the screen. With a few clicks and keystrokes, Access can order records, calculate or pick records meeting certain criteria.\n\nThe pro/con of point/click\n\nThe interface makes it simple to fine tune, according to Dick O'Reilly, director of computer analysis for the Los Angeles Times. \"It's easier to do an ad hoc kind of hacking project than with FoxPro,\" he said.\n\nOn the flip side, it's easy to goof during queries that use aggregate functions, such as counting, summing and averaging.\n\nJeff Kummer of St. Paul Pioneer Press said that the seemingly endless pointing and clicking can be a pain. He sometimes finds himself going back to FoxPro and writing a query in structured query language, which Access supports.\n\nUnder the hood, Access will push your computer's limits. \"Run it on a fast machine,\" said George Landau of the St. Louis Post Dispatch.\n\nFoxPro vs. Access vs. dBase\n\nDespite Access' shortcomings, O'Reilly and others like its new speed. For a column last year, O'Reilly stacked it up against FoxPro for Windows, and Borland's dBase for Windows. The test was a real-world situation: rummaging through 107,000 building inspection records. FoxPro ran his query the quickest. Access placed second, trailing by just a few seconds. That left dBase last.\n\nHowever, as the database table sizes grow, Access slows. Queries did recently on a 350-mb table were noticeably slower in Access than FoxPro for Windows 2.6.\n\nMicrosoft has deployed aspects of FoxPro to improve Access' speed. Microsoft acquired FoxPro's parent company, Fox Software Inc., in June 1992. Access developers then took a key part of FoxPro - the Rushmore indexing technology to rev up Access, according to product manager Jeff Graff.\n\nWhen Microsoft introduced Access three years ago, users complained about its heel-dragging. Also, many found it weak for database developers. The current version includes a grab bag of tools for everyone from end users to developers. Question and answer boxes walk the user through normally complex operations, such as creating a crosstab query or designing and printing a report. Also, Microsoft included quick import and export links to FoxPro, Excel spreadsheets and other programs.\n\nWith millions of Access sales, it's fair to ask what will happen to FoxPro. \"We have no intention of phasing one or the other out,\" said Microsoft's Graff. \"The strategy is to move them together into a tool that is better than any single one.\" Expect Access and FoxPro to share more features, he said. Though Graff declined to be specific, he said that within 3 to 5 years Microsoft wants the programs to have common elements, such as a form package and database engine.\n\nAccess 2.0 and the Access Developer's Toolkit, handy for building applications, is each sold for less than $300. Shop around for the lowest prices.\n\nDavid Herzog can be reached at (610) 820-6584, or send e-mail to dherzog@mcall.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-nicar-web-site-expands","slug":"1995-october-nicar-web-site-expands","url":"https://dwillis.github.io/uplink/article/1995-october-nicar-web-site-expands.html","year":1995,"month":"October","monthNum":10,"headline":"NICAR web site expands","kicker":"See you at www.reporter.org","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR has moved its website to a new address at reporter.org and is expanding its online resources, including a new Russian journalism site and connections to other journalism organizations like IRE and the Hillman Project.","keywords":["journalism","website","resources"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web","Internet"],"full_text":"The NICAR site on the World Wide Web is at a new address. Visit NICAR at http://www.reporter.org. The change reflects the growing list of non-profit journalism organizations with home pages at the web site.\n\nNow under construction is a Russian journalism site, which will help reporters cover the former Soviet Union. Plans are underway to include information in Cyrillic.\n\nAdministered by NICAR's web master, Wallace Winfrey, the web site offers detailed information about NICAR's activities. This includes upcoming events, training seminars, the newsletter Uplink, and NICAR's database library.\n\nIn addition, you can search the journalism resource center run by Investigative Reporters and Editors, at www.ire.org The web site also includes a menu for IRE and for the Hillman Project, which is developing a site for tips and resources for journalists around the world.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-growing-collection-of-federal-databases","slug":"1995-october-growing-collection-of-federal-databases","url":"https://dwillis.github.io/uplink/article/1995-october-growing-collection-of-federal-databases.html","year":1995,"month":"October","monthNum":10,"headline":"Growing collection of federal databases","kicker":"From the NICAR library","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR maintains a comprehensive collection of federal government databases covering elections, aviation, finance, crime, and other sectors available to journalists for investigative reporting. The organization offers data transfer services and continuously updates its growing library of federal and state databases.","keywords":["databases","federal","journalists"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"Growing collection of federal databases\n\nFrom the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nFederal Election Commission contributions data, including donations by individuals and political action committees.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1994, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records.\nNational Bridge Inventory System data, includes inspection grades.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1994. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nU.S. Department of Education's Common Core of Data runs from the 1987-1988 to the 1992-1993 school year. It includes data on school enrollments, finances, staffing and dropouts.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nNICAR also offers inexpensive data transfer for journalists, and houses an expanding library of state databases. NICAR is also constantly updating data.\n\nFor up-to-date prices and more information, call (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-get-computer-training","slug":"1995-october-get-computer-training","url":"https://dwillis.github.io/uplink/article/1995-october-get-computer-training.html","year":1995,"month":"October","monthNum":10,"headline":"Get computer training","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Multiple journalism organizations are offering computer training bootcamps and workshops for journalists in fall 2024 and spring 2025, with programs focusing on computer-assisted reporting at MIT, Medill School, and NICAR.","keywords":["training","journalism","bootcamp"],"topics":["Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Get computer training:\n\nSociety for Environmental Journalists, Oct. 26-29, Massachusetts Institute for Technology, Boston. The Medill School of Journalism, four-day computer-assisted reporting bootcamp, Dec. 7-10. Medill's Washington, D.C., campus.\nNICAR Bootcamps, week-long intensive training seminars, Jan. 7-12, and March 10-15, Columbia, Mo.\n\nThese dates are open to all journalists. For more information, call NICAR, (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-missed-cleveland","slug":"1995-october-missed-cleveland","url":"https://dwillis.github.io/uplink/article/1995-october-missed-cleveland.html","year":1995,"month":"October","monthNum":10,"headline":"Missed Cleveland?","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Information on how to obtain tape recordings and handouts from CAR Rock '95, a journalism conference, through Sound Images and Investigative Reporters and Editors.","keywords":["conference","recordings","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Missed Cleveland? To order tape recordings of CAR Rock '95 sessions, call Sound Images at (303) 649-1811. For handouts, call Investigative Reporters and Editors at (314) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-being-inventive-go-on-line-to-find-designs","slug":"1995-october-being-inventive-go-on-line-to-find-designs","url":"https://dwillis.github.io/uplink/article/1995-october-being-inventive-go-on-line-to-find-designs.html","year":1995,"month":"October","monthNum":10,"headline":"Being inventive: Go on-line to find designs","kicker":"","author_name":"Heather Newman","author_title":"","affiliation":"The (Nashville) Tennessean","summary":"The article explains how to search the U.S. Patent Office's database to find patents from your state or region, using Tennessee as an example, and demonstrates the search techniques and tools needed to access this free online resource.","keywords":["patents","database","search"],"topics":["Technology","Investigative Methods"],"technologies":["WAIS","Internet","Netscape","Word","Internet Assistant","HTML"],"full_text":"The world's biggest collection of new ideas, products and designs - including those in your city and state- is literally at your fingertips.\n\nThe U.S. Patent Office has run a full-text database of the abstract and summary of every patent issued since Jan. 1, 1994. It is free and on the Internet, waiting for you to discover your area's inventors.\n\nWe rummaged through this database to see how Tennessee compared to the rest of the country. We also wanted to see what inventions were coming from our area and look at who's doing the inventing.\n\nA few searches and a little legwork made this an easy weekend project that could be done in any city in the United States.\n\nWe stumbled on the site maintained by Internet Multicasting Service when surfing the 'net. We tried calling IMS to get the whole database directly, but it was not willing to pass it along. We probably could have received the database straight from the patent office, but because we were on deadline, we opted to use IMS's searcher and bolster our findings with summary numbers from patent office officials. Internet Multicasting stopped the service Oct. 1. The U.S. Patent Office announced it will open its own searchable site on Nov. 9.\n\nGetting local patents\n\nThe IMS database used WAIS search criteria, which means you have a few more options than the standard Boolean set. We found the most useful are the wildcard and ADJacent operators. Here's how we used them to get all our local patents:\n\nWe searched all the patents in each year (you have to search 1994 and 1995 separately) using Inventor ADJ State: ADJ TN (There is a \"field search\" option, but we did not find it to be helpful). That's when we hit our first snag. IMS only allows you to retrieve 150 hits at a time.\n\nBecause there were more than 150 patents in the state for each year, we were not getting a complete list. At this point, we faced a choice: select by city (by adding AND Inventor ADJ City ADJ Nashville, for example) or divide the database by date. We decided to do the latter, gathering each month's new issues one at a time.\n\nThe issue date was in YYYYMMDD format, so we used wildcards. For example, we added Issue ADJ Date: ADJ 199409* to our query to get all the patents issued in Tennessee in September 1994. Because the answer \"page\" gave us the total hits, we also had an easy count of patents by month in the state.\n\nUnfortunately, there is no easy way to save those patents to disk. We ended up opening every one of them and using Netscape's SAVE AS command to save them to a directory. We used original file names (which were the patent numbers).\n\nClick tip\n\nA tip: We also saved the answer \"pages\" in their original HTML code and pasted them together in one long document using Word for Windows 6.0c (with the free Internet Assistant add-on). We could look at the answer pages, and if we needed to look at the patent itself, we could still click on the link and go directly to the patent.\n\nLeaving the file names as the patent numbers also let us check the number of each type of patent. Invention patents are just numbers. Design patents start with \"D,\" plant patents start with \"PP,\" patent reissues are labeled \"RE\" and statutory patents start with \"H.\" A quick count in our directory gave totals.\n\nThe patent texts themselves were clean, chock full of information and up-to-date. We found some in our collection were issued less than a week prior to our search. They included the date when the original application was made, giving us a quick count on how long the patenting process took. They also included an abstract and summary that explain the invention in reasonably clear English, and the name and address of every inventor.\n\nThis meant that, with a little phone book work, we interviewed a pile of folks with interesting inventions. Their descriptions of how they got their ideas and their patents added color and life to our story.\n\nAbout the only thing we wanted that was not in the database were the illustrations that went with each application. The descriptions of what the illustrations depicted were entered, but the pictures themselves were not included.\n\nOverall numbers\n\nFor overall numbers, we decided that hunting through the IMS database would take too long, so we called the patent office directly. They sent us a list of how many patents of each type each state received in 1990 through 1994 (The most recent numbers at sub-state level, unfortunately, are from 1992).\n\nWe plugged the 1990 and 1994 numbers into a spreadsheet with the state population estimates from those years to see which states won out in patents per capita. Then we asked a few patent experts to help us interpret those numbers.\n\nWe included a sidebar on how to get your own patent, a where-to-call box on a local inventors' group and a list of some recent Nashville inventions. The result was a nice, well-rounded package with local inventors in a national context.\n\nHeather Newman can be reached at (615) 259-8075, or send e-mail to hnewman@tennessean.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-here-are-books-available-from-ire","slug":"1995-october-here-are-books-available-from-ire","url":"https://dwillis.github.io/uplink/article/1995-october-here-are-books-available-from-ire.html","year":1995,"month":"October","monthNum":10,"headline":"Here are books available from IRE","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE offers several investigative journalism guides including books on reporting techniques, finding sources, and computer-assisted reporting methods, with prices ranging from $5 to $26.","keywords":["journalism","investigative","reporting"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Here are books available from IRE: \"Have I Got a Tip for You And Other Tales of Dirty Secrets, Political Payoffs and Corporate Scam.\" Written by Stanley Penn, retired Wall Street Journal reporter, this guide to investigative reporting is published by Dow Jones & Co. The price is $5. \"The Reporter's Handbook.\" Now in its third edition, this invaluable guide to documents and investigative techniques is written by Steve Weinberg, and published by St. Martin's Press. Order it in bookstores, or from IRE. The price is $21 for members, and $26 for non-members, plus postage. '100 Computer-Assisted Stories\" is published by IRE and NICAR. Newspaper and television reporters recount how they got the story. The price is $20. To order, call IRE, (314) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-patented-on-the-net","slug":"1995-october-patented-on-the-net","url":"https://dwillis.github.io/uplink/article/1995-october-patented-on-the-net.html","year":1995,"month":"October","monthNum":10,"headline":"Patented on the net","kicker":"Search for ideas","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute","summary":"A guide to online resources for learning about patents, including the U.S. Patent Office, searchable databases, FAQs, law school resources, and patent search tools.","keywords":["patents","database","searching"],"topics":["Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"The Internet offers good sites to learn more about patents. Here are the best. The U.S. Patent Office offers the expected press releases, speeches and updates about patent law changes. There are also some useful files explaining patents and trademarks.\n\nCheck out the link to the searchable database of AIDS-related Patents (developed in cooperation with the Clearinghouse for Networked Information Discovery and Retrieval). It is a model of web searching, providing the ability to search by fields and retrieve not only the text of the patents but also the images submitted along with the patent application. http://www.uspto.gov/\n\nYes, there is a place for patent frequently asked questions (FAQ). The first rule of Internet searching is find the relevant FAQ. This one for patents offers history, background and definitions of patenting terms. http://www.sccsi.com/DaVinci/patentfaq.html\n\nFor patent law information, the Villanova University Law School offers a list of sources. There is even a section on patent law for non-lawyer detailing, in laymen's terms, the rights and responsibilities of patent filing. It's a great site for background information on terms and legalities. http://www.law.vill.edu/~rgruner/patent2.htm\n\nPatents and patent searching is offered as a companion to the previous site. This page links to all the options for patent searching and retrieval on the Internet. http://www.law.vill.edu/~rgruner/patent3.htm\n\nPatent issues\n\nFor documents dealing with patenting issues, the University of North Carolina offers yet another of its many great sites. This one links to articles about patent topics - check out the article titled, \"A really trivial educational patent.\" http://sunsite.unc.edu/patents/document.html\n\nThe Shadow Patent Office offers free searching of recently issued patents. Register for full patent search and retrieval. http://www.spo.eds.com/patent.html\n\nNora Paul can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-us-patent-and-trademark-office-announcement","slug":"1995-october-us-patent-and-trademark-office-announcement","url":"https://dwillis.github.io/uplink/article/1995-october-us-patent-and-trademark-office-announcement.html","year":1995,"month":"October","monthNum":10,"headline":"U.S. Patent and Trademark Office announcement","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The U.S. Patent and Trademark Office announced plans to launch its own searchable patent database on November 9, and the SEC took over management of the EDGAR corporate financial reports database from Internet Multicasting Service.","keywords":["patents","database","SEC"],"topics":["Technology","Business & Economy"],"technologies":["Internet","World Wide Web"],"full_text":"The U.S. Patent and Trademark Office announced in late September that it plans to open its version of the searchable patent site on Nov. 9. It will replace what had been run by Internet Multicasting Service. The patent office is at http://www.uspto.gov/ Meanwhile, the EDGAR database of corporate financial reports is also no longer run by Internet Multicasting. On Oct. 1, the U.S. Securities and Exchange Commission took over the effort that is putting company filings to the government on the Internet. The address is http://www.sec.gov/edgarhp.htm","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-look-who-gets-city-hall-jobs","slug":"1995-october-look-who-gets-city-hall-jobs","url":"https://dwillis.github.io/uplink/article/1995-october-look-who-gets-city-hall-jobs.html","year":1995,"month":"October","monthNum":10,"headline":"Look who gets City Hall jobs","kicker":"Dialing direct","author_name":"David Crowder","author_title":"reporter","affiliation":"El Paso Times","summary":"The El Paso Times analyzed a city employee database to uncover workplace trends, discovering that while city government mirrored the city's ethnic diversity, significant gender disparities existed with women comprising only 23% of the workforce and holding few top-paying positions.","keywords":["database","gender","reporting"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"The first example of a database most computer-assisted reporters in training hear about is the telephone book. We nod OK, but can't imagine using it for any reason other than that for which it was produced.\n\nBut a more focused roster of a community of sorts with some additional information can turn up facts and trends and, thus, stories that no one in your city may have thought about before.\n\nThat's what the El Paso Times learned from a database of city government's 6,500 employees recently. We got names, pay, ethnicity, gender, marital status, department and job titles. We also got Social Security numbers just by asking for a unique identifier.\n\nIn a city of more than 600,000 people, about 72 percent of whom are Hispanic, we expected to find that city government was out of whack ethnically. Instead we found that city government to be almost a mirror image of the city in terms of ethnicity and, not surprisingly, income.\n\nGender was another matter. We found that women make up only 23 percent of the city's full-time work force and that only 17 women are among the 100 best-paid employees. That led to a lead story about the apparent \"glass ceiling\" and about what women who made it to the top went through.\n\nFor instance, comparing the 100 best-paid men and to the 100 top women, we found that 71 percent of the men are married compared with 39 percent of the women.\n\nThe city employees roster was an easy little hit, and the stories it produced got people talking. Even before the stories came out, we used that database to analyze four city departments for other stories. I expect we'll be digging into that data again and again in coming months.\n\nDavid Crowder can be reached at (800) 351-6007, ext. 6194, or send e-mail to 75352.2352@compuserve.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-office-of-economic-conversion-and-information","slug":"1995-october-office-of-economic-conversion-and-information","url":"https://dwillis.github.io/uplink/article/1995-october-office-of-economic-conversion-and-information.html","year":1995,"month":"October","monthNum":10,"headline":"Office of Economic Conversion and Information","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Office of Economic Conversion and Information, a joint Pentagon and Commerce Department office, provides resources on military base closures including conversion projects, job loss projections, and guides for communities dealing with base shutdowns.","keywords":["bases","conversion","closure"],"topics":["Government & Politics","Business & Economy","Data & Databases"],"technologies":["World Wide Web"],"full_text":"The Office of Economic Conversion and Information, run jointly by the Pentagon and the Commerce Department, offers background on military base conversion projects, including speeches, testimony and press releases. It also offers a state-by-state rundown of the bases that have been shut down, projected numbers of lost jobs, and guides that may help you evaluate whether your local officials are doing everything they can when faced with a closing base. http://www.doc.gov/ecix/ecixhomepage.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-access-to-electronic-records","slug":"1995-october-access-to-electronic-records","url":"https://dwillis.github.io/uplink/article/1995-october-access-to-electronic-records.html","year":1995,"month":"October","monthNum":10,"headline":"Access to Electronic Records","kicker":"","author_name":"","author_title":"","affiliation":"Reporters Committee for Freedom of the Press","summary":"A guide is available from the Reporters Committee for Freedom of the Press on how to access and report on computerized government records at the state and local level.","keywords":["records","government","access"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"For information on getting computerized records, get \"Access to Electronic Records: A Guide to Reporting on State and Local Government in the Computer Age.\" It's published by the Reporters Committee for Freedom of the Press. To order, call (202) 466-6313. The price is $5 plus postage.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-bits-bytes-and-barks","slug":"1995-october-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1995-october-bits-bytes-and-barks.html","year":1995,"month":"October","monthNum":10,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A journalism newsletter highlighting Profnet's new email address, NICAR job openings, warnings about fake presidential websites, and examples of how reporter Amy Bayer uses Internet sources to cover breaking stories effectively.","keywords":["Profnet","Internet","reporting"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nProfnet changes address\n\nProfnet, the outfit that will route your search for experts to universities and think tanks, recently changed its e-mail address. Now, send your inquiry to: profnet@vyne.com Include a short heading that describes what you need, the nature of the assignment and what kind of questions you need answered, your deadline, name, newspaper, phone number and e-mail address. Let Profnet know if you prefer e-mail or to be called - and when you'll be in the office to take a phone call.\n\nNICAR wants you\n\nThe National Institute for Computer-Assisted Reporting is hiring for two positions.\n\nNICAR seeks a training director who will oversee on-the-road classes in newsrooms across the country; and a systems director for data analysis and coordinating NICAR's database library. Both positions include sharing duties for training and data analysis.\n\nFor more information, call Brant Houston, NICAR's managing director, at (314) 882-1984, or send e-mail to brant@nicar.org.\n\nBeware on the 'net\n\nThe Kansas City Star reported on Oct. 2 how unofficial presidential candidate sites are popping up on the World Wide Web with addresses that may seem like the real thing. But beware: www.dole96.org is not controlled by Bob Dole, while www.dole96.com is the Republican's official site. For Pat Buchanan, www.buchanan.org is official, but www.buchanan96.org is not. The story cautioned how unaffiliated sites may be used for everything from satire to campaign dirty tricks.\n\nCatching Washington in the net\n\nMore proof that you can cover breaking stories with the Internet: Amy Bayer, who covers the White House and national affairs for Copley News Service, said she uses Internet sources \"on just about every story.\"\n\nHere are some recent examples.\n\nWhen she covered the United Nations conference on women, and wanted to probe a platform controversy, she found draft versions of the platform on the Internet, and compared those documents with remarks by delegates.\n\nFor finding sources on Ross Perot's recent appearance on Larry King Live, she got phone numbers for the United We Stand state officials at http://www.uwsa.org.\n\nAfter covering President Clinton's visit to Moscow, she checked the searchable Congressional Record at http://thomas.loc.gov to document the lack of support from fellow Democrats for the trip.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-nicar-wants-you","slug":"1995-october-nicar-wants-you","url":"https://dwillis.github.io/uplink/article/1995-october-nicar-wants-you.html","year":1995,"month":"October","monthNum":10,"headline":"NICAR wants you","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"The National Institute for Computer-Assisted Reporting is hiring a training director and a systems director to oversee journalism education and data analysis services. Both positions involve shared responsibilities for training and managing NICAR's database resources.","keywords":["hiring","journalism","training"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"NICAR wants you\n\nThe National Institute for Computer-Assisted Reporting is hiring for two positions.\n\nNICAR seeks a training director who will oversee on-the-road classes in newsrooms across the country; and a systems director for data analysis and coordinating NICAR's database library. Both positions include sharing duties for training and data analysis.\n\nFor more information, call Brant Houston, NICAR's managing director, at (314) 882-1984, or send e-mail to brant@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-october-join-nicar-on-the-internet","slug":"1995-october-join-nicar-on-the-internet","url":"https://dwillis.github.io/uplink/article/1995-october-join-nicar-on-the-internet.html","year":1995,"month":"October","monthNum":10,"headline":"Join NICAR on the Internet","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"Instructions for subscribing to NICAR and IRE listservs where journalists can discuss reporting techniques and professional development.","keywords":["listserv","subscribe","journalists"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet"],"full_text":"Don't forget to keep up with NICAR on the Internet. Subscribe to our listserve and join in as reporters talk about how to do the job better. E-mail listserv@mizzou1.missouri.edu In the message, on the first line, write: subscribe nicar-1 your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-uplink-update","slug":"1995-november-uplink-update","url":"https://dwillis.github.io/uplink/article/1995-november-uplink-update.html","year":1995,"month":"November","monthNum":11,"headline":"Uplink update","kicker":"Mining the tape","author_name":"","author_title":"","affiliation":"","summary":"This journalism newsletter discusses upcoming election coverage resources through an IRE and NICAR conference focused on campaign contributions and polling analysis, while also featuring investigative techniques for workplace safety stories using OSHA databases and supplementary sources.","keywords":["election","investigation","workplace"],"topics":["Investigative Methods","Government & Politics","Business & Economy"],"technologies":["Internet"],"full_text":"It's beginning to look a lot like election season.\n\nYes, we're getting closer to the New Hampshire presidential primaries.\n\nA January conference by IRE and NICAR will help you delve into campaign contributions, analyze polls and dig deeper for stories. See Page 3 for more information.\n\nMeanwhile, this issue of Uplink is designed to help you cover what goes on in the office and factory.\n\nThe stories look back at the seminal series by the Dayton Daily News, and jump to today, where reporters continue to mine the Occupational Health and Safety Administration's database. Journalists recount how to go beyond OSHA records, using court records, state worker's compensation data and more.\n\nBy popular demand, this issue also recaptures highlights on the NICAR listserv, where reporters debated selling data directly to readers. And we include an update on the SEC, which may change its free Internet site popular with reporters.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-osha-falls-flat","slug":"1995-november-osha-falls-flat","url":"https://dwillis.github.io/uplink/article/1995-november-osha-falls-flat.html","year":1995,"month":"November","monthNum":11,"headline":"OSHA falls flat","kicker":"","author_name":"Jo Craven","author_title":"Staff","affiliation":"NICAR","summary":"An exploration of how government fines for workplace safety violations often undervalue the loss of life and limb. The article references reports uncovering minimal fines for fatal workplace accidents and the ongoing feature of OSHA records by journalists.","keywords":["OSHA","fines","safety"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Hazardous working conditions can cost employees an arm and a leg - or worse. But until recently, the government put little value on the loss of life and limb. In 1986, Cleveland King and a co-worker were killed in Atlanta, Ga., when an unsupported concrete-block wall collapsed, crushing them both to death. The company was fined $2. In 1989, four workers were injured in an explosion at the Phillips 66 Houston Chemical Complex in Pasadena, Texas. Two of the four died. The company was fined $720. In 1989, Richard Spencer was crushed to death by a forklift at Bethlehem Steel Corp. in Sparrows Point, Md. The company was fined $585. These were among the many facts unearthed in Occupational Safety and Health Administration data by Mike Casey and Russell Carollo in a series that ran four years ago in the Dayton Daily News. This year, Virginia Baldwin Hick and Natalya Shulyakovskaya of the St. Louis Post-Dispatch found there are still good stories in OSHA records. Hick first used OSHA data in a spring St. Louis Post-Dispatch series that examined local workplace falls that resulted in fatalities. Hick said, \"I cut my teeth in computer-assisted reporting by using OSHA data.\" In a Sept. 4 St. Louis Post-Dispatch story, Hick and Shulyakovskaya used OSHA data to document the organization's progress nationally in deterring workplace accidents. In 1991, after OSHA violations had peaked in 1990, Congress increased fines. Since then, accident and fatality rates have been declining.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-november-workplace-inspections-short-on-surprises","slug":"1995-november-workplace-inspections-short-on-surprises","url":"https://dwillis.github.io/uplink/article/1995-november-workplace-inspections-short-on-surprises.html","year":1995,"month":"November","monthNum":11,"headline":"Workplace inspections Short on surprises","kicker":"","author_name":"Sarah Christian","author_title":"","affiliation":"The Associated Press","summary":"An AP investigation found that three-quarters of U.S. workplaces where serious accidents occurred had never been inspected by OSHA during the 1990s, with the agency spending more time on union complaints than conducting preventive surprise inspections. The study of 778,000 OSHA inspections revealed systemic gaps in oversight, particularly affecting small, non-union businesses.","keywords":["OSHA","workplace","inspections"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Three-quarters of U.S. work sites where serious accidents occurred in 1994 and early 1995 had never been inspected during this decade by the Occupational Safety and Health Administration. Together these accidents killed more than 1,800 workers and injured thousands more. This fall, AP studied the records of 778,000 inspections conducted by OSHA between January 1989 and May 1995 and examined the most serious accidents in 1994 through April 1995. The resulting story, \"OSHA Fails to Check Three Fourths of Places Where Workers Die,\" was released Sept. 5 and 6. Some of what AP found included: OSHA inspectors were not doing a large number of surprise inspections. These unannounced \"program inspections\" are supposed to convince businesses to follow safety regulations because OSHA staff could, in theory, show up at any time and slap fines and citations on companies that don't follow the rules. OSHA staff often spent its time chasing down union complaints, many of which resulted in no serious violations. Nationally, 54 percent of OSHA inspections were surprise inspections during the period studied by AP. However, several states, \"inundated with complaints that must be inspected first, lag far behind the national average,\" AP reported. Of the 6,411 workplaces AP concentrated on, 75 percent had not been visited by OSHA in the previous five years. Seventy-six percent of those were non-union companies and more than half were small shops with fewer than 50 employees.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-november-selling-data-to-readers","slug":"1995-november-selling-data-to-readers","url":"https://dwillis.github.io/uplink/article/1995-november-selling-data-to-readers.html","year":1995,"month":"November","monthNum":11,"headline":"Selling data to readers","kicker":"On the NICAR-L listserv","author_name":"Naomi Aoki","author_title":"","affiliation":"Columbia (Mo.) Missourian","summary":"Journalists debate whether selling databases of government records aligns with their mission and ethical standards, with some arguing it's a legitimate way to distribute information while others worry about privacy violations, credibility damage, and compromising journalistic integrity for profit.","keywords":["databases","journalism","privacy"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"In the information age, information itself is one of the hottest commodities. Selling information, of course, is nothing new. That is the business of journalism: But the question facing journalists today is whether to sell databases of government records, made more accessible by journalists, or compiled by reporters. \"There are lots of ways to meet our mission of giving people the information they need to survive in the world,\" said Steve Doig of the Miami Herald during a recent conversation on the NICAR-L listserv. \"We should not be so narrowly focused that the only way to accomplish that mission is on newsprint through a journalist's words.\" Doig argues that as long as the same standards of accuracy and newsworthiness are applied to databases, selling them is no different from selling newspapers. He is not alone. Bob Warner at the Philadelphia Daily News added, \"If people want the information, we should try to make it available, in formats convenient to them, for an amount that covers our out-of-pocket costs but does not exploit our readers.\" Some fear selling databases will inspire a public outcry against invasion of privacy, further damage the media's credibility, and make it harder for reporters to get access to data. Rich Gordon, also of the Miami Herald, believes people's privacy concerns must be weighed in every decision on what to publish. Dan Browning of the St. Paul Pioneer Press agreed, adding that the impossibility of verifying the accuracy of some databases - particularly sensitive ones like driver's license records and criminal records - could hurt a newspaper's credibility. Other reporters believe the profit motive would compromise journalism. Currently, journalists are often charged less for databases than those in other industries. Gregg Leslie, a staff attorney at the Reporters Committee for Freedom of the Press, added, \"Reselling government data - even value-added data and even at break-even rates that subsidize other operations - will not start us down a slippery slope away from the public interest defense; it will banish it.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1995-november-data-on-the-net","slug":"1995-november-data-on-the-net","url":"https://dwillis.github.io/uplink/article/1995-november-data-on-the-net.html","year":1995,"month":"November","monthNum":11,"headline":"Data on the 'Net","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"News organizations like the Virginian Pilot and Detroit's WDIV-TV are providing publicly accessible data online, reflecting a growing trend of transparency in journalism.","keywords":["data","journalism","transparency"],"topics":["Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Among the news organizations that offer data on the Internet are the Virginian Pilot at http://data.pilotonline.com and Detroit's WDIV-TV http://www.wdiv.com/index.htm","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-cover-campaigns-better","slug":"1995-november-cover-campaigns-better","url":"https://dwillis.github.io/uplink/article/1995-november-cover-campaigns-better.html","year":1995,"month":"November","monthNum":11,"headline":"Cover campaigns better","kicker":"IRE and NICAR sessions in January","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Investigative Reporters and Editors and the National Institute for Computer-Assisted Reporting are hosting a weekend training workshop on January 20-21 at the University of Maryland to teach journalists how to cover the 1996 presidential election, including campaign finance analysis, polling expertise, and investigative reporting techniques.","keywords":["campaigns","training","journalism"],"topics":["Government & Politics","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Investigative Reporters and Editors and the National Institute for Computer-Assisted Reporting kick off the 1996 presidential election year with a weekend of comprehensive training in covering federal, state and local campaigns. From understanding polls and their flaws to tracing special interest money that flows into campaign warchests, this workshop will give you the basics and the nuances.\n\nThe conference will be Jan. 20-21 at the University of Maryland in College Park.\n\nThe program is scheduled to include:\n\nTraining in using computers to analyze federal, state and local campaign finances.\nTalks by experts on how polls succeed and fail.\nTips on how to come up with investigative stories while doing the daily coverage.\nSaving all your hard work during the campaign to keep the winners honest afterwards.\nCall IRE at (314) 882-2042 for more information and a registration form.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-from-the-nicar-library","slug":"1995-november-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1995-november-from-the-nicar-library.html","year":1995,"month":"November","monthNum":11,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering elections, aviation, finance, crime, safety, transportation, and education for investigative journalism purposes.","keywords":["databases","government","journalism"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["CD-ROM"],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nFederal Election Commission contributions data, including donations by individuals and political action committees.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\n\nFederal procurement data, 1992-1994, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records.\n\nNational Bridge Inventory System data, includes inspection grades.\n\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration loan guarantees, 1989-1994. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nU.S. Department of Education's Common Core of Data runs from the 1987-1988 to the 1992-1993 school year. It includes data on school enrollments, finances, staffing and dropouts.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nU.S. Department of Transportation fatal roadway accidents, 1988-1993.\n\nNational Endowment for the Arts, grants, 1989-1993.\n\nFor up-to-date prices and more information, call (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-get-computer-training","slug":"1995-november-get-computer-training","url":"https://dwillis.github.io/uplink/article/1995-november-get-computer-training.html","year":1995,"month":"November","monthNum":11,"headline":"Get computer training:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Medill School of Journalism and NICAR are offering computer-assisted reporting training programs for journalists in December and January-March at their respective locations.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Get computer training:\n\nThe Medill School of Journalism, four-day computer-assisted reporting bootcamp, Dec. 7-10. Medill's Washington, D.C., campus.\nNICAR Bootcamps, week-long intensive training seminars, jan. 7-12, and March 10-15, Columbia, Mo.\nThese dates are open to all journalists. For more information, call NICAR, (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-missed-cleveland","slug":"1995-november-missed-cleveland","url":"https://dwillis.github.io/uplink/article/1995-november-missed-cleveland.html","year":1995,"month":"November","monthNum":11,"headline":"Missed Cleveland?","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Information on how to order tape recordings and handouts from the CAR Rock '95 conference sessions through Sound Images and Investigative Reporters and Editors.","keywords":["conference","recordings","resources"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Missed Cleveland? To order tape recordings of CAR Rock '95 sessions, call Sound Images at (303) 649-1811. For handouts, call Investigative Reporters and Editors at (314) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-beyond-osha","slug":"1995-november-beyond-osha","url":"https://dwillis.github.io/uplink/article/1995-november-beyond-osha.html","year":1995,"month":"November","monthNum":11,"headline":"Beyond OSHA","kicker":"Other places to look for workplace data","author_name":"Cassandra Sweet","author_title":"","affiliation":"University of Missouri","summary":"Journalists used databases and public records to investigate workplace safety violations and deaths, revealing that OSHA failed to investigate most workplace fatalities and that labor laws were unenforced, leading to agency policy changes.","keywords":["investigation","workplace","databases"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["PC","Nutshell","DataTimes","EX IQ"],"full_text":"Paul Van Osdol of WJXT-TV in Jacksonville, Fla., was reporting a story on an industrial accident last year, when he decided to research deaths investigated by the Occupational Safety and Health Administration.\n\nThose inspections are required by law. With only an IBM-compatible 286 PC and a program called Nutshell, Van Osdol compiled information from death reports and OSHA investigative reports.\n\nBecause state law prohibited him from obtaining complete electronic versions of death reports, Van Osdol tracked down victims' families to get details. He accessed state vital statistics death records, probate court and circuit court records and GAO reports. Van Osdol's team also used the on-line resource, Data Times, to research companies where deaths occurred.\n\nVan Osdol found that OSHA investigated fewer than one-third of all workplace deaths in Florida during a three-year period. When OSHA did investigate, the average fine imposed was $2,900. After WJXT-TV's report, OSHA changed its procedures for investigating deaths involving longshoremen. The agency also promised to get tougher with companies that failed to report accidents.\n\nDateline NBC's David Hinchman, then at KOMU-TV in Columbia, Mo., reviewed electronic databases of OSHA inspections and Missouri workers compensation claims when doing a similar story on workplace fatalities.\n\nHinchman compared electronic records from OSHA and Missouri's Division of Workers Compensation. OSHA documented only 39 percent of the workplace fatalities that fell into its jurisdiction. He also found repeated on-the-job injuries at one plant did not lead to changes in workplace conditions.\n\nUsing compensation records\n\nWhile reporting for the Houston Chronicle in 1993, Nancy Stancill acquired tapes from the Texas Worker's Compensation Commission to examine 13,328 injuries to Texas teenage workers from January 1989 to October 1992. Stancill discovered that Texas' child labor law, on the books since 1981, had never been enforced. Although required by law, the state had never actually hired inspectors to monitor children's work conditions. And while the law sets 14 as the minimum working age, Stancill found some teen-agers who said they started working when they were 10 years old.\n\nStancill acquired names of businesses cited for violations from the Department of Labor. She then analyzed all the data using EX IQ and found the majority of injured teens were hurt in restaurants and grocery stores. Injuries ranged from sprains and cuts to amputations and deaths. She also found many teen-age farm workers became ill after being sprayed with pesticides.\n\nOne thing veterans of workplace investigations warn is that seemingly related information might not be stored in one place. Many government documents are stored on different PCs among various public agencies.\n\nCheck other oversight agencies\n\nDavid Armstrong of the Boston Globe suggests reporters check local industrial accident boards or equivalent administrative agencies that resolve workers compensation claims. \"They're a good resource for finding out why people are getting hurt on the job and what injured workers are getting in terms of compensation.\"\n\nMost states have an agency that handles claims independently, Armstrong said. These agencies' accident records include workers' descriptions of what happened, what kind of work they were doing, how long they've been off the job and what company they were working for. Information can be dug up including abuses of the system, people filing multiple claims, people who win multiple claims for the same kind of injuries at different companies and other fraud.\n\nOther sources include labor unions, some of which do studies on workplace issues, and state governmental records on their own employees and how they get hurt.\n\nNICAR Director Brant Houston suggests that if electronic accident, death or medical records are not readily available, civil and circuit court records can often fill in the information gap. Most cities and counties have computerized litigation records and can copy records onto diskettes for easy data analysis.\n\nCassandra Sweet can be reached at (314) 449-9368 or send e-mail to :656776@showme.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-tale-of-osha-tapes","slug":"1995-november-tale-of-osha-tapes","url":"https://dwillis.github.io/uplink/article/1995-november-tale-of-osha-tapes.html","year":1995,"month":"November","monthNum":11,"headline":"Tale of OSHA tapes","kicker":"","author_name":"Jo Craven","author_title":"Staff","affiliation":"NICAR","summary":"OSHA provides four interconnected data tables tracking workplace accidents, hazardous substances, company violations, and site information that can be analyzed together using activity numbers.","keywords":["OSHA","accidents","violations"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"OSHA data includes four tables, which may be analyzed separately or joined using the activity number field, which appears in each. Here are the tables and some of the information they contain: Accidents: Names those injured, identifies the task that was being performed when the accident occurred, indicates whether a hazardous substance contributed to the accident, lists which body parts were injured, and indicates the degree of the injury, including whether it was fatal. Hazardous Substances: Lists accidents involving hazardous substances not involving people. OSHA: Gives company sites, previous violations, the number of workers employed at the site in the previous 12 months and other OSHA activity at the sites. Violations: Details penalties, types of violations and the number of employees exposed to the danger.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-tips","slug":"1995-november-tips","url":"https://dwillis.github.io/uplink/article/1995-november-tips.html","year":1995,"month":"November","monthNum":11,"headline":"Tips","kicker":"","author_name":"Jo Craven","author_title":"Staff","affiliation":"NICAR","summary":"Journalists offer practical strategies for analyzing OSHA workplace safety data, including using ZIP codes for location specificity, comparing data with state records to identify gaps, and carefully examining multiple penalty fields that may show different amounts.","keywords":["OSHA","workplace","database"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":[],"full_text":"Here are some tips Carollo, Hick and Shulyakovskaya offer for handling OSHA data.\n\nUse ZIP codes. \"Some companies had many, many locations,\" Carollo said. \"If you used company names, like General Motors, you'd end up with a zillion hits. We used ZIP codes to identify locations.\"\n\nLook for deaths. \"One of the more interesting queries is to look for citations that result from fatalities,\" Carollo said. \"Once you've got those, you can get all the paperwork on it because somebody's dead. Then rank those in order of monetary fines.\"\n\nFind the survivors. \"Look through the data field of fatalities, find the names and look up the survivors early on,\" Hick said. \"It helped me focus my story and focus my interviews.\"\n\nKeep it simple. \"It's a pretty complicated database,\" Carollo said. \"Definitely make the queries simple.\"\n\nLook for what is NOT in the data. Casey and Carollo suspected that not all workplace deaths were recorded with OSHA. They confirmed their suspicion by comparing the data with state medical examiners' reports in Oklahoma and Arkansas. \"I picked states that had a coroner's office that had good records,\" Carollo said.\n\nExamine penalty fields carefully. \"There are actually three different amounts of penalties that you will see for each inspection,\" Shulyakovskaya said. \"The first comes from the OSHA table; it's called 'total current penalties' and represents the initially assessed fine. Second is the 'current penalty reflecting changes and modifications,' which indicates the fine negotiated between the company and OSHA. It appears in the violations table. Third is the 'total penalty dollars remitted for the inspection,' which is what was actually paid. This appears in the OSHA table.\n\n\"Often, the difference is drastic,\" she said. \"There are two obvious reasons: Employees abate regulations or go into litigation.\"\n\nLook out for \"deleted\" records. \"OSHA has a habit of leaving records marked for deletion in the database with a flag,\" Shulyakovskaya said. \"Look for the 'delete' field in the violations table where (existing) records are marked for deletion.\"\n\nInspected or not? \"We discovered that about 4,000 records a year have no inspections,\" Hick said. The records are listed in the OSHA table, but the companies have not been inspected. The lack of inspection — which may result, for example, because the business closed or because it refused entry to the OSHA inspector — is noted in the field referred to in the record layout as \"Why-no inspec.\" While this may not alter overall trends, not noting it may skew specific calculations.\n\nTake advantage of SIC codes. \"I didn't get into looking at OSHA for dangerous occupations because the occupation field is rarely filled in,\" Hick said, \"but standard industrial codes (SIC) are listed. These are standard codes used throughout federal government. SIC codes indicate how the industry is classified in the federal bureaucracy. You can find out which types of industries have the most fatalities.\"\n\nUse clipping services. \"You'll get clippings from tiny, tiny weeklies that otherwise get ignored. If you're doing a national project, there is no better way to find anecdotes,\" Carollo said. \"It might be a brief in some paper in South Dakota and you get this great material.\"\n\nJo Craven can be reached at (314) 882-0684, or send e-mail to c655034@mizzoul.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-sec-ponders-edgars-fate","slug":"1995-november-sec-ponders-edgars-fate","url":"https://dwillis.github.io/uplink/article/1995-november-sec-ponders-edgars-fate.html","year":1995,"month":"November","monthNum":11,"headline":"SEC ponders EDGAR's fate","kicker":"On the internet","author_name":"Robert Anderson","author_title":"","affiliation":"University of Missouri","summary":"The SEC is considering privatizing EDGAR, its free public database of company financial documents, and is soliciting public input through a survey with an extended deadline of November 30. The move has sparked concerns from advocates of free public access who believe for-profit companies are lobbying to eliminate the free service.","keywords":["EDGAR","privatization","database"],"topics":["Data & Databases","Government & Politics","Business & Economy"],"technologies":["EDGAR","Internet","World Wide Web","Nexis"],"full_text":"Although the U.S. Securities and Exchange Commission has put a key database of company financial documents on the Internet, the agency is hinting that free access to the database might come to an halt. Among the possibilities the SEC raises is privatization the database, known as EDGAR. \"We want to throw the net as wide as we can\" for suggestions about how to run EDGAR, said Mike Bartell, an official in the SEC's Office of Information Technology. He was referring to a 12-question survey that was placed on the SEC's web page (http://www.sec.gov) and in the publication Commerce Business Daily on Oct. 9. The No. 1 question in the survey is: \"Should all, or a portion of EDGAR be privatized?\" Another question refers to the operation of EDGAR at the SEC's Web site: \"Should the agency continue to maintain and operate this service or should this service be provided by the private sector either on the Internet or via some other means?\" The questionnaire, called a request for information, had an original response deadline of Oct. 31 - but that has been extended to Nov. 30, according to the SEC. Bartell said the main reason for the extension is that \"there's an awful lot of interest in this.\" But he declined to say what the volume of the response or what responders were suggesting. EDGAR, which stands for Electronic Data Gathering, Analysis and Retrieval system, was phased in by the SEC in 1993 for access to filings of public companies. Among the documents is the 10-K, the annual financial report for a public company. Investors, journalists and the general public use 10-K's and other EDGAR documents to assess the financial well-being of companies.\n\nMalamud's service (http://www.town.hall.org) had carried the EDGAR documents on-line until Sept. 28, under a two-year pilot program underwritten by the National Science Foundation. The SEC took over control of the EDGAR system on that date.\nThe pilot program also allowed Internet Multicasting to carry full-text versions of patent documents filed with the U.S. Patent and Trademark Office. The funding for carrying that database ended at the same time as funding EDGAR. The Patent Office has promised to put up a free on-line system of searchable-but not full-text patent information by Nov. 9.\nDuring the pilot program, persons exploring EDGAR scrutinized an average of 17,000 documents per day, for a total of 3 million documents.\nThe SEC states that almost three-quarters of the 15,580 publicly traded domestic companies now use EDGAR to file their reports. Under a U.S. Congressional mandate, the remainder of the companies will file by May 1996.\nSince 1985, the SEC also has given out millions of dollars in contracts to such private firms as LEXIS-NEXIS of Dayton, Ohio, which packages raw EDGAR documents for resale.\nMalamud has said that for-profit companies tried to kill free on-line access to EDGAR when the Internet Multicasting project ended and the SEC took over the online service.\n\"Lobbyists for the Information Industry Association (IIA) have actively opposed this project,\" said Malamud in a September statement on the Internet Multicasting Web page. He invited people to write SEC Chairman Arthur Levitt with their complaints.\nIn an interview, Malamud praised the actions of the SEC to keep the free online service.\n\"My feeling is that the database is on-line with the SEC and it will be difficult to take a step backward by removing it,\" he said. \"However, the SEC has an obligation to find out what people and the industry think, hence the RFI.\"\nThe main SEC contracts for EDGAR expire at the end of 1996. SEC officials have said they may use responses from the questionnaire to put together a request for proposals from companies that want to take over the EDGAR system.\nRobert Anderson can be reached at (314) 449-6714, or send e-mail to Robert43@delphi.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-survey-no-problem","slug":"1995-november-survey-no-problem","url":"https://dwillis.github.io/uplink/article/1995-november-survey-no-problem.html","year":1995,"month":"November","monthNum":11,"headline":"Survey: 'No problem'","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Carl Malamud of Internet Multicasting Service dismisses the SEC's survey on EDGAR access, emphasizing that the actual online database implementation matters more than the agency's information request.","keywords":["EDGAR","access","SEC"],"topics":["Government & Politics"],"technologies":["Internet"],"full_text":"One of the key proponents for free access to EDGAR said he is not worried about the SEC's survey.\n\n\"Actions speak louder than words,\" said Carl Malamud of the non-profit Internet Multicasting Service. \"I think the on-line database is of more significance than an RFI (request information).\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-visit-nicar-at-httpwwwreporterorg","slug":"1995-november-visit-nicar-at-httpwwwreporterorg","url":"https://dwillis.github.io/uplink/article/1995-november-visit-nicar-at-httpwwwreporterorg.html","year":1995,"month":"November","monthNum":11,"headline":"Visit NICAR at http://www.reporter.org","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR's website, administered by Wallace Winfrey, provides comprehensive information about the organization's events, training seminars, newsletters, and database library, while also offering access to journalism resources from IRE and the Hillman Project.","keywords":["journalism","training","resources"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Visit NICAR at http://www.reporter.org. Administered by NICAR's web master, Wallace Winfrey, the web site offers detailed information about NICAR's activities. This includes upcoming events, training seminars, the newsletter Uplink, and NICAR's database library. In addition, you can search the journalism resource center run by Investigative Reporters and Editors, at www.ire.org The web site also includes a menu for IRE and for the Hillman Project, which is developing a site for tips and resources for journalists around the world.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-bits-bytes-and-barks","slug":"1995-november-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1995-november-bits-bytes-and-barks.html","year":1995,"month":"November","monthNum":11,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A newsletter covering recent internet and journalism news, including the defacement of the Nation of Islam's website, job moves by computer-assisted reporting professionals, and updates on professional journalist networks and listservs.","keywords":["hacking","journalism","internet"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","Netscape","Listserv","Email"],"full_text":"Bits, Bytes and Barks\n\nNation of Islam site defaced\n\nThe Nation Of Islam's Internet home page was defaced within two days after the Million Man March.\n\nA hacker electronically broke into the Internet site and peppered it with cyber-graffiti files.\n\nThe page opened with its Netscape browser title changed to \"The Bigots Of Islam HOmeb0y pAg3!@#.\"\n\nOn the page itself, the opening read, \"In the name of some goofy god that does not exist! @#.' The site continued with previous welcome message of \"As-Salaam Alaikum (Peace be unto you),\" defaced by another set of parentheses: \"(As if we really belive that)\" (sic).\n\nA source at Object Oriented Information Systems (02IS), the Internet consulting company that runs Afrinet Central, home of the Nation of Islam page, said Nation of Islam staffers alerted 02IS to the break-in two days after the march. The source added that the company is working on its computer security to prevent future attacks.\n\nThe Cambridge, Mass.-based company had taken Nation's Home Page off the Internet while it eliminated the hacker files. The site, which includes speeches by Minister Louis Farrakhan, resumed at the same address, http://www.afrinet.net/~islam/\n\nTimothy K. Maloy, The Internet Newsroom, editors@DGS.dgsys.com\n\nBound for California\n\nJennifer LaFleur, who has helped hundreds of journalists learn computer-assisted reporting, recently left as NICAR's training director. In late October, she started her new job as database editor for the San Jose Mercury News. She can be reached at (408) 920-5728. or send e-mail jenster@aol.com\n\nBound for Omaha\n\nCarol Napolitano, formerly computer-assisted reporting coordinator at The Times in Munster, Ind., will be joining the Omaha World-Herald on Nov. 20. She will bring her CAR skills to the paper as a member of its newly created public affairs reporting team.\n\nSwedish IRE begins listserv\n\nSwedish IRE recently began MEDIA, a European listserv on journalism, investigative reporting and computer-assisted reporting. The list includes bilingual journalists able to help English-speaking reporters covering Europe. To subscribe, send e-mail to listserv@sokrates.mip.ki.se. If you have questions, send e-mail to Michel Bajuk, head of the Swedish IRE-secretariate, at bajuk@jmk.su.se\n\nJoin NICAR on the Internet\n\nDon't forget to keep up with NICAR on the Internet. Subscribe to our listserve and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURIEDU. In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-nation-of-islam-site-defaced","slug":"1995-november-nation-of-islam-site-defaced","url":"https://dwillis.github.io/uplink/article/1995-november-nation-of-islam-site-defaced.html","year":1995,"month":"November","monthNum":11,"headline":"Nation of Islam site defaced","kicker":"","author_name":"Timothy K. Maloy","author_title":"","affiliation":"The Internet Newsroom","summary":"The Nation of Islam's website was hacked and defaced with cyber-graffiti shortly after the Million Man March, with the hosting company removing the site to eliminate malicious files before restoring it.","keywords":["hacker","website","security"],"topics":["Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","Netscape"],"full_text":"Nation of Islam site defaced\n\nThe Nation Of Islam's Internet home page was defaced within two days after the Million Man March.\n\nA hacker electronically broke into the Internet site and peppered it with cyber-graffiti files.\n\nThe page opened with its Netscape browser title changed to \"The Bigots Of Islam HOmeb0y pAg3!@#.\"\n\nOn the page itself, the opening read, \"In the name of some goofy god that does not exist! @#.' The site continued with previous welcome message of \"As-Salaam Alaikum (Peace be unto you),\" defaced by another set of parentheses: \"(As if we really belive that)\" (sic).\n\nA source at Object Oriented Information Systems (02IS), the Internet consulting company that runs Afrinet Central, home of the Nation of Islam page, said Nation of Islam staffers alerted 02IS to the break-in two days after the march. The source added that the company is working on its computer security to prevent future attacks.\n\nThe Cambridge, Mass.-based company had taken Nation's Home Page off the Internet while it eliminated the hacker files. The site, which includes speeches by Minister Louis Farrakhan, resumed at the same address, http://www.afrinet.net/~islam/\n\nTimothy K. Maloy, The Internet Newsroom, editors@DGS.dgsys.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-bound-for-california","slug":"1995-november-bound-for-california","url":"https://dwillis.github.io/uplink/article/1995-november-bound-for-california.html","year":1995,"month":"November","monthNum":11,"headline":"Bound for California","kicker":"","author_name":"","author_title":"database editor","affiliation":"San Jose Mercury News","summary":"Jennifer LaFleur, a prominent computer-assisted reporting educator who trained hundreds of journalists at NICAR, has transitioned to a new role as database editor at the San Jose Mercury News.","keywords":["journalism","training","databases"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":[],"full_text":"Jennifer LaFleur, who has helped hundreds of journalists learn computer-assisted reporting, recently left as NICAR's training director. In late October, she started her new job as database editor for the San Jose Mercury News. She can be reached at (408) 920-5728. or send e-mail jenster@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-bound-for-omaha","slug":"1995-november-bound-for-omaha","url":"https://dwillis.github.io/uplink/article/1995-november-bound-for-omaha.html","year":1995,"month":"November","monthNum":11,"headline":"Bound for Omaha","kicker":"","author_name":"","author_title":"computer-assisted reporting coordinator","affiliation":"Omaha World-Herald","summary":"Carol Napolitano is joining the Omaha World-Herald as a computer-assisted reporting specialist on their new public affairs reporting team, bringing her CAR expertise from her previous role at The Times in Munster, Indiana.","keywords":["reporting","journalism","technology"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Carol Napolitano, formerly computer-assisted reporting coordinator at The Times in Munster, Ind., will be joining the Omaha World-Herald on Nov. 20. She will bring her CAR skills to the paper as a member of its newly created public affairs reporting team.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-swedish-ire-begins-listserv","slug":"1995-november-swedish-ire-begins-listserv","url":"https://dwillis.github.io/uplink/article/1995-november-swedish-ire-begins-listserv.html","year":1995,"month":"November","monthNum":11,"headline":"Swedish IRE begins listserv","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Swedish IRE has launched MEDIA, a European listserv connecting journalists interested in investigative reporting and computer-assisted reporting, with bilingual members available to assist English-speaking reporters covering European stories.","keywords":["listserv","journalism","European"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Listserv","Email"],"full_text":"Swedish IRE recently began MEDIA, a European listserv on journalism, investigative reporting and computer-assisted reporting. The list includes bilingual journalists able to help English-speaking reporters covering Europe. To subscribe, send e-mail to listserv@sokrates.mip.ki.se. If you have questions, send e-mail to Michel Bajuk, head of the Swedish IRE-secretariate, at bajuk@jmk.su.se","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-november-join-nicar-on-the-internet","slug":"1995-november-join-nicar-on-the-internet","url":"https://dwillis.github.io/uplink/article/1995-november-join-nicar-on-the-internet.html","year":1995,"month":"November","monthNum":11,"headline":"Join NICAR on the Internet","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR and IRE offer listserve subscriptions for journalists to discuss reporting techniques and professional development through email-based communities.","keywords":["journalism","listserve","networking"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"Don't forget to keep up with NICAR on the Internet. Subscribe to our listserve and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURIEDU. In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-hitching-a-free-ride","slug":"1995-december-hitching-a-free-ride","url":"https://dwillis.github.io/uplink/article/1995-december-hitching-a-free-ride.html","year":1995,"month":"December","monthNum":12,"headline":"Hitching a free ride","kicker":"","author_name":"Jo Craven","author_title":"Staff","affiliation":"NICAR","summary":"A U.S. News and World Report investigation reveals that 1.1 million nonprofit organizations in the U.S. control $1.475 trillion in assets while avoiding taxes on property, income, and capital gains, costing municipalities and the federal government billions in potential tax revenue.","keywords":["nonprofits","taxes","investigation"],"topics":["Government & Politics","Business & Economy"],"technologies":[],"full_text":"Can you say nonprofit, kids? The National Football League can. So can the Professional Golf Association. So can the National Rifle Association. And so can the Motion Picture Association of America, the folks who put on the lavish Academy Awards ceremony each year. These are just a few of the wealthy organizations listed as nonprofit by the Internal Revenue Service. U.S. News and World Report recently spent four months investigating so-called non-profit organizations. In stories published Oct. 2, reporters Penny Loeb, Edward Pound and Gary Cohen revealed that 1.1 million \"non-profit\" organizations in the United States control $1.475 trillion is assets while claiming tax-exempt status.\n\nFreebies\n\nThat means they do not pay taxes on property. They do not pay taxes on income. They do not pay taxes on profits from capital gains, royalties, interest, dividends and rents from debt-free real estate. They get reduced postal rates.\n\nThey do compete with for-profit enterprises, which claim that their own tax dollars underwrite the products of their non-profit competitors. And they do get to keep a chunk of change that could otherwise benefit the U.S. Treasury and various municipalities and schools.\n\nFor example, financially ailing Washington, D.C., is home to 1,800 nonprofits. If those groups paid taxes on their land and buildings, Loeb reported, the district would collect nearly $94 million more each year. New York City, which recently cut $754 million from its school budget to balance the city budget, could collect an additional $524 million from nonprofits. Albany, N.Y., could collect $25 million more from taxing nonprofits. The cost to the U.S. Treasury is impossible to calculate, the magazine concluded.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-who-profits-at-nonprofits","slug":"1995-december-who-profits-at-nonprofits","url":"https://dwillis.github.io/uplink/article/1995-december-who-profits-at-nonprofits.html","year":1995,"month":"December","monthNum":12,"headline":"Who profits at nonprofits","kicker":"Look at the fine print","author_name":"Jo Craven","author_title":"NICAR staff","affiliation":"NICAR","summary":"Journalists provide guidance on how to investigate nonprofit organizations for executive enrichment, hidden compensation, and misuse of tax-exempt status through examination of salaries, political contributions, and business operations.","keywords":["nonprofits","executives","compensation"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":["IPEDS"],"full_text":"Who profits at nonprofits\n\nNon-profits make a lot of promises. Some promise to help the poor. They are not supposed to enrich their executives. They all face certain rules if they run businesses on the side. U.S. News and World Report's Penny Loeb and the Philadelphia Inquirer's Neill Borowski have spent a lot of time examining nonprofits. Here are some of their tips:\n\nCheck executive income. The Internal Revenue Code stipulates that nonprofit revenues should not be used to enrich executives. Yet, U.S. News and World Report found that the NFL paid Commissioner Paul Tagliabue $2.5 million in salary and gave him a $950,000 loan.\n\nLook for disguised executive incomes. \"You might be looking at one form for an administrator's salary, and it's, say, $90,000,\" Borowski said. \"That's very low compared to the norm. Then, when you start to put together related organizations, you have to total up the salaries to give an overall umbrella salary.\"\n\nExamine political contributions. U.S. News and World Report found that the National Education Association, the nation's largest union with 2.2 million members and a nonprofit, takes in $200 million a year and has made $8.9 million in political contributions.\n\nIdentify subsidiary companies. The Unrelated Business Income Tax act of 1950 requires nonprofits to pay taxes on income from subsidiary businesses unrelated to their tax-exempt mission. Loeb, Pound and Cohen reported that fewer than 5 percent of nonprofits reported unrelated business income. However, what is \"unrelated\" income is ill-defined.\n\n\"Most people think of nonprofits as primarily getting donations from the public, but they get most of their revenue from operations,\" Borowski said. \"You have to step back and say, 'Which of these operations are like businesses and are competing with for-profit business?'\"\n\nWhen examining hospitals, confirm indigent care. Hospitals qualify for tax exemption based on such care. Yet, U.S. News and World Report found that six hospitals examined did not spend significant percentages of gross income on indigent care.\n\nWhen examining educational institutions, also check out the Integrated Post-Secondary Education Data System (IPEDS), a database from the U.S. Department of Education on the nation's colleges. \"It is a big database that comes out every year,\" Borowski said. \"It is really rich in financial, enrollment and scholarship information.\"\n\nWho profits at nonprofits\n\nNon-profits make a lot of promises. Some promise to help the poor. They are not supposed to enrich their executives. They all face certain rules if they run businesses on the side. U.S. News and World Report's Penny Loeb and the Philadelphia Inquirer's Neill Borowski have spent a lot of time examining nonprofits. Here are some of their tips:\n\nCheck executive income. The Internal Revenue Code stipulates that nonprofit revenues should not be used to enrich executives. Yet, U.S. News and World Report found that the NFL paid Commissioner Paul Tagliabue $2.5 million in salary and gave him a $950,000 loan.\n\nLook for disguised executive incomes. \"You might be looking at one form for an administrator's salary, and it's, say, $90,000,\" Borowski said. \"That's very low compared to the norm. Then, when you start to put together related organizations, you have to total up the salaries to give an overall umbrella salary.\"\n\nExamine political contributions. U.S. News and World Report found that the National Education Association, the nation's largest union with 2.2 million members and a nonprofit, takes in $200 million a year and has made $8.9 million in political contributions.\n\nIdentify subsidiary companies. The Unrelated Business Income Tax act of 1950 requires nonprofits to pay taxes on income from subsidiary businesses unrelated to their tax-exempt mission. Loeb, Pound and Cohen reported that fewer than 5 percent of nonprofits reported unrelated business income. However, what is \"unrelated\" income is ill-defined.\n\n\"Most people think of nonprofits as primarily getting donations from the public, but they get most of their revenue from operations,\" Borowski said. \"You have to step back and say, 'Which of these operations are like businesses and are competing with for-profit business?'\"\n\nWhen examining hospitals, confirm indigent care. Hospitals qualify for tax exemption based on such care. Yet, U.S. News and World Report found that six hospitals examined did not spend significant percentages of gross income on indigent care.\n\nWhen examining educational institutions, also check out the Integrated Post-Secondary Education Data System (IPEDS), a database from the U.S. Department of Education on the nation's colleges. \"It is a big database that comes out every year,\" Borowski said. \"It is really rich in financial, enrollment and scholarship information.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-legal-griefs","slug":"1995-december-legal-griefs","url":"https://dwillis.github.io/uplink/article/1995-december-legal-griefs.html","year":1995,"month":"December","monthNum":12,"headline":"Legal griefs","kicker":"When agencies don't deliver records","author_name":"Sig Splichal","author_title":"Assistant Professor","affiliation":"School of Communication","summary":"Public records laws written for paper documents are inadequate for the digital age, creating access challenges that fall into three categories: what government agencies cannot do due to lack of training or technical capability, what they will not do due to unclear legal standards around data compilation and format, and what they should not do regarding privacy concerns with centralized digital databases.","keywords":["access","digital","privacy"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"Most laws governing public and media access to government records were written and tested in court when most records had four corners and fit neatly into file cabinets.\n\nPublic record laws were designed and applied by courts to strike a balance. On one side are government efficiency and personal privacy. On the other are the needs of the public and media for reasonable access.\n\nAs governments shifted from paper to digital record systems during the past three decades, many laws are no longer adequate to ensure access. Problems confronting the public and media fall into three categories based on what government records custodians can't do, won't do and shouldn't do.\n\nCan't do\n\nOften records custodians have little training, especially at the local level, or are not \"computer persons.\" They don't understand requests or are unwilling to attempt to meet them. An oft-heard refrain: \"But our computers can't do that.\" Reporters lacking computer skills often contribute to the problem, not knowing where and how to obtain information, or what questions to ask.\n\nWon't do\n\nIn the paper era, records custodians were not required to gather information from different sources and put it together in a meaningful way. Records either existed or they didn't. But does information gleaned from a computer at a reporter's request constitute the \"creation\" of a new record? Is sorting a database akin to searching a file cabinet. Courts disagree among themselves.\n\nShould a custodian be required to create a computer program at government expense to extract information to meet a reporter's request? Courts disagree among themselves.\n\nWhat about the format in which a computer record is released? Who should decide, the agency or the requester? Is it reasonable access when an agency decides to give a requester a computer database on hundreds of pages of printouts when the same could be provided at much less cost to taxpayers, and in a much more usable form on computer tape? Courts disagree among themselves.\n\nFinally, who should pay the cost of ensuring that government agencies use their computers to provide public records? The agency or the requesters? Courts disagree among themselves.\n\nShouldn't do\n\nA more fundamental question deals with the very nature of computerized records and how such records differ substantively from their paper counterparts. Government information about individuals used to be less threatening to privacy.\n\nThe U.S. Supreme Court, in the influential 1989 opinion Justice Department vs. Reporters Committee for Freedom of the Press, lamented the demise of this so-called \"practical obscurity.\" The court, in reasoning that threatens to limit access to computerized records, suggested that public records gathered from many public sources compiled in a central government database enjoyed a rejuvenated privacy interest.\n\nIn other words, the way in which records are stored could become as important as their content. This reasoning has not been lost on records custodians, who arguably could raise privacy concerns to shield information that rightly belongs in the public domain.\n\nThe potential dangers of computerized records have not been lost on legislatures. Protecting individual privacy is attractive politically. Several states, in the wake of a federal anti-stalking law, limited access to drivers license records.\n\nThe computerization of government records creates access problems on a number of levels. None has been resolved adequately to both facilitate access and ensure government efficiency and privacy. Journalists must understand these issues if they are to maintain continued levels of access and properly frame the issues in the ongoing legal and political debates.\n\nIronically, there is a downside to the resounding success of journalists who have mined government databases for stories rich in detail and social and political relevance. As journalists become more adept with computers, custodians likewise become more sophisticated at using computers and legal confusion over open records laws to thwart access.\n\nSig Splichal is assistant professor at University of Miami's School of Communication.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-training-mexican-reporters","slug":"1995-december-training-mexican-reporters","url":"https://dwillis.github.io/uplink/article/1995-december-training-mexican-reporters.html","year":1995,"month":"December","monthNum":12,"headline":"Training Mexican reporters","kicker":"IRE wins grant to start operation","author_name":"","author_title":"","affiliation":"","summary":"Investigative Reporters and Editors (IRE) received a $544,000 grant to establish a Mexican Reporting Institute focused on training journalists in advanced reporting skills and computer-assisted data analysis. The initiative aims to develop a cooperative network of Mexican journalists and improve access to information in the country.","keywords":["training","journalism","Mexico"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Investigative Reporters and Editors (IRE) recently won a $544,000 three-year grant from the Robert T. McCormick Tribune Foundation of Chicago to work with Mexican journalists in launching a Mexican Reporting Institute.\n\nThe main objectives will be training Mexican reporters, broadcasters and editors in advanced reporting skills, especially computer-assisted acquisition and analysis of data; developing a cooperative network of journalists, and supporting efforts to improve access to information.\n\n\"The Mexican Reporting Institute could not have come at a better time,\" said Rossana Fuentes-Berain, an IRE member at Reforma in Mexico City. She will serve on the search committee looking for leadership of the Institute and on the Institute Advisory Council.\n\nThe Robert R. McCormick Tribune Foundation has assets of more than $700 million. One of the priorities of the foundation's journalism grants program is support of freedom of expression in the Americas.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-from-the-nicar-library","slug":"1995-december-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1995-december-from-the-nicar-library.html","year":1995,"month":"December","monthNum":12,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to numerous federal government databases covering elections, aviation, finance, crime, transportation, and small business data, with instructions for connecting to the IRS bulletin board system IRIS.","keywords":["databases","government","federal"],"topics":["Data & Databases","Investigative Methods"],"technologies":["modem/BBS","Telnet","FTP","Internet"],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nFederal Election Commission contributions data, including donations by individuals and political action committees.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1994, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records.\nNational Bridge Inventory System data, includes inspection grades.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nNEW U.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nNEW U.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1994.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nNEW National Endowment for the Arts, grants, 1989-1993.\n\nFor up-to-date prices and more information, call (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.\n\nThere are several ways to get into the IRS' bulletin board, known as IRIS. You can dial direct with a modem. The phone number is (703) 321-8020. Set your modem to no parity, 8 bits, 1 stop bit. The system runs at a 9600 baud rate. The terminal emulation should be set to ANSI. Set the duplex connection to full. If you see odd graphics, you can turn them off by typing this at a command line:\n\nYou can also connect to IRIS via telnet and ftp. The telnet address is fedworld.gov. The ftp address is ftp.fedworld.gov. Indicate at the prompt that you want to hook into IRIS. The logon for the public is \"anonymous.\" The password is your e-mail address. The IRS keeps the site running 24 hours a day, seven days a week. The systems operator is at (202) 874-0277. Once you're in the system, there are instructions on how to use it.\n\nIn 1913, when income tax began, nonprofits were born. Since then, Loeb, Pound and Cohen reported, Congress has recognized 24 categories of tax-exempt organizations, besides charities. Covered are professional and trade groups that promote the welfare of their members, colleges, health-care providers, retirement funds, real-estate boards, credit unions, civic leagues, and more.\n\nInquirer's analysis\n\nPhiladelphia Inquirer reporters Gilbert Gaul and Neill Borowski were the first reporters to analyze nonprofits as a whole sector. The result was a seven-part series published in 1993, which became a book called \"Free Ride.\"\n\nGaul and Borowski spent two years poring over federal budgets, tax returns and federal and state economic statistics. They went through 50 years of economic data to track the growth of nonprofits. And they examined 6,000 organizations picked from the IRS' master list of nonprofits.\n\nTheir selections included the nation's largest nonprofits (those with revenues and assets valued at $10 million), those of regional interest to their readers and others, such as the NFL, that were \"just plain interesting.\"\n\n'Grow your own'\n\nGaul and Borowski gathered paper 990 forms, which nonprofits are required to file with the IRS, and built numerous spreadsheets and databases.\n\n\"I call it 'grow your own,'\" Borowski said. But, several years later, Loeb found the quickest way to get data is to download it from the IRS bulletin board.\n\n\"It's free, but you have to dial it directly,\" Loeb said. \"The IRS data is in seven files, one for each region of the country, and someone doing their own city can get their own region zipped in ascii.\"\n\nIt took two to three hours to download one region, Loeb said.\n\n\"It's very basic data: assets, their most recent year of income, name and address, category of nonprofit, their three chief interests - what they support,\" Loeb said. \"I would say it's some of the cleanest data I have seen. The queries were very simple, sorting by category and adding up; summing total for assets and incomes.\"\n\nGathering 990s\n\nLike the Inquirer reporters, Loeb, Pound and Cohen also spent time collecting 990s from the nonprofits. They all found that getting that data can be a little tricky.\n\n\"The rule is the nonprofits have to allow you to read the three most recent years of their 990s,\" Borowski said. \"They don't have to make copies for you. They don't have to do anything but let you see them and copy them down.\"\n\nBoth Loeb and Borowski said the best way to deal with this is to take blank 990 forms to the organizations you are interested in.\n\n\"Simply take that copy in and fill out the form based on the way they filled it out,\" Borowski said. Blank 990s can be obtained from the IRS or from the IRS web site at http://www.ustreas.gov/treasury/bureaus/irs/fwforms.html. You also need to download the Adobe Acrobat reader to open the forms.\n\nBorowski cautioned reporters to view completed 990s skeptically. \"You have to realize that some organizations clearly did not take the forms very seriously and weren't that careful in the way they filed them,\" he said. \"Each 990 is a mystery, and you have to unravel it.\"\n\nBut the result is worth the trouble.\n\n\"Every tax exempt organization has a cost to the taxpayer,\" Borowski said. \"They are not benefiting the public; there is a cost involved. You have to read the form with that in mind.\"\n\nJo Craven can be reached at (314) 882-0684 or send e-mail to c6.55034@mizzoul.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-get-computer-training","slug":"1995-december-get-computer-training","url":"https://dwillis.github.io/uplink/article/1995-december-get-computer-training.html","year":1995,"month":"December","monthNum":12,"headline":"Get computer training","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"Multiple computer-assisted reporting training seminars and bootcamps are available for journalists throughout early 1996, with registration open to all news professionals. Additionally, recordings and materials from the CAR Rock '95 conference are available for purchase.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Get computer training:\nNICAR Bootcamps, week-long intensive training seminars, Jan. 7-12, March 10-15, and May 19-24, Columbia, Mo.\nNICAR/Medill Broadcast Seminar, March 23-24, Chicago\nNorth Carolina Advanced Computer-Assisted Reporting Seminar, May 5-10, Chapel Hill, N.C.\n\nThese dates are open to all journalists. For more information, call NICAR, (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.\n\nMissed Cleveland? To order tape recordings of CAR Rock '95 sessions, call Sound Images at (303) 649-1811. For handouts, call Investigative Reporters and Editors at (314) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-check-out-irs-web-site","slug":"1995-december-check-out-irs-web-site","url":"https://dwillis.github.io/uplink/article/1995-december-check-out-irs-web-site.html","year":1995,"month":"December","monthNum":12,"headline":"Check out IRS web site","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This article directs readers to the IRS website on the U.S. Treasury Department's official domain.","keywords":["IRS","website","Treasury"],"topics":["Technology"],"technologies":["World Wide Web"],"full_text":"Check out IRS web site\n\nThe Internal Revenue Service can be found at http://www.ustreas.gov/treasury/bureaus/irs.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-tracking-nonprofits","slug":"1995-december-tracking-nonprofits","url":"https://dwillis.github.io/uplink/article/1995-december-tracking-nonprofits.html","year":1995,"month":"December","monthNum":12,"headline":"Tracking nonprofits","kicker":"On the Internet","author_name":"Nora Paul and Andy Lehren","author_title":"","affiliation":"Poynter Institute / NICAR","summary":"A guide for journalists on how to research and background nonprofit organizations using online resources, including financial evaluation tools, trade publications, and comprehensive nonprofit directories.","keywords":["nonprofits","research","journalists"],"topics":["Journalism & Newsroom","Technology","Business & Economy"],"technologies":["Internet","World Wide Web","gopher","Email","Listserv"],"full_text":"Tracking nonprofits\n\nBy Nora Paul and Andy Lehren\n\nThe Internet offers a growing list that can help reporters background a nonprofit organization. Learn how to better read their financial reports. Tap into the trade press. See how they should be run. And, of course, you can find all kinds of nonprofits and foundations.\n\nBe prepared\n\nLearn how to evaluate nonprofits. This site offers a good introduction on some of the financial measures to evaluate. This includes a section on how to read a nonprofit's financial statement. Visit http://www.human.com:80/inc/lib/eval.html\n\nThe National Charities Information Bureau wants to promote informed giving to charities. Part of that effort includes evaluations of more than 300 nonprofits. For instance, Accuracy in Media does not meet all standards. The group also outlines its standards to better help you evaluate a nonprofit. Go http://www.give.org/\n\nLearn more about how nonprofits should run from the National Center for Nonprofit Boards. It offers advice on how boards of directors should work, sample job descriptions for chief executives, and case studies on running nonprofits. Visit gopher://ncnb.org:7002/1\n\nTrade press offerings\n\nSee how the trade press covers nonprofits. This list offers links to a variety of publications. There is The NonProfit Times, which recently profiled how nonprofits use databases to raise money. The Philanthropy News Digest, run by the Foundation Center, offers tidbits compiled from the nation's media, like how Minnesota nonprofits are worried about 3M's restructuring. Go to this link list at http://www.duke.edu/~ptavern/Pete.Philanthropic.html#Press\n\nOne publication not on the above list is The Chronicle of Philanthropy. But you can subscribe to the magazine's listserv. It's a one-way conversation, where every two weeks the magazine sends you a summary of issues facing nonprofits. Subscribe by sending e-mail to chronicle-request@nonprofit.com. Write \"subscribe chronicle\" as your message.\n\nChronicle editors also mail out a listing of forthcoming conferences, workshops, and other events. The publication cautions that the list is not a substitute for the paper version. Postings to the list are not allowed.\n\nLinks\n\nThere's also a list that links to hundreds of nonprofits with web sites, from repertory theaters to cat rescue groups to the Children's Defense Funds. Information at each of the sites differs. For example, the Children's Defense Fund has statistics, like \"Each day in America\" and information on how much it costs to raise a child. City Harvest has videos about some of their projects, and Debt Counselors of America has information on scams to avoid. Go to http://www.vive.com/www/virtual/all.htm\n\nThe Meta-Index for Non-Profit Organizations is among the most comprehensive Internet listings, broken down by categories, of resources by, for and about nonprofits. http://www.duke.edu/~ptavern/Pete.meta-index.html\n\nInternet Resources for Non-profit Public Service Organizations offers an excellent compilation of links to discussion groups, public records, legislative acts and government agencies of interest those who run nonprofits. http://asa.ugl.lib.umich.edu/chdocs/nonprofits/nonprofits.html\n\nPhilip A. Walker, an information systems manager for Emergency Food and Shelter National Board Program at United Way of America, runs a series of links to nonprofits. These include a variety of United Way sites. Go to http://www.clark.net/pub/pwalker/home.html\n\nFor health-related nonprofits, the University of Kentucky offers gopher://UKCC.uky.edu:70/1menu%0UKR200%21191/HLTHMNTL.INFO\n\nCampaign Central's page on Political Action Committees, trade associations and nonprofits is at http://www.clark.net/ccentral/nonprofs.htm\n\nAlso available is the Contact Center, which does just what the name says. It lets you get in contact with a lot of nonprofits and foundations. Go to http://www.contact.org/\n\nFor a listing of toll-free phone numbers of non-profit organization, go to gopher://woca7.nwoca.ohio.go 70/00gopher_root:%5b_other%5d_1800HELP\n\n\"Have I Got A Tip For You...\" a guide to investigative reporting by Stanley Penn, is available from IRE for $5. Penn is a Pulitzer Prize-winning reporter who retired in 1990 from the Wall Street Journal. IRE is also now selling guides to public records from BRB Publications Inc. The volumes include nationwide guides on where to find tax liens, bankruptcy records, and drivers license records. Prices vary. Call IRE for more information.\n\nNora Paul of the Poynter Institute can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org.\n\nAndy Lehren of NICAR can be reached at (314) 882-0684, or send e-mail to andy@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-cover-campaigns-better","slug":"1995-december-cover-campaigns-better","url":"https://dwillis.github.io/uplink/article/1995-december-cover-campaigns-better.html","year":1995,"month":"December","monthNum":12,"headline":"Cover campaigns better","kicker":"Workshop programs","author_name":"","author_title":"","affiliation":"","summary":"IRE, NICAR, and the University of Maryland are sponsoring a Campaign Coverage Workshop on January 20-21, 1996, offering comprehensive training in covering federal, state, and local elections with sessions on online coverage, candidate profiling, databases, and polling.","keywords":["campaign","workshop","journalism"],"topics":["Government & Politics","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"To register for the Campaign Coverage Workshop, call IRE at (314) 882-2042. The price for IRE or SPJ members is $96. The price for non-members is $136. The price for students who are not IRE members is $121. The price for international reporters who are not members is $151. For all those who are not members, the registration fee includes IRE membership. You may register by phone using a Visa or Master Card. The conference is at the Inn and Conference Center, University of Maryland, College Park, MD. Hotel rooms are available there. The rates are $74 for single occupancy, and $89 for double occupancy. Call the hotel at either (800) 727-8622 or (301) 985-7300.\n\nGet a jump start on the 1996 presidential election with a weekend of comprehensive training in covering federal, state and local campaigns. Investigative Reporters and Editors, the National Institute for Computer-Assisted Reporting and the University of Maryland, with support from Gannett News Service, are sponsors of the conference Jan. 20-21 at the University of Maryland in College Park. Here is the preliminary program:\n\nSaturday, January 20:\n\n9-10 a.m. Plenary session and overview: Combining Issues Coverage, Candidate Profiles and the High-Tech Campaign\nRosemary Armao, IRE\nBrant Houston, NICAR\nChris Callahan, University of Maryland\n10-11 a.m. On-line Campaign Coverage Lessons\nNora Paul, Poynter Institute\nBill Loving, Minneapolis Star-Tribune\n11 a.m.-noon Michael Dickerson of the FEC tells how to get the data you need to cover campaigns\nNoon-1 p.m. Lunch with Gene Roberts, managing editor, New York Times\n1-2 p.m. (Concurrent Sessions)\nA. Getting Through the Spin\nGloria Borger or Steve Roberts, U.S. News & World Report\nGlenn Totten, political consultant\nSteve Geimann, SPJ president-elect\nB. Writing Issue Stories into a Must Read\nJim Neff, Ohio State University\nJill Abramson, Wall Street Journal\n2-3 p.m. (Partly Concurrent Sessions)\nDeveloping Campaign (\"Off-line\") Sources\nJohn King, The Associated Press\n2-4 p.m. Building your own Campaign Database\nDavid Morris, The Associated Press\nThe Center for Responsive Politics\n3-4 p.m. (Concurrent Sessions)\nA. Backgrounding and profiling candidates\nDavid Maraniss, Washington Post\nB. Polls and Focus Groups\nEvans Witt, The Associated Press\n\nHands-on CAR sessions will run at 10-noon, 1-3 p.m. and 3-5 p.m. While we've given each of these two-hour time slots, we expect them to run closer to 90 minutes, which will keep down interference with lectures and workshops.\n\nSunday, January 21\n\n9-10 a.m. (Concurrent Sessions)\nA. On-line Campaign Coverage Lessons\nMargot Williams, Washington Post\nWendell Cochran, American University\nB. Dissection of TV Campaign Ads\nTodd Purdum, New York Times\n10-11 a.m. Polls and Focus Groups Repeat Session\nEvans Witt, The Associated Press\n10 a.m.-noon (Two Concurrent Sessions)\nA. Covering Virginia State Elections:\nLise Olsen, Virginian-Pilot\nPeter Baker, Washington Post\nB. Covering Maryland State Elections:\nFrazier Smith, Baltimore Sun\n11 a.m.-noon Stories from the Data\nBrooks Jackson, CNN\nJonathan Salant, Congressional Quarterly\nSara Fritz, Los Angeles Times\nNoon-1 p.m. Lunch with Doug Bailey, American Political Network Inc.\n1-2 p.m. (Concurrent Sessions)\nA. Opposition Researchers\nB. Campaign Finance\nDwight Morris, consultant\nChris Schmitt, San Jose Mercury News\nCenter for Public Integrity\n2-4 p.m. What are the Big Issues that will make Headlines in Campaign 96?\nBob Schieffer, CBS\nJack Germond, Baltimore Sun\nLee Thornton, Howard University\n4-5 p.m. What's Wrong with Campaign Coverage?\nRem Reider and Carl Sessions Stepp, University of Maryland\nHands-on CAR sessions will run at 10-noon, 1-3 p.m. and 3-5 p.m.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-baby-you-cant-drive-my-car","slug":"1995-december-baby-you-cant-drive-my-car","url":"https://dwillis.github.io/uplink/article/1995-december-baby-you-cant-drive-my-car.html","year":1995,"month":"December","monthNum":12,"headline":"Baby, you can't drive my car","kicker":"Do instructors practice what they preach?","author_name":"Renita Coleman","author_title":"","affiliation":"University of Missouri","summary":"A St. Paul Pioneer Press investigation found that driving instructors in Minnesota have similar or slightly worse traffic violation records compared to the general population, despite expectations that they would be better drivers. The reporter compiled data from state databases and commercial driving school records to compare instructor violations against the broader driving population.","keywords":["instructors","violations","drivers"],"topics":["Investigative Methods","Data & Databases"],"technologies":["FoxPro","Excel"],"full_text":"Do instructors practice what they preach?\n\nBaby, you can't drive my car\n\nHere's a scary thought: Driving teachers apparently teach what they know. Dan Browning of the St. Paul Pioneer Press found out what they know is nothing special. They drive as bad as the rest of us.\n\nBrowning used several state drivers databases. In addition to the driving records from the general population which contained about 3.8 million records - Browning used driving records for the 400 or so public school and commercial driving instructors in Minnesota.\n\n\"The public schools didn't have the data and the commercial school instructor data was on rolodex cards,\" he said.\n\nBrowning got the paper records for all instructors, and compiled them into a database. Some instructors worked at more than one school, so he checked for double entries. On his 486 with a gigabyte hard drive, he used FoxPro and Excel to cull through the data.\n\nBrowning compared instructor violations against the general driving population. \"We found they were slightly more likely to have two, three, or four tickets. It was not statistically significant, but we took it seriously because you expect driving instructors to be better drivers,\" he said. Also, driving teachers can't teach after a certain number of tickets a year.\n\nDriving tips\n\nThe Pioneer Press started looking into driving instructors because an editor was curious after it turned out a child molester in the community had owned a driving school.\n\nFor those considering such a story, Browning suggested that reporters talk with driver education associations.\n\nHe also recommended: \"You can do the story with or without the driving records of everybody in the state you can just get them for the driving instructors. The computer file doesn't show warning letters, but the certified paper copy would.\"\n\nRenita Coleman can be reached at (314) 642-6765, or send e-mail to c673328@showme.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1995-december-bits-bytes-and-barks","slug":"1995-december-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1995-december-bits-bytes-and-barks.html","year":1995,"month":"December","monthNum":12,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"The Tennessean","summary":"The new U.S. Patent and Trademark Information site offers improved search capabilities over its predecessor but has limitations, including the ability to search only one year at a time and potentially incomplete search results. The article also includes job announcements for journalism positions at various newspapers and universities.","keywords":["patents","database","journalism"],"topics":["Technology","Investigative Methods"],"technologies":["Internet"],"full_text":"A new patent site with new problems\n\nThe new U.S. Patent and Trademark Information site is much prettier than its Internet Multicasting Service predecessor, and it professes to include patents back to 1985 where IMS was limited to 1994 and 1995. The interface supports everything from simple point-and-click to full Boolean language searching by field, and is easy to use.\n\nOne problem of the IMS site that shows up again here is that you may only search one year at a time. A warning: a quick search for inventors in Nashville (a search I performed often at the IMS site) turned up just 18 patents, much less than the 50-60 I was used to experiencing. And the latest of those patents was from June. This could have been a bug in the system, which was still being worked on when I logged on (Nov. 13). But beware of assuming that any search is pulling up \"all\" patents on that subject in the given year.\n\nHeather Newman, The Tennessean; send e-mail to hnewman@tennessean.com\n\nChicago Sun-Times hires Schmid\n\nJon Schmid recently left as database editor for the Raleigh News and Observer to join the Chicago Sun-Times in November as a reporter.\n\nDown in El Paso\n\nThe El Paso Times is seeking applicants for its systems editor position. The systems editor supervises all computer-related operations in the newsroom, including maintenance, training, networking, hardware, software and upgrades for all computer systems including SII, AP LeafDesk, Macintosh and PCs. Serves as primary liaison between the newsroom and technical personnel. Advises management on the impact and potential of new technology in news gathering, news production and other newsroom projects. Apply to: Don Flores, editor, El Paso Times, P.O. Box 20, El Paso, Texas, 79999, (915) 546-6149. For more jobs in the Gannett chain, visit the company's web site at http://www.gannett.com.\n\nCronkite school looking for professor\n\nThe Walter Cronkite School of Journalism and Telecommunication at Arizona State University is seeking applicants for a continuing full professor to occupy the Knight Chair in Journalism. The school seeks a working journalist who would play a leadership role in teaching students how to use electronic retrieval technology to better analyze and present news stories. Please contact them by Feb. 1, for complete job description and requirements. Direct inquiries to Dr. John Craft or Dr. Fran Matera, co-chairs of the search committee, at (602) 965-5011, or e-mail your snail mailing address to: atfrm@asuvm.inre.asu.edu\n\nJoin NICAR on the Internet\n\nDon't forget to keep up with NICAR on the Internet. Subscribe to our listserve and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURI.EDU. In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-january-campaigns","slug":"1996-january-campaigns","url":"https://dwillis.github.io/uplink/article/1996-january-campaigns.html","year":1996,"month":"January","monthNum":1,"headline":"Campaigns","kicker":"Uplink update","author_name":"","author_title":"","affiliation":"","summary":"Computers and the Internet are transforming election coverage, enabling journalists to provide voters with real-time campaign contribution data and election results. The article highlights innovative reporting practices like Oregon reporters securing early computerized contribution disclosures and the Syracuse Post-Standard using online platforms for live election night coverage.","keywords":["elections","computers","reporting"],"topics":["Government & Politics","Investigative Methods","Technology"],"technologies":["Internet"],"full_text":"Computers are emerging as a new-age voting machine, and elections may never be the same.\n\nThis issue of Uplink profiles computer-assisted efforts, and offers tips for reporters to better cover campaigns. That includes using the Internet as well as using campaign contribution data.\n\nFor instance, in Oregon, reporters cajoled U.S. Senate candidates to release computerized contribution data before the Federal Election Commission's deadline. That meant voters knew days before they went into the booth who was paying for each candidate's campaign.\n\nAt the Syracuse (N.Y.) Post-Standard, the newspaper used the Internet on election night to give up-to-the-minute tallies for every local race.\n\nWe also offer a smorgasbord of web sites for campaign reporting everything from voting records to speeches.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-finding-the-best-politicians-money-can-buy","slug":"1996-january-finding-the-best-politicians-money-can-buy","url":"https://dwillis.github.io/uplink/article/1996-january-finding-the-best-politicians-money-can-buy.html","year":1996,"month":"January","monthNum":1,"headline":"Finding the best politicians money can buy","kicker":"Money-go-round","author_name":"David Royse","author_title":"","affiliation":"Associated Press","summary":"Campaign finance data provides journalists with rich story opportunities to reveal the disconnect between candidates' public messaging and their actual donor relationships, potentially exposing inconsistencies between their stated positions and the interests that fund them.","keywords":["donors","legislation","accountability"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Campaign finance data. To paraphrase Jimmy Buffett: We all want it, we've all got it. Now, what do we do with it? \"There are a thousand stories that will jump off anybody's desk,\" says Josh Goldstein, research director at the Center for Responsive Politics, which tracks campaign giving. \"You don't have enough time in the day to do all the stories that are out there.\"\n\nLarry Makinson, a former Anchorage Daily News reporter who also works at the Center for Responsive Politics, wrote the book \"Follow the Money.\" He says looking at campaign giving can shed light on the real candidate. \"The issues that a candidate talks about in targeting these contributors are often totally different from the issues they talk about to the public,\" Makinson writes. \"Exposing the details of this phantom campaign - and identifying the players - provides a much more fully-rounded picture of the candidate and the positions he or she will take once they attain office.\"\n\nDebunking candidates\n\nThe most important money profiles can debunk candidates' attempts to portray themselves as something they are not. For example, consider this lede from a 1988 story about presidential candidate Richard Gephardt:\n\n\"Rep. Richard Gephardt, who is seeking to craft an image as an anti-establishment, anti-corporate crusader, has raised far more money from political action committees than any of his Democratic rivals - much of it from the very interests he rails against on the stump.\"\n\nDivulging the corporate support for Gephardt revealed his populism as a born-again-type, the writer implied, thus preventing Gephardt from running two campaigns - one for the readers of the stump speeches, and one for the PAC representatives and Washington lobbyists.\n\nA closely related type of story is one that exposes what Makinson calls the \"cash constituents.\" Are these large contributors the people the elected officials really serve? There's one way to find out, Goldstein suggests.\n\n\"Look at who carried water for what industries on specific amendments?\" For example, Goldstein suggests, \"take a small element of the farm bill and look at who (among contributors) got something back.\"\n\nBangs for the bucks\n\nSome of the most interesting political stories are those that open up the \"back rooms\" and point to connections between campaign financing and precise pieces of legislation.\n\nOne such story, and one of the best examples of cash constituent coverage is an October 1995 Boston Globe story by Michael Kranish. The piece begins:\n\n\"Three weeks ago, Sen. Bob Dole launched his latest drive to save tax preferences worth billions of dollars for the ethanol industry. He sent letters, held private meetings with lobbyists, pressured fellow Republicans and ultimately helped turn back a House revolt against the tax break.\"\n\nKranish then outlines the cozy relationship between Dole and Dwayne Andreas, chairman of agri-business giant Archer Daniels Midland. ADM and a couple of other companies would see more than $600 million in savings from the ethanol tax breaks. \"Over the past 14 years,\" writes Kranish, \"Andreas' family and company have given more than $250,000 to Dole's campaign committees and foundations.\"\n\nSuch in-depth reporting is not always needed. Simply reporting lists of campaign contributors and their affiliations, although not necessarily as captivating as deeper investigative pieces, can give the reader an idea of where a candidate gets support.\n\nGo local\n\nThe Courier Journal of Louisville, Ky., recently began routinely reporting the sources of all campaign contributions to statewide candidates. Sometimes the stories may seem a little dull. Consider this headline from the coverage of the 1995 governor's race: \"Optometrists are leading Patton donors; also gave heavily to Forgy.\" But they show the reader who are the cash constituents.\n\nThis year, another good local story, says Goldstein, is to look at the money going from the circulation area to the presidential campaigns. \"It's time for folks to start making that a local story showing who the national cash constituents are locally. That's not going to win you a Pulitzer, but it's a good local story.\"\n\nGoldstein also suggests looking at the campaign expenditures of members of Congress. \"You're looking for stuff that looks unusual,\" Goldstein says. He cited the case of one member whose campaign had spent $8,500 on clocks, according to her FEC expenditure reports. \"What is a campaign doing spending $8,500 on clocks,\" Goldstein asks. \"An enterprising reporter asked the store that, and they said, 'No, nobody's spent that much here.' That's a story.\"\n\nstump speeches, and one for the PAC representatives and Washington lobbyists. A closely related type of story is one that exposes what Makinson calls the \"cash constituents.\" Are these large contributors the people the elected officials really serve? There's one way to find out, Goldstein suggests.\n\n\"Look at who carried water for what industries on specific amendments?\" For example, Goldstein suggests, \"take a small element of the farm bill and look at who (among contributors) got something back.\"\n\nBangs for the bucks\n\nSome of the most interesting political stories are those that open up the \"back rooms\" and point to connections between campaign financing and precise pieces of legislation.\n\nOne such story, and one of the best examples of cash constituent coverage is an October 1995 Boston Globe story by Michael Kranish. The piece begins:\n\n\"Three weeks ago, Sen. Bob Dole launched his latest drive to save tax preferences worth billions of dollars for the ethanol industry. He sent letters, held private meetings with lobbyists, pressured fellow Republicans and ultimately helped turn back a House revolt against the tax break.\" Kranish then outlines the cozy relationship between Dole and Dwayne Andreas, chairman of agri-business giant Archer Daniels Midland. ADM and a couple of other companies would see more than $600 million in savings from the ethanol tax breaks. \"Over the past 14 years,\" writes Kranish, \"Andreas' family and company have given more than $250,000 to Dole's campaign committees and foundations.\"\n\nSuch in-depth reporting is not always needed. Simply reporting lists of campaign contributors and their affiliations, although not necessarily as captivating as deeper investigative pieces, can give the reader an idea of where a candidate gets support.\n\nGo local\n\nThe Courier Journal of Louisville, Ky., recently began routinely reporting the sources of all campaign contributions to statewide candidates. Sometimes the stories may seem a little dull. Consider this headline from the coverage of the 1995 governor's race: \"Optometrists are leading Patton donors; also gave heavily to Forgy.\" But they show the reader who are the cash constituents.\n\nThis year, another good local story, says Goldstein, is to look at the money going from the circulation area to the presidential campaigns. \"It's time for folks to start making that a local story showing who the national cash constituents are locally. That's not going to win you a Pulitzer, but it's a good local story.\"\n\nGoldstein also suggests looking at the campaign expenditures of members of Congress. \"You're looking for stuff that looks unusual,\" Goldstein says. He cited the case of one member whose campaign had spent $8,500 on clocks, according to her FEC expenditure reports. \"What is a campaign doing spending $8,500 on clocks,\" Goldstein asks. \"An enterprising reporter asked the store that, and they said, 'No, nobody's spent that much here.' That's a story.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-pulitzer-winner-calculates-justice","slug":"1996-january-pulitzer-winner-calculates-justice","url":"https://dwillis.github.io/uplink/article/1996-january-pulitzer-winner-calculates-justice.html","year":1996,"month":"January","monthNum":1,"headline":"Pulitzer winner calculates justice","kicker":"Virgin territory","author_name":"Andrew W. Lehren","author_title":"NICAR staff","affiliation":"NICAR","summary":"A Pulitzer Prize-winning investigation by Melvin Claxton at the Virgin Islands Daily News exposed systemic failures in the Caribbean territory's criminal justice system by using computer spreadsheet analysis of over 5,000 court cases to reveal that the vast majority of crimes went unsolved, uninvestigated, or resulted in no prison time.","keywords":["investigation","corruption","data"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Excel","Macintosh"],"full_text":"Sometimes, for a Pulitzer Prize-winning story, you need to bang your computer with a screwdriver. That's what Melvin Claxton found out at the Virgin Islands Daily News. But this gets ahead of the tale. The newspaper won a 1995 Pulitzer for exposing how the criminal justice system failed to work in the Caribbean paradise. Police and prosecutors failed time and again to put criminals in jail. Most major crimes went unsolved. Claxton found rampant corruption and incompetence. Even when prosecutors did win convictions, criminals rarely did time in jail.\n\nClaxton also used something government officials had ignored: a computer. His tabulations were startling. Fewer than five of every 100 violent crimes reported to police led to an arrest. Fewer than 10 of every 100 were even investigated. About two-fifths of all violent crime cases were dismissed before going to trial, often at the request of prosecutors. Fewer than a third of all convicted criminals went to prison. Claxton wove these facts into compelling stories that dramatized how the system failed in the U.S. territory. His series was rich with anecdotes and detail. But the computer analysis, he said, was the most important part. \"It was the bottom line,\" the reporter said. The series began after a new editor, Lowe Davis, talked with Claxton about possible long-term projects. Claxton already knew of countless tales of police work gone wrong, how evidence got lost, how unqualified people got key police jobs.\n\nTo widen the scope of the story, Claxton began looking at the system itself. That included examining stacks of court records that tracked charges, disposition and sentences. The government system was in such disarray that officials could not recount how many rapes they prosecuted in 1994.\n\nClaxton answered those questions by building his own spreadsheet. Claxton, an Antigua native, received his college degree in economics. He knew something about numbers and, he said, \"I've always been a Mac buff and always keep my personal bills in the computer.\"\n\nPaper records\n\nWith help from two interns, the reporter got paper records from the U.S. territorial courts in St. Thomas and St. Croix. In one case, because court clerks photocopied the wrong pages, he needed to revisit court files to make sure he got the right information.\n\nClaxton used Excel to record more than 5,000 cases. The information included 10 columns - all the information kept in the court disposition listings. The records included a case number, the charges, the offender's sex, whether the case was decided by a judge or a jury, and what happened. The most useful information flowed from the last column, labeled \"Dispositions.\" It was a collection of notes. It included notes like \"Gov't not prepared for trial,\" \"Dismissed by motion of the Government,\" and \"Dismissed by the Government.\" In addition, the reporter gathered police records, including arrest logs. He had planned beforehand how to build the spreadsheets. \"We decided what we are looking for, and what averages we would love to know.\" Those figures included the crime rate, the number of criminal court cases, and the percentage of convictions. Claxton typed in the records. \"I sure would have loved for the government to have done this,\" he said. But few public records are computerized in the U.S. Territory. He had no choice if he wanted the story. Claxton worked on an old Macintosh SE30. It was the best computer at the newspaper. That's not saying much, considering what is in many big U.S. newsrooms. To make things worse, the computer kept crashing. The problem was a hard drive that got stuck when it should have been spinning around for data. \"That was a dark side of the story,\" Claxton recalled. \"It gave us some frightening moments.\" A repairman, after several failed attempts to fix it, offered sage advice: Take off the back panel and whack the hard drive with a screwdriver. He offered them a shiny screwdriver with a black, rubber handle. The solution was brutal, but got the job done. Just to make sure, Claxton printed out every day's work. The results were shocking - so unusual that Claxton recounted everything twice before discussing them with public officials. Police officers said they had no way to verify or dispute the findings. Judges and prosecutors, at first disbelieving, sought printouts for review. The calculations brought depth to the anecdotes. When he started the story, he said, \"what we didn't have was the quantified numbers.\" But with those figures, the result \"was a quantified rather than a subjective story.\" The newspaper has since replaced the troubled hard drive. Claxton kept it as a souvenir. The repairman got his screwdriver back. In December, Claxton left the newspaper to finish a true-crime book set in the Virgin Islands.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-from-page-one-calculating-justice","slug":"1996-january-from-page-one-calculating-justice","url":"https://dwillis.github.io/uplink/article/1996-january-from-page-one-calculating-justice.html","year":1996,"month":"January","monthNum":1,"headline":"From page one: Calculating justice","kicker":"","author_name":"Andy Lehren","author_title":"reporter","affiliation":"NICAR","summary":"A reporter in the U.S. Virgin Islands used computer analysis of court records to expose systemic failures in the criminal justice system, finding that few violent crimes resulted in arrests or convictions. By building his own spreadsheet from over 5,000 court cases, he uncovered critical statistics that government officials couldn't provide, creating a data-driven investigation that revealed how the system failed.","keywords":["investigation","data","justice"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Excel","Macintosh"],"full_text":"Claxton also used something government officials had ignored: a computer. His tabulations were startling. Fewer than five of every 100 violent crimes reported to police led to an arrest. Fewer than 10 of every 100 were even investigated. About two-fifths of all violent crime cases were dismissed before going to trial, often at the request of prosecutors. Fewer than a third of all convicted criminals went to prison. Claxton wove these facts into compelling stories that dramatized how the system failed in the U.S. territory. His series was rich with anecdotes and detail. But the computer analysis, he said, was the most important part. \"It was the bottom line,\" the reporter said. The series began after a new editor, Lowe Davis, talked with Claxton about possible long-term projects. Claxton already knew of countless tales of police work gone wrong, how evidence got lost, how unqualified people got key police jobs.\n\nTo widen the scope of the story, Claxton began looking at the system itself. That included examining stacks of court records that tracked charges, disposition and sentences. The government system was in such disarray that officials could not recount how many rapes they prosecuted in 1994.\n\nClaxton answered those questions by building his own spreadsheet. Claxton, an Antigua native, received his college degree in economics. He knew something about numbers and, he said, \"I've always been a Mac buff and always keep my personal bills in the computer.\"\n\nPaper records\n\nWith help from two interns, the reporter got paper records from the U.S. territorial courts in St. Thomas and St. Croix. In one case, because court clerks photocopied the wrong pages, he needed to revisit court files to make sure he got the right information.\n\nClaxton used Excel to record more than 5,000 cases. The information included 10 columns - all the information kept in the court disposition listings. The records included a case number, the charges, the offender's sex, whether the case was decided by a judge or a jury, and what happened. The most useful information flowed from the last column, labeled \"Dispositions.\" It was a collection of notes. It included notes like \"Gov't not prepared for trial,\" \"Dismissed by motion of the Government,\" and \"Dismissed by the Government.\" In addition, the reporter gathered police records, including arrest logs. He had planned beforehand how to build the spreadsheets. \"We decided what we are looking for, and what averages we would love to know.\" Those figures included the crime rate, the number of criminal court cases, and the percentage of convictions. Claxton typed in the records. \"I sure would have loved for the government to have done this,\" he said. But few public records are computerized in the U.S. Territory. He had no choice if he wanted the story. Claxton worked on an old Macintosh SE30. It was the best computer at the newspaper. That's not saying much, considering what is in many big U.S. newsrooms. To make things worse, the computer kept crashing. The problem was a hard drive that got stuck when it should have been spinning around for data. \"That was a dark side of the story,\" Claxton recalled. \"It gave us some frightening moments.\" A repairman, after several failed attempts to fix it, offered sage advice: Take off the back panel and whack the hard drive with a screwdriver. He offered them a shiny screwdriver with a black, rubber handle. The solution was brutal, but got the job done. Just to make sure, Claxton printed out every day's work. The results were shocking - so unusual that Claxton recounted everything twice before discussing them with public officials. Police officers said they had no way to verify or dispute the findings. Judges and prosecutors, at first disbelieving, sought printouts for review. The calculations brought depth to the anecdotes. When he started the story, he said, \"what we didn't have was the quantified numbers.\" But with those figures, the result \"was a quantified rather than a subjective story.\" The newspaper has since replaced the troubled hard drive. Claxton kept it as a souvenir. The repairman got his screwdriver back. In December, Claxton left the newspaper to finish a true-crime book set in the Virgin Islands. Andy Lehren can be reached at (314) 882-0684, or send e-mail to andy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-building-your-own-database","slug":"1996-january-building-your-own-database","url":"https://dwillis.github.io/uplink/article/1996-january-building-your-own-database.html","year":1996,"month":"January","monthNum":1,"headline":"Building your own database","kicker":"Hard-copy hints","author_name":"Jennifer LaFleur","author_title":"Reporter","affiliation":"San Jose Mercury News","summary":"Journalists can build their own databases from hard-copy documents when government agencies won't provide electronic data, using options like professional data-entry services, temps, or manual entry, while following key planning and consistency guidelines to ensure accuracy and usability.","keywords":["database","journalism","data"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["Paradox","FoxPro","Access"],"full_text":"Build your own database. The phrase strikes terror in the minds of journalists who suddenly picture themselves locked in a padded room typing numbers for weeks on end, never seeing daylight.\n\nAs much as we strive to get government offices to provide information electronically, it is not always possible. Sometimes journalists must build their own database from hard-copy documents.\n\nCarolyn Tufts, when she was a reporter for the Belleville (Ill.) Democrat, spent her free time entering information from thousands of traffic tickets into a database.\n\nRobert Imrie, an Associated Press reporter in Wausau, Wis., wanted to analyze deer hunting accidents, but only had hard-copy forms—so he build his own database. Ralph Frammolino, a reporter for the Los Angeles Times built a database from juror questionnaires in the Simpson trial.\n\nBuilding your own database also can be useful to keep track of information we get into our newsrooms regularly: accidents, homicides or other crime incidents. If you input these items as they come in, eventually you'll have a database you can use to look at trends.\n\nInput options\n\nIn cases where large amounts of data must be entered, the reporter doesn't always have the time to become a full-time data entry person. Here are some options:\n\nUse a professional data-entry company. They are usually quite efficient and not expensive. In many cases, you can have these companies \"double-enter\" the data. This means that two people enter the data, then cross-check the two files.\nUse a temporary service. This requires that you train the temp on how to properly enter the data or create a simple data-entry form. This also requires regular spot-checks of the data.\nDo it yourself in your spare time. Especially on your first projects, this will help you interview the data.\n\nOnce you get ready to build the database, here are some dos and don'ts to keep in mind:\n\nDos and Don'ts\n\nBreak up names into separate fields for first, last, and middle or middle initial.\nPlan. Plan. Plan. If your set up your database for U.S. addresses, will there ever be a point when you'll be putting foreign addresses in the database? If yes, create a field for country. In many database programs you can give these fields a \"default\" value. That means the computer would automatically put USA in the country field unless you specified something different.\nWill you want to look at the data by day of the week or month? You may want to break up the date into three parts or add a separate field for day of the week.\nBe consistent. If you're typing in city names or streets, use the same format each time. Use all upper case to avoid problems with case-sensitivity.\nEnter some of the data, then have a colleague look at the data. This is a good way to test the database to make sure it's easy to understand and they may suggest other information to include.\nBalance. Don't include so much that it takes you the next 10 years to enter the data, but include that data you think you \"might\" use: It's easier to input it the first time around.\nIf you're entering the data yourself, have someone else spot check from the original form.\nGet a summary report based on the forms you're using. This can help you check the totals in your database.\nDo some basic integrity checks on the data: Check for duplicates, count fields and look for inconsistencies. For example, if you were inputting accidents, does your database use I-70 and I 70 and Interstate 70?\nIn Paradox: check mark these fields individually and CALC COUNT ALL, then sort alphabetically.\nIn FoxPro and Access: group by these fields individually, COUNT them, then sort or order by alphabetically.\nBackup your database frequently.\n\nJennifer LaFleur can be reached at (408) 920-5728, or send e-mail to JENSTER@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-get-computer-training","slug":"1996-january-get-computer-training","url":"https://dwillis.github.io/uplink/article/1996-january-get-computer-training.html","year":1996,"month":"January","monthNum":1,"headline":"Get computer training","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers several computer training programs for journalists in spring and early summer, including week-long bootcamps, broadcast seminars, and advanced reporting courses at various locations.","keywords":["training","journalism","conferences"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Get computer training:\n\nNICAR Bootcamps, week-long intensive training seminars, March 10-15 and May 19-24, Columbia, Mo. NICAR/Medill Broadcast Seminar, March 23-24, Medill School of Journalism, Evanston, Ill. Advanced Computer-Assisted Reporting Seminar, May 5-10, Chapel Hill, N.C. North Carolina National IRE Conference, June 13-16, Convention Center, Providence, R.I.\n\nThese dates are open to all journalists. For more information, call NICAR, (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-public-interest-contributions-and-the-candidates","slug":"1996-january-public-interest-contributions-and-the-candidates","url":"https://dwillis.github.io/uplink/article/1996-january-public-interest-contributions-and-the-candidates.html","year":1996,"month":"January","monthNum":1,"headline":"Public-interest contributions and the candidates","kicker":"Linking money before the vote","author_name":"Dan Mihalopoulos","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"Oregon journalists from The Oregonian and Willamette Week partnered with the Center for Responsive Politics to conduct real-time analysis of campaign contributions before the 1996 Senate election, using database techniques to identify contributors' interest groups and trace money sources ahead of the FEC deadline.","keywords":["campaign","contributions","journalism"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["Access","Excel"],"full_text":"It's good to know who gave how much and to which candidate, even when an election is over. It's much better still when you can have that information before voters decide.\n\nThat's a luxury Oregon journalists enjoyed during the campaign to replace Sen. Bob Packwood.\n\nWith help from the Center for Responsive Politics, reporters at The Oregonian and Willamette Week, an alternative weekly, linked candidates with contributions from interest groups and industries. All of this was done before voters needed to turn in their special, mail-in ballots.\n\n\"This was the first attempt to do real-time reporting and get a handle on the money before elections,\" said Larry Makinson, deputy director of the center in Bandon, Ore. \"I'm not aware that anyone has done that before in a federal race.\"\n\nWorking independently, Steve Suo of The Oregonian and Bob Young of Willamette Week convinced the candidates to disclose their contribution records ahead of the Federal Election Commission's deadline.\n\nIdentifying contributors\n\nThis allowed Makinson to do what he does best - identify individual contributors with interest groups or industries and determine where the money really is coming from.\n\n\"The only way is to browbeat the candidates into releasing the information,\" Makinson said. \"They don't like to share information if they can get away with it.\"\n\nThe Oregonian and Willamette Week then analyzed the databases and did stories that identified one candidate, Rep. Ron Wyden, as the favorite of former Packwood contributors.\n\n\"We took some hits for that story,\" Suo said. \"Some felt it was an unfair comparison, but that's what the numbers showed. That's not to say that Wyden is a sexual harasser. Packwood had an incredible ability to raise money, and we showed that Wyden will inherit that legacy.\"\n\nThe Oregonian's Washington bureau had compiled Packwood's FEC reports from 1979. Suo used Access to join the table of Packwood contributors to the candidate tables. The query yielded nearly 200 hits.\n\nAt Willamette Week, Young used an Excel spreadsheet in a similar story on Wyden's Packwood connection. He superimposed the Packwood data onto the recent list of campaign contributors. Young then scrolled down the new list, noting the one-time Packwood contributors that had been active in the campaign.\n\nSimple analysis of the data also showed which candidates received the most money from women's groups, pro-Israel PACs and the timber, insurance and health-care industries.\n\n\"Normally, we follow the FEC deadlines and there is a mad scramble to get a list of the biggest contributors,\" Suo said. \"There's never been an attempt to make a comprehensive analysis of where the money was coming from. This allowed us to go a little deeper.\"\n\nScanning sees the light\n\nThe Oregonian had first crack at the data because Suo scanned in the paper records. Only one candidate handed over information in an electronic format.\n\n\"It wasn't difficult to scan individual donations into the database,\" Suo said. \"Using simple macros, we got the information in tab-delimited format and got it into the database. It was much, much faster than data entry by keyboard.\"\n\nSome candidates were unwilling to give the journalists data on disk although they had all the information on a computer, Suo said. The campaigns feared that the competition would acquire their lists and use them to raise money.\n\nWith scanning technology, however, \"handing it to us on paper is as good as on disk,\" Suo said. \"They were in the dark on the state of technology.\"\n\nIf data is unavailable on disk, he suggested, the next best thing is to convince the campaign to create a paper form that is scanner-friendly.\n\nRegardless, duplicating the impressive effort in Oregon is not difficult with a little bit of computer savvy, Makinson said.\n\n\"A small paper with one inquisitive reporter can unravel patterns that were totally invisible a few years ago,\" he said. \"You can have an incalculable impact, especially if you do it before the election.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-on-the-world-wide-web","slug":"1996-january-on-the-world-wide-web","url":"https://dwillis.github.io/uplink/article/1996-january-on-the-world-wide-web.html","year":1996,"month":"January","monthNum":1,"headline":"On the World Wide Web","kicker":"Many happy returns","author_name":"Tom Foster","author_title":"","affiliation":"Syracuse Newspapers","summary":"Syracuse Newspapers used an integrated technology system combining an audiotext service and website to deliver real-time 1995 election results to central New York readers, with data flowing from county election mainframes through a Paradox database to multiple outputs.","keywords":["elections","technology","journalism"],"topics":["Investigative Methods","Technology","Government & Politics"],"technologies":["Paradox","modem/BBS","mainframe","Novell","HTML","Internet","World Wide Web","Netscape"],"full_text":"The Syracuse Newspapers used a multilayered approach to report 1995 election results to their central New York readers in real time.\n\nUsing NewsLine, the newspapers' audiotext service, and Syracuse OnLine, the newspapers' World Wide Web site (www.syracuse.com), the newspapers delivered up-to-the minute returns on everything from the county executive campaign to dozens of town council races.\n\nThe newspapers used three terminals connected via modem to Onondaga County's Board of Elections mainframe to monitor results in real time. Those were transferred by typists to a master Paradox database of results that resided on the newsroom's Novell network.\n\nGetting results\n\nOnce the results were on the newspapers' PC network, those single keystrokes did a lot of things at the same time. Results were posted at our web site. The newsroom as a whole was able to track the evening's events. And reporters got instant printouts for the races they were covering. Here are the details:\n\nInternet documents: Paradox's report writing functions allow HTML tags to be built into templates. Using templates makes it easy to accommodate changing data. The functions let us create HTML documents containing the latest results on the fly by telling Paradox to make the right web page.\n\nWe had to prepare. Prior to the election, fields in the Paradox table had been filled in with the names of the HTML files for the fact boxes, profiles and preview stories for each race. That let us publish more than just numbers. On election night, the files were sent to an update folder on the newsroom network, where they were transferred to the web server.\n\nNewsroom updates: Real-time electronic updates were available to all reporters and editors. One newsroom PC running Netscape was hooked up to a TV mounted above the city desk. That allowed point-and-click access to results that could be viewed by dozens of people at once. To speed things up, the PC connected to the television was used to monitor the results file from the newsroom network rather than our web server.\n\nCustomized reports: keep reporters who were working on the more than 140 races efficiently updated, the Paradox table also included a field listing the name of the reporter covering the race. A Paradox script triggered paper printouts so each reporter received a report listing results on just the races that reporter covered.\n\nA Paradox script running in the background generated automatically all of the HTML and paper reports. That script monitored the system clock and triggered updates every 10 minutes. It allowed the typists, reporters and editors to work without interruption throughout the night.\n\nWhat we learned\n\nGenerally, the process worked smoothly. We usually were able to pass results from the board of elections mainframe to our staff and online users within five minutes. The ability to update all races simultaneously tended to put us ahead of broadcast media. A local public radio station used our web page as a source of its report to listeners. The Board of Elections also monitored our page and referred callers to our site. There is always room for improvement. We confined our updates for Onondaga County to one file that was fairly large, about 49K. That's with no images. Because we only had to replace one file, that sped up our ability to update our site. Its size meant reloading via modem took an annoyingly long time in some cases. This was particularly true when using America OnLine's browser.\n\nOn the newsroom side, we learned the hard way that low-tech approaches can be more effective. On election night in 1994, reporters got an icon on their PCs that allowed them to monitor the master Paradox results table as it was updated. Two problems arose: Some reporters weren't comfortable multitasking, so they would view the results, take notes by hand and then exit that application to write. And when about 35 reporters tried to run Paradox over the network at the same time, the load brought traffic to a crawl and finally a crash.\n\nWe used paper reports this year to guard against network overload, but it turned out that most reporters were more comfortable with that system anyway.\n\nTom Foster can be reached at (315) 470-3071, or send e-mail to tsfoster@mailbox.syr.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-from-page-four-linking-contributions","slug":"1996-january-from-page-four-linking-contributions","url":"https://dwillis.github.io/uplink/article/1996-january-from-page-four-linking-contributions.html","year":1996,"month":"January","monthNum":1,"headline":"From page four: Linking contributions","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Campaign finance researchers advocate for voluntary candidate disclosure of donations before legal requirements force compliance, citing success with early adoption creating a domino effect among other candidates.","keywords":["disclosure","donations","campaigns"],"topics":["Government & Politics","Investigative Methods"],"technologies":[],"full_text":"But the key is to get the candidates to disclose before the law forces them to. Young and Suo recommend approaching the most amenable campaign first.\n\n\"Peter DeFazio agreed on a Monday morning. Once I got him, it was like dominoes,\" Young recalled. \"It was not hard to do it. I had it sewn up in one day.\"\n\nMakinson promised to apply much of what he learned in the special Senate race to the 1996 elections. Contributions to presidential hopefuls and candidates for a few Senate seats will be likely subjects of pre-election scrutiny next year.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-on-the-internet","slug":"1996-january-on-the-internet","url":"https://dwillis.github.io/uplink/article/1996-january-on-the-internet.html","year":1996,"month":"January","monthNum":1,"headline":"On the Internet","kicker":"Campaign coverage","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute","summary":"A guide for journalists covering political campaigns, providing web resources for researching candidate positions, voting records, speeches, and campaign finances across multiple stages of campaign reporting.","keywords":["campaigns","journalism","candidates"],"topics":["Government & Politics","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web","gopher","FTP"],"full_text":"Campaign coverage requires different kinds of information at different stages of coverage and for particular types of stories. Here are some excellent web sites for campaign reporting.\n\nMake an issue of it\n\nGet background on an issue from Project Vote Smart site at http://www.vote-smart.org/ It links to organizations interested in issues ranging from abortion to environment to international trade and welfare, poverty and homelessness.\n\nCapWeb's Political Page at http://policy.net/capweb/political.html has links to PACs and political interest groups, as well as all of the political parties.\n\nDemocracy Place USA, a new service of the Soundprint Media Center, promises weekly updates on key issues around the country. It offers polls, and a discussion list only for journalists. It's at http://soundprint.org/-democracy/\n\nThe On-line Political Information Network, provided by the Kennedy School of Government at Harvard University, http://ksgwww.harvard.edu/~ksgpress/opinhome.htm, links to campaign and election information. The home page also offers experts.\n\nCheck what they said\n\nPolitics USA, from the National Journal and the American Political Network - http://politicsusa.com/PoliticsUSA/ - allows for selective comparisons of candidates' positions on several key issues.\n\nThe texts of speeches and public announcements made by candidates can be a great way to check for conflicting, or convenient, statements. Search archives of Presidential documents, speeches and information located on the Internet at http://www.whitehouse.gov and the text of legislation and the Congressional Record on Thomas at http://thomas.loc.gov/\n\nMost of the candidates are also putting together websites with background information and speeches. Find a list of the candidates' websites at Primary Destination: New Hampshire's Candidate InfoBank at http://www.fosters.com/FOSTERS/cgi-bin/walk.cgi/FOSTERS/info/d2/\n\nComprehensive candidates' pages, by party then candidate name can be found at http://www.webpub.com/prez/ Use PR Newswire at http://www.quote.com/info/prnews.html to read news releases from U.S. government agencies, members of Congress, and congressional committees.\n\nSee how they voted\n\nTrack Voting Records. Contact the legislator's office; it should keep a list of voting attendance and the vote cast. For a compilation of contact information for the 104th Congress members go to http://ast1.spa.umn.edu/juan/congress.html\n\nAlso, find out how your congresspeople voted at the \"How did your senators and representative vote?\" site, http://www.timeinc.com/cgi-bin/congress-votes. Fill in your zip code and the voting records of your senators and representatives will be listed.\n\nThe 1994 National Environmental Scoreboard lists the voting records on environmental issues at http://www.econet.apc.org/lcv/lcv94/scorecard.html\n\nFollow the money\n\nMother Jones put together a wonderful site on following campaign spending: \"The Best Congress Money Can Buy\" which lets you plug in the name of a candidate and search 1991-1994 PAC contributions and '91-'92 individual contributions. The results come in a bar graph with the special interest group's contribution charted with other PACs. This great application is at http://www.mojones.com/coinop_congress/data_viewer/data_viewer.html\n\nCheck out C-Span's campaign contribution site at gopher://c-span.org:70/11/Resource/fec. It includes tables that show PAC contributions to legislators.\n\nAlso, the Federal Election Commission's new ftp site offers extensive summary data in zipped files http://ftp.fec.gov. Use the program PKUnzip to pull out the tables that show, for instance, how much each PAC spent. Save the text files that explain how the data was put together.\n\nLocate discussion areas on the Internet on particular campaign topics or candidates. There are also discussion areas on issues such as abortion, disarmament, and military.\n\nThe following list of web site and Internet resources is extracted from the Poynter's Guide to Resources for Campaign and Election Coverage. For the complete story, go to http://www.poynter.org/poynter/elect2.html For links to sources on hot news stories, go to http://www.nando.net/prof/poynter/hrintro.html\n\nNora Paul is the author of \"Computer Assisted Research,\" available from the Poynter Institute and NICAR. She can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-bits-bytes-and-barks","slug":"1996-january-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1996-january-bits-bytes-and-barks.html","year":1996,"month":"January","monthNum":1,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"New York Daily News reporters investigated the firefighter disability pension system using freedom of information requests and data analysis, revealing that half of ex-firefighters collected disability pensions and nearly 40% received lucrative line-of-duty benefits, with some questionable cases included. The National Institute for Computer-Assisted Reporting appointed Neil Reisner as training director to oversee journalism training seminars across the country.","keywords":["pensions","investigation","journalists"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["FoxPro"],"full_text":"Bits, Bytes and Barks\n\nPutting the heat on pensions\n\nAt New York Newsday, reporter Joe Calderone wrote about certain firefighters who seemed to have won excessive disability pensions. When the newspaper closed, he moved over to the New York Daily News, and continued his investigation with another ex-Newsday reporter, Russ Buettner. Together, they decided to probe the entire disability pension system for firefighters. \"We wanted to look at the context of what was going on,\" Buettner said.\n\nThe reporters negotiated more than three months with the New York State Comptroller, using the state freedom of information law, for firefighter pension data. The database included more than 14,000 retirees, and about seven columns of information.\n\nThe reporters, already familiar with much about how the pension system worked and how the information was coded, needed about two days to analyze the data using FoxPro. \"The analysis was simple,\" according to Buettner.\n\nThe result was a story that ran Dec. 18 showing how one of every two ex-firefighters collected disability pensions. That included the fire department's own retired doctors.\n\nAlso, nearly four of every 10 pocketed lucrative line-of-duty pensions. That included one firefighter who injured his ankle after tripping on a crack in the sidewalk outside a firehouse.\n\nThe reporters also showed how the Fire Department brass won line-of-duty benefits more than front-line firefighters. And New York firefighters won line-of-duty pensions more often than their counterparts in the state's other big cities. That point was highlighted with a chart that showed New York's wide gap with Yonkers, Buffalo, Rochester and Syracuse. Russ Buettner can be reached at (212) 210-1964, or send e-mail to rbnews@pipeline.com.\n\nReisner new training director\n\nThe National Institute for Computer-Assisted Reporting has appointed Neil Reisner as its training director. Reisner will be in charge of NICAR's on-the-road training seminars that last year provided training for more than 3,000 journalists throughout the country. He will report to Brant Houston, NICAR's managing director.\n\nAn award-winning journalist with two decades of experience and a journalism educator, Reisner most recently served as database editor with The Bergen Record of Hackensack, N.J. During six years there he served in The Record's statehouse bureau and as a regional reporter. Reisner previously worked at The Home News in New Brunswick, N.J.\n\nHe has taught journalism at Columbia University's Graduate School of Journalism and at Rutgers University.\n\nKeep up with NICAR on-line\n\nSubscribe to our listserve and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURIEDU In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.\n\nIRE/NICAR is also accessible through CompuServe's Journalism Forum. Go to the JForum, Section 19. Also look into the IRE/NICAR files in Library 19.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-putting-the-heat-on-pensions","slug":"1996-january-putting-the-heat-on-pensions","url":"https://dwillis.github.io/uplink/article/1996-january-putting-the-heat-on-pensions.html","year":1996,"month":"January","monthNum":1,"headline":"Putting the heat on pensions","kicker":"","author_name":"Russ Buettner","author_title":"reporter","affiliation":"New York Daily News","summary":"A guide to using the Internet for campaign coverage, highlighting key websites and resources for issue backgrounds, voting records, and financial disclosures.","keywords":["Internet","campaign","resources"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["FoxPro"],"full_text":"At New York Newsday, reporter Joe Calderone wrote about certain firefighters who seemed to have won excessive disability pensions. When the newspaper closed, he moved over to the New York Daily News, and continued his investigation with another ex-Newsday reporter, Russ Buettner. Together, they decided to probe the entire disability pension system for firefighters. \"We wanted to look at the context of what was going on,\" Buettner said.\n\nThe reporters negotiated more than three months with the New York State Comptroller, using the state freedom of information law, for firefighter pension data. The database included more than 14,000 retirees, and about seven columns of information.\n\nThe reporters, already familiar with much about how the pension system worked and how the information was coded, needed about two days to analyze the data using FoxPro. \"The analysis was simple,\" according to Buettner.\n\nThe result was a story that ran Dec. 18 showing how one of every two ex-firefighters collected disability pensions. That included the fire department's own retired doctors.\n\nAlso, nearly four of every 10 pocketed lucrative line-of-duty pensions. That included one firefighter who injured his ankle after tripping on a crack in the sidewalk outside a firehouse.\n\nThe reporters also showed how the Fire Department brass won line-of-duty benefits more than front-line firefighters. And New York firefighters won line-of-duty pensions more often than their counterparts in the state's other big cities. That point was highlighted with a chart that showed New York's wide gap with Yonkers, Buffalo, Rochester and Syracuse.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-january-reisner-new-training-director","slug":"1996-january-reisner-new-training-director","url":"https://dwillis.github.io/uplink/article/1996-january-reisner-new-training-director.html","year":1996,"month":"January","monthNum":1,"headline":"Reisner new training director","kicker":"","author_name":"","author_title":"Training Director","affiliation":"National Institute for Computer-Assisted Reporting (NICAR)","summary":"The National Institute for Computer-Assisted Reporting has appointed Neil Reisner as training director to oversee seminars that train thousands of journalists annually. Reisner brings two decades of journalism experience and teaching credentials from Columbia University and Rutgers University.","keywords":["training","journalism","appointment"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":[],"full_text":"The National Institute for Computer-Assisted Reporting has appointed Neil Reisner as its training director. Reisner will be in charge of NICAR's on-the-road training seminars that last year provided training for more than 3,000 journalists throughout the country. He will report to Brant Houston, NICAR's managing director.\n\nAn award-winning journalist with two decades of experience and a journalism educator, Reisner most recently served as database editor with The Bergen Record of Hackensack, N.J. During six years there he served in The Record's statehouse bureau and as a regional reporter. Reisner previously worked at The Home News in New Brunswick, N.J. He has taught journalism at Columbia University's Graduate School of Journalism and at Rutgers University.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-january-keep-up-with-nicar-on-line","slug":"1996-january-keep-up-with-nicar-on-line","url":"https://dwillis.github.io/uplink/article/1996-january-keep-up-with-nicar-on-line.html","year":1996,"month":"January","monthNum":1,"headline":"Keep up with NICAR on-line","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Instructions for subscribing to journalism listservs and accessing IRE/NICAR resources for reporters to discuss best practices in the profession.","keywords":["journalism","listserv","subscription"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","Listserv","CompuServe"],"full_text":"Subscribe to our listserve and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURIEDU In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.\n\nIRE/NICAR is also accessible through CompuServe's Journalism Forum. Go to the JForum, Section 19. Also look into the IRE/NICAR files in Library 19.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-lost-your-way-in-car-maybe-a-map-would-help","slug":"1996-february-lost-your-way-in-car-maybe-a-map-would-help","url":"https://dwillis.github.io/uplink/article/1996-february-lost-your-way-in-car-maybe-a-map-would-help.html","year":1996,"month":"February","monthNum":2,"headline":"Lost your way in CAR? Maybe a map would help.","kicker":"Uplink update","author_name":"","author_title":"","affiliation":"","summary":"This issue of Uplink explores how newsrooms use desktop mapping programs to identify geographic patterns and create analytical maps that help visualize stories, covering available tools, costs, and data sources.","keywords":["mapping","newsrooms","data"],"topics":["Technology","Investigative Methods","Journalism & Newsroom"],"technologies":["ArcView","MapInfo","Atlas GIS"],"full_text":"This issue of Uplink looks at how desktop mapping programs can help you identify geographic patterns and create images to help readers better understand your stories.\n\nNewsrooms have used desktop mapping to locate clusters of blighted properties, pinpoint pollution and analyze demographics. News artists then convert the results into often startling analytical maps.\n\nOur stories tell you about these successes, walk you through downloading mapping data from the Internet, describe what programs are available, how much they cost and where you can get them.\n\nAlso inside are stories on the results of two national surveys revealing what hardware and software newsrooms are using - including mapping programs.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-may-i-draw-you-a-map","slug":"1996-february-may-i-draw-you-a-map","url":"https://dwillis.github.io/uplink/article/1996-february-may-i-draw-you-a-map.html","year":1996,"month":"February","monthNum":2,"headline":"May I draw you a map?","kicker":"","author_name":"David Herzog","author_title":"","affiliation":"The (Allentown, Pa.) Morning Call","summary":"Desktop mapping software is becoming an increasingly valuable tool for journalists conducting computer-assisted reporting, allowing them to visualize geographic patterns in data that might otherwise be overlooked during traditional analysis. Despite its proven effectiveness in uncovering major stories, the technology remains underutilized in newsrooms due to its relative newness and high costs.","keywords":["mapping","analysis","data"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["Atlas GIS","MapInfo","Excel","Data Map","GIS","mainframe"],"full_text":"After months of wrangling, you finally get that database you wanted from the county courthouse. You crunch the data with your database manager. You slice and dice it with a spreadsheet. You think you're done analyzing and ready to start interviewing.\n\nStop there and think again. Unless you run your data through a desktop mapping program to look for geographic patterns you could be missing the big picture.\n\n\"The magic of mapping software is that it provides a visual interpretation of your data,\" said Bill Loving, editor for computer-assisted reporting at the Minneapolis Star Tribune. \"Mapping is just such a quick and useful way of taking what could be an otherwise unintelligible pile of information and finding the patterns in it,\" says Steve Doig, research editor at the Miami Herald.\n\nIn the aftermath of 1992's Hurricane Andrew, Doig used AtlasGIS for DOS to analyze 60,000 damage inspection reports. In the Herald's Pulitzer Prize-winning series, Doig's analysis showed how homes built after 1980 fared much worse than sturdier older homes.\n\nDesktop mapping growing\n\nMore and more, journalists like Doig are using desktop mapping software to supplement traditional data analysis. One paper, the Oregonian in Portland, hired a master's student in geography as an intern for this school year to delve into desktop mapping, says Gail Hulden, assistant librarian and research director. In one of the clearest signs of desktop mapping's arrival, Microsoft Excel for Windows 95 includes Data Map, a tool for quickly displaying spreadsheet data.\n\nNewsrooms that use desktop mapping are finding great stories. They've been able to locate clusters of blighted properties, pinpoint pollution and analyze demographics. News artists have turned these analytical maps into eye-grabbing graphics.\n\nStill a CAR frontier\n\nYet, desktop mapping remains one of computer-assisted reporting's frontiers. The 1994 University of Miami national CAR study found just 17 percent of the 208 responding newsrooms used mapping for analysis.\n\nChalk that up to a couple of reasons: Desktop mapping on the PC is relatively new, and the investment can run into the thousands of dollars.\n\nHigh-end mapping analysis programs, called geographic information systems (GIS), have been on mainframes for two decades. But it wasn't until around 1990 that desktop mapping caught on in newsrooms across the country, thanks to two factors.\n\nFirst, in the late 1980s, GIS program makers introduced easy-to-use PC software. Second, the results from the 1990 Census became widely - and inexpensively - available in computer form, along with the public domain geographic files.\n\nIn fact, the Census provided the initial push for many reporters and editors to learn desktop mapping.\n\nShortly after the Census, New York Times database editor John Freed used MapInfo to analyze the racial makeup of proposed city council districts. Under the Voting Rights Act, New York had to submit its redistricting plan to the U.S. Justice Department.\n\nWhen the Justice Department rejected the plan, the Times used its own findings to show why: East Harlem and the South Bronx's Hispanic population had been splintered into numerous districts - putting nearly all of the Hispanics in the minority, Freed said. \"It was very obvious.\"\n\nMapping tools\n\nHigh-end mapping programs, such as AtlasGIS and MapInfo give users a number of tools for analyzing data. Less expensive programs likely will have fewer of these features.\n\nThematic mapping. This common feature takes your data and converts it to a map that uses colors or symbols to show variations. The Minneapolis Star Tribune used AtlasGIS for DOS to show changes in property values in the Twin Cities. Seeing the extent and location of the areas in decline, marked in red, was stunning, Loving said.\n\nThe Morning Call in Allentown, Pa., and other newspapers have used thematic mapping and Home Mortgage Disclosure Act data to find inner-city Census tracts with high home loan rejection rates.\n\nAddress matching or geocoding. You know those pin maps that police use to track murders? This feature allows you to create the computer equivalent. However, you must first use the mapping program's geocoder to assign latitude and longitude coordinates to each record in your database.\nThe Morning Call geocoded branch offices of merging banks. After we mapped the branches, we looked for overlap and speculated which ones might be shuttered.\n\nSelecting. A good mapping program allows you to select records that meet certain geographic criteria. MapInfo for Windows, the program I use, lets users select records that fall within boundaries or circles drawn on the map with a mouse. The top programs also have an SQL database manager that lets you find areas that meet certain criteria.\n\nFlexibility. The program should be able to work with a variety of geographic, text, spreadsheet and database files. In addition, the programs should be able to export to a wide range of formats.\n\nGeographic and 'attribute' data\n\nMapping programs use two kinds of data: geographic data and \"attribute\" data. The former defines features on the map (streets, county boundaries, rivers). The latter is what you analyze. When you use a desktop mapping program, you add layers of geographic and attribute data.\n\nMost mapping programs come bundled with some geographic data. However, you'll probably need to buy other geographic data to match the locations in your databases. If you want to analyze Federal Election Commission data, you'll need ZIP boundaries because the database contains the donor's ZIP code. If you're thinking about demographic analysis, get boundaries for census tracts or block groups that constitute tracts.\n\nDesktop mapping can cost thousands of dollars, but it doesn't have to, Doig says. Anyone who looks hard enough can find inexpensive geographic data sources. For instance, the Herald paid just $30 for election districts from Dade County. Local planning agencies may have the data you need. Also, check with university geography professors.\n\nSmaller newsrooms, or those with a tight budget, should talk to other departments (especially marketing or research) at their news organizations and see if they can share a desktop mapping program.\n\nThat's what the Flint Journal did about five years ago. The newsroom and two other departments chipped in for AtlasGIS for DOS, says Mary Ann Chick Whiteside, a former editor who's now the paper's new media manager.\n\nSince then, the Journal has printed a number of mapping-based stories, including one showing clusters of homes for the mentally ill. \"We knew they were all in the same ZIP code, but we plotted them and found them on streets back-to-back,\" Whiteside says.\n\nA reporter could have taken a list of group home addresses and stuck pins in a map to find the patterns, she said. \"But what reporter has time to do that?\"\n\nDavid Herzog can be reached at (610) 820-6584, or send e-mail to herzog@pulitzer.mcall.com\n\nNewsrooms that use desktop mapping are finding great stories. They've been able to locate clusters of blighted\n\nYet, desktop mapping remains one of computer-assisted reporting's frontiers. The 1994 University of Miami national CAR study found just 17 percent of the 208 responding newsrooms used mapping for analysis.\n\nChalk that up to a couple of reasons: Desktop mapping on the PC is relatively new, and the investment can run into the thousands of dollars.\n\nHigh-end mapping analysis programs, called geographic information systems (GIS), have been on mainframes for two decades. But it wasn't until around 1990 that desktop mapping caught on in newsrooms across the country, thanks to two factors.\n\nFirst, in the late 1980s, GIS program makers introduced easy-to-use PC software. Second, the results from the 1990 Census became widely - and inexpensively - available in computer form, along with the public domain geographic files.\n\nIn fact, the Census provided the initial push for many reporters and editors to learn desktop mapping.\n\nShortly after the Census, New York Times database editor John Freed used MapInfo to analyze the racial makeup of proposed city council districts. Under the Voting Rights Act, New York had to submit its redistricting plan to the U.S. Justice Department.\n\nWhen the Justice Department rejected the plan, the Times used its own findings to show why: East Harlem and the South Bronx's Hispanic population had been splintered into numerous districts - putting nearly all of the Hispanics in the minority, Freed said. \"It was very obvious.\"\n\nMapping tools\n\nHigh-end mapping programs, such as AtlasGIS and MapInfo give users a number of tools for analyzing data. Less expensive programs likely will have fewer of these features. Thematic mapping. This common feature takes your data and converts it to a map that uses colors or symbols to show variations. The Minneapolis Star Tribune used AtlasGIS for DOS to show changes in property values in the Twin Cities. Seeing the extent and location of the areas in decline, marked in red, was stunning, Loving said.\n\nThe Morning Call in Allentown, Pa., and other newspapers have used thematic mapping and Home Mortgage Disclosure Act data to find inner-city Census tracts with high home loan rejection rates.\n\nAddress matching or geocoding. You know those pin maps that police use to track murders? This feature allows you to create the computer equivalent. However, you must first use the mapping program's geocoder to assign latitude and longitude coordinates to each record in your database.\nThe Morning Call geocoded branch offices of merging banks. After we mapped the branches, we looked for overlap and speculated which ones might be shuttered.\nSelecting. A good mapping program allows you to select records that meet certain geographic criteria. MapInfo for Windows, the program I use, lets users select records that fall within boundaries or circles drawn on the map with a mouse. The top programs also have an SQL within boundaries or circles drawn on the map with a mouse. The top programs also have an SQL database manager that lets you find areas that meet certain criteria.\nFlexibility. The program should be able to work with a variety of geographic, text, spreadsheet and database files. In addition, the programs should be able to export to a wide range of formats.\n\nGeographic and 'attribute' data\n\nMapping programs use two kinds of data: geographic data and \"attribute\" data. The former defines features on the map (streets, county boundaries, rivers). The latter is what you analyze. When you use a desktop mapping program, you add layers of geographic and attribute data.\n\nMost mapping programs come bundled with some geographic data. However, you'll probably need to buy other geographic data to match the locations in your databases. If you want to analyze Federal Election Commission data, you'll need ZIP boundaries because the database contains the donor's ZIP code. If you're thinking about demographic analysis, get boundaries for census tracts or block groups that constitute tracts.\n\nDesktop mapping can cost thousands of dollars, but it doesn't have to, Doig says. Anyone who looks hard enough can find inexpensive geographic data sources. For instance, the Herald paid just $30 for election districts from Dade County. Local planning agencies may have the data you need. Also, check with university geography professors.\n\nSmaller newsrooms, or those with a tight budget, should talk to other departments (especially marketing or research) at their news organizations and see if they can share a desktop mapping program.\n\nThat's what the Flint Journal did about five years ago. The newsroom and two other departments chipped in for AtlasGIS for DOS, says Mary Ann Chick Whiteside, a former editor who's now the paper's new media manager.\n\nSince then, the Journal has printed a number of mapping-based stories, including one showing clusters of homes for the mentally ill. \"We knew they were all in the same ZIP code, but we plotted them and found them on streets back-to-back,\" Whiteside says.\n\nA reporter could have taken a list of group home addresses and stuck pins in a map to find the patterns, she said. \"But what reporter has time to do that?\"\n\nDavid Herzog can be reached at (610) 820-6584, or send e-mail to herzog@pulitzer.mcall.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-atlas-gis-has-site","slug":"1996-february-atlas-gis-has-site","url":"https://dwillis.github.io/uplink/article/1996-february-atlas-gis-has-site.html","year":1996,"month":"February","monthNum":2,"headline":"Atlas GIS has site","kicker":"Get mapping data on-line","author_name":"Dan Keating","author_title":"Reporter","affiliation":"Miami Herald","summary":"CIESIN is a free online repository offering detailed census maps and demographic data that can be imported into Atlas GIS, with step-by-step instructions for downloading map files and matching data from specific directories.","keywords":["CIESIN","maps","census"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["Atlas GIS","MapInfo","FTP","Internet","World Wide Web","CSV","PKZipped"],"full_text":"Even after you spend hundreds of dollars, you won't find detailed local maps in your brand new mapping program. Most of the mapping companies will be glad to sell them to you for hundreds of dollars more. Atlas GIS users are in luck, because there's an online repository that has tons of detailed maps and matching data. It's called CIESIN (pronounced see'-son), Consortium for International Earth Science Information Network. (Unfortunately, MapInfo doesn't import the mapfile format used at CIESIN).\n\nGet mapping data for your area, you can find census tract maps, block group maps, block maps, county maps, municipal boundaries and Zip code boundaries. You can also find 1990 Census data broken into the same groupings with ID labels already matched up to the maps. It has the 225 most-requested variables.\n\nHere's a detailed breakdown on how to get this free information into your system.\n\nLog onto the CIESIN file transfer site at http://ftp.ciesin.org. (You can check out the Web page, also, at http://www.ciesin.org, but the data is on the ftp site.) Go to pub/ and then census/ and then USA/ You need to pick up map files and then get matching data files.\n\nGo to tiger/ and then pick your state from the two-letter abbreviation. If you do not know the Census code for your Metropolitan Statistical Area, you should pick up msacodes.txt from the tiger/ directory before moving on.\n\nOnce in your state, take bna_st/ to get local area maps files. The “bg” files are block group maps - these are the smallest detail area that also has vast demographic information. If you're working with large areas, you can use the “t” tract or “C” county files for the state. I find the block group files extremely useful. Download the file for your MSA. You'll get a PKZipped file. We'll get back to that file in a minute.\n\nIf you want the smallest geographic level, take the bnablk/ directory instead of bna_st/. Download the “b” block file for your MSA. These are for very small areas, for which you can get population and household counts, but no detailed demographic data.\n\nNow you have to get the matching data. Back up through the ftp site back to pub/census/USA/. Now take stf/ and pick your state. Then get the “CONTENTS.XX* file, in which “XX” is the two-letter abbreviation for your state. It will help guide you to get the correct data files. There are “bg” block group data files, “II” tract data files, “zip” Zip code area files, “pl” files with data for cities and other places and more. If you need the population counts for blocks, go to the csvblk/directory and get the file matching the map file you got.\n\nDownload whatever files you need to match the geographic files you got. These will be PKZipped CSV files - meaning the data is separated by commas. Most spreadsheets and databases can import these files, as can Atlas GIS. Now you can sign off the Internet.\n\nYou'll need one special tool to be able to import the maps. It's called Import/Export or I/E and costs about $250 from Strategic Mapping. That's expensive, but it's a lot cheaper than buying the maps. It also allows you to import many other map formats, which you might come across if you try to get data from your county GIS department or other agencies.\n\nPut the PKZipped files into a directory. Unzip them. You'll have *.CSV data files and .bna map files. BNA is the Atlas transfer format.\n\nAt a DOS command line prompt, go to the directory where you have Atlas GIS and use the command line iemapfile.bnamymap.agf/na4\"\n\nImport/Export will create the geographic maps for Atlas GIS. You can then open the maps in the program and then open the .CSV files. The program will ask if the data files can be linked to the maps, and ask which field in the data file to use for matching. Select the “POLID” field except for the block groups, which should be linked with the “_Name3” field.\n\nNow you've got detailed local maps for whatever geographic area you want, along with a mountain of demographic data.\n\nDan Keating can be reached at (954) 985-4571, or send e-mail to dtkeats@ibm.net","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-february-maptitude-its-a-deal","slug":"1996-february-maptitude-its-a-deal","url":"https://dwillis.github.io/uplink/article/1996-february-maptitude-its-a-deal.html","year":1996,"month":"February","monthNum":2,"headline":"Maptitude: It's a deal","kicker":"Less sophisticated, but more affordable","author_name":"Carol Napolitano","author_title":"","affiliation":"The Omaha World-Herald","summary":"Maptitude is an affordable mapping software offering extensive U.S. and world geographic data with Census information, though it has limitations including no Mac version, lower map quality than competitors, and tedious manual editing requirements.","keywords":["mapping","software","cartography"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["Maptitude","FoxPro","Atlas GIS","MapInfo"],"full_text":"Maptitude is a great deal for all but the most sophisticated of mappers and it costs a fraction of other similar programs. One drawback: There is no Macintosh version.\n\nFor the United States, you get: county and state boundaries, Census designated places and metropolitan statistical areas boundaries, cities and populated places boundaries, roads and highways, local streets, landmarks, railroads, water, ZIP codes, Census tracts, Census block groups, designated market areas, areas of dominant influence and TIFF images of a few dozen major U.S. metropolitan areas. You also get world boundaries, places, highways, railroads and latitude and longitude grids.\n\nYou get selected Census data, including the basics of population, gender, race, marital status, age, ancestry, occupation, household income, poverty status, and more. In some instances, as when you join an outside database to your map, Maptitude will do some math for you, giving you raw numbers as well as highs, lows and averages. So, for example, if you had a FoxPro database on accidents in your area, you could join that to a Maptitude ZIP code map, and it would figure out the average number of accidents per ZIP code.\n\nYou get what you pay for. First, the entire program is designed for a screen operating at a 1024x768 resolution. If you are like me and operate at 640x480, you cannot see or use half or more of the toolbar, which means going in and out of menus more often than you might like. This resolution means that even with a 17-inch monitor, I often cannot see my whole map at once unless I make it very small and, consequently, lose labels and other details.\n\nSecond, map and label quality is poorer than that of Atlas GIS or MapInfo. Lines are fuzzier and so forth. These maps probably shouldn't be reproduced in the paper without some enhancement in a graphics program.\n\nMaptitude offers a nice range of thematic mapping options, but nothing extravagant. You can choose from colors and patterns, pin mapping, scaled symbol, dot density, pies, horizontal and vertical bars and stacked bars. You can modify and hide legends, as well as colors, patterns, dot shapes, etc.\n\nAs you build maps, you can save each layer as an editable or non-editable map. However, you have ZIP codes that extend beyond your city boundary, for example, you cannot tell Maptitude to split the ZIP area using the city boundary as its guideline. You must manually edit the ZIP code boundary, dragging its polygon lines down to match the city boundary. Or, you can trace the city line with the drawing tools and then delete the area outside of it. This can be extremely tedious.\n\nWhen you delete a portion of a geographic area, Maptitude estimates what percentage of the data is deleted and updates the remaining data accordingly. So, if you delete 60 percent of a ZIP code area because it is outside your city, then 60 percent of the population will be deleted in the attached Census data for that map. Of course, this isn't the greatest way to split data, because it assumes the population is evenly distributed in the ZIP code. It works the other way too. If you join three counties together in a map, it will add their three populations.\n\nOne annoying thing is that when you edit a map (for example, you select your city from the United States for mapping), the data does not automatically go with the map lines. Several times I created maps, forgetting to check that little box, and all I created were empty shells with no attached data tables. And you can't undo it. You have to start all over again.\n\nAnother problem: Thematic mapping rounds off numbers by default. You can check a box to turn this default off and you should in many cases. Rounding often messes up thematic mapping when you are plotting numbers in equal intervals because not all of the numbers will fall at equal intervals, making your map inaccurate. Turning off the rounding function fixes this.\n\nThis is an area where I am not very impressed. You can use an automatic labeling tool or a manual labeling process. With either choice, label layers cannot be treated differently when it comes to overlapping or not overlapping them. Therefore, if you have a map with ZIP code boundaries and then streets over that, and you want to overlap ZIP code labels but not street labels, you can't do it. They all must be overlapped or not overlapped. Automatic labels also cannot be moved around, so if they overlap, you can't pull them away from each other. Manual labels can be moved around, but manual labeling can be tedious and time consuming.\n\nYou can save maps in Maptitude as editable or non-editable. Editable versions can be copied from your PC and pasted into something like Paint Shop Pro (which you can download from the Internet as shareware and pay $69) and then saved in a variety of formats, including Adobe Photoshop and MacPaint. These can be opened up by graphics programs on Macs and sharpened, edited and published. If you have a network, you can move the files. If you don't have a network, your maps will have to be small enough to fit on a disk.\n\nDocumentation is in a single book that could be more detailed, but is decent. Technical service is knowledgeable, honest about the program's inadequacies, polite and helpful. Technicians are often all busy, but the company takes messages and gets them to the techs call back within a couple of hours. The customer service rep was courteous, knowledgeable and took the hint when I told him I wasn't interested in a hard sell. Additionally, the company gave us a 30-day unconditional money-back guarantee so I could put the program through the paces and decide whether it fit our needs.\n\nCarol Napolitano can be reached at (402) 444-1000, ext. 2403, or send e-mail to CarolNap@aol.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-february-get-computer-training","slug":"1996-february-get-computer-training","url":"https://dwillis.github.io/uplink/article/1996-february-get-computer-training.html","year":1996,"month":"February","monthNum":2,"headline":"Get computer training","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"NICAR offers multiple computer-assisted reporting training opportunities for journalists, including week-long bootcamps, a broadcast seminar, and a national conference scheduled throughout spring and summer.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Get computer training:\n\nNICAR Bootcamps, week-long intensive training seminars, March 10-15 and May 19-24, Columbia, Mo.\nNICAR/Medill Broadcast Seminar, March 23-24, Medill School of Journalism, Evanston, Ill.\nNorth Carolina Advanced Computer-Assisted Reporting Seminar, May 5-10, Chapel Hill, N.C.\nNational IRE Conference, June 13-16, Convention Center, Providence, R.I.\n\nThese dates are open to all journalists. For more information, call NICAR, (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-visit-nicar-at-httpwwwreporterorg","slug":"1996-february-visit-nicar-at-httpwwwreporterorg","url":"https://dwillis.github.io/uplink/article/1996-february-visit-nicar-at-httpwwwreporterorg.html","year":1996,"month":"February","monthNum":2,"headline":"Visit NICAR at http://www.reporter.org","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR's website, administered by Wallace Winfrey, provides comprehensive information about the organization's events, training seminars, newsletter, and database library, while also offering access to IRE's journalism resource center and the Hillman Project's global journalist resources.","keywords":["NICAR","journalism","resources"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Visit NICAR at http://www.reporter.org. Administered by NICAR's web master, Wallace Winfrey, the web site offers detailed information about NICAR's activities. This includes upcoming events, training seminars, the newsletter Uplink, and NICAR's database library. In addition, you can search the journalism resource center run by Investigative Reporters and Editors, at www.ire.org The web site also includes a menu for IRE and for the Hillman Project, which is developing a site for tips and resources for journalists around the world.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-pick-a-mapping-program","slug":"1996-february-pick-a-mapping-program","url":"https://dwillis.github.io/uplink/article/1996-february-pick-a-mapping-program.html","year":1996,"month":"February","monthNum":2,"headline":"Pick a mapping program","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A comparison of four geographic information system (GIS) software products including ArcView, Atlas GIS, MapInfo, and Maptitude, with pricing, features, and contact information for journalism and research applications.","keywords":["software","mapping","GIS"],"topics":["Technology"],"technologies":["ArcView","Atlas GIS","MapInfo","Maptitude","CD-ROM"],"full_text":"ArcView ($995)\nUnix workstation, Windows and Macintosh versions. Environmental Systems Research Institute; (800) 447-9778\nIncludes Avenue, a tool for building front-end interfaces. Also comes with U.S. state and county boundary files, ZIP-5 boundary and point files and point files for state capitals and cities with more than 10,000 people.\n\nAtlas GIS (Call for price)\nWindows and DOS versions. Strategic Mapping Inc.; (800) 472-6277\n\nMapInfo ($1,295)\nWindows and Macintosh versions. MapInfo Inc.; (518) 285-6000\nLatest Windows version, MapInfo Professional, includes U.S. state and county boundaries, ZIP code boundaries and points; 200,000 landmarks, 9,000 cities and state capitals.\n\nMaptitude ($395)\nWindows version only. Caliper Corp. (617) 527-4700\nIncludes CD-ROM of U.S. streets, useful for street address matching. A second CD includes U.S. state, county and census tract boundaries, as well as ZIP code boundaries and points. Census demographics are included for each geographic boundary type.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-newsrooms-flock-to-the-net","slug":"1996-february-newsrooms-flock-to-the-net","url":"https://dwillis.github.io/uplink/article/1996-february-newsrooms-flock-to-the-net.html","year":1996,"month":"February","monthNum":2,"headline":"Newsrooms flock to the 'Net","kicker":"Research shows 7 percent gain","author_name":"Bruce Garrison","author_title":"Professor","affiliation":"University of Miami","summary":"Online service usage in U.S. newsrooms grew 7 percent from 1994 to 1995, with 64 percent of daily newspapers now using some form of online tools, though large newspapers adopted them much more rapidly than smaller ones.","keywords":["newsrooms","online","growth"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet"],"full_text":"Use of on-line services in newsrooms is growing rapidly, up about 7 percent in just one year.\n\nIn 1994, reporters, news researchers, and editors at about 57 percent of daily newspapers in the United States used some form of on-line services.\n\nA year later, 1995 data from a national survey show a considerable 7 percent growth to 64 percent of daily newspapers using some type of on-line services-ranging from government bulletin board systems to commercial services to the Internet.\n\nThe findings are part of a two-year research project about CAR conducted in the School of Communication at the University of Miami that included data collected from 208 daily newspapers in 1994 and 287 in 1995. The response rates were about 41 percent in 1994 and 56 percent in 1995.\n\nThe differences in a newsroom's capability to use CAR may not be any more dramatic than in the use of on-line information services. Distinctions in large and small newspaper use of these on-line services are quite evident when basic use is analyzed by size.\n\nLarger dailies, which were defined as those with daily circulation over 52,800 in 1995, are far more resource rich, have almost completely moved into on-line tools for reporting. While 64 percent used on-line services overall, more than 90 percent of the nation's large newspapers reported using on-line tools in 1995, overshadowing the 40 percent use level of smaller dailies.\n\nIf a news organization subscribed to an on-line service, chances were good that it was widely used. A total of 33 percent of daily newspapers searched one or more times daily in 1995. The figure was 27 percent in 1994.\n\nIn 1995, 25 percent of dailies searched once a week or more often, but not daily. Another 11 percent searched once a month or more often, but not weekly. A total of 6 percent searched less than monthly, and 24 percent said they had never used on-line services.\n\nBruce Garrison can be reached at (305) 284-2846, or send e-mail to bgarriso@umiami.ir.miami.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-a-one-day-conference-on-satellite-imagery-and-the-news-media","slug":"1996-february-a-one-day-conference-on-satellite-imagery-and-the-news-media","url":"https://dwillis.github.io/uplink/article/1996-february-a-one-day-conference-on-satellite-imagery-and-the-news-media.html","year":1996,"month":"February","monthNum":2,"headline":"A one-day conference on 'Satellite Imagery and the News Media'","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A one-day conference on satellite imagery and its applications in news media will be held March 7 at American University in Washington, D.C., sponsored by multiple organizations including the Poynter Institute and the Miami Herald.","keywords":["satellite","journalism","imagery"],"topics":["Journalism & Newsroom","Technology"],"technologies":[],"full_text":"A one-day conference on \"Satellite Imagery and the News Media\" will be held March 7 at American University in Washington, D.C. The conference is being sponsored by American University, the Poynter Institute, the Miami Herald, Earthwatch, the U.S. Geological Survey, and For more information, call Professor Chris Simpson at (202) 885-2037, or fax him at (202) 885-2099.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-analytical-software-elbows-in","slug":"1996-february-analytical-software-elbows-in","url":"https://dwillis.github.io/uplink/article/1996-february-analytical-software-elbows-in.html","year":1996,"month":"February","monthNum":2,"headline":"Analytical software elbows in","kicker":"","author_name":"Bruce Garrison","author_title":"Professor","affiliation":"University of Miami","summary":"A 1995 survey of newsroom computer-assisted reporting (CAR) software usage reveals that Xy Write dominated word processing, Excel led spreadsheets, Paradox was preferred for databases, and MapInfo emerged as the top mapping tool, while most newspapers had not yet adopted statistical or mapping software.","keywords":["software","newsrooms","spreadsheets"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["XyWrite","Excel","Paradox","askSam","MapInfo","SPSS","Procomm","Word","WordPerfect","Lotus 1-2-3","Quattro Pro","FoxPro","Access","Atlas GIS","SAS"],"full_text":"Analytical software elbows in\n\nBy Bruce Garrison\nUniversity of Miami\n\nWhat was the most popular combination of software used for CAR in 1995? It was Xy Write, Excel, Paradox, askSam, MapInfo, SPSS and Procomm. Here are the details:\n\nWord Processing. Xy Write (27 percent) remains the most-used word processor and text editor in newsrooms for purposes related to CAR as well as general newsroom writing tasks.\n\nMicrosoft Word (20 percent) and Novell WordPerfect (14 percent) are becoming more common as newsrooms move to PC-based networks and adopt the leading office \"suites\" produce by Microsoft, Novell and Lotus.\n\nSpreadsheets. Microsoft's Excel has become the dominant spreadsheet (31 percent) in newsrooms that reported using spreadsheets, but many journalists also use Lotus 1-2-3 (20 percent) and Novell's Quattro Pro (17 percent). Noticeable proportions did not use spreadsheets or did not respond, however.\n\nExcel's use grew 9 percent from 1994 to 1995, while Quattro Pro and 1-2-3 use grew very little - about 1 percent each. The fact that there was a 16 percent drop in \"missing/none\" responses in the single year indicates more awareness of products in use as well as more use of spreadsheets that began in the past year.\n\nRelational databases. Relational database management systems, or relational database software, are very popular. In 1995, Borland Paradox was the preferred (24 percent) software, but Microsoft's FoxPro (21 percent) was also widely used. Another Microsoft relational database program, Access, is growing in popularity as part of the Microsoft Office suite \"Professional\" edition. It was used by 5 percent of newspapers in 1995.\n\nAnalytical Mapping. Mapping programs have not yet gained widespread use. One of the major reasons, until recently, has been the high cost of the software. Some producers lowered their prices in 1995.\n\nMost newspapers (71 percent) were not yet using computer mapping products. But the decrease in this figure shows signs of some growth in use. MapInfo (15 percent) and Atlas GIS (9 percent) are the most widely used.\n\nThere were also differences in use by size. Larger newspapers (43 percent) used the product far more than smaller ones (10 percent). MapInfo gained in use over the year, about 7 percent, while Atlas GIS did not change in proportion of use over the year.\n\nStatistics. Not many newspapers - just 16 percent - used statistical software in 1995. The Statistical Package for the Social Sciences (SPSS) is the most popular CAR statistics tool with just 9 percent use. The Statistical Analysis System (SAS), was used by 3 percent, and other products by 4 percent.\n\nCommunications. Procomm Plus dominates newsroom preferences for general on-line connections. In 1995, about 45 percent of newsrooms using communications software reported preferring Procomm Plus for DOS or Windows. All other programs - such as Crosstalk, Windows Terminal, Smartcom - constitute only about 39 percent of use. However, one in six newspapers, 16 percent still did not use any communications software or simply did not know what was in use.\n\nBruce Garrison can be reached at (305) 284-2846, or send e-mail to bgarriso@umiami.ir.miami.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-pcs-dominate-hardware","slug":"1996-february-pcs-dominate-hardware","url":"https://dwillis.github.io/uplink/article/1996-february-pcs-dominate-hardware.html","year":1996,"month":"February","monthNum":2,"headline":"PCs dominate hardware","kicker":"Striking differences among large, small newspapers","author_name":"Bruce Garrison","author_title":"Professor","affiliation":"University of Miami","summary":"A 1994-1995 University of Miami survey reveals that U.S. daily newspapers predominantly used Intel 486 processors, DOS with Windows operating systems, and increasingly adopted CD-ROM drives and larger hard drives for computer-assisted reporting, with notable differences between large and small publications.","keywords":["hardware","newspapers","survey"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["PC","Pentium","Macintosh","DOS","Windows","CD-ROM"],"full_text":"What types of PC hardware are most used by U.S. dailies in computer-assisted reporting? Findings from the 1994 and 1995 national surveys conducted by the School of Communication at the University of Miami suggest preference patterns are emerging. They include:\n\nProcessors. The most commonly used central processing unit for CAR projects is an Intel 80486-type that has been on the market since 1993. A total of 43 percent of all newspapers used \"486s\" for CAR in 1995. Only a handful of newsrooms percent) used the then-new Intel Pentium processor.\n\nA little more than one in 10 newspapers used any type of Macintosh for CAR (12 percent) and another one in 10 still used older 80386-type processors.\n\nDifferences in small and large newspapers are striking. Larger newspapers, for example, defined as those with daily circulation over 52,800 copies, rarely use Macintoshes for CAR, but are widely known to use them for graphics. Almost one-quarter of smaller newspapers use them for CAR. In fact, it is the second-largest category of CPU for the smaller dailies and only by a small margin.\n\nTop operating system\n\nOperating Systems. The most-used operating system in 1995 was the popular combination of DOS and a Windows overlay. More than half of the newspapers responding to the 1995 survey (52 percent) reported using some form of DOS with a version of Windows. Since the data in the 1995 survey were collected early in the year, prior to the debut of the national Windows 95 beta program or the final release, the newest form of Windows was not included or found to be in use.\n\nCD-ROMS. In the 1995 survey, more than half of the newspapers reported using one or more CD-ROM drives for CAR. A total of 38 percent reported using one CD-ROM drive and another 33 percent reported using two or more CD-ROM drives, or more than 71 percent used a CD-ROM drive. Large dailies typically have more than one CD-ROM drive, while smaller dailies are more likely to have only one drive and a much larger proportion had none.\n\nAbout 39 percent of the reporting newspapers used one or more CD-ROM drive in newsrooms. Of that, 36 percent reported using one CD-ROM reader, and just 4 percent used more than one CD-ROM reader. This reflects a 23 percent growth in the number of newspapers with two or more CD-ROM drives and a 19 percent growth in the number of newsrooms with CD-ROMs in use for CAR overall.\n\nGreater capacity\n\nHard drives. Hard drive manufacturers in 1994 and 1995 were making larger and faster drives and, at the same time, lowering their prices. This meant that newsrooms using personal computers for CAR had greater capacity available to them. And they were taking advantage of the newer drives. The mean CAR hard-drive size was over a gigabyte (1,055 MB) in early 1995. A year earlier, the mean hard-drive size was considerably less at 853 MB for the newspapers reporting.\n\nThe most frequent reported sizes in early 1995 were 1 GB (10 percent), 500 MB drives (5 percent), and 2 GB (4 percent). The sizes ranged from a 20 MB drive still in use to a 6.7 GB server in use for CAR.\n\nThese data suggest that news organizations acquire new CAR hardware as it becomes available on the market, but not very quickly. This suggests budgetary influences and that CAR is not yet a high spending priority in most newsrooms. Equipment is acquired as spending limitations permit it, but often a year or more after the new technology debuts on the market.\n\nNevertheless, there seemed to have been steady hardware advances in daily newspaper newsrooms between 1994 and 1995, suggesting similar growth between 1995 and 1996 as well.\n\nBruce Garrison can be reached at (305) 284-2846, or send e-mail to bgarriso@umiami.ir.miami.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-here-are-some-academic-studies","slug":"1996-february-here-are-some-academic-studies","url":"https://dwillis.github.io/uplink/article/1996-february-here-are-some-academic-studies.html","year":1996,"month":"February","monthNum":2,"headline":"Here are some academic studies","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"This collection of scholarly sources documents the evolution of computer technology adoption in newspaper journalism, from early database utilization in the 1980s through computer-assisted reporting methods in the 1990s.","keywords":["journalism","computers","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"\"Daily Newspaper Utilization of Computer Data Bases,\" Frederic F. Endres, Fall 1985, Newspaper Research Journal. \"Daily Use of Computers to Analyze Data,\" Cecilia Friend, Winter 1994, Newspaper Research Journal. \"Computer-Assisted Reporting,\" Bruce Garrison, 1995. Lawrence Erlbaum Associates.\n\nFor a full text, see NICAR's website at http://www.nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-ire-l-and-nicar-l-mailing-list-archives","slug":"1996-february-ire-l-and-nicar-l-mailing-list-archives","url":"https://dwillis.github.io/uplink/article/1996-february-ire-l-and-nicar-l-mailing-list-archives.html","year":1996,"month":"February","monthNum":2,"headline":"IRE-L and NICAR-L mailing list archives","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR have made their mailing list archives publicly accessible on their websites in multiple formats (HTML and plain text), allowing users to browse posts by thread, author, and date.","keywords":["archives","mailing","accessible"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","HTML"],"full_text":"Check out the IRE-L and NICAR-L mailing list archives on our websites at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in HTML or in plain text format, which is useful for downloading a particular month. If you have any comments on how we can make these archives more useful, send e-mail to wally@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-the-abcs-of-car","slug":"1996-february-the-abcs-of-car","url":"https://dwillis.github.io/uplink/article/1996-february-the-abcs-of-car.html","year":1996,"month":"February","monthNum":2,"headline":"The ABCs of CAR","kicker":"Practical guide to computer-assisted reporting","author_name":"","author_title":"","affiliation":"","summary":"Brant Houston's 'Computer-Assisted Reporting: A Practical Guide' is an introductory handbook that teaches journalists how to use computer tools and online databases for reporting, including accessing government data and detecting analytical flaws.","keywords":["reporting","databases","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet"],"full_text":"\"Computer-Assisted Reporting: A Practical Guide\" by Brant Houston specifically written for the beginner is a concise handbook providing students with an accessible guide to the essentials of computer-assisted reporting. The first text of its kind, \"Computer-Assisted Reporting: A Practical Guide\" can be used in a wide variety of introductory courses. Its focused approach also makes it suitable as a supplementary text for more advanced courses or as a handy reference for the working journalist. Emphasizing the most useful tools in on-line reporting, it springboards students into the computer age. The book provides, in a concise format, the knowledge students need to understand-and effectively use-computer-assisted reporting. The text helps students locate and negotiate government databases, examines how to use descriptive statistics to perform an initial analysis of those databases, and explains how to detect flaws. Other feature includes: Includes numerous practical examples and step-by-step procedures on the basics of computer-assisted reporting. Gives students a straightforward, practical, guided tour through the Internet. Accompanied by data diskette. Houston is managing director of the National Institute for Computer-Assisted Reporting at the University of Missouri. The book is available for $24 from IRE and St. Martin's Press Inc., New York.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-nicar-telephone-numbers-have-a-new-area-code","slug":"1996-february-nicar-telephone-numbers-have-a-new-area-code","url":"https://dwillis.github.io/uplink/article/1996-february-nicar-telephone-numbers-have-a-new-area-code.html","year":1996,"month":"February","monthNum":2,"headline":"NICAR telephone numbers have a new area code","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR telephone numbers are transitioning to a new 573 area code to replace 314, with both codes functioning until July when 314 is eliminated for Mid-Missouri regions outside St. Louis.","keywords":["NICAR","areacode","Missouri"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"NICAR telephone numbers have a new area code: 573 replaces 314; however, both area codes will work until July, when 314 will be eliminated for all of Mid-Missouri areas except St. Louis. The new numbers are: Brant Houston, (573) 882-1984 Rosemary Armao, (573) 882-1982 NICAR, (573) 882-0684 IRE, (573) 882-2042","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-from-the-nicar-library","slug":"1996-february-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-february-from-the-nicar-library.html","year":1996,"month":"February","monthNum":2,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering transportation safety, financial transactions, crime statistics, and regulatory compliance from various agencies. The databases span multiple decades and include detailed records useful for investigative journalism on topics ranging from accidents and fraud to discrimination patterns.","keywords":["databases","federal","government"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Election Commission contributions data, including donations by individuals and political action committees.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1994, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records.\nNational Bridge Inventory System data, includes inspection grades.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1994.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nNational Endowment for the Arts, grants, 1989-1993.\nFor up-to-date prices and more information, call (314) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-the-fec-has-made-available-free-of-charge-campaign-finance-data","slug":"1996-february-the-fec-has-made-available-free-of-charge-campaign-finance-data","url":"https://dwillis.github.io/uplink/article/1996-february-the-fec-has-made-available-free-of-charge-campaign-finance-data.html","year":1996,"month":"February","monthNum":2,"headline":"The FEC has made available free of charge campaign finance data","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"The FEC has launched a free internet service providing public access to campaign finance data including candidate information, downloadable databases, and historical election cycle summaries, with monthly updates beginning with 1995-1996 cycle data.","keywords":["campaign","finance","data"],"topics":["Government & Politics","Data & Databases"],"technologies":["Internet","World Wide Web","FTP"],"full_text":"The FEC has made available free of charge campaign finance data via the Internet. Data posted by the FEC includes: information about candidates, parties and other committees; downloadable databases containing data about candidates, parties and other committees; and summary files for past election cycles. The 1995-1996 cycle data will be updated monthly henceforth and posted on-line the first of each month. The data can be accessed via the World Wide Web at http://www.fec.gov or via anonymous ftp (file transfer protocol) at ftp.fec.gov.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-ire-seeks-training-director","slug":"1996-february-ire-seeks-training-director","url":"https://dwillis.github.io/uplink/article/1996-february-ire-seeks-training-director.html","year":1996,"month":"February","monthNum":2,"headline":"IRE seeks training director","kicker":"","author_name":"","author_title":"","affiliation":"IRE (Investigative Reporters and Editors)","summary":"Investigative Reporters and Editors is recruiting a training director for a new nonprofit educational organization in Mexico aimed at professional journalist development, with the position potentially leading to a coordinator role within two years.","keywords":["journalism","training","Mexico"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Investigative Reporters and Editors is searching for a training director for the new Instituto Mexicano de Investigacion Periodistica, a not-for-profit educational organization for Mexican journalists funded with a three-year grant from the McCormick Tribune Foundation.\n\nThe training director will report to the Insituto Coordinator with the goal of assuming that top position within two years and will help develop and carry out professional training programs for journalists. Applicants should be Mexican journalists who are fluent in English with an interest in investigative reporting and computer-assisted reporting. Send a letter, resume and references to IRE Executive Director Rosemary Armao, 138 Neff Hall Annex, University of Missouri School of Journalism, Columbia, Mo., 65211. The deadline is May 1.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-mapping-out-the-web","slug":"1996-february-mapping-out-the-web","url":"https://dwillis.github.io/uplink/article/1996-february-mapping-out-the-web.html","year":1996,"month":"February","monthNum":2,"headline":"Mapping out the Web","kicker":"On the Internet","author_name":"David Herzog","author_title":"","affiliation":"The Morning Call","summary":"A guide to useful websites for learning about and working with geographic information systems (GIS), ranging from beginner resources to professional tools and data sources.","keywords":["GIS","mapping","data"],"topics":["Technology","Data & Databases"],"technologies":["ArcView","MapInfo","Atlas GIS","Maptitude","World Wide Web","FTP","CD-ROM"],"full_text":"Whether you're a novice who wants to learn more about geographic information systems or a power user looking for files, you'll find a number of useful sites on the Internet's World Wide Web. Here are some of the best:\n\nCaliper Corp. This Massachusetts company makes Maptitude, a GIS package that comes with a load of useful data and maps. Cost is $395. www.caliper.com\n\nU.S. Census Bureau. This official Census site has all kinds of economic and demographic data on-line. In addition, you can get information about the Census' data products on CD-ROM and tape. www.census.gov\n\nConsortium for International Earth Sciences Information Network U.S. Demography page. CIESIN has 4 GB of compressed data and TIGER mapping files available for free download. www.ciesin.org/datasets/us-demog/us-demog-home.html\n\nEnvironmental Systems Research Institute. ESRI's web site has information about its ArcView GIS programs. Also, it has good general information for the beginner, including a glossary and \"Getting to Know Desktop GIS.\" www.esri.com\n\nGeographic Information Systems FAQ. This large (128 KB) file answers frequently asked questions regarding GIS. The subject matter, however, tends to be on the scholarly and arcane side. www.lib.ox.ac.uk/internet/news/faq/archive/geography.infosystems-faq.html\n\nGIS World Magazine. This magazine and its sister publication, Business Geographics (www.gisworld.com/mag/bg/index.html), put their indexes and sample articles on-line. www.gisworld.com/mag/gw/index.html\n\nMapInfo. The official home of MapInfo, maker of widely used GIS programs. Its web site includes product information, tech tips, training guides and user group lists. Also, you can download demo programs. www.mapinfo.com\n\nStrategic Mapping Inc. The home of Strategic Mapping, maker of the popular GIS package, AtlasGIS. Get product information and technical support files. You can download program demos. www.stratmap.com\n\nXerox Map download archive. You can find some TIGER and Census demographic files in this free download site maintained by Xerox Corp. ftp://spectrum.xerox.com\n\nDavid Herzog can be reached at (610) 820-6584, or send e-mail to herzog@pulitzer.mcall.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-journalists-needed-for-on-line-service","slug":"1996-february-journalists-needed-for-on-line-service","url":"https://dwillis.github.io/uplink/article/1996-february-journalists-needed-for-on-line-service.html","year":1996,"month":"February","monthNum":2,"headline":"Journalists needed for on-line service","kicker":"","author_name":"","author_title":"","affiliation":"The Miami Herald","summary":"The Miami Herald and El Nuevo Herald are hiring journalists and technical staff to develop bilingual English and Spanish online news services, with positions including web designer, managing editor, producer, and database programmer.","keywords":["journalists","online","bilingual"],"topics":["Journalism & Newsroom","Technology"],"technologies":["HTML","Photoshop","Adobe Illustrator","Unix","Perl","Java"],"full_text":"Journalists needed for on-line service\n\nThe Miami Herald and El Nuevo Herald need journalists to help develop on-line English and Spanish news and information services. Because content will be presented in two languages, fluency in both English and Spanish is a plus. Among the positions:\n\nWeb Designer/Illustrator with digital design skills and experience with HTML, Photoshop and Adobe Illustrator.\nOn-line Managing Editor responsibile for organizing online editorial content, building web pages and identifying web links.\nOn-line Producer to build web pages, organize and edit previously published material, create new content to be delivered online, and seek out web links. Experience in news research and computer-assisted reporting techniques would be valuable.\nDatabase/Unix Analyst Programmer to design databases for web publishing. Prefer experience in designing and implementing client/server relational database applications and familiarity with HTML, Unix platforms and utilities, including Perl. Expect to learn other programming tools, including Java.\n\nTo apply, send resume and statement of online philosophy to Elizabeth W. Ruiz, Employment Recruiter, The Miami Herald Publishing Co., 1 Herald Plaza, Miami, Fla., 33132.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-russell-clemings-websites-for-journalism-job-openings","slug":"1996-february-russell-clemings-websites-for-journalism-job-openings","url":"https://dwillis.github.io/uplink/article/1996-february-russell-clemings-websites-for-journalism-job-openings.html","year":1996,"month":"February","monthNum":2,"headline":"Russell Clemings websites for journalism job openings","kicker":"","author_name":"Russell Clemings","author_title":"computer-assisted reporting reporter","affiliation":"The Fresno Bee","summary":"Russell Clemings from The Fresno Bee recommends several websites for finding journalism job openings, including resources from the California Society of Newspaper Editors, Editor and Publisher, Gannett, and a tech job board that also lists technical writing positions.","keywords":["jobs","journalism","openings"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Russell Clemings, computer-assisted reporting reporter, The Fresno Bee, recently shared the following websites as good sources of journalism job openings: The California Society of Newspaper Editors has job listings at: http://www.ccnet.com/CSNE/jobs/postings.html Editor and Publisher classifieds are online at: http://www.mediainfo.com/edpub/ep/classi.htm Gannett has openings posted at: http://www.gannett.com/newswatch/nw0128-8.htm And while this one is mainly for computer job openings, it also includes technical writing and similar positions: http://www.espan.com/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-bits-bytes-and-barks","slug":"1996-february-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1996-february-bits-bytes-and-barks.html","year":1996,"month":"February","monthNum":2,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The National Institute of Computer-Assisted Reporting and Medill School of Journalism are hosting a two-day workshop on March 23-24 at Northwestern University for broadcast journalists to learn data analysis techniques, including internet research, databases, spreadsheets, and investigative reporting methods.","keywords":["workshop","journalists","broadcast"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","CD-ROM","spreadsheet"],"full_text":"Radio and TV journalists, mark your calendars for March 23-24, when The National Institute of Computer-Assisted Reporting and the Medill School of Journalism will offer a special two-day workshop at Northwestern University.\n\nThe workshop will focus on the special needs of broadcast journalists when using electronic data and will include discussions, demonstrations, and hands-on training from some of the best teachers and CAR practitioners in the country. Among the topics to be covered:\n\nHow computer-assisted reporting makes for better stories.\nA review of some of the best work done by broadcasters.\nUsing the Internet for breaking stories and references.\nBest World Wide Web sites for broadcasters.\nChoosing good commercial on-line services.\nBest on-line discussion groups.\nComputer-assisted reporting for investigative pieces.\nCD-Roms that every broadcast journalist should have.\nUsing computer-assisted reporting for radio.\nConvincing your station manager of the value of computer-assisted reporting.\nHow electronic databases turn into great stories.\nWhat's a spreadsheet and how to use it.\nWhat's a database manager and how to use it.\nTurning netsurfing into daily stories.\nThe ethics of computer-assisted reporting.\nHow to make the data visual.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-send-us-your-ideas","slug":"1996-february-send-us-your-ideas","url":"https://dwillis.github.io/uplink/article/1996-february-send-us-your-ideas.html","year":1996,"month":"February","monthNum":2,"headline":"Send us your ideas","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE/NICAR National Conference is scheduled for June 13-16 in Providence, with organizers seeking panel and speaker suggestions from members across all journalism sectors. Additional conference details, pricing, and Providence area information are available on the Providence Journal's conference website.","keywords":["conference","journalism","Providence"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Planning is underway for the June 13-16 IRE/NICAR National Conference in Providence. We are intent on making this a program appealing to all our members - broadcast, print, freelance, big and small markets, alternatives, etc. Send suggestions for panels and speakers now to Rosemary Armao, IRE executive director, at jourrca@mizzoul.missouri.edu, or call her at (573) 882 1982. Meanwhile, check out the Providence Journal's National Conference Homepage at: http://www.projo.com/ire/. You'll find dates, prices, hotel numbers, information on the shape of the program so far and fun things to do in New England.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-february-keep-up-with-nicar-on-line","slug":"1996-february-keep-up-with-nicar-on-line","url":"https://dwillis.github.io/uplink/article/1996-february-keep-up-with-nicar-on-line.html","year":1996,"month":"February","monthNum":2,"headline":"Keep up with NICAR on-line","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Instructions for subscribing to journalism listservs NICAR-L and IRE-1 via email, with additional information about accessing IRE/NICAR resources through CompuServe's Journalism Forum.","keywords":["listserve","journalism","subscribe"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","CompuServe","Listserv","Email"],"full_text":"Subscribe to our listserve and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURI.EDU In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.\n\nIRE/NICAR is also accessible through CompuServe's Journalism Forum. Go to the JForum, Section 19. Also look into the IRE/NICAR files in Library 19.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-computer-assisted-reporting-turns-on-more-tv-journalists","slug":"1996-march-computer-assisted-reporting-turns-on-more-tv-journalists","url":"https://dwillis.github.io/uplink/article/1996-march-computer-assisted-reporting-turns-on-more-tv-journalists.html","year":1996,"month":"March","monthNum":3,"headline":"Computer-assisted reporting turns on more TV journalists","kicker":"On the air raves","author_name":"Mike Wendland","author_title":"","affiliation":"WDIV-TV Detroit","summary":"The article discusses the growing adoption of computer-assisted reporting (CAR) among broadcasters, illustrating through anecdotes how TV journalists are applying these techniques to enhance storytelling and investigative reports.","keywords":["broadcast","journalists","CAR"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"NICAR trainer Jennifer LaFleur just might have the defining anecdote about the problem of implementing computer-assisted reporting in broadcast newsrooms. It involves a telephone call she received a couple years ago from a West Coast television station \"The news director said he had heard about this 'computer reporting thing' and thought it would be pretty 'promotable',\" said LaFleur, the database editor of the San Jose Mercury News. He wanted LaFleur to come and train his whole news staff. But the news director didn't want the full \"bootcamp\" of six days. He didn't want a two-day seminar. He wanted her to do it all in one day. And he had a condition. \"He expected me to come out there, teach his staff everything they needed to know to be CAR reporters in about five or six hours, and he expected them all to have found a major story that would air that very night,\" LaFleur said, shaking her head. She told him, sorry, that would be tough to do. He never called back. LaFleur's story of the impatient and unrealistic news director has been all too typical. Although broadcasters are rushing like never before to get on the CAR bandwagon, electronic journalists still lag far behind their print counterparts in using these new skills. News managers are cautious by nature, skeptical by experience and stingy out of necessity. They are so overwhelmed with the day-to-day responsibilities of producing two to three hours of news that anything that isn't immediate, isn't practical. Besides, they've seen newspaper CAR projects and the rows of statistics, graphs and charts. Putting that kind of stuff on the air flies in the face of everything the consultants say about the need to make TV stories punchy and compelling. Then there's their concern over training and time. Managers worry that the learning curve is so steep that it'll be long weeks before CAR stories start showing up on air. Fortunately, thanks to the growing success of CAR projects at stations across the country, the managerial resistance is slowly giving way. For just as newspapers and broadcasters tell their regular stories in very different ways, using different techniques, so it is with CAR stories - which are not boring, don't require long away-from-work training and can quickly start showing up on the air. Revving up CAR I just finished a five-part investigative series and a half-hour documentary on widespread abuse, neglect and fraud in the nursing home industry that never would have been possible without having CAR skills. The only thing near to a graph or chart was a short Chyron roll of the 20 nursing homes we identified with the highest number of health and safety violations - a list that thousands of viewers downloaded from our World Wide Web page and even more wrote in and requested by mail. The series and documentary had hidden video, powerful interviews with nursing home patients, relatives, past and present workers, and confrontations with balky state officials. It was a consultant's dream. It never would have happened without CAR because it would have taken two to three times longer to research and report. CAR broadcast stories usually take less, rather than more, time to produce because CAR techniques are efficient and add depth to broadcast reporting and researching that most journalists simply are too busy to achieve. Take the case of Mark Sauter, an investigative reporter with the nationally-syndicated news magazine \"The American Journal.\" Last June, Sauter attended some CAR workshops at the annual IRE national conference in Miami. On his return to New York, he immediately started using CAR and helping teach his co-workers. In just a few weeks, Sauter and his unit produced a blockbuster report that detailed how airlines routinely return planes to service after they've sustained serious damage, sometimes in fatal disasters. Sauter's team found 90 passenger airliners the National Transportation Safety Board said suffered \"substantial\" damage in accidents since 1985, 52 of which were put back in service. FAA computer data, on-line research, and contacts, victims and experts found through the 'net led the team to the people they needed to interview and to video of some of the \"rehabbed\" passenger jets landing and taking off. \"CAR gave us a significant advantage. It made this story possible,\" Sauter said. Not all stories have to be \"big.\" Wes Williams, at KPNX-TV in Phoenix, found, wrote and edited a CAR story in 30 minutes. Last fall, he spotted a routine Associated Press roundup about state-by-state home sales from the National Association of Realtors. The wire piece simply recounted the raw numbers on how many homes were sold in each state for a three-month period. After accessing Bureau of the Census data, using a spreadsheet to compare the AP stats and his state's population figures, Williams found that Arizona had the highest number of home sales per capita in the nation. Fleshed out with interviews with Realtors and homebuyers, he literally plucked a story out of cyberspace. 'Exotic condoms' In Raleigh, N.C., reporter Dave Boliek, obtained a database of state government contracts and expenditures. Most of it was very boring. But as he scrolled down the alphabetized listings, he noted something for \"birth control devices.\" Under that, he found \"condoms.\" \"I came across tens of thousands of dollars being spent for hundreds of thousands of condoms,\" he said. found contracts for colored condoms, ribbed condoms, rainbow-colored, ultra-sensitive condoms. Even a contract for mint-flavored condoms.\" Total amount of state money spent for 2 million condoms given out by public health departments in North Carolina: $150,000. It was a story made for TV, and the controversy over the \"exotic condoms\" continued for weeks. In Minneapolis, Alan Cox of WCCO is one of maybe a half-dozen local TV reporters who spend most of their time as CAR specialists. Cox has used his computers to track and map political spending, crime trends, and to expose a controversial government program that allows out-of-state criminals to be paroled to Minneapolis. Cox said the training is not that hard. Before his CAR assignment, the only thing he had ever done on a computer was write a wedding invitation on a rented machine at an office supply shop. \"You can get the basic skills in a couple of days of hands-on training,\" he said. \"After that, start using it. You just get better.\" There are other encouraging signs about CAR taking root in broadcast journalism. At the Car Rock '95 computer-assisted reporting conference in Cleveland last fall, well over 50 broadcast journalists showed up. The year before, only a handful attended. More than 300 showed up, overfilling the room, lining up against the walls, spilling out into the hallways. So, the broadcasting boom in CAR has finally begun. And probably a good bet that the unreasonable West Coast news director who called LaFleur either has been or soon will be replaced after getting beaten by CAR-trained competitors. Management is also changing. I spoke about CAR to news directors at the RTNDA convention in New Orleans last year. Organizers told me to expect 50, maybe 75.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-march-a-look-at-old-decaying-dams","slug":"1996-march-a-look-at-old-decaying-dams","url":"https://dwillis.github.io/uplink/article/1996-march-a-look-at-old-decaying-dams.html","year":1996,"month":"March","monthNum":3,"headline":"A look at old, decaying dams","kicker":"Frankly, my dear","author_name":"Stan Dorsey","author_title":"NICAR staff","affiliation":"NICAR","summary":"A Dateline NBC investigation reveals that thousands of aging American dams pose serious safety risks to nearby communities, with many lacking proper emergency plans and regular inspections despite their hazard ratings.","keywords":["dams","inspection","safety"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["Nexis","CD-ROM"],"full_text":"Everyone hates a leaky faucet. And we all know the hassles involved with repairing a flooded basement. But have you ever wondered what would happen if a dam holding back thousands of tons of water decided to break right into your own back yard? A Dateline NBC report revealed this threat does exist for thousands of unsuspecting residents across the country. In need of story ideas, Dateline producer Chris Scholl acted on a hunch that the nation's dams are old and decaying. A quick Nexis search found Congressional testimony by Brad Iarossi, a legislative affairs representative from the Association of State Dam Safety Officials. Iarossi lamented not only the condition of many dams but also the lack of funding and immense workload many state inspectors face. Scholl knew he was onto something and asked Dateline reporter David Hinchman to look deeper into the numbers. Hinchman's findings formed the foundation of a June 27, 1995, piece titled \"Swept Away.\" Looking deeper Hinchman knew he had to find the data to support his producer's suspicions and to make the story hit home with viewers. His first move was to contact the Federal Emergency Management Agency and obtain its free National Inventory of Dams CD-ROM database. With information on more than 75,000 dams, the database compiled by the U.S. Army Corps of Engineers contains various facts about the location, stability and inspection history of dams. Hinchman, focused on six fields to get the hard-hitting facts he needed. Dam age: Hinchman looked in the \"Year Completed\" field and focused specifically on dams that were more than 50 years old, assuming that these dams were more likely than newer dams to be in poor shape. He was forced to rely on inferences of this sort because the database contains no field that specifically identifies the quality or condition of a dam. Believe it or not, there are more than 20 active dams erected as long ago as the 18th century. Dam location: One major concern of Hinchman and Scholl's was how many dams pose a direct threat to human life simply by their proximity to communities. The \"Distance Nearest City\" and \"Nearest City\" fields helped them determine this by indicating, in miles, how close the nearest city is to a dam. Hazard rating: Another field that helps determine whether human life may be in jeopardy is the \"Downstream Hazard\" field. It rates exactly how dangerous a dam is to those living near it if the dam fails. Dams are coded H (high), S (significant) or L (low) based on the potential threat they pose. Hinchman said this field is much like the \"Sufficiency Rating\" field found in the National Bridge Inventory database. Emergency action plan: This was one of the most telling fields for Hinchman when it came to determining the culpability of local dam officials. While some smaller, more remote dams are not required to have an emergency plan, more than 80 percent of the dams rated as high hazard had no plan in place. Inspection date: A final clue in helping to identify dam neglect and workload of state dam officials was the \"Last Inspection Date\" field. Hitting the Road Inspecting the database and finding specific information made the Dateline team's job of \"cruising around the country looking for dams with problems or that were literally crumbling\" much easier, Hinchman said. It also allowed the team to focus on making some human connections for the story. Hinchman emphatically noted that when doing broadcast CAR stories this step is crucial. \"You have to have a human face with the story,\" he said. \"Strictly survey stories tend not to work well for broadcast. In print, for example, you could go to the courtroom and get quotes from court records. In broadcast, you must find the person then fly or drive to photograph them.\" Use local resources Of particular help to Hinchman in finding the sources for the story were the engineers employed by the Association of State Dam Safety Officials. Not only were they extremely helpful in locating victims for Dateline to interview, but in many cases, they were \"angry because they didn't have enough money to go inspect all the dams they needed to,\" Hinchman said. And, as Hinchman and Dateline found out, such grudges can lead to good information. All told, the project took about four months to complete and produced some impressive results. Some of the most shocking, however, were completely unexpected. In the weeks following the story, several major dams failed, including one at the Folsom Lake Dam above Sacramento, Calif. The Association of State Dam Safety Officials decided to use the piece to lobby Congress for increased federal funding of dam inspection programs nationwide. The association presented Dateline with a public awareness award for the investigation.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-march-get-computer-training","slug":"1996-march-get-computer-training","url":"https://dwillis.github.io/uplink/article/1996-march-get-computer-training.html","year":1996,"month":"March","monthNum":3,"headline":"Get computer training:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Multiple computer-assisted reporting training opportunities are available for journalists, including NICAR bootcamps, broadcast seminars, and conferences scheduled across the spring and summer.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Get computer training:\n\nNICAR Bootcamps, week-long intensive training seminars, May 19-24, Columbia, Mo.\nNICAR/Medill Broadcast Seminar, March 23-24, Medill School of Journalism, Evanston, Ill.\nNorth Carolina Advanced Computer-Assisted Reporting Seminar, May 5-10, Chapel Hill, N.C.\nNational IRE Conference, June 13-16, Convention Center, Providence, R.I.\n\nThese dates are open to all journalists. For more information, call NICAR, (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-check-out-recent-broadcast-projects","slug":"1996-march-check-out-recent-broadcast-projects","url":"https://dwillis.github.io/uplink/article/1996-march-check-out-recent-broadcast-projects.html","year":1996,"month":"March","monthNum":3,"headline":"Check out recent broadcast projects","kicker":"Need inspiration?","author_name":"Mary Flegas","author_title":"","affiliation":"Missouri School of Journalism","summary":"This article highlights two investigative journalism projects by KSTP-TV using Computer-Assisted Reporting (CAR) techniques: one exposing unlicensed home remodelers defrauding Minnesota homeowners and the state's weak enforcement of penalties, and another investigating a firefighters union officer's abuse of sick leave and mismanagement of a museum trust fund.","keywords":["investigation","fraud","enforcement"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["FoxPro","Excel","CompuServe","CDB Infotek","Westlaw","COBOL","Nexis","Internet"],"full_text":"In \"Dreams Destroyed,\" a November 1995 story by KSTP-TV in St. Paul, Minn., reporter Joel Grover exposed how state-licensed remodelers swindled thousands of Minnesotans by tearing apart their homes during harsh winters and then threatening to leave if they weren't paid more than promised. This is among broadcast projects to air using CAR techniques. The following is a sampling of some of the most recent stories. The KSPT-TV investigative team, for instance, also found unlicensed contractors working illegally and ripping off homeowners. Minnesota law calls for punishing unlicensed contractors with 90 days in jail and with a $700 fine. The investigative team looked at whether this law was effective and found that it was seldom enforced. In only three of 63 cases in three years did the state push for criminal penalties. Instead, most received letters requesting them to stop their illegal contracting businesses. The data: The state of Minnesota's Department of Commerce maintains two large relational databases that are used to regulate a number of industries, including home remodeling. Unit 5 received the data in askii comma delimited files, which were loaded into FoxPro and Excel. The investigative team also reviewed lawsuits from federal and district courts, state and city licensing and enforcement records, complaints filed with the state and city agencies, secretary of state filings and Dun and Bradstreet business reports. The team retrieved some of these documents through Compuserv, CDB Infotek and Westlaw on-line services. In \"Big Burn,\" a May 1995 story by KSTP-TV in St. Paul, Minn., reporter Jay Kolls looked at whether the firefighters union was abusing its sick leave policy. Union officers were drinking at bars when they were supposed to be either working or sick. One particular union officer for the firefighters association, Gerald Brown, was targeted for his wrongdoings. The analysis indicated that Brown was out of the office 78 percent of the time at the taxpayers' expense and had used all of his available sick leave. KSTP also uncovered that a trust fund set up by a retired firefighter for a new firefighters' museum had been mismanaged, with $20,000 of the fund being spent on other items. Brown was a trustee to the fund. Reconciling the matter cost the fund another $18,000. Also, donations for the Welfare Association for Firefighters were skimmed. The Charities Review Council said the group must donate 70 percent of its money, but it had only donated 34 percent. As a result of the report, the attorney general's office and the mayor investigated. A building was found for the new museum. The data: The investigative team looked at 16 months of the city's union records, focusing on sick leave, vacation time and union time. The team compiled personnel records and time sheets from several fire stations into Excel and checked it against Minneapolis payroll records. The payroll records had to be converted from the original COBOL to FoxPro. In \"Cyberspace Casinos,\" a November 1995 story by WLS-TV News in Chicago, Ill., reporter Chuck Goudie looked into numerous gambling sites on the Internet and backtracked various Internet addresses to determine who was behind the gambling product. The multimillion-dollar gambling providers said they were getting around U.S. laws by setting up shop on Caribbean Islands, but WLS uncovered evidence that the top gambling service was putting people on-line through equipment in the United States. The data: The Internet was used to trace computer addresses, along with Lexis-Nexis. Cumulative reports from Attorney General's Offices in various states were examined to investigate the legality of such activities. In a February 1995 story by WJXT-TV in Jacksonville, Fla., reporter Paul Van Osdol revealed that nearly 400 Jacksonville area restaurants were considered unsanitary by state standards. Problems included roaches in kitchens and rats in storage rooms. The investigative team also found that inspectors rarely fined restaurants for health and safety violations, even after repeated citations. The data: WJXT obtained the restaurant inspection database for $65 from the state of Florida. In a February 1994 story by WCCO-TV in Minneapolis, Minn., reporter Kevyn Burger studied milk and juice in Minnesota schools. The station tested more than 1,000 containers and found companies shorting schools on their contracts, in one case by as much as 12 percent. State inspectors went into schools and confirmed underfilled cartons. As a result of the investigation, the dairies purchased new equipment, enforced stricter quality control and pledged to put the correct amount milk in the cartons. One company reimbursed the schools and donated 100,000 underfilled containers of juice to charity. State officials now say school beverages will be tested regularly. The data: The investigative team used the test procedures outlined in the National Bureau of Standards Handbook 133 (Third Edition) and NIST Handbook 133 (Supplement 4). These are the standards for testing procedures in the world of weights and measures. Federal weights and measures officials helped write the standards. The team used computers to organize the data and track the milk and juice lots to determine patterns. In a March 1994 story by WJXT-TV in Jacksonville, Fla., reporter Paul Van Osdol looked at whether school teachers in the public school system at Duval County had criminal records. Of the system's 6,000 educators, nearly 200 had a record. In many of these cases, the charges were significant - there were teachers who had been involved in shootings, stabbings and theft. In many of the cases, the Duval County School System knowingly allowed the offenders to keep teaching. Among the offenders who were still teaching was a driver's education teacher who had several DUI convictions, an elementary teacher who had been judged criminally insane, and a counselor who had several convictions for indecent exposure. Some teachers had lied about their occupation when they were arrested. The data: A search of records from the Duval County courthouse. The original plan was to build a database of these records to cross-check teachers' names with criminal records, but the city charged $3,000 for its criminal database. The records were checked manually. The electronic records did not match occasionally with the paper records.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-facts-and-findings-from-national-inventory-of-dams","slug":"1996-march-facts-and-findings-from-national-inventory-of-dams","url":"https://dwillis.github.io/uplink/article/1996-march-facts-and-findings-from-national-inventory-of-dams.html","year":1996,"month":"March","monthNum":3,"headline":"Facts and findings from \"National Inventory of Dams\"","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The National Inventory of Dams reveals critical infrastructure vulnerabilities, with nearly half of flood deaths in Georgia linked to dam failures and a significant portion of U.S. dams lacking proper inspection, maintenance, and emergency preparedness measures.","keywords":["dams","safety","infrastructure"],"topics":["Public Safety","Data & Databases","Health & Environment"],"technologies":["National Inventory of Dams"],"full_text":"Facts and findings from \"National Inventory of Dams\"\n\nAlmost half of the people killed in the 1994 Georgia floods lived downstream from failed dams or dams that failed to operate properly.\nNearly 20 percent of the nation's dams are over 50 years old.\nMore than 2,000 of the country's dams have gone more than five years without inspection.\nOne-third of the nation's most dangerous dams have no emergency action plan in place.\nThirty-two percent of the dams in the inventory have a high or significant hazard potential.\nOnly 35 percent of all dams rated as high-hazard have emergency action plans.\nNinety-nine percent of the dams in the inventory are owned by municipal or private owners.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-radio-tunes-into-car","slug":"1996-march-radio-tunes-into-car","url":"https://dwillis.github.io/uplink/article/1996-march-radio-tunes-into-car.html","year":1996,"month":"March","monthNum":3,"headline":"Radio tunes into CAR","kicker":"Making numbers work on air","author_name":"Glaston Ford","author_title":"NICAR staff","affiliation":"NICAR","summary":"Radio reporters at NPR and WBUR use computer-assisted reporting (CAR) and spreadsheets to analyze numerical data for campaign finance and political stories, translating complex numbers into compelling narrative angles for broadcasts.","keywords":["reporting","data","radio"],"topics":["Journalism & Newsroom","Investigative Methods","Data & Databases"],"technologies":["Excel","FoxPro","spreadsheet"],"full_text":"Computer-driven stories and the numbers they produce challenge radio reporters.\n\n\"You can't just box or graph a bunch of numbers,\" said Andrew Caffrey, managing editor at WBUR radio in Boston. \"You have to translate it into English as if you were telling a story.\"\n\nNational Public Radio's Peter Overby agreed. He said it is hard to work numbers into a radio piece. To make matters worse, there usually is not time for in-depth stories.\n\nYet Caffrey and Overby have found CAR to be a valuable skill for radio reporters.\n\n\"I mostly use it for quick and dirty type of stuff, like the percent of campaign contributions that came from PACs,\" said Overby, who is on NPR's money, power and influence beat.\n\nWhen Phil Gramm announced his candidacy for president, Overby looked at who had supported Gramm over the years.\n\nHe gathered information on the senator's fund raising between 1989 and 1995 and organized it in a spreadsheet. Only the grand total, $133.9 million was used on the radio.\n\n\"What I usually do,\" Overby said, \"is analyze the numbers and talk about the trends. Most of what I do is working with Excel on my spreadsheets.\"\n\nIn other instances, he used spreadsheets to:\n\nTrack the big swing in PAC money from House Democrats to House Republicans in early 1995.\nTally the fund raising by the two parties and by Speaker of the House Newt Gingrich after he and President Clinton agreed to establish a blue ribbon commission on campaign finance reform.\nDo a deadline count of donations to Senator Bob Packwood's campaign and legal defense fund by one PAC.\n\nStory anchors\n\nCaffrey, who uses spreadsheets more than database managers, said CAR helps him track and look at numbers. \"I use spreadsheets like a large calculator,\" he said.\n\nWhen he was a reporter at WBUR, Caffrey did several CAR projects. He used spreadsheets to look at the 351 towns in Massachusetts. Caffrey said it is hard to compare towns, but with a spreadsheet, it was easy to analyze tax bases and water rates. His station has also analyzed lottery sales and revenue.\n\n\"Good computer work gives you an anchor for your story,\" Caffrey said. \"It finds things you wouldn't ordinarily see. It sharpened my critical thinking.\"\n\nHe said CAR allowed him to go after stories he couldn't go after before. He repeatedly asked himself, \"What more could I do with these numbers?\"\n\nIt doesn't have to be difficult or time-consuming.\n\nMaking connections\n\nWhile looking at lobbyist's contributions to state representatives, Caffrey noticed that one lobbyist, Tom Joyce, had made several large contributions. Caffrey sorted them by date and found they were all on June 7.\n\n\"What's so important about this day?\" he asked himself.\n\nCaffrey looked at the legislative activity on that day and a few days before and after.\n\nWhat he found made for a strong lead into his story:\n\n\"Last June, on the eve of a landmark vote to reform the role of money in politics, a select group of Massachusetts lawmakers got a tantalizing last taste of what they were about to give up.\n\n\"On June 7, lobbyist Tom Joyce donated $13,300 to 21 legislators.\n\n\"And the very next day, those lawmakers joined their colleagues in passing the nation's most restrictive campaign finance law,\" Caffrey wrote.\n\nJoyce gave more money in one day than the new law allowed him to give in an entire year.\n\n\"It was an easy little thing,\" Caffrey said of the analysis. \"It gave the story a nice pop.\"\n\nCaffrey uses Excel 5.0 and FoxPro 2.6 on an 486 PC with a couple hundred meg hard drive. His PC is run off a server. He downloads some of his data from state government bulletin boards and has even resorted to typing it in himself in the past.\n\nOverby runs Excel on an IBM PC with a 75 mhz Pentium processor. He gets much of his data from his FEC on-line account.\n\nThe Center for Responsive Politics has released a new analysis of soft money contributions made to the national parties in 1995. The release and three-page report are available on disk or as a printout from the center by calling (202) 857-0044. Or, if you have questions, send e-mail to Research Director Josh Goldstein at jgoldstein@crp.org.\n\nGlaston Ford can be reached at (573) 882-0684, or send e-mail to glaston@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-excel-paints-picture-for-kpnx","slug":"1996-march-excel-paints-picture-for-kpnx","url":"https://dwillis.github.io/uplink/article/1996-march-excel-paints-picture-for-kpnx.html","year":1996,"month":"March","monthNum":3,"headline":"Excel paints picture for KPNX","kicker":"CAR trips: First adventures","author_name":"Wes Williams","author_title":"","affiliation":"KPNX","summary":"KPNX investigated lead paint hazards in Phoenix by analyzing Census data to identify ZIP codes with the highest percentage of older homes built before 1978, then tested playgrounds, parks, and schools in those 'danger zones' to confirm elevated lead levels.","keywords":["lead","Phoenix","data"],"topics":["Data & Databases","Investigative Methods","Health & Environment"],"technologies":["Excel","Internet","World Wide Web"],"full_text":"The federal government banned lead paint in 1978 because of the health threat it posed to children in particular, but KPNX wanted to show Phoenix residents where the threat is today. Wes Williams, who attended the NICAR Bootcamp in January, tells how the series, which was begun in January, unfolded.\n\nWe tested playground equipment and city parks and schools across the area and found many contained chipping lead paint. Our I-team reporter, Lonni Leavitt, gave me a list from the state showing the number of homes built in each decade for every Census tract in our county. We wanted to be able to tell people whether they lived in an area full of mostly older homes (built before 1978), which therefore might contain lead paint.\n\nFirst problem\n\nOur problem was that no one knows in which Census tract they live, so we decided to convert this to ZIP codes. I found a table on the Internet to convert them through one of the groups that works with the Census bureaus, but I found there was a problem with some Census tracts covering more than one ZIP code, and vice versa.\n\nFortunately, there proved to be an easier solution. I went to the Census Bureau home page (http://www.Census.gov) and found the Summary Tape File 3B search tool under the Data Access Tools/Lookup section. The STF3B classifies the 1990 Census data by ZIP code. I found a list of ZIP codes for our area at the U.S. Postal Service home page (http://www.usps.gov) and plugged them into the STF3B search tool.\n\nThe STF3B gives an extended list of categories. One of them happened to be \"year dwelling built.\" I also had it give me the total number of homes for each ZIP code. I ended up with a text file of the total numbers of homes built in each decade by ZIP code.\n\nIt was easy to save this, parse it into Excel, and start crunching numbers. I figured out which ZIP codes had the highest percentage of older homes, and we did a story on where those \"danger zones\" are (areas where at least 80 percent of the homes in each ZIP code were built before 1990).\n\nOf course, so far, this sounds like a newspaper story, not one for broadcast. But we checked out homes and schools in those areas, found higher lead in both, and used the \"danger zones\" for a graphic in the story. We then included the full table with other lead paint information in a brochure we printed for our viewers.\n\nWe were surprised by some of the results. We expected the downtown areas to show up at the top of our list, but we didn't think about Sun City, where almost all of the homes were built back when lead paint was in use, and the two local Air Force bases, one of which has been turned over for civilian use. We included since people still work in those same buildings.\n\nTroubleshooting\n\nI should have looked for another source of the data by ZIP code first rather than trying to convert it from the Census tract list the state gave us.\nBecause the Census data worked by decades, I could only determine how many homes were built before 1980, not before 1978, when lead paint was banned.\nThree of our current ZIP codes did not appear in the Census data because they didn't exist in 1990. They're in newly-developed areas, but we had to catch that they were missing to put them on our list. We listed them as 0 percent.\nSome ZIP codes I got from the Postal Service home page belonged to just mail boxes, so they did not show up in the Census data. We didn't include those in any of our work.\nIt wasn't really a problem, but I didn't start my \"lab notebook\" detailing every step of work until my next project. I realize now just how much that might help me in the future. Take the time to jot a quick note. It will save you a lot of time later. It's already saved me a couple of times in the past month.\n\nWes Williams can be reached at (602) 261-6152, or send e-mail news@indirect.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-search-and-seize-our-abstracts","slug":"1996-march-search-and-seize-our-abstracts","url":"https://dwillis.github.io/uplink/article/1996-march-search-and-seize-our-abstracts.html","year":1996,"month":"March","monthNum":3,"headline":"Search and seize our abstracts","kicker":"Web Talk","author_name":"Brant Houston","author_title":"managing director","affiliation":"IRE and NICAR","summary":"NICAR and IRE have expanded their websites with new searchable databases of investigative journalism resources, including the IRE Resource Center and Uplink abstracts, while also hosting pages for various journalism organizations and international journalist resources.","keywords":["database","journalism","resources"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["PostgreSQL","Internet","World Wide Web"],"full_text":"Services at NICAR and IRE's web site continue to expand with the addition of new ways to search through our on-line resources while we also work to upgrade the design of the site and the information we offer.\n\nYou can now search through the IRE Resource Center (which contains abstracts of thousands of investigative stories) using a free relational database manager called Postgres95. By typing a few key words, you can find citations of articles on organized crime, environment, medical malpractice or just about any subject.\n\nOnce you identify the articles you want, call IRE at (573) 882-3364 to order the full-text or video of the articles. Charges are an inexpensive 10 cents a page and video dubs are $10 per tape, plus postage and handling.\n\nThe address for the resource center is http://www.ire.org/resource/morgue.html\n\nYou also can now search abstracts of Uplink to find tips and resources on how to do computer-assisted reporting. Check it out at http://www.nicar.org/uplink/abstracts.html\n\nYou will soon be able to search our database library, too.\n\nMeanwhile, we are helping other nonprofit journalism groups develop their pages at our site. Among them are the National Association for Black Journalists, the Asian American Journalists Association, and the Education Writers Association.\n\nWe also are publishing pages on our recent Campaign Coverage workshop in Washington, D.C., and on the June IRE-NICAR national conference in Providence, R.I.\n\nYou might also view one of our sites for international journalists that contains resources for U.S. and Russian journalists reporting on Russia. That address is http://www.reporter.org/hillman/russian\n\nThe hillman in the address refers to one of our of more generous donors, Greg Hillman, who, with the Freedom Forum, has made this web site possible.\n\nIn the coming months, we will add pages on Mexican issues and resources as NICAR and IRE crank up the Mexico project, which is an effort to help Mexican journalists set up an organization similar to IRE.\n\nIf you have any suggestions, please contact our web master Wallace Winfrey at (573) 884-7321, or send e-mail to wally@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-postgres95-lets-multiple-users-in","slug":"1996-march-postgres95-lets-multiple-users-in","url":"https://dwillis.github.io/uplink/article/1996-march-postgres95-lets-multiple-users-in.html","year":1996,"month":"March","monthNum":3,"headline":"Postgres95 lets multiple users in","kicker":"","author_name":"Wallace Winfrey","author_title":"NICAR staff","affiliation":"NICAR","summary":"NICAR has added Postgres95, a free relational database manager, to its website as a backend SQL server that offers advantages over single-user systems like Access and FoxPro for multi-user environments. Unlike PC-focused databases, Postgres95 allows hundreds or thousands of users to simultaneously access and manipulate datasets, making it ideal for busy websites.","keywords":["database","Postgres95","website"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["PostgreSQL","SQL","FoxPro","Access","Oracle","SyBase","Informix","Unix","World Wide Web"],"full_text":"A lot of exciting things have happened on NICAR's web site in the last few months. But probably the most exciting thing isn't a web page we've put up, although there have been plenty of those. The most exciting thing to hit our web site has been the addition of a free relational database manager called Postgres95 to serve as a backend SQL server.\n\nWhat's the difference between Postgres95 and FoxPro or Access? While there are many similarities - such as the ability to import and export data, store datasets in tables and run SQL queries to ask questions of the data - there are also some big differences.\n\nAccess and FoxPro are geared more toward the PC database user, while Postgres95 is more for a multi-user environment such as the UNIX operating system. Relational Database Management Systems such as Postgres95 (or its commercial cousins Oracle, SyBase and Informix) allow several hundred (or even several thousand) users to work with and manipulate the same datasets at the same time.\n\nThis distinction has some real advantages over single-user systems such as FoxPro, particularly when it comes to interfacing databases to a WWW front-end. The multi-user functionality of an RDBMS is ideal for a busy web site, where several users may need to access the data at once.\n\nIf you'd like to see the power of Postgres95 at work on our web site, check out such things as the IRE Morgue Online, located at http://www.ire.org/resource/morgue.htm)\n\nWe'll be back next month with more news about the latest developments on our web site.\n\nWallace Winfrey can be reached at (573) 884-7321, or send e-mail to wally@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-how-to-get-a-date","slug":"1996-march-how-to-get-a-date","url":"https://dwillis.github.io/uplink/article/1996-march-how-to-get-a-date.html","year":1996,"month":"March","monthNum":3,"headline":"How to get a date","kicker":"Tech tips","author_name":"Andrew Lehren","author_title":"NICAR Staff","affiliation":"NICAR","summary":"This article teaches journalists how to convert improperly formatted date data in FoxPro databases into proper date columns that enable calendar calculations and date-based queries without complex workarounds.","keywords":["database","dates","FoxPro"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["FoxPro"],"full_text":"This is for those whose lives have gotten to the point where they want to get a date out of a database.\n\nA lot of databases include information that looks like a date, but instead is a character. Variations on this theme include separate columns for the month, day and year. Or single fields with 19940131. Or 103195. Or 9931231.\n\nBut database managers don't know these are dates unless we help them. Here is how to help FoxPro without writing contorted queries every time. Once this is done, reporters can run calculations using FoxPro's calendar brain.\n\nMake a date column\n\nThe first step is to create a new column in your table. Go to Database on the menu bar. Pick Setup Then Modify Insert a new column where you want it. Under Name, let's call it Newdate. And, under Type, choose Date. FoxPro automatically picks the width. Click OK. Then agree to make the changes permanent. Close out of the Setup box. Congratulations, you've created the target date column, filled with blank spaces and / marks, waiting to be filled with things like 11/12/94 and 01/10/60.\n\nFill the dates\n\nNow, we will fill Newdate with what we want. For the sake of unoriginality, let's call the oldcolumn with the faux character dates Olddate. Get the command window, and get ready to write. The first step is REPLACE ALL NEWDATE WITH. This is pretty clear. We are getting ready to put stuff into Newdate.\n\nThe next phrase is CTOD( This is FoxPro's function for changing characters to dates. The parenthesis is open at this point because there is more to come. We will close it at the end. For the sake of clarity, add and go the next line, where we will stuff the CTOD() function.\n\nThis part depends on how the Olddate character dates appear. We are about to pull out the month, then the day, then the year. That's how dates appear among the slashes for Americans. This trips up European journalists who are used to the day/month/year syntax.\n\nLet's say Olddate contains 9931023 and 9960914 and such. The month is in the middle, starting in the fourth spot, and filling two places.\n\nFor this, we will use the SUBSTR() function. To pluck them out, type this: SUBSTR(Olddate,4.,2 Now, for the first slash, type + \"/\" + For the day, beginning in the sixth spot, we will use the same function: SUBSTR(Olddate,6,2) Add another slash with \"/\" + And finish with the year, which starts on the second space: SUBSTR(Olddate,2,2). To keep things clear, don't forget you can break the line at any time with Don't forget to close all this with a).\n\nRemember: The first digit indicates the starting position of the substring of numbers you want to pluck out, and the second digit indicates how many places you want to pluck out.\n\nPut this all together and you get:\nReplace all Olddate with ctod(\nsubstr(Olddate,4,2) +\"/\"+;\nsubstr(Olddate,6,2) substr(Olddate,2,2);\n)\n\nHit enter, and FoxPro will do all the work. There are variations. For the three separate month, day, year columns, you would supplant the middle three lines of this command with:\noldmonth + oldday + \"/\" oldyear\nIf you face a year in the first two spaces, you could save a couple key strokes with the LEFT() function, which knows to start pulling things from the left. You just have to tell it how far. For 071496, use this for the middle three lines:\nleft(Olddate,2) + \"/\" +:\nsubstr(Olddate,4,2) + \"p\" ;\nsubstr(Olddate,6,2)\nYou may also see another variation in your data travels, where you get the month and the year in separate columns, but not the day. You can still coax FoxPro to help with calendar calculations. Just punch in a fake date, like 01 or 15. As with so many data problems, there are several solutions. Just remember that, in this case, you really don't know the day. The middle three lines in this example are:\noldmonth + \"/\" \"15\"+\"\";\noldyear ;\nNow, you're set to start getting dates out of FoxPro. Andrew Lehren can be reached at (573) 882-0684, or send e-mail to andy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-miss-something","slug":"1996-march-miss-something","url":"https://dwillis.github.io/uplink/article/1996-march-miss-something.html","year":1996,"month":"March","monthNum":3,"headline":"Miss something?","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR mailing list archives are now available online organized by thread, author, and date in both HTML and plain text formats for easy access and download.","keywords":["archives","mailing","lists"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web","Internet"],"full_text":"Miss something?\n\nCheck out the IRE-L and NICAR-L mailing list archives on our websites at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format, which is useful for downloading a particular month. If you have any comments on how we can make these archives more useful, send e-mail to wally@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-from-the-nicar-library","slug":"1996-march-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-march-from-the-nicar-library.html","year":1996,"month":"March","monthNum":3,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering transportation accidents, financial transactions, crime statistics, and regulatory compliance records for investigative journalism purposes.","keywords":["databases","government","journalism"],"topics":["Data & Databases"],"technologies":[],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Election Commission contributions data, including donations by individuals and political action committees.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1994, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records.\nNational Bridge Inventory System data, includes inspection grades.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1994.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nNational Endowment for the Arts, grants, 1989-1993.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-ire-seeks-training-director","slug":"1996-march-ire-seeks-training-director","url":"https://dwillis.github.io/uplink/article/1996-march-ire-seeks-training-director.html","year":1996,"month":"March","monthNum":3,"headline":"IRE seeks training director","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"Investigative Reporters and Editors is seeking a training director for a new nonprofit educational organization in Mexico focused on journalist training, with the position offering a path to executive leadership within two years.","keywords":["journalism","training","Mexico"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Investigative Reporters and Editors is searching for a training director for the new Instituto Mexicano de Investigacion Periodistica, a not-for-profit educational organization for Mexican journalists funded with a three-year grant from the McCormick Tribune Foundation.\n\nThe training director will report to the Insituto Coordinator with the goal of assuming that top position within two years and will help develop and carry out professional training programs for journalists. Applicants should be Mexican journalists who are fluent in English with an interest in investigative reporting and computer-assisted reporting. Send a letter, resume and references to IRE Executive Director Rosemary Armao, 138 Neff Hall Annex, University of Missouri School of Journalism, Columbia, Mo., 65211. The deadline is May 1.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-e-tv-stations-on-line","slug":"1996-march-e-tv-stations-on-line","url":"https://dwillis.github.io/uplink/article/1996-march-e-tv-stations-on-line.html","year":1996,"month":"March","monthNum":3,"headline":"E-TV: Stations on-line","kicker":"On the Internet","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute for Media Studies","summary":"Television stations are increasingly establishing online presence with rich content including news scripts, investigative reports, editorials, photos, and searchable archives, moving beyond initial public relations-only websites. The article provides examples of innovative e-TV sites and resources for accessing broadcast news content.","keywords":["television","internet","news"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"Television stations have been considered late-bloomers in the use of computers both for reporting and research, as well as in the area of delivering alternative news products. But they are catching up quickly in both areas.\n\nWhen television stations first came to the 'net, it was primarily as a public relations tool. There was information about the news anchors and some of the programs.\n\nNow, many television stations are developing rich and unique deposits of information, reports and graphics, which supplement their nightly newscasts. In increasing areas of the country, researchers interested in how local news organizations have covered certain stories can check not just the print organizations, but the broadcast offerings as well.\n\nFinding E-TV\n\nThe best site for locating both national and international television sites on the Internet is TV Net (http://tvnet.com/TVnet.html). There are links to U.S. and World TV Home Pages.\n\nA national archive of network television news transcripts can be found at Vanderbilt University (http://tvnews.vanderbilt.edu/). Searchable abstracts and summaries of the daily network news programs.\n\nWhat TV offers on-line\n\nScripts of news programs: Didn't catch the name of the doctor mentioned in the package on a new cancer treatment? Get the text from the TelePrompTers and find it:\n\nWISC-TV NEWS, Madison, Wis. http://www.wisctv.com/news.htm News scripts from the 6 p.m. news program for the past week.\n\nKLAS-TV, Las Vegas, Nev. http://www.infi.net/vegas/KLAS-TV/news.html Scripts from all the various daily newscasts for the past week and a very informative guide on how to read TV scripts.\n\nInvestigative Packages: The text and, sometimes, frames from the video from large investigative stories are being posted on some E-TV sites.\n\nWDIV-TV, Detroit, Mich. http://www.wdiv.com/special.html Investigative reporter Mike Wendland is providing the text of the stories he's investigated and the background data used in reporting the story. The explanations of how the data were found and analyzed are great lessons in computer-assisted reporting. Wendland has just completed a book on Internet research for broadcast journalists.\n\nEditorials:\nKCBS-TV, Los Angeles, Calif. http://www.kcbs2.com/ Weekly editorials are posted (with a box for sending comments attached at the end) and there is an archive of the editorials for the past eight months.\n\nPhotos: Some sites are making interesting use of photos:\n\nWHNT-TV, Huntsville, Ala. http://www.whnt19.com/weather/towercam.html Hourly photos of the weather in the hill country of Huntsville from the Tower Cam (and it will show radar system images in severe weather). There is also a nice link of other sites with camera shots available.\n\nArchives of news stories:\nKPIX-TV, San Francisco, Calif. http://www.kpix.com:80/news-archives/ News databases, accessible by sophisticated search engines, are new developments for most television stations. This one lets you search the text of news stories.\n\nThis column is taken from \"Computer Assisted Research: A guide to tapping on-line information,\" third edition. If you would like a copy of the guide, contact the Poynter Institute for Media Studies: 801 3rd St. S., St. Petersburg, Fla. 33701, or call (813) 821-9494. There is a paper version: $5 for the pages, $10 for the pages and a three ring binder, and there is hypertext version on disc for $5 (indicate Mac or PC).\n\nNora Paul can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-check-out-the-following-urls-for-newspapers","slug":"1996-march-check-out-the-following-urls-for-newspapers","url":"https://dwillis.github.io/uplink/article/1996-march-check-out-the-following-urls-for-newspapers.html","year":1996,"month":"March","monthNum":3,"headline":"Check out the following urls for newspapers","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article highlights publicly available newspaper databases on websites, including school databases from various news organizations, and invites readers to share additional examples via email to Russell Clemings.","keywords":["databases","newspapers","schools"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"Check out the following urls for newspapers that have made databases available to the public on a web site: http://bors.nando.net/ schoolstreets/ lookupSchools9495.html http:// data.pilotonline.com/ http:// www.phillynews.com/ cgi-bin/njschools If you know of others, send them via e-mail, to Russell Clemings at clemings@CRIS.COM","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-bits-bytes-and-barks","slug":"1996-march-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1996-march-bits-bytes-and-barks.html","year":1996,"month":"March","monthNum":3,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Paul Adrian joins WBNS-TV as an investigative reporter using computer skills, while Writer-L listserv relocates to nicar.org for feature and literary journalism discussions.","keywords":["investigative","journalism","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","CompuServe"],"full_text":"Bits, Bytes and Barks\n\nCAR training lands investigative job\n\nPaul Adrian has been hired as a full-time investigative reporter at WBNS-TV in Columbus, Ohio, a CBS affiliate. He starts April 1. \"I think my computer skills caught their attention,\" Adrian said. The station is investing $10,000 in computer equipment for the investigative team, Adrian said. He will be working with a full-time investigative producer on longer term projects.\n\nAdrian, who has a bachelor's degree in journalism from the University of Texas at Austin, leaves WAVE-TV in Louisville, Ky., where he was a general assignment reporter for three years.\n\nWriters discussion group\n\nWriter-L, a listserv tailored for the discussion of feature writing, explanatory journalism, and literary journalism, has found a new electronic home at nicar.org. Topics include writing techniques, markets, jobs, privacy law, criticisms of stories and media, and other related issues. The list is moderated by Jon Franklin, coordinator of the University of Oregon's creative nonfiction program.\n\nSubscribers are required to submit short professional bios that are made available to the membership. The purpose of the bio is not to exclude anybody but to know who the subscribers are.\n\nTo subscribe, send the following information to jonfrank@nicar.org. Follow the format carefully, including the X's and skipping the lines between paragraphs. Limit the bio to five, 65-character lines. Your completed subscription message will look something like:\n\nsubscribe WRITER-L joeblow@pdq.com\n\nXXXBlow, JoeXXX XXXjoeblow@pdq.comXXX\n\nXXXReporter/free-lancer for 30 years. 4 books, most recent WHO AMI AND WHY DO CARE? Credits in NYT mag, Vogue. Specialize in weird stuff. BA Syracuse; MA Podunk U.XXX\n\nFranklin asks for a voluntary donation of $5 for students and $20 for others. Make checks payable to Jon Franklin and send to: P.O. Box 929, Philomath, Ore., 97370.\n\nCheck out IRE conference on the Web\n\nPlanning is underway for the June 13-16 IRE/NICAR National Conference in Providence. Check out the Providence Journal's National Conference Homepage at: http://www.projo.com/ire/. You'll find dates, prices, hotel numbers, information on the shape of the program so far and fun things to do in New England.\n\nKeep up with NICAR on-line\n\nSubscribe to our listserv and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURI.EDU. In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-l your name.\n\nIRE/NICAR is also accessible through CompuServe's Journalism Forum. Go to the JForum, Section 19. Also look into the IRE/NICAR files in Library 19.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-car-training-lands-investigative-job","slug":"1996-march-car-training-lands-investigative-job","url":"https://dwillis.github.io/uplink/article/1996-march-car-training-lands-investigative-job.html","year":1996,"month":"March","monthNum":3,"headline":"CAR training lands investigative job","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Paul Adrian joins WBNS-TV in Columbus as a full-time investigative reporter, starting April 1, with his computer skills and technical expertise helping to secure the position. The station is investing $10,000 in computer equipment for the investigative team, where Adrian will collaborate with a full-time producer on in-depth projects.","keywords":["investigative","reporter","Columbus"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Paul Adrian has been hired as a full-time investigative reporter at WBNS-TV in Columbus, Ohio, a CBS affiliate. He starts April 1. \"I think my computer skills caught their attention,\" Adrian said. The station is investing $10,000 in computer equipment for the investigative team, Adrian said. He will be working with a full-time investigative producer on longer term projects.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-writers-discussion-group","slug":"1996-march-writers-discussion-group","url":"https://dwillis.github.io/uplink/article/1996-march-writers-discussion-group.html","year":1996,"month":"March","monthNum":3,"headline":"Writers discussion group","kicker":"","author_name":"","author_title":"","affiliation":"University of Oregon","summary":"Writer-L, a moderated listserv for feature and literary journalists, has relocated to nicar.org and requires subscribers to submit professional bios to join the community. The list is coordinated by Jon Franklin and requests voluntary donations to support its operation.","keywords":["journalism","listserv","writing"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"Writer-L, a listserv tailored for the discussion of feature writing, explanatory journalism, and literary journalism, has found a new electronic home at nicar.org. Topics include writing techniques, markets, jobs, privacy law, criticisms of stories and media, and other related issues. The list is moderated by Jon Franklin, coordinator of the University of Oregon's creative nonfiction program.\n\nSubscribers are required to submit short professional bios that are made available to the membership. The purpose of the bio is not to exclude anybody but to know who the subscribers are.\n\nTo subscribe, send the following information to jonfrank@nicar.org. Follow the format carefully, including the X's and skipping the lines between paragraphs. Limit the bio to five, 65-character lines. Your completed subscription message will look something like:\n\nsubscribe WRITER-L joeblow@pdq.com\n\nXXXBlow, JoeXXX XXXjoeblow@pdq.comXXX\n\nXXXReporter/free-lancer for 30 years. 4 books, most recent WHO AMI AND WHY DO CARE? Credits in NYT mag, Vogue. Specialize in weird stuff. BA Syracuse; MA Podunk U.XXX\n\nFranklin asks for a voluntary donation of $5 for students and $20 for others. Make checks payable to Jon Franklin and send to: P.O. Box 929, Philomath, Ore., 97370.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-check-out-ire-conference-on-the-web","slug":"1996-march-check-out-ire-conference-on-the-web","url":"https://dwillis.github.io/uplink/article/1996-march-check-out-ire-conference-on-the-web.html","year":1996,"month":"March","monthNum":3,"headline":"Check out IRE conference on the Web","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The IRE/NICAR National Conference is scheduled for June 13-16 in Providence, with planning underway and details available on the Providence Journal's dedicated homepage including dates, pricing, hotel information, and program updates.","keywords":["conference","Providence","journalism"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"Planning is underway for the June 13-16 IRE/NICAR National Conference in Providence. Check out the Providence Journal's National Conference Homepage at: http://www.projo.com/ire/. You'll find dates, prices, hotel numbers, information on the shape of the program so far and fun things to do in New England.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-march-keep-up-with-nicar-on-line","slug":"1996-march-keep-up-with-nicar-on-line","url":"https://dwillis.github.io/uplink/article/1996-march-keep-up-with-nicar-on-line.html","year":1996,"month":"March","monthNum":3,"headline":"Keep up with NICAR on-line","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Instructions for journalists to subscribe to NICAR-L and IRE-L listservs for professional discussion and networking, with additional information about accessing IRE/NICAR resources through CompuServe's Journalism Forum.","keywords":["listserv","journalists","networking"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","Listserv","CompuServe"],"full_text":"Subscribe to our listserv and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURI.EDU. In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-l your name.\n\nIRE/NICAR is also accessible through CompuServe's Journalism Forum. Go to the JForum, Section 19. Also look into the IRE/NICAR files in Library 19.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-uplink-update","slug":"1996-april-uplink-update","url":"https://dwillis.github.io/uplink/article/1996-april-uplink-update.html","year":1996,"month":"April","monthNum":4,"headline":"Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Orange County Register won the Pulitzer Prize for investigative reporting on fertility fraud, while Raleigh News & Observer reporters won for public service journalism examining health risks in the hog industry. The next issue of Uplink will feature these and other Pulitzer Prize winners and finalists who used computer-assisted reporting.","keywords":["Pulitzer","journalism","reporting"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Congratulations to the Orange County Register staff, who won the Pulitzer Prize for investigative reporting for \"Fertility Fraud,\" and to Raleigh News & Observer reporters Pat Stith, Joby Warrick and Melanie Sill, who won the Pulitzer Prize for public service for \"The Price of Pork,\" a look at the health risks of waste disposal in the hog industry. The next issue of Uplink will feature these and other contest winners and finalists who used computer-assisted reporting.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-stats-add-muscle-to-computer-assisted-reporting","slug":"1996-april-stats-add-muscle-to-computer-assisted-reporting","url":"https://dwillis.github.io/uplink/article/1996-april-stats-add-muscle-to-computer-assisted-reporting.html","year":1996,"month":"April","monthNum":4,"headline":"Stats add muscle to computer-assisted reporting","kicker":"Feel the power","author_name":"Stan Dorsey","author_title":"NICAR Staff","affiliation":"NICAR","summary":"This article provides a comprehensive overview of how statistical techniques like linear regression and logistic regression can be applied to journalistic investigations.","keywords":["statistics","linear model","regression"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["FoxPro","Paradox","Access","Excel"],"full_text":"In the fast-moving world of computer-assisted reporting, journalists each day defy the widely held notion that reporters and numbers don't mix. They break down databases and computer record layouts with virtual ease, using programs such as FoxPro, Paradox, Access and Excel. Just when it seems like they've done it all, something new comes along something like statistics. The use of statistics for research and analysis in the social sciences and in most medical fields has been going on for years. But it is a tool that is relatively unfamiliar to journalists, even those who are savvy in the general applications of CAR. That does not have to be the case, says Philip Meyer, author of \"The New Precision Journalism\" and William Rand Kenan, Jr., professor of journalism at the University of North Carolina at Chapel Hill. The power of statistical analysis can be tapped by any journalist who is willing to learn, Meyer says. Both Meyer and John Bare, a media consultant who works closely with Meyer and specializes in the use of statistics, agree that the first step in interpreting statistical analyses is to understand the basic model on which it based - the linear model. \"Understanding the linear model,\" Bare says, \"is the key to understanding other tools like linear regression or logistic regression. You should always know it first.\" The linear model, Meyer says, \"is based on the idea that many relationships in life fit a straight line, even when they don't seem to at first.\" One way to conceptualize the linear model is to think back to the old days of high school or college algebra. On a basic graph with a y axis and an X axis, the linear model posits that as the X, or independent variable, changes, and so changes the y ,or dependent variable. In other words, the amount that y increases or decreases is directly dependent on changes in the X variable. Consider the real life application offered by Meyer, which deals with state SAT score averages. \"The scores from each state would be directly dependent upon how many students took the test. The fewer students who take it, generally that means a higher score, and vice versa,\" Meyer says. The degree to which a comparison fits a straight line is referred to as a linear correlation. So, for example, there is a strong linear correlation between the temperature outside and how much you sweat. Linear correlation leads to the discussion of the first and simplest type of statistical analysis journalists use. It is called linear regression and follows the exact concept explained by the linear model. Regression refers to the relationship between two or more correlated variables. A more complicated application of statistics is called logistic regression. The main difference between linear and logistic regression is in the nature of the X and y (independent and dependent) variables. Meyer says that a good use of linear regression is for dealing with economics. It allows you to make a prediction about something based on information you are sure about. For example, it is known that when the interest increases, the demand for housing decreases. Based on this linear correlation, we should be able to predict a rise or fall in the demand for housing by simply looking for a trend in interest rates. In linear regression the two variables are continuous, meaning that there is a great range in their potential value. In the example of the SAT scores, the dependent variable can range from 400 to 1600, and the independent variable can be any number of students. In logistic regression, however, at least one of the variables involved is referred to as a categorical or binary variable. In stark contrast to continuous variables, categorical variables have dichotomous values such as male/female, yes/no or dead/alive. Logistic regression still involves finding a relationship between two or more factors, but it allows you to do it in a far more powerful, meaningful way. \"The beauty of logistic regression,\" Meyer says, \"is that you are able to take several independent variables and use them to estimate the odds that your dependent variable will be one or zero.\" Using the different independent variables, Meyer says, enables you to have control over certain variables that you ordinarily may not. A case where this is most evident is in medical reporting. Very often reporters are interested in investigating mysterious death rates (death is a categorical variable) at a particular medical institution. Using logistic regression allows them to control for certain independent factors such as the patient's age, level of income, health history or insurance coverage. Controlling for these factors in a comparison of death rates has the effect of leveling the playing field for all the institutions being examined. This is precisely what New York Times reporter Josh Barbanel attempted to do in his investigation of the unusually high rate of newborn deaths at certain New York City hospitals. Multiple regression is very similar to logistic regression with the exception that you are able to include more than one dependent variable. Many education beat reporters have found this technique particularly valuable, Meyer says, when attempting to measure the efficiency of particular schools in a district by using graduate rates and test scores of students. In these studies, variables such as the amount of money per pupil and the level of income per family can be controlled for. Dave Davis of The Plain Dealer+ found this out when he embarked upon a study of the test scores for 600 Ohio public schools. Davis and three other education reporters controlled for variables such as poverty, parents' education level, and the percentage of renters in a district as a measure of how often families move. Their study found that these factors these account for nearly 60 percent of the difference in test scores between districts. But more importantly, they were able to target schools that were not performing up to the standards which Davis' demographic analysis predicted they should. Some schools, on the other hand were performing well above predicted levels, despite difficult social circumstances. A final tool, which Meyer says he finds very useful in making his stories relevant to their readers, is reliability analysis. It is based on the principle of intercorrelation, which measures the degree to which several different items measure the same thing. This is particularly helpful in doing surveys. \"Reliability analysis allows you to take several different survey questions and find out how related they are to each other,\" he says. \"Once you do that, you can compute a single number, which expresses your outcome.\" By expressing complete survey results with only one figure, reporters can easily compare results from different segments of the population without drowning the reader in numbers. Regardless of which method reporters decide is best for their story, the use of statistics is not as scary as it may sound. But Davis is quick to note that it is only part of the process. \"It's great for getting numbers crunched,\" he says, \"but you still have to put everything in a relevant context for your readers.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-april-sas-vs-spss","slug":"1996-april-sas-vs-spss","url":"https://dwillis.github.io/uplink/article/1996-april-sas-vs-spss.html","year":1996,"month":"April","monthNum":4,"headline":"SAS vs. SPSS","kicker":"Playing with the big guns","author_name":"Steve Doig","author_title":"","affiliation":"The Miami Herald","summary":"The article recommends that computer-assisted reporting specialists learn statistical analysis software like SAS or SPSS despite their high cost and steep learning curve, due to their powerful capabilities for sophisticated data analysis beyond what Excel and basic programs can offer.","keywords":["statistics","software","analysis"],"topics":["Data & Databases","Technology","Investigative Methods"],"technologies":["SAS","SPSS","Excel","FoxPro","SQL"],"full_text":"If you can touch-type your way around a spreadsheet, and you speak fluent SQL to your database program, it's time to add a new power tool to your CARpentry workbench: One of the major statistical analysis software packages, either SAS or SPSS.\n\nHere's where I'd like to tell you that, despite what you've heard, SAS and/or SPSS are cheap and easy to master. I'd like to, but I'd be telling a lie.\n\nActually, they're damned expensive, and they can do things you'll have trouble pronouncing (say \"partial canonical correlation analysis\" three times), much less readily understand how to use.\n\nSoftware with strength\n\nSo why would I recommend that CAR specialists who already are happy with programs like Excel and FoxPro begin learning such powerful software? For that very reason the power. As you might expect, both SAS and SPSS are loaded with a huge array of functions for doing sophisticated statistical analyses. With them, you can do just about anything from simple crosstabs and frequency tables up through multiple linear regression and on to such exotica as cluster, fac... you'll be glad you have either one.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-suggested-books-on-statistics","slug":"1996-april-suggested-books-on-statistics","url":"https://dwillis.github.io/uplink/article/1996-april-suggested-books-on-statistics.html","year":1996,"month":"April","monthNum":4,"headline":"Suggested books on statistics","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Philip Meyer recommends essential statistics and data analysis books for journalists, and provides instructions for subscribing to online discussion forums for SAS and SPSS software users.","keywords":["statistics","software","data"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":["SPSS","SAS"],"full_text":"Here are several books suggested by Philip Meyer: \"SPSS Guide to Data Analysis\" by Marija J. Norusis, whom Meyer says is currently the best writer on statistics; \"Data Analysis for Politics and Policy\" by Edward Tufte; \"Primer of Statistics for Political Scientists\" by V.O. Key Jr.\n\nFor a lively discussion on the use of SAS software, subscribe to SAS-L by sending e-mail to listserv@uga.cc.uga.edu Your message should read: subscribe SAS-L yourname. To tune into an SPSS discussion, subscribe to SPSSX-L by sending e-mail to listserv@uga.cc.uga.edu Your message should read: subscribe SPSSX-L yourname.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-it-ought-to-be-a-crime","slug":"1996-april-it-ought-to-be-a-crime","url":"https://dwillis.github.io/uplink/article/1996-april-it-ought-to-be-a-crime.html","year":1996,"month":"April","monthNum":4,"headline":"It ought to be a crime","kicker":"Looking at police stats","author_name":"Bill Bell","author_title":"","affiliation":"Missouri School of Journalism","summary":"Jeff Green of The Oakland Press built his own crime database by collecting data directly from 60 local police departments in Oakland County, finding it cleaner and more accurate than FBI statistics. He used the data to analyze the correlation between commercial growth and crime while emphasizing the importance of statistical significance in reporting.","keywords":["database","crime","statistics"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["Excel","FoxPro","PC"],"full_text":"Last Christmas, Jeff Green of The Oakland Press in Pontiac, Mich., learned an important lesson: Do it yourself. In the Christmas to New Year's lull, Green wrote a series of stories on growth and crime with fellow reporter Kathy Gray. Green got the jump on his competitors by circumventing the FBI and going straight to his local police departments.\n\nOver the course of several weeks, Green asked about 60 law enforcement agencies in Oakland County for their tallies in eight crime categories: murder, rape, robbery, assault, burglary, larceny, auto theft and arson. Green pooled his numbers with reporters from a local television station.\n\nUsing his IBM laptop, Green put together his own database on Excel 5.0 and Fox Pro 2.6. He estimated it took seven hours to clean up and track down errors in the data. Green found that many data discrepancies stemmed from the structure of the police department.\n\nOverall, Green said, his data was cleaner and more accurate than the crime statistics from the FBI. \"It's better than the FBI database because the FBI has blanks and stuff,\" Green said. \"If you are going to start comparing Community A to Community B, you better go to Community A and Community B.\"\n\nGreen also researched the correlation between commercial growth and crime. He used data on property tax values from the Oakland County Planning Department.\n\nGreen, a science, demographics and environment reporter with a background in statistics, said he is planning to follow up the series by seeing if the number of police officers is keeping up with community growth. Research last year showed the fire department increasingly understaffed.\n\nGreen advised reporters to look at the statistical significance of the data. Reporters are notorious for writing stories about startling numbers without looking at the whole statistical picture, he said. Small communities may experience a large percentage increase in crime but still have relatively low crime rates.\n\n\"A real pitfall in these type of stories is paying too much attention to the change and not to the actual numbers,\" Green said. \"A lot of places with the fastest growing crime rates were also still the safest communities.\"\n\nJeff Green can be reached at (810) 745-4647, or send e-mail to user@cris.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-economic-analysis-on-the-fly","slug":"1996-april-economic-analysis-on-the-fly","url":"https://dwillis.github.io/uplink/article/1996-april-economic-analysis-on-the-fly.html","year":1996,"month":"April","monthNum":4,"headline":"Economic analysis on the fly","kicker":"A look at widgetmaking","author_name":"Neill A. Borowski","author_title":"reporter","affiliation":"The Philadelphia Inquirer","summary":"Neill Borowski explains how to use 'location quotients' to measure industry concentration in local economies compared to national averages.","keywords":["location quotient","economic analysis","employment data"],"topics":["Data & Databases","Business & Economy","Investigative Methods"],"technologies":["spreadsheet","County Business Patterns"],"full_text":"When it comes to local economies, percep- tions may not be realities. Community leaders may THINK their re- gion is a leader in computer manufacturing, or financial services or other industries. But we don't really know this simply by looking at employment totals. Enter the \"location quo- tient\" û a well-worn tool in regional economic analysis. We recently used this statistic to see just how important computer industries are to the Philadelphia area. Location quotients are fairly simple to com- pute. They show the concentration of employ- ment in a region compared to the United States. The quotients, which are measures of con- centration, also can be used with total payroll dollars (from County Business Patterns) rather than employment. Or they can be used to calcu- late the concentration of one population (race, etc.) compared with the nation (or other areas). Let's look at widgetmaking. The quotient calculation first requires calculating what per- centage widget employment is in this metro and the percentage widgetmaking is nationally. If there are 10,000 widget workers and 100,000 workers in a metro, the percent is 10 percent. If there are 5 million widget workers nationally and the total workforce is 100 mil- lion, 5 percent of the national workforce is in widgetmaking. Just by looking at the data (10 percent vs. 5 percent) you can see widget em- ployment is twice as concentrated in the metro than in the nation as a whole. You convey this with another quick calcula- tion: (10 percent/5 percent)*100 = 200. This is the location quotient, or index, of the metro, with the U.S. average = 100. With a spreadsheet or database manager you can quickly compute the quotients for all sec- tors in your local economy. If your quotients are around 100 (cut some slack for data reporting error), the concentration is equal to the national concentration. If your quotient is, say, 50, then your concentration is half that of the nation. We ran a bar chart with the Philadelphia metro vs. San Jose in several computing and other industries. A broken line through the charts marked U.S.=100. Why use index numbers when you could run a table showing 10 percent and 5 percent? For widgetmaking, you could. But what about ce- ramic-plated widgetmaking? The numbers are 0.0043 percent vs. 0.0021 percent. Still twice as concentrated, but a lot more difficult to under- stand in percentages (the index would be 205). In general, the percentages will be relatively small. By the way, round the index numbers; don't use decimals, which serve to confuse. Of course, there are assumptions and prob- lems with this. I used only non-agricultural businesses. Government employment wasn't a factor. If the metro is small enough, it could have a high concentration (index of 200 or more) but actually have only a few thousand employees in that industry. Or, the employment might be all at one company. (However, the County Business Patterns will suppress employment and payroll data if there are only one or two employers in a given industry.) Some purists might also say you have to subtract your metro from the nation before making the national computations. You don't have to use County Business Pat- terns. Your regional Bureau of Labor Statistics office can supply establishment employment (series 790) statistics. Or you could use data from the Bureau of Economic Analysis (U.S. Commerce Department). I chose the Patterns because they are seen as a pretty accurate mea- sure. Another measure is the series of economic censuses from the U.S. Census Bureau (they're done every five years in years ending in \"2\" and \"7\"; the data for 1992 came out last fall). Some possibilities: If agriculture is big in your area, use the Census Bureau's Census of Agriculture to com- pute location quotients (dairy revenues in your metro vs. other leading dairy metros). If the federal government is talking of more cuts, determine how concentrated federal em- ployment is in your metro. Is it much more concentrated than the nation as a whole? Is upscale retail hot in your area? Use the Census of Retail Trade to compute the relevant location quotients. For those fortunate enough to have motor vehicle databases, is there a concentration of luxury cars in your county vs. other counties?","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-april-statistics-resources-online","slug":"1996-april-statistics-resources-online","url":"https://dwillis.github.io/uplink/article/1996-april-statistics-resources-online.html","year":1996,"month":"April","monthNum":4,"headline":"Statistics resources online","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Carnegie Mellon University's Statistics Department offers reporters a comprehensive resource for learning data analysis through real-world examples, including sports statistics, public health, and environmental data.","keywords":["statistics","data","reporting"],"topics":["Data & Databases","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"Practice your skills on the sport that loves statistics: baseball. You can look at baseball player's salaries and performance figures. Or, if you prefer, see an example of a simple regression on the taste of cheddar cheese, and look at how to use a scatterplot to probe the history of Olympic winners. That's just one of the offerings from Carnegie Mellon University's Statistics Department. A great resource for reporters just starting is the Data and Story Library, a collection of real-life ways that experts crunched data. There are serious subjects, too from breast cancer to crime to air pollution - and examples about how to investigate them with stats. Start at http://lib.stat.cmu.edu/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-exposing-little-white-lies","slug":"1996-april-exposing-little-white-lies","url":"https://dwillis.github.io/uplink/article/1996-april-exposing-little-white-lies.html","year":1996,"month":"April","monthNum":4,"headline":"Exposing 'Little White Lies'","kicker":"CAR trips: First adventures","author_name":"Aaron Elstein","author_title":"","affiliation":"Illinois Times","summary":"Aaron Elstein describes using FoxPro to analyze Illinois state workforce reports, uncovering racial disparities in government hiring and management.","keywords":["affirmative action","state agencies","FoxPro"],"topics":["Government & Politics","Demographics & Census","Investigative Methods"],"technologies":["FoxPro"],"full_text":"Every state agency in Illinois is supposed to file a \"work-force report\" with the secretary of state's office by Jan. 1. The reports detail the past year's hirings and promotions and catego- rizes employees by race, gender and salary level. I had written about the issue a year earlier, but this year, I had the benefit of FoxPro. There are almost 100 state agencies, com- missions and boards, and 65 of them had filed their reports. The records were not available electronically, SO I typed in the information I wanted into about 20 different fields. I wanted to look at percentages of minorities in every agency, and at how salaries for minorities compared from agency to agency and track hiring and promotion trends. It took about three days to enter this using my own Mac laptop for the project. Affirmative action is a hot issue, and state employment is always a lively topic around Spring- field, where one-third of the city's population works for the government. There's a law on the books saying each state agency's work force should reflect the state's population as a whole (which is 25 percent nonwhite), but the groups who represent minority employees believe the state has been lax in enforcing this. My data analysis revealed that, in some ways, the state has done pretty well hiring minorities and women: 23 percent of the work force is minority, and Illinois Gov. Jim Edgar has ap- pointed some women and minorities as cabinet directors. Despite that, the number of minorities work- ing for the government has actually declined since his administration started five years ago. And right away, the analysis showed just how few minorities can get a job outside the urban social service agencies that have traditionally provided them with patronage work in welfare offices and mental hospitals. Outside a few well-paid directors, few mi- norities could be found in management-level jobs in any state agency. Blacks actually repre- sented a greater percentage of state employees than they did in the overall state population. But Hispanics, who now comprise at least 8 percent of Illinois' population, have yet to get state government jobs.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-april-riding-the-a-train","slug":"1996-april-riding-the-a-train","url":"https://dwillis.github.io/uplink/article/1996-april-riding-the-a-train.html","year":1996,"month":"April","monthNum":4,"headline":"Riding the A-train","kicker":"CAR trips: First adventures","author_name":"Jim Morris","author_title":"","affiliation":"CNN","summary":"Jim Morris recounts his experience utilizing Federal Railroad Administration safety databases to identify hazardous railroad crossings.","keywords":["railroad safety","FRA","data cleaning"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["FTP","Internet"],"full_text":"Like a bolt of lightning, it struck. It was my first day at NICAR Bootcamp, and I had just had a revelation of Biblical proportions: I really could use this stuff. I was working on a railroad safety story, and I was anxious to use my new skills on my return to Atlanta. I got on the phone with the systems operator at the Federal Railroad Ad- ministration. I knew the FRA maintained sev- eral databases on train safety, and I wanted to know how I could access the information. I was expecting a bureaucratic nightmare, but what I found was computer nirvana. The FRA maintains a page on the Department of Transportation's website. The page includes ftp versions of all of its safety databases for 1990-95. First, I downloaded the file layout and code sheets. Then Idownloaded the databases. When I opened the first file, voila, nothing. Something was wrong with the layout. I began experimenting with the layout values and quickly discovered that things didn't add up. My record lengths were all wrong. The layout sheet listed fields that just didn't exist! After much fussing, I finally got it all worked out. Until I checked the codesheet. Ahh, the codesheet. Not even close to reality. I got back on the horn with the FRA and had them fax me a copy of the incident/accident report sheet from which the data is entered. Having the report form saved me from near insanity. By now, my eyes were burning, my fingers cramped. But I finally was ready to ask some questions. I wanted to know where the most deadly railroad crossing in the United States was. So I asked the computer to count the number of accidents at each highway crossing. That was a mistake. As I've since realized (I feel stupid admitting I didn't catch this right away), high- ways often cross railroad tracks more than once. Then I noticed the column marked \"GXID,\" or Grade Crossing ID. That was the magic field. Within seconds, I had the correct answer. IfI' ve learned anything from this, it is DON'T GIVE UP. Whenever I got stumped, I'd take a break, get some air. If I was still stumped, I'd call NICAR or the FRA and ask for help. I even believe this experience changed my life. As my wife will attest, I'm no longer afraid to ask for directions or admit I'm lost while driving my car. That's progress.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-april-two-years-at-pentium-speed","slug":"1996-april-two-years-at-pentium-speed","url":"https://dwillis.github.io/uplink/article/1996-april-two-years-at-pentium-speed.html","year":1996,"month":"April","monthNum":4,"headline":"Two years at Pentium speed","kicker":"We've come a long way, Baby","author_name":"Brant Houston","author_title":"Managing Director","affiliation":"National Institute for Computer-Assisted Reporting","summary":"The National Institute for Computer-Assisted Reporting reviews its accomplishments over two years, including expanded staff, training of 5,000 journalists through 125 seminars, and development of online resources and database libraries for news organizations. The institute has established itself as a leader in promoting computer-assisted reporting through partnerships with universities and professional organizations.","keywords":["training","database","journalists"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"We're taking a moment here at the National Institute for Computer-Assisted Reporting to hit the break key and review what we've accomplished in the past two years with the help of our staff and volunteers, the funding from foundations and the support from the farsighted and adventurous throughout the news profession.\n\nBegun in 1989 by the Missouri School of Journalism, the institute was revitalized in 1994 after the school and Investigative Reporters & Editors joined forces to promote computer-assisted reporting (CAR) through expanded training and services.\n\nWhen I arrived in February 1994, there were two graduate assistants at NICAR. Since then, our full-time and part-time staff has increased to a dozen. We have a training director, several graduate assistants, a World Wide Web site administrator and web designer, an assistant systems administrator, and an office coordinator. We also share support staff with IRE, of which NICAR is an administrative division, including a bookkeeper, office manager and seminar coordinator.\n\nWith this staff, we have conducted more than 125 seminars throughout the United States for more than 5,000 print and broadcast journalists during a two-year period. With funding from the Freedom Forum and Knight Foundation, we have initiated a minority training program.\n\nOur database library has increased from a handful of offerings to more than 20 different databases that we have distributed to more than 90 news organizations. And we have done data work for organizations ranging from small newspapers to national TV magazines.\n\nOn the paper publishing side, this newsletter's circulation has more than quadrupled from 80 subscriptions to more than 400. We also have published a book on CAR stories, \"100 Computer-Assisted Stories,\" that shows how such stories are done, and IRE and NICAR supported my work on a basic primer on CAR, \"Computer-Assisted Reporting: A Practical Guide,\" which is published by St. Martin's Press.\n\nWhile all this activity was going on, we have developed online offerings for the profession with the financial help of IRE supporter Greg Hillman and the Freedom Forum. You can go to our web site at http://www.reporter.org and find your way not only to NICAR's and IRE's web pages (where you can obtain sample data and do research on investigative reporting), but also to web pages for minority journalists' organizations. You can also join our listservs to discuss CAR and investigative reporting. In the next few months, we hope to unveil new web pages that will provide even more help to journalists. Most important, we have stuck to our goal of working with other groups to form alliances to do better work. In the past two years, we have offered seminars with Northwestern University, New York University, the University of North Carolina, the University of Maryland and Indiana University. We worked, too, with many other societies including the Society for Professional Journalists, the Society of Environmental Journalists, and the Unity Conference. We hope to raise our work to new and higher levels in the coming months with your help and the continued financial support of the profession and its foundations. If you have suggestions for the future or observations about the present, please send them to brant@nicar.org And thanks again from NICAR for a terrific two years.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-new-books-for-online-journalism","slug":"1996-april-new-books-for-online-journalism","url":"https://dwillis.github.io/uplink/article/1996-april-new-books-for-online-journalism.html","year":1996,"month":"April","monthNum":4,"headline":"New books for online journalism","kicker":"","author_name":"Brant Houston","author_title":"Managing Director","affiliation":"NICAR","summary":"Brant Houston reviews two new books designed to help broadcast and student journalists navigate the Internet and online resources for reporting.","keywords":["books","online journalism","internet"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"Just in time for NICAR's newest training push computer-assisted reporting for broad- casters - come two books for using online resources for news stories. The first book, \"Wired Journalist\" by long- time broadcast and print reporter Mike Wendland, is aimed at TV and radio journalists. Wendland is a full-time reporter at WDIV-TV in Detroit where he practices CAR on a daily basis. He also teaches at NICAR seminars and knows how to make computer technology easy to understand. His book, published by the Radio and Tele- vision News Directors Foundation, is practical and to the point. He avoids \"geek speak,\" using arelaxed and friendly tone to explain the Internet and how to use it for news stories. It's must read for broadcast journalists just starting to Net surf, and it's a handy reference book for those who have already begun. Also out is Randy Reddick and Elliot King's revision of \"The Online Journalist,\" which is now called \"The Online Student.\" Reddick and King, two university professors, had the misfor- tune to publish \"The Online Journalist\" right before the World Wide Web exploded. Now they are back with a valuable excursion into online resources. More detailed than Wendland's book, this would be a good follow-up compan- ion. For professional journalists, the chapters targeting student concerns might seem unneces- sary, but this still is a good introductory text. Meanwhile, don't forget Nora Paul's excel- lent web book, which you can find at poynter.org, or order the hard copy from the Poynter Institute in St. Petersburg, Fla.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-april-ire-mailing-list-archives-notice","slug":"1996-april-ire-mailing-list-archives-notice","url":"https://dwillis.github.io/uplink/article/1996-april-ire-mailing-list-archives-notice.html","year":1996,"month":"April","monthNum":4,"headline":"IRE mailing list archives notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR provide searchable mailing list archives on their websites, accessible by thread, author, and date in both HTML and plain text formats for easy reference and download.","keywords":["archives","mailing","searchable"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web","HTML"],"full_text":"Miss something? Check out the IRE-L and NICAR-L mailing list archives on our websites at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format, which is useful for downloading a particular month.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-parsing-is-the-answer","slug":"1996-april-parsing-is-the-answer","url":"https://dwillis.github.io/uplink/article/1996-april-parsing-is-the-answer.html","year":1996,"month":"April","monthNum":4,"headline":"Parsing is the answer","kicker":"Tech tips: It may look like a table but it's not","author_name":"Natalya Shulyakovskaya","author_title":"Staff","affiliation":"NICAR","summary":"This article provides a step-by-step guide for downloading and parsing columnar text files from the Census Bureau website into Excel, explaining how to use Excel's Text Import Wizard to convert flat text files into organized tables with proper rows and columns.","keywords":["parsing","Excel","Census"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Excel"],"full_text":"Parsing comes into play when you need to turn a text file into a table with rows and columns. There are plenty of those text files scattered all over the Web with numbers and other information organized into columns, or, once in a while, you get a disk from a bureaucrat with data in report format: The files look like tables because the information is organized into separate columns, but they are flat text files. They may have a .txt extension, the extension .dat or some other creative three-letter extension - or no extension at all. You can open these files with a text editor, but no database manager or a spread sheet will recognize them as tables. At this moment, you know it's time to parse. One word about downloading files using Netscape: If you are saving a table-looking file from the Web, make sure to save it as a text file and not as an html file. You don't want to waste time fishing out html coding from the document; let the browser do it for you. If you are not using Netscape, you may have to strip out the html coding using an html browswer or text editor before proceding.\n\nStraight to the business\n\nLets download and parse one of the tables from the Census Bureau Web site ( http:// www.census.gov/) The easiest tool to use for parsing is Excel, but remember: Excel can only handle up to 16,000 records in one table. Table 1. Projections of the Population of Voting Age, for States, by Sex, Race, and Selected Ages: November 1996: http:// www.census.gov/ftp/pub/population/ socdemo/voting/proj/votepgl.asc Go to the File pull-down menu, pick Save As and for File Type, specify All files. Save the file, and go to Excel. In Excel, under the File pull-down menu, choose Open. Find your file, making sure to specify the correct file type - it should be text only(??). Excel will bring you to the Text Import Wizard. On Step 1, you have an option of specifying Original Data Type. Here are your choices:\n\nFixed Width: Fields are aligned in columns, with spaces between each column of text or numbers.: Choose Fixed Width (see Variation below). If your file has a lot of text on the top that you don't need in your table, note a small Start Import at Row box. You can browse your file in a window below your text with the numbers of each line. For our Census Projection file, choose Row 8 to start importing. If your file has some junk at the bottom, we will kill it after the parsing by deleting rows. The Wizard adjusts the view in the window. Click on Next. In Step 2, you will slice your file into the columns: Use the mouse and click once to insert a new line; double click to delete a line. In some cases, Excel will put black vertical lines subdividing the columns for you. Often, though, it doesn't guess right. Click and drag the lines within the window. If you see any repeating junk characters, such as vertical bars, you can separate them from the other data with the lines, or you can clean them out later using Excel's search and replace function under the Edit pull-down menu. In Step 3, specify in what format to import the data. Excel usually sets all the data types to General and guesses the data types; however, it doesn't always guess right. We will import all the columns as text. Now, save your table using Save As under the File pull-down menu. Under Save As Type, pick Microsoft Excel Workbook.\n\nDelimited: All elements in the text are separated by special characters, such as quote marks, ampersands, tabs, spaces or commas.\n\nVariation\n\nThis file also is a good example of how you can use the Delimited option. Before, when you opened the file, you specified Fixed Width. To see how the Delimited option works, repeat the process with the following exceptions: During Step 1, examine your file - each column is separated by a series of vertical bars. Let's use them as delimiters. Check the Delimited button. In Step 2 in the Delimiters part of the dialog window, check the Other box and type (vertical bar) in the provided window.\n\nIn Step 3, Excel puts the separating lines in the place of those pesky vertical bars. Let's let Excel do the data format. Never trust it completely, though, especially with dates.\n\nNow, let's give the columns their names just type them into the cells above the appropriate columns of information. If you plan to use your file later in FoxPro, limit the names to eight characters and don't use spaces.\n\nClean-up\n\nLet's get rid of periods in our first column: Highlight the column by clicking on the gray bar above the column, and use the Replace option from the Edit pull-down menu. Put a period into the Find box and leave the Replace With box empty. Click Replace and see the periods dissapear into the ether. Let's save this file as Excel Workbook, too.\n\nWhat we just did with Excel is possible with Visual Foxpro 3.0. In FoxPro 2.6, in order to import a text file, you must build a corresponding table structure and then append a text file to the table as an SDF (standard data-file format), herding the data into correct chutes.\n\nVisual Fox lets you do the parsing without going to Excel for help. Use Import under the File pull-down menu. Choose Import Wizard from the pop-up menu and follow the steps. Visual FoxPro will ask you to name the columns and set up the data format within the Wizard. Natalya Shulyakovskaya can be reached at (573) 882-0684, or send e-mail to natalya@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-thats-stats","slug":"1996-april-thats-stats","url":"https://dwillis.github.io/uplink/article/1996-april-thats-stats.html","year":1996,"month":"April","monthNum":4,"headline":"That's stats","kicker":"On the Internet","author_name":"Andy Lehren","author_title":"NICAR staff","affiliation":"NICAR","summary":"This article provides a comprehensive guide to online resources for learning statistics, including educational websites, courses, software tools, and reference materials designed specifically to help journalists understand and apply statistical concepts in their reporting.","keywords":["statistics","journalism","education"],"topics":["Data & Databases","Journalism & Newsroom","Technology"],"technologies":["SPSS","SAS","SYSTAT","Statistica","KwikStat","Xlisp-Stat","StatView","Internet","World Wide Web"],"full_text":"There's a good chance you can learn a few things about statistics on the Internet. It's just a matter of where you weigh in. A significant amount is at universities. Some of this has a tendency to drive beginners into fits of apoplectic boredom. Don't get mean. Odds are, if your patient, you'll get some good ideas for how statistics can help you're reporting.\n\nLearning about statistics\n\nA good starting point is Chance, designed to help teach statistics. For those who think number crunching has no place in the news, check out the archive of stories that use statistics. There's Stephen Jay Gould's \"The Median Isn't The Message,\" plus a lot more. Visit http://www.geom.umn.edu/docs/snell/chance/welcome.htm!\n\nFor reality checks, visit How to Avoid Lies and Damned Lies at http://maddog.fammed.wisc.edu/pitfalls/\n\nThere are a lot of college statistic classes with some course material on the Internet. Some are actually worthwhile. One of the best is at Arizona State University, with everything from a simple program you can download for free to help learn about probability, to a clear course outline that you can actually learn from. Don't miss its concise links collection, too. It's at http://olam.ed.asu.edu/-glass/502/home.html\n\nAnother good course is from Concordia University at http://artsci-ccwin.concordia.ca/inte/inte298s/courses/index.htm\n\nFrom the land down under, at the University of Newcastle, is http://frey.newcastle.edu.au/Stats/surfstat/noframes surfstat.html\n\nAmerican Demographics does a lively job analyzing numbers. Visit http://www.marketingtools.com/ad_current/default.htm\n\nA thorough dictionary of statistics is Prophet Statguide at http://www-prophet.bbn.com/statguide/sg_glos.htm Scroll through the interactive dictionary, or do keyword searches. A short list written for journalism students is at http://excellent.com.utk.edu/~foley/stat.html\n\nDownload a program that will help you learn exactly how regressions work, and how to run them. Visit Click&Learn Regression at http://nsns.com/click/ From there, you can get a program that will walk you through regression-and even quiz you along the way. The site offers a free demo. The full program is $49.\n\nFor an online book about probability, check out the University of Maryland's Resampling Project at http://www.statistics.com/text.html It offers clear writing about some basics in sampling and statistics.\n\nTooling around\n\nWhen it comes to getting the software to run statistical tests, the Internet harbors everything from freebie programs to how to contact the big software makers. For SPSS and SYSTAT, visit http://www.spss.com For SAS, go to http://www.sas.com/ For Statistica, go to http://www.statsoftinc.com/\n\nDownload and sample Resampling Stats software at http://www.statistics.com/software.html Right nearby, at http://www.statistics.com/resample/vendors.html, is a handy price guide for statistical software.\n\nGet a review copy of KwikStat at http://www.dfw.net/homepage/\n\nFor Xlisp-Stat, download free copies starting at http://stat.umn.edu/~rcode/xlispstat-resources.html#win31 For StatView, try http://www.abacus.com/\n\nThe Journal of Statistics Education Information Service offers a variety of services, including software and the ability to check the archives of SAS and SPSS listservs. It's at http://www2.ncsu.edu/ncsu/pams/stat/info/infopage.html\n\nThe University of Minnesota's School of Statistics offers everything from free software to guides on running statistical tests at http://stat.umn.edu:80/ARCHIVES/archives.html\n\nFor those with time, check out extensive lists of universities and other organizations around the world that offer information on statistics at http://www.stat.cmu.edu/otherplaces/ Or try http://www.geom.umn.edu/docs/snell/chance/sources.html\n\nAndy Lehren can be reached at (573) 882-0684, or send e-mail to andy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-statistics-web-resources-sidebar","slug":"1996-april-statistics-web-resources-sidebar","url":"https://dwillis.github.io/uplink/article/1996-april-statistics-web-resources-sidebar.html","year":1996,"month":"April","monthNum":4,"headline":"Statistics web resources sidebar","kicker":"","author_name":"Andy Lehren","author_title":"staff","affiliation":"NICAR","summary":"A guide to resources for calculating and understanding inflation rates, including the U.S. Bureau of Labor Statistics tools, comparative inflation analysis methods, and salary and housing cost calculators across different regions.","keywords":["inflation","calculator","resources"],"topics":["Data & Databases","Technology"],"technologies":["Excel","Internet","World Wide Web"],"full_text":"Want to use U.S. Bureau of Labor Statistics to calculate inflation rates? Check out this straightforward primer at http://www.science.gmu.edu/csi779/drope/csi804/theory.html The BLS homepage on the consumer price index is at http://stats.bls.gov/cpihome.htm Oregon State's Prof. Robert Sahr offers a variety of ways to look at inflation - from blockbuster movies to Congressional pay. There is even a handy Excel table you can download. http://www.orst.edu/Dept/pol_sci/sahr/sahr.html You can point and click your way to compare housing prices at http://fly.HiWAAY.net/~rhuff/Postma/Cald Or, if you're moving, point and click to compare what your salary buys elsewhere in the U.S. at http://www.homefair.com/homefair/cmr/salcalc.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-from-the-nicar-library","slug":"1996-april-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-april-from-the-nicar-library.html","year":1996,"month":"April","monthNum":4,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides a comprehensive collection of federal government databases covering campaign finance, healthcare, transportation, crime, loans, and other public records available for journalistic research.","keywords":["databases","government","journalism"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["CD-ROM"],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nA monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1994; includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records.\nNational Bridge Inventory System data, includes inspection grades.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1994.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nNational Endowment for the Arts, grants, 1989-1993.\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-grants-available-for-investigative-journalism","slug":"1996-april-grants-available-for-investigative-journalism","url":"https://dwillis.github.io/uplink/article/1996-april-grants-available-for-investigative-journalism.html","year":1996,"month":"April","monthNum":4,"headline":"Grants available for investigative journalism","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE is donating $2,000 from Awards Contest entry fees to the Fund for Investigative Journalism, which has supported IRE members' investigative reporting for 25 years.","keywords":["donation","journalism","grants"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"IRE is donating $2,000 of proceeds from the IRE Awards Contest Entry fee to the Fund for Investigative Journalism.\n\nIRE members Robert Baskin of CBS, author David Burnham, Brooks Jackson of CNN, Margaret Engle of the Alicia Patterson Foundation and Gene Roberts of the New York Times, sit on the board of the fund. For 25 years, IRE members have gone to the fund for support to do stories that otherwise would not be done.\n\nFor information about applying for a grant, call (202) 462-1844, or write to the fund at: 1755 Massachusetts Ave., NW, Room 324, Washington. D.C. 20036","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-two-day-seminars-listing","slug":"1996-april-two-day-seminars-listing","url":"https://dwillis.github.io/uplink/article/1996-april-two-day-seminars-listing.html","year":1996,"month":"April","monthNum":4,"headline":"Two-day seminars listing","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Two-day journalism seminars offering panel discussions and hands-on training in spreadsheet and database skills are being held at seven locations across the United States in May and June, with registration costs ranging from $50-$135.","keywords":["seminars","journalism","training"],"topics":["Journalism & Newsroom","Data & Databases","Investigative Methods"],"technologies":["spreadsheet"],"full_text":"The following two-day seminars, including one day of panel discussions and demonstrations and one day of hands-on training in basic spreadsheet and database skills, are open to all journalists. Cost ranges from $50-$75 for Day I and from $100-$135 for both days. Here is a list of dates, locations and phone numbers to call for more information: East Lansing, Mich., May 14-15, (313) 259-0650. Concord, N.H., May 14-15, (603) 224-3327. Aberdeen, S.D., May 17-18, (605) 332-2111 Albany, N.Y., May 20-21, (518) 458-7821 Rochester, N.Y., May 22-23, (518) 458-7821 Vancouver, Wash., May 31-June I, (206) 682-1812 State College, Penn., June 7-8, (215) 561-1133","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-hate-crimes","slug":"1996-april-hate-crimes","url":"https://dwillis.github.io/uplink/article/1996-april-hate-crimes.html","year":1996,"month":"April","monthNum":4,"headline":"Hate crimes","kicker":"Bits, Bytes and Barks","author_name":"John Sullivan","author_title":"","affiliation":"The Chicago Reporter","summary":"The Chicago Reporter created a custom database to track hate crimes in Chicago, revealing trends and failures in state data collection.","keywords":["hate crimes","database","police records"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"The Chicago Reporter, an investigative monthly, built its own database of hate crimes from police records obtained from the Chicago Police Department. The paper chose 12 fields including type of crime, location, victim's age and race, offender's age and race. The analysis found that hate crimes against whites had dropped, while those against blacks had risen, and there was a slight decrease in hate crimes overall. Some information was missing, but even that turned into a story. The paper discovered the state of Illinois was violating state law by not collecting data on hate crimes.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-april-jobs-systems-editor-in-des-moines","slug":"1996-april-jobs-systems-editor-in-des-moines","url":"https://dwillis.github.io/uplink/article/1996-april-jobs-systems-editor-in-des-moines.html","year":1996,"month":"April","monthNum":4,"headline":"JOBS: Systems editor in Des Moines","kicker":"","author_name":"","author_title":"systems editor","affiliation":"The Des Moines Register","summary":"The Des Moines Register is hiring a systems editor to manage newsroom technical operations, supervise staff training, and oversee IT infrastructure for the 179,000-circulation newspaper. The position requires a bachelor's degree and 3-5 years of newsroom technology experience, with proficiency in multiple publishing systems and platforms.","keywords":["technology","journalism","management"],"topics":["Journalism & Newsroom","Technology"],"technologies":["SII/Coyote","Quark","Digicol","Photoshop","AP/Leaf","Cumulus","Apple OS","Windows","Unix","AppleScript","Internet"],"full_text":"The Des Moines Register seeks a systems editor, part of the newsroom's management team, to supervise daily newsroom technical operations and planning.\n\nThe Register uses an SII/Coyote editing system, Quark pagination, networked Mac and IBM/Windows computers, a Digicol archive system, Photoshop and AP/Leaf imaging systems and Cumulus archiving.\n\nThe systems editor leads newsroom technology training, help-desk operations, installations and budgeting. Applicants must have a bachelor's degree in journalism, computer science or related studies plus three to five years experience at a newsroom or periodical publisher. Work with Apple's OS, Windows, traditional front-end systems and modem communications is a must. Experience with SII Styl, Unix, AppleScript or Internet applications is an asset.\n\nThe Register is a Gannett paper with 179,000 daily circulation, 302,000 Sundays. Send a resume and cover letter to: David Rhein, Deputy Managing Editor, The Des Moines Register, 715 Locust St., Des Moines, IA 50313.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-jobs-the-corpus-christi-caller-times","slug":"1996-april-jobs-the-corpus-christi-caller-times","url":"https://dwillis.github.io/uplink/article/1996-april-jobs-the-corpus-christi-caller-times.html","year":1996,"month":"April","monthNum":4,"headline":"JOBS: The Corpus Christi Caller-Times","kicker":"","author_name":"","author_title":"","affiliation":"Corpus Christi Caller-Times","summary":"The Corpus Christi Caller-Times is hiring a computer-assisted reporter/coordinator with at least two years of experience in database management, data cleaning, and investigative journalism to produce CAR projects and support other newsrooms in the Harte-Hanks newspaper group.","keywords":["journalism","database","investigative"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet","Internet","modem/BBS"],"full_text":"The Corpus Christi Caller-Times has an opening for a computer-assisted reporter/coordinator. Applicants should have at least two years experience; knowledge of database, spreadsheet, Internet, modem communication and word processing software; basic hardware skills, including replacement of motherboards and memory chips; and experience negotiating for data and cleaning it up for CAR use. Expect to produce investigative and enterprise projects, help other reporters with projects and help other Harte-Hanks newspapers and the Harte-Hanks Austin Bureau with CAR projects.\n\nThe Caller-Times is a 65,000-circulation daily. An Internet version of the newspaper is at http://www.caller.com. To apply, send a cover letter, resume and clips to: Scott Rothschild, Metro Editor, Corpus Christi Caller-Times, P.O. Box 9136, Corpus Christi, TX 78469. Send questions via e-mail to Rothschild at scottr@caller.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-april-keep-up-with-nicar-online","slug":"1996-april-keep-up-with-nicar-online","url":"https://dwillis.github.io/uplink/article/1996-april-keep-up-with-nicar-online.html","year":1996,"month":"April","monthNum":4,"headline":"Keep up with NICAR online","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Instructions for subscribing to IRE/NICAR listservs via email and accessing journalism resources through CompuServe's Journalism Forum.","keywords":["listserv","journalism","IRE"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","CompuServe","Listserv","Email"],"full_text":"Subscribe to our listserv and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURIEDU. In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.\n\nIRE/NICAR is also accessible through CompuServe's Journalism Forum. Go to the JForum, Section 19. Also look into the IRE/NICAR files in Library 19.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-winners-use-car","slug":"1996-may-winners-use-car","url":"https://dwillis.github.io/uplink/article/1996-may-winners-use-car.html","year":1996,"month":"May","monthNum":5,"headline":"Winners use CAR","kicker":"Uplink update","author_name":"","author_title":"","affiliation":"Uplink","summary":"This issue of Uplink highlights 1996 Pulitzer Prize winners and finalists who used computer-assisted reporting (CAR) to produce their award-winning journalism, featuring interviews with reporters from the Raleigh News & Observer, Orange County Register, and other publications.","keywords":["Pulitzer","reporting","journalism"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"We always push CAR for deadline stories, beat reporting and special projects; now, we would like to highlight CAR reporters who have reaped the rewards: This issue of Uplink features the 1996 Pulitzer Prize winners and finalists who used computer-assisted reporting to get their stories.\n\nWe hear from Pat Stith, Joby Warrick and Melanie Sill of the Raleigh News & Observer, who won the Pulitzer for public service for \"Boss Hog,\" a series detailing abuses by corporate pig farmers. We talk with reporters from California's Orange County Register, whose staff won the Pulitzer for investigative reporting. And we look at work by the Dayton Daily News, The Fort Lauderdale Sun-Sentinel and Kansas City Star, who all had Pulitzer finalist stories.\n\nAlso, check out Tech Tips for helpful information on searching the Net, and On the Internet for a look at journalism awards and where you can find information on all types of honors and prizes.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-pulitzer-winner-for-public-service-boss-hog","slug":"1996-may-pulitzer-winner-for-public-service-boss-hog","url":"https://dwillis.github.io/uplink/article/1996-may-pulitzer-winner-for-public-service-boss-hog.html","year":1996,"month":"May","monthNum":5,"headline":"Pulitzer winner for public service 'Boss Hog'","kicker":"","author_name":"Anna Clark","author_title":"","affiliation":"Missouri School of Journalism","summary":"The (Raleigh) News & Observer's 'Boss Hog' series won a Pulitzer Prize for exposing environmental, economic, and political issues related to hog farming in North Carolina, revealing unregulated waste disposal and political connections that led to significant regulatory changes.","keywords":["Pulitzer","hog farming","environment"],"topics":["Health & Environment","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"The (Raleigh) News & Observer's 1996 Pulitzer prize-winning public service series \"Boss Hog\" sparked a public debate on hog farming that continues more than a year after the series ran.\n\nAt a population of 8 million and growing, hogs outnumber people in North Carolina. Pigs are taking over the economy where tobacco has left off, making North Carolina second only to Iowa in pig production. The economic boon to the state has not been without costs.\n\nPigs can produce up to four times as much waste as humans, yet North Carolina let waste disposal go largely unregulated. Open lagoons, many of which have no clay liners to prevent seepage into the sandy soil, hold the waste until it can be sprayed as fertilizer on fields.\n\nSacred pig\n\nThe News & Observer ran \"Boss Hog,\" a five-part series from Feb. 19-Feb. 26, 1995, exposing the environmental, economic and political impact of the state's new sacred cow - make that sacred pig. Reporters Pat Stith and Joby Warrick brought to light research about hog waste's threat to the water quality. They pinpointed leaks and spills hitherto covered up.\n\nStith and Warrick also showed the human side of the manure problem - the stench. People living near hog farms find themselves sealing their houses, running to get into cars when they have to go out and still having the odor stick to their clothes throughout the day. Perhaps the most condemning aspect of the series was the direct link Stith and Warrick uncovered between politicians and big-time hog producers. Phone records tracked legislators' calls to the Murphy Family Farms on the day regulation bills were up for vote. \"Boss Hog\" spotlights Wendell H. Murphy's cozy relationship with Gov. Jim Hunt. Murphy became the No. 1 hog producer in the country while a state legislator.\n\nbecame the No. 1 hog producer in the country while a state legislator.\n\nPig in a poke\n\nThe seven-month project began with a tip about a state-employed veterinarian accepting a hunting trip from Murphy Family Farms. The News & Observer asked for phone records of all long-distance calls placed by all state employees for the previous two years, although initially they were interested only in the records for one person.\n\n\"Generally, it is easier to get much rather than little,\" Stith said of records requests.\n\nThey waited six months for the database. The News & Observer has a strict records-request policy: \"If we ask you for it, we're going to get it or sue,\" Stith said.\n\nIn the end, the veterinarian story was a \"throwaway,\" Stith said. It did not run as part of the series. The bigger story was in calls between government officials and Murphy Family Farms. The News & Observer calculated an average of 50 calls a week over two years from state employees to the Murphy company and family, including calls from legislators and the governor's office.\n\nStory goes whole hog\n\n\"Boss Hog\" developed from a story about one person abusing a system to a business story (corporate farms swallow independent farmers) to an environmental story (what are the big farms doing with all that waste?). It turned into a series with political, economic, environmental and human impact. The series received the attention of the N.C. General Assembly, which authorized the governor and legislative leaders to form a task force to investigate problems uncovered by the report.\n\n\"We're shocked,\" Warrick said of the series' impact.\n\nAfter the series ran, legislators who had wanted to push for regulations were emboldened to introduce a half dozen bills, Warrick said. Several major lagoon spills during the summer of 1995 highlighted the environmental risks \"Boss Hog\" had forewarned, and some regulatory bills passed.\n\nBetween The News & Observer's coverage and the summer accidents, N.C. Gov. Hunt has begun cracking down on the industry's polluters, Warrick said. The blue-ribbon commission he helped appoint, of which the majority of members have financial interests in the hog industry, has come up with significant proposals for regulation.\n\n\"Pressure has been such that they knew they had to do something,\" Warrick said.\n\nDatabases\n\nThe News & Observer used the following resources: Its own database of political contributions, started in 1990, that now contains the records of 100,000 contributors and 250,000 contributions.\n\nA database of records of all long-distance calls from state officials for a two-year period, including more than 40 million records.\n\nThe team used Nine Track Express with FoxPro and R&R Report Writer on a Zeos 486-66 PC. They had a Contech tape drive to download the tapes.\n\nOnline data\n\nFor online data, the newspaper tapped: FEC data of campaign contributions. General Assembly public terminal with legislation on the hog industry. Lexis/Nexis and Information Resource Service Inc. (IRSC) to track property owners in Florida. AVCOMM BBS, an FAA database for aircraft and pilot information. LINC, a N.C. state government database with economic data on the hog industry's growth. Dun's Market Identifiers and Dun's Financial Records Plus through Dialog for companies' financial and historical information. Disclosure/Spectrum Ownership through Dialog for stockholder information. Insider Trading Monitor through Dialog for information on corporate insiders. Securities Exchange Commission through the Internet and Dialog. USDA through the Internet. Secretary of State and Limited Partnership databases from the state on N.C. companies. PhoneDi for phone numbers. Pat Stith and Joby Warrick can be reached at (919) 829-4500\n\nbecame the No. 1 hog producer in the country while a state legislator.\n\nPig in a poke\n\nThe seven-month project began with a tip about a state-employed veterinarian accepting a hunting trip from Murphy Family Farms. The News & Observer asked for phone records of all long-distance calls placed by all state employees for the previous two years, although initially they were interested only in the records for one person.\n\n\"Generally, it is easier to get much rather than little,\" Stith said of records requests.\n\nThey waited six months for the database. The News & Observer has a strict records-request policy: \"If we ask you for it, we're going to get it or sue,\" Stith said.\n\nIn the end, the veterinarian story was a \"throwaway,\" Stith said. It did not run as part of the series. The bigger story was in calls between government officials and Murphy Family Farms. The News & Observer calculated an average of 50 calls a week over two years from state employees to the Murphy company and family, including calls from legislators and the governor's office.\n\nStory goes whole hog\n\n\"Boss Hog\" developed from a story about one person abusing a system to a business story (corporate farms swallow independent farmers) to an environmental story (what are the big farms doing with all that waste?). It turned into a series with political, economic, environmental and human impact. The series received the attention of the N.C. General Assembly, which authorized the governor and legislative leaders to form a task force to investigate problems uncovered by the report.\n\n\"We're shocked,\" Warrick said of the series' impact.\n\nAfter the series ran, legislators who had wanted to push for regulations were emboldened to introduce a half dozen bills, Warrick said. Several major lagoon spills during the summer of 1995 highlighted the environmental risks \"Boss Hog\" had forewarned, and some regulatory bills passed.\n\nBetween The News & Observer's coverage and the summer accidents, N.C. Gov. Hunt has begun cracking down on the industry's polluters, Warrick said. The blue-ribbon commission he helped appoint, of which the majority of members have financial interests in the hog industry, has come up with significant proposals for regulation.\n\n\"Pressure has been such that they knew they had to do something,\" Warrick said.\n\nDatabases\n\nThe News & Observer used the following resources: Its own database of political contributions, started in 1990, that now contains the records of 100,000 contributors and 250,000 contributions.\n\nA database of records of all long-distance calls from state officials for a two-year period, including more than 40 million records.\n\nThe team used Nine Track Express with FoxPro and R&R Report Writer on a Zeos 486-66 PC. They had a Contech tape drive to download the tapes.\n\nOnline data\n\nFor online data, the newspaper tapped: FEC data of campaign contributions. General Assembly public terminal with legislation on the hog industry. Lexis/Nexis and Information Resource Service Inc. (IRSC) to track property owners in Florida. AVCOMM BBS, an FAA database for aircraft and pilot information. LINC, a N.C. state government database with economic data on the hog industry's growth. Dun's Market Identifiers and Dun's Financial Records Plus through Dialog for companies' financial and historical information. Disclosure/Spectrum Ownership through Dialog for stockholder information. Insider Trading Monitor through Dialog for information on corporate insiders. Securities Exchange Commission through the Internet and Dialog. USDA through the Internet. Secretary of State and Limited Partnership databases from the state on N.C. companies. PhoneDi for phone numbers. Pat Stith and Joby Warrick can be reached at (919) 829-4500","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-top-prize-for-investigation-fertility-fraud","slug":"1996-may-top-prize-for-investigation-fertility-fraud","url":"https://dwillis.github.io/uplink/article/1996-may-top-prize-for-investigation-fertility-fraud.html","year":1996,"month":"May","monthNum":5,"headline":"Top prize for investigation Fertility fraud","kicker":"","author_name":"Stan Dorsey","author_title":"NICAR Staff","affiliation":"NICAR","summary":"The Orange County Register's investigation into fraudulent egg transfers at UC-Irvine's fertility center required innovative reporting techniques to locate victims and obtain medical records in a state with largely unlisted phone numbers. The team used databases, public records, and creative source management to build a case that won the 1996 Pulitzer Prize.","keywords":["investigation","fertility","records"],"topics":["Investigative Methods","Crime & Justice","Health & Environment"],"technologies":["Access","Paradox","World Wide Web"],"full_text":"In a state where nearly 60 percent of all phone numbers are unlisted, reaching out and touching someone can be far more difficult than it sounds.\n\nThis frustrating fact of life confronted the staff of the Orange County Register as it tried to piece together its 1996 Pulitzer Prize-winning series on fraudulent egg transfers at the internationally renowned University of California-Irvine Center for Reproductive Health.\n\nA hospital employee tipped staff medical reporter Susan Kelleher that doctors at the center might be taking eggs from their patients during fertility procedures and later transferring these eggs to other barren women. The catch: The eggs were being transferred without the patients' consent, and some of the illegal transfers had resulted in live births.\n\nHospital roadblocks\n\nRegister reporters Kim Christensen, Michelle Nicolosi, Ernie Slone and Kelleher had to go to great lengths to obtain the medical records of the allegedly criminal procedures that the doctors were performing.\n\nSome sources returned records anonymously by way of regular mail. Others used what investigations editor Jim Mulvaney called a \"modern age Transom system,\" returning records to members of the team by way of self-addressed Federal Express envelopes. Kelleher even worked out a system where sources left records in the unlocked trunk of her car.\n\nOnce validated, the records were turned over to Slone, who, using Access and Paradox, built a database of patients who had donated or received eggs through procedures performed by the accused doctors. Some records clearly showed corruption, while others took more investigation. But without Slone's database, the information would have been unwieldy.\n\nUse local resources\n\nSlone's work was particularly handy when it came to identifying patients. Using local resources such as voter registration records, motor vehicle records and Social Security numbers, Slone was able to find matches for a good number of the patients in his database.\n\n\"When we went to these women, we were going to be changing their lives,\" said Mulvaney. \"Most of the people didn't even know they were being victimized.\"\n\nThat meant the Register staff would have to make absolutely sure they were contacting the right women.\n\nThe team used commercial databases, such as CDB Infotek and Information America to further identify patients. This allowed them access to local census and advertising databases.\n\nSlone was then able to target certain women, such as those between the ages of 25 and 45 and those in higher income neighborhoods, to find additional matches among them. These women were far more likely candidates for in vitro fertilization and could possibly afford the $10,000 price tag. The extra step helped save the reporters from the potential disaster of contacting the wrong patient. \"It's the ultimate invasion of privacy,\" added Mulvaney. \"You have to be relatively certain.\"\n\nCaught in the web\n\nAs the team continued to follow the case, the reporters began posting their articles on the Register's World Wide Web site (http://www.ocregister.com).\n\nThey expected it to be an excellent resource for anyone who had heard about the case and wanted to know more. What they didn't expect was for it to help them locate Ricardo Asch, the primary doctor under investigation.\n\nAsch and one of his partners had fled the country under the pressure of the Register articles and five federal probes into the allegations. The health minister of Mexico called saying he had seen the Register's Web site and knew that Asch was trying to apply for a license to practice in Mexico.\n\nThe team subsequently sent an undercover reporter to check out the story.\n\nNot just for numbers\n\nAfter eight months, the Register's project produced some startling numbers: 61 women were identified as fraud victims, 10 children were born from stolen eggs, and 32 patients had filed lawsuits against the doctors.\n\nOn Jan. 19, in Tijuana, Asch gave a four-day deposition in response to those lawsuits, and Slone drew on his computer-assisted instincts once again: He electronically transferred transcripts from the depositions to the Register, where they were entered into a text database program called askSam.\n\nReporters queried the database for specific information from the deposition. This afforded the Register a major advantage over the competition.\n\n\"We were able to get straight to the exact facts of the day,\" said Mulvaney, who helped edit the 300 pages of transcripts per day. \"Meanwhile, the Los Angeles Times' reporters were still writing off of the lawyers' claims.\"\n\nErnie Slone can be reached at (714) 835-1234, or send e-mail to eslone@link.freedom.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-pulitzer-finalist-for-national-reporting-military-secrets","slug":"1996-may-pulitzer-finalist-for-national-reporting-military-secrets","url":"https://dwillis.github.io/uplink/article/1996-may-pulitzer-finalist-for-national-reporting-military-secrets.html","year":1996,"month":"May","monthNum":5,"headline":"Pulitzer finalist for national reporting 'Military secrets'","kicker":"","author_name":"Rex W. Huppke","author_title":"","affiliation":"Missouri School of Journalism","summary":"A Pulitzer Prize finalist investigation by Dayton Daily News reporters revealed that the U.S. military systematically shielded hundreds of sex offenders, including child molesters, by allowing resignations instead of prosecution and failing to report crimes to the FBI. The series prompted congressional inquiries, reopened cases, and forced the Army to release previously withheld court-martial records.","keywords":["military","investigation","accountability"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Paradox"],"full_text":"Tailhook was just the tip of the iceberg. An eight-month investigation by Russell Carollo, Jeff Nesmith and Carol Hernandez of the Dayton (Ohio) Daily News revealed that the U.S. military has shielded hundreds of sex offenders within their ranks, including child molesters.\n\nIn a series that was a 1996 Pulitzer Prize finalist for national reporting, the reporters told readers about:\n\nNorman Michael Fletcher, who, as a Navy recruiter, was accused of sexually assaulting a 14-year-old girl. Fletcher received an honorable discharge from the Navy in lieu of a trial. He now helps oversee a home for unwed teen-age mothers.\n\nCortney A. Riley, who was accused of raping a 16-year-old girl. A witness heard the girl screaming as she was assaulted, but a panel of Riley's Navy co-workers concluded that she \"wanted sex.\" Six months later, Riley raped and choked a 15-year-old girl at gunpoint.\n\nChristopher Matthews, who was charged with two rapes. At trial time, the Navy claimed it could not \"find\" the first victim. Matthews' own attorney said this was instrumental in winning his client's acquittal. It took the Dayton reporters about 15 minutes to locate the \"missing\" accuser using her Social Security number and a publicly available computer database.\n\nSpotting the trend\n\nCarollo noticed the trend of military sex offenders while researching another series titled \"Prisoners on Payroll,\" which showed that many imprisoned servicement collect paychecks while in jail. He next turned his attention to the sexual charges.\n\nUsing FOIA requests, he obtained court martial databases from the Navy and Air Force. The Army refused to cooperate. The Coast Guard did not have a database, so Carollo built his own from Coast Guard files.\n\nThe reporters analyzed the data and found evidence of unusual judicial practices:\n\nHundreds of military personnel charged with sex offenses managed to get around prosecution or criminal punishment. Numerous military sex offenders returned to civilian life with clean records because the military often allows sex offenders to resign rather than face prosecution. They also found that military members who were victims of sexual assault and spoke up were frequently ostracized or fired. And, they reported, the military often refused to report criminal records to the FBI, as required.\n\nThe series, which ran in October 1995, got results.\n\nU.S. Rep. Robert Dornan, R-Calif., requested that the Department of Defense investigate several issues the series uncovered.\n\nA 1992 case in which two Navy sailors allegedly sexually assaulted a 14-year-old girl in Alaska was reopened by a civilian prosecutor in Sitka, Alaska.\n\nThe Dayton Daily News filed a federal lawsuit against the Army after it refused to provide records of public courts-martial. The Army wound up giving Carollo most of the information.\n\nCarollo said Paradox was used almost exclusively for the data analysis.\n\nThe Air Force data was very detailed and clean, Carollo said. The Navy database, on the other hand, contained many discrepancies. For example, \"Memphis Naval Base\" was abbreviated more than 15 different ways.\n\nAnother problem Carollo encountered with the Navy information was that defendants' names were not given. The locations of the trials, however, were in the database. The reporters managed to contact the civilian attorneys who had been involved in the various cases and were thus able to obtain the defendants' names.\n\nLooking back, Carollo would recommend several tips for someone trying to deal with obtaining military information.\n\n\"The most important thing is to file your FOI way in advance,\" he said. \"Give yourself four to six months.\" He also suggests starting off confidently and asking for the right people - the computer technicians.\n\n\"Assume there is a computer database that you're looking for,\" Carollo said. \"Find the person who operates that system. If you deal with public affairs, you are just putting another layer in an already bureaucratic system.\"\n\nRussell Carollo can be reached at (513) 225-2000, or send e-mail to russell_carollo@dni.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-pulitzer-finalist-for-explanatory-journalism-divided-we-sprawl","slug":"1996-may-pulitzer-finalist-for-explanatory-journalism-divided-we-sprawl","url":"https://dwillis.github.io/uplink/article/1996-may-pulitzer-finalist-for-explanatory-journalism-divided-we-sprawl.html","year":1996,"month":"May","monthNum":5,"headline":"Pulitzer finalist for explanatory journalism 'Divided We Sprawl'","kicker":"","author_name":"Glaston Ford","author_title":"NICAR staff","affiliation":"NICAR","summary":"Kansas City reporters investigated suburban sprawl and found that many homes in the metro area failed to keep pace with inflation between 1980-1990, while also discovering systemic lending discrimination and high governmental costs associated with sprawl.","keywords":["sprawl","inflation","housing"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Atlas GIS"],"full_text":"Your home may not be the inflation-beating investment it is sold to be. Ask Marvin Blevins. In 1981 he bought his three-bedroom ranch house in a suburb of Kansas City for $62,550. If the value of the home had kept up with the rising cost of living, it would have sold for more than $103,000. He sold it last fall for $73,000.\n\nIn the Kansas City metro area, the housing values in more than half the census tracts did not keep up with inflation between 1980 and 1990. So, even though the cost of homes in these tracts might have increased, the real value of the home declined.\n\nThat is one of the key discoveries of Kansas reporters Chris Lester, Jeffrey Spivak and Greg Reeves in their six-part series examining the cost and character of sprawl. The series, \"Divided We Sprawl,\" ran Dec. 17-22, 1995. It was a 1996 Pulitzer finalist for explanatory journalism.\n\nStart your engines\n\nFrom the beginning, Lester knew the project would require a lot of number crunching. \"The computer work was the key to analyzing white flight and income flight,\" said Reeves. He found that Kansas City lenders reject applications from blacks 2.5 more times than they do from whites. Even high-income blacks are rejected more than low-income whites.\n\nThey also wanted to come up with a way to measure the cost of sprawl. Many people leave the urban core for a promise of a better life in the suburbs. This spread creates a maze of government, from school districts, sewer districts, library districts, to cities and counties. The cost is staggering.\n\nA computer analysis of the 1987 Census of Government Finance Statistics found that Kansas City ranked third in the nation in governmental units per person. The total regional cost is $3 billion with an average annual cost per household of $5,000.\n\nAnother question they had was, \"What is the pattern of housing value in the entire metropolitan area between 1980 and 1990?\"\n\nReeves massaged the numbers. Then Lester and Spivak used the results to stimulate their investigation.\n\nThey analyzed every census tract in metropolitan Kansas City and looked at median housing values in each tract from 1980 to 1990. The increase seemed small, so they decided to check it against inflation. The results were surprising: Many homes lost value, even in wealthier suburbs.\n\nThe Golden Rainbow\n\nLester discovered some research done in Ohio that suggested people who make money on their homes tend to purchase homes farther from the urban core. The tax code encourages this behavior.\n\nIf a homeowner makes a profit, he can avoid paying a capital gains tax by buying homes of equal or greater value. These homes tend to be in the suburbs. \"That got us all agitated about the codification of sprawl in the tax code,\" Lester said.\n\nHe wanted to see what was happening in Kansas City.\n\nReeves used Atlas GIS to map housing values in the metro area. He then calculated the average housing value of owner-occupied units within a two-mile radius of the downtown intersection of 12th and Main. Then, he determined the average value for the next ring and the next ring, and so on.\n\nAverage housing values increased incrementally until peaking in a ring 14 to 16 miles from downtown - the golden rainbow.\n\nHe did the same analysis using census figures from previous years and discovered the golden rainbow of peak property values had been creeping away from downtown about two miles a decade. This figure became one of the measures of sprawl in the story.\n\nCities tend to grow in directions, not circles. So they picked the quadrant with the greatest wealth, Johnson County, and used that figure in the series.\n\nLester and Spivak used the computer results, together with interviews with more than 700 sources, to try to explain sprawl. It wasn't easy.\n\n\"This wasn't about investigating one public official who did something wrong and sending them to jail,\" Lester said. \"This is about trying to explain in some rational way why American society came out the way it did over the last 50 years.\"\n\nThe reporting team got the 1980 and 1990 Census housing tables online. They typed in the\n\nvalues for 1950, 1960 and 1970. This wasn't as much work as it seems: Each year had only 200 to 300 records.\n\nThe Census of Government Finance Statistics came from the Missouri state data center. Each state should have one, but the file is also available directly from the Census Bureau. \"This is the database from hell,\" Reeves said. The file contains 70 to 80 fields, many of them coded.\n\n\"You'll get used to the codes,\" he said. They obtained HMDA data from NICAR and checked parts of their analysis against published government reports.\n\nComparing census tracts over several decades was tricky. Tracts change and split as the population changes. Be prepared to spend a lot of time massaging the numbers if you plan to do this.\n\nThe following week-long bootcamps offer hands-on training in computer-assisted reporting skills:\n\nNICAR Bootcamps, featuring training in the use of spreadsheets and database managers, accessing data in various media, such as nine-track tapes, and negotiating for data; May 19-24, July 14-19, and Aug. 11-16. Columbia, Mo. For more information, call NICAR","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-pulitzer-finalist-for-beat-reporting-medicaid-hmo-game","slug":"1996-may-pulitzer-finalist-for-beat-reporting-medicaid-hmo-game","url":"https://dwillis.github.io/uplink/article/1996-may-pulitzer-finalist-for-beat-reporting-medicaid-hmo-game.html","year":1996,"month":"May","monthNum":5,"headline":"Pulitzer finalist for beat reporting 'Medicaid HMO Game'","kicker":"","author_name":"Dawn Hobbs","author_title":"IRE staff","affiliation":"IRE","summary":"A Fort Lauderdale Sun-Sentinel investigation by reporters Fred Schulte and Jenni Bergal exposed widespread fraud and poor care in Florida Medicaid HMOs, revealing that executives enriched themselves while patients suffered inadequate services. The 1996 Pulitzer finalist series documented enrollment fraud, unqualified providers, and executive misconduct that cost taxpayers millions.","keywords":["Medicaid","fraud","investigation"],"topics":["Health & Environment","Investigative Methods","Business & Economy"],"technologies":["As Easy As","File Express","Excel","SPSS"],"full_text":"Pulitzer finalist for beat reporting 'Medicaid HMO Game'\n\nBy Dawn Hobbs, IRE staff\n\nDoctor, doctor give me the news; I got a bad case of - cheating the taxpayers.\n\nA Fort Lauderdale Sun-Sentinel investigation revealed that HMOs designed to provide cost-cutting health care for the poor often fail to deliver services the state paid them to provide.\n\nThe investigation also indicated that while patients may suffer, some Medicaid HMO executives live lavishly off the taxpayers leasing luxury cars and awarding themselves generous salaries or service contracts.\n\n\"The Medicaid HMO Game: Poor Care, Big Profits\" is the Sun-Sentinel's fourth investigative series exposing quality problems in tax-funded Florida HMOs. The series, by Fred Schulte and Jenni Bergal, was a 1996 Pulitzer finalist for beat reporting. It ran Nov. 26-29, 1995. The reporters began investigating HMOs in the early 1990s.\n\nContractors see green\n\nThe investigation exposed physicians with histories of providing poor patient care, convicted felons and trash collectors-all of whom had applied for HMO contracts or had already received them. The point is to sign up as many people as possible, Schulte explained, because the state pays the HMO contractor per person.\n\n\"They were out signing people up under bridges and forging people's names on applications,\" Schulte said. \"In Miami, they'd drive up in a big van and unload a bunch of sales agents dressed up as nurses with white coats and stethoscopes. They'd knock on people's doors and tell them about this wonderful program. When the people said they were already on Medicaid, the agents told them they must sign up or lose their Medicaid benefits.\"\n\nThe enrollment fraud proved to be so outrageous, that Schulte and Bergal investigated matters further.\n\nThe team began collecting every sort of document available: the Florida Agency for Health Care Administration general correspondence files, HMO hospital and physician utilization data, disenrollment data, medical peer review reports, annual incident reports and grievance reports and administrative hearing judgments and final reports.\n\nNone of these documents came in electronic form. And even worse, Schulte and Bergal found thousands of pages of medical peer review records and other reports sitting in boxes in a closet in the Tallahassee state offices.\n\nBuilt to order\n\nAfter retrieving all the boxed data, Schulte and Bergal met with Jay Mendell, professor of public administration at Florida Atlantic University in Ft. Lauderdale. Mendell, who has taught computer systems and the Internet to graduate students for 20 years, has provided technical support to Schulte and Bergal on several assignments.\n\nMendell set up data templates with as much error rejection as possible in which Bergal then entered some 5,000 to 7,000 pages of peer reviews. As Easy As, shareware available through Trius, and File Express, another shareware put out by Expressware, were used for entering the data on an IBM clone.\n\nPick a program\n\nOnce the data was entered, Mendell explained, it went into a program that best expressed the results-sometimes Excel and sometimes SPSS.\n\n\"Sometimes simple summary statistics and ratios will do the trick, and then you use Excel,\" Mendell said. \"Other times, when the data is more complicated, you need to produce three-dimensional plots and look at them from different angles. This is when you use SPSS, a heavy-duty statistical program.\n\nAs a result of this series, key Florida legislators promised new laws to crack down on Medicaid HMO abuses during the current session, rewrote contracts for Medicaid HMOs and banned door-to-door marketing. State officials also refused to license two HMOs whose ties to state Sen. Alberto Gutman were documented by the newspaper. The series disclosed that the senator had attempted to broker the sales of HMOs for fees as high as $1 million and had gone into the HMO business outside Florida with some health industry executives with business before his committee. Gutman has resigned from the health-care committee and faces state charges.\n\nFred Schulte can be reached at (305) 356-4591, or send e-mail to freds@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-car-trips-first-adventures-debunking-gas-claims","slug":"1996-may-car-trips-first-adventures-debunking-gas-claims","url":"https://dwillis.github.io/uplink/article/1996-may-car-trips-first-adventures-debunking-gas-claims.html","year":1996,"month":"May","monthNum":5,"headline":"CAR trips: First adventures Debunking gas claims","kicker":"","author_name":"Chris Heinbaugh","author_title":"Investigative reporter","affiliation":"News 15","summary":"An investigative journalist uncovers that Circle K's advertised partnership to sell UNOCAL 76 gasoline was misleading, as the gas was actually still coming from the same non-UNOCAL suppliers as before, proven through public database records of inspection reports.","keywords":["investigation","gasoline","fraud"],"topics":["Investigative Methods","Business & Economy","Data & Databases"],"technologies":["FoxPro"],"full_text":"It sounded too crazy to be true. After all, it involved two huge firms, the Phoenix-based Circle K chain, and UNOCAL 76 Gasoline. In a well-publicized announcement, last year they announced that Circle K would now carry Unocal 76 gas in the pumps at its stores in Arizona and Nevada. It was followed by a major advertising blitz: full-page ads, billboards, TV, radio. But the News 15 investigative unit began getting calls from tanker drivers and Circle K employees, all saying the same thing: The gas at Circle K, wasn't coming from UNOCAL's terminals. We checked it out, following tanker trucks, and sure enough, the gasoline at Circle K, was coming from the same gas terminals it was using before the agreement. Nothing had changed, except now \"76\" was slapped on the pumps.\n\nFoxPro is my friend\n\nBut we wanted more proof. So I called the Arizona Department of Weights and Measures, the agency responsible for testing quality. An inspector there would not get specific, but dropped hints about inspections reports. He mentioned they were kept on a database. BOING! Having just returned from NICAR Bootcamp, I was salivating. He began bragging about this great FoxPro program.\n\n\"FoxPro is my friend,\" he said. \"Can I get a copy of your database?\" \"Sure,\" he said. \"Just fax me an Open Records request.\" One fax and a diskette later, I had my database (easiest I've ever acquired), and I had the hard facts to back up my story.\n\nThe proof\n\nThe database helped me verify what our insiders had been telling us. As part of the inspection, the stations must list which terminal the gas came from. This field not only proved the Circle K gas wasn't coming from UNOCAL, but because the records stretched back to well before the agreement, we could prove these were the same suppliers as before.\n\nAnother field showed the elements to oxygenate the fuel. UNOCAL only uses ethanol in Phoenix, but the gas at Circle K used MTBE as well as ethanol, clearly demonstrating a difference.\n\nThe database also helped us establish a serious difference in the quality control between real UNOCAL gas, and the stuff at Circle K. I ran queries pulling out all the violations inspectors handed UNOCAL and Circle K since January. UNOCAL only had three. Circle K had 33 more than any other chain.\n\nThe database also helped debunk Circle K's claim that swapping gas is something everyone does. It showed that the gas at Chevron stations was coming only from Chevron terminals, Mobil from Mobil terminals, Unocal from Unocal. Circle K has lots of suppliers, and few got their gas from UNOCAL. The bottom line: The gas I buy at Circle K is not the same gas I would get at UNOCAL.\n\nLoosening tongues\n\nFaced with this, state inspectors now felt quite free to spell out their concerns that consumers were being misled. And the state Attorney General's office made it clear, if the companies are saying it's UNOCAL gas, it better be UNOCAL, not something like UNOCAL, and the ads could be violating consumer protection laws. Circle K could not refute that it was different gas. They could only say that it met the standards set by UNOCAL. But even this didn't hold up because the database showed the problems with quality control.\n\nData backbone\n\nThe story was important to consumers, and might not have flown if it wasn't for the database. Circle K had brushed off concerns as no big deal, insisting the gas meets UNOCAL's standards. At least one of our print competitors reportedly shelved their story because they bought Circle K's explanation. We might have too if we hadn't found and used the database. While it was only one element our viewers saw in a colorful story, it provided backbone and gave the report the teeth it needed to stand in the face of some major corporate pressure to refute it.\n\nChris Heinbaugh can be reached at (602) 232-6314, or send e-mail to heinknxv@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-nicar-seminars-taking-the-show-on-the-road","slug":"1996-may-nicar-seminars-taking-the-show-on-the-road","url":"https://dwillis.github.io/uplink/article/1996-may-nicar-seminars-taking-the-show-on-the-road.html","year":1996,"month":"May","monthNum":5,"headline":"NICAR seminars Taking the show on the road","kicker":"","author_name":"Brant Houston","author_title":"managing director","affiliation":"NICAR","summary":"The National Institute for Computer-Assisted Reporting will conduct 52 seminars across more than 25 states between January and June 1996, training over 1,000 journalists in computer-assisted reporting tools and techniques. The organization is also expanding its reach to minority journalists through fellowships, regional seminars, and partnerships with journalism associations.","keywords":["seminars","journalists","training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"Fifty-two. Count 'em. 52. That's how many seminars the National Institute for Computer-Assisted Reporting will give between Jan. 1, 1996, and June 30, 1996.\n\nIn six months, NICAR instructors will have gone to more than 25 states to teach the rudiments of CAR to more than 1,000 print and broadcast journalists. The NICAR seminars last from one to six days, ranging from a CAR overview to intense hands-on work in statistics and mapping software.\n\nIn the past few months, NICAR has conducted customized on-site seminars for 10 to 20 reporters and editors at Newsday, The Washington Post, the TV magazine American Journal, Alabama Public TV and Money Magazine.\n\nThe institute also held a special, first-ever seminar for 25 broadcast journalists in March at Northwestern University and a week-long advanced seminar in statistics and mapping at the University of North Carolina in Chapel Hill with precision journalism guru Philip Meyer.\n\nAt the same time, NICAR has trained journalists from smaller news organizations on the needs of minority journalists. In a joint effort with the Associated Press, NICAR will give at least 16 regional seminars in six months, with several more planned. Journalists from papers such as Paducah Sun and Kalamazoo Gazette have been able to afford the low fees and convenience of these regional, two-day meetings.\n\nMinority fellowships\n\nMinority journalists have been offered fellowships to the week-long bootcamps at the University of Missouri in Columbia. In the fall, NICAR will hold three two-day regional seminars for minority journalists. NICAR hopes to hold the seminars in Richmond, Va.; Dallas, Texas; and San Francisco, Calif. NICAR also has teamed up with the Multicultural Management Program at the Missouri School of Journalism to hold a week-long seminar for minority managers dealing with the new technology.\n\nNICAR also plans to give presentations and sample training at the national conventions of the associations of Hispanic, Black, and Asian American journalists.\n\nNICAR's basic curriculum covers practical use of the Internet (not to surf but to produce stories), spreadsheets, and database managers — the three basic tools of CAR. NICAR instructors can also teach mapping, data managing and statistics.\n\nProviding a backbone of instruction are exercises created by practicing journalists and the NICAR staff, the handbook \"Computer-Assisted Reporting: A Practical Guide\" and two books of synopses of CAR stories, and a collection of videotapes.\n\nPost praises ambassadors'\n\n\"Ambassadors to the newsroom\" is what Washington Post executive editor Leonard Downie Jr. called the journalists at the NICAR seminar at the Post this spring. In a brief talk with the 10 seminar participants, Downie said he expected them to take their new knowledge back to the newsroom and lead the way on computer-assisted reporting. Downie said he wanted computer-assisted reporting to become a routine occurrence in the Post newsroom, saying that it could be used for deadline stories, on-the-beat stories and projects. The Post invited NICAR to teach three days of a five-day seminar put together by Post CAR experts and trainers. William Casey, who heads CAR at the Post, led the first day of training on the Internet and spreadsheets. NICAR trainers then appeared on the second day and spent three days on spreadsheets, database managers, data acquisition and data cleaning. On the fifth day, Casey finished the seminar.\n\nCasey said the training journalists received before NICAR's arrival made the seminar extremely productive. Trainers at the Los Angeles Times, the Boston Globe, and Newsday have also found that kind of preparation helpful.\n\nDiane Weeks, manager of computer training for the newsroom, said the seminar worked \"really well\" and plans are underway for Post trainers to spread the CAR word further.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-post-praises-ambassadors","slug":"1996-may-post-praises-ambassadors","url":"https://dwillis.github.io/uplink/article/1996-may-post-praises-ambassadors.html","year":1996,"month":"May","monthNum":5,"headline":"Post praises 'ambassadors'","kicker":"","author_name":"Brant Houston","author_title":"","affiliation":"NICAR","summary":"The Washington Post hosted a five-day computer-assisted reporting seminar led by Post experts and NICAR trainers, aiming to make CAR a routine practice in the newsroom. Executive editor Leonard Downie Jr. called the participating journalists \"ambassadors to the newsroom\" who would lead the way in spreading CAR skills throughout the organization.","keywords":["training","journalism","technology"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"\"Ambassadors to the newsroom\" is what Washington Post executive editor Leonard Downie Jr. called the journalists at the NICAR seminar at the Post this spring. In a brief talk with the 10 seminar participants, Downie said he expected them to take their new knowledge back to the newsroom and lead the way on computer-assisted reporting. Downie said he wanted computer-assisted reporting to become a routine occurrence in the Post newsroom, saying that it could be used for deadline stories, on-the-beat stories and projects. The Post invited NICAR to teach three days of a five-day seminar put together by Post CAR experts and trainers. William Casey, who heads CAR at the Post, led the first day of training on the Internet and spreadsheets. NICAR trainers then appeared on the second day and spent three days on spreadsheets, database managers, data acquisition and data cleaning. On the fifth day, Casey finished the seminar.\n\nCasey said the training journalists received before NICAR's arrival made the seminar extremely productive. Trainers at the Los Angeles Times, the Boston Globe, and Newsday have also found that kind of preparation helpful.\n\nDiane Weeks, manager of computer training for the newsroom, said the seminar worked \"really well\" and plans are underway for Post trainers to spread the CAR word further.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-miss-something","slug":"1996-may-miss-something","url":"https://dwillis.github.io/uplink/article/1996-may-miss-something.html","year":1996,"month":"May","monthNum":5,"headline":"Miss something?","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR maintain searchable mailing list archives organized by thread, author, and date, available in both HTML and plain text formats for easy access and downloading.","keywords":["archives","mailing","lists"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Miss something? Check out the IRE-L and NICAR-L mailing list archives on our websites at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format, which is useful for downloading a particular month.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-tech-tips-a-guide-to-search-engines-seek-and-ye-shall-find","slug":"1996-may-tech-tips-a-guide-to-search-engines-seek-and-ye-shall-find","url":"https://dwillis.github.io/uplink/article/1996-may-tech-tips-a-guide-to-search-engines-seek-and-ye-shall-find.html","year":1996,"month":"May","monthNum":5,"headline":"Tech tips: A guide to search engines Seek and ye shall find","kicker":"","author_name":"Dan Keating","author_title":"","affiliation":"Miami Herald","summary":"This guide explains the key differences between web indexes and search engines, helping reporters understand when to use each tool: indexes organize sites by general topic and require registration, while search engines scan actual page content and can return deep links to specific information.","keywords":["indexes","search","content"],"topics":["Technology","Investigative Methods"],"technologies":["World Wide Web","Internet","Usenet"],"full_text":"A typical area of confusion for new online users especially reporters - is searching. People hear about \"search engines\" and indexes, but seldom recognize the important differences between them, and when to use the different things. Here's a guide:\n\nWhat's out there?\n\nIndexes - They work by general descriptions of content. There are indexes, largely of World Wide Web sites. Yahoo and Galaxy are well organized to quickly get to the kind of information you want. They are comprehensive of major Web sites.\n\nThe owner of a Web site has to \"register\" with the indexes, so they are for finding sites to which someone would want to draw attention. They tend to be sites of major educational, government, organization and commercial interests. They are organized based on the general topic of their content. So if a site is about movies, it'll be organized there. The Web site may or may not have a section about Marilyn Monroe. All you know is that it has movies.\n\nYou find sites by checking the general descriptions. You can also search the index itself, so you can look quickly for a particular business name. But you are searching only the name of the link and description of the link, not the content of what's on the link. Links will almost always bring you to the main homepage for a site. You will also find pointers to some discussion groups and a handful of listservs.\n\nYahoo! http://www.yahoo.com Galaxy http://galaxy.einet.net\n\nInternet Search Engines - They look at the exact content of each: There are \"search engines\" for searching the content of all the Web pages on the Internet. Obviously, it would be impractical for them to scan the entire network after you put in your request. They work by sending out \"robot\" programs that automatically read every page of every Web site they can find (there are things administrators can do to prevent pages from being read). They then index every word on every site.\n\nSo, the search engines have a giant database listing words and where they were found. When you search, it looks in the database and brings back the matching sites.\n\nUnlike an index, you're not just searching for words in the title or description of a site; you're actually searching the contents of every page on a site. And you might get linked to a page deep within a Web site, rather than just the front page. For instance, if you searched for \"Marilyn Monroe\" you might get a link to a reference to her on a page with an address such as \"http://www.moviespot.com/classics/comedy/some-like-it-hot/stars/mm.html\". That is extremely useful for getting exactly to what you want. But you won't be at the \"welcome\" screen for that Web site, so you might have trouble figuring out where you are.\n\nAlso, you are likely to find a lot of obscure things when you search here, since just one reference on a page somewhere might match your search, even if the general contents of the Web site at that link aren't of interest.\n\nSo, when you search these Net-wide engines, you tend to get a huge number of responses and may get buried in them. Lycos http://www.lycos.com WebCrawler http://www.webcrawler.com AltaVista http://altavista.digital.com (my favorite)\n\nUsenet Discussion Group Search - Searching online conversation. The indexes and search engines generally point you at sites where information has been posted.\n\nBut there's a lot of discussion online, too. You can search the content of the discussion on 10,000 usenet groups. If you search for a common term, such as \"Microsoft,\" you'll get buried. But if you're trying to find an expert on some obscure topic, such as \"hermaphroditic mammalian albinos,\" the discussions are probably your best chance.\n\nYou can find out who's talking about something and what they're saying. DejaNews http://www.dejanews.com AltaVista http://altavista.digital.com\n\nWhat do I use?\n\nIf you're looking for a specific site online that is not obscure and whose creators would want it to be found, then you are best off at the indexes.\n\nIf you are looking for the various sites on a general topic, the indexes are by far the best choice. So, if you want to find all the sites online about classical music or computer-assisted design or reproductive rights, you'll almost certainly find a heading on that subject in the index. You'll find the related sites. If you are looking for something that isn't major and whose creator wouldn't register it with the indexes, then try the search engines. Don't do this with general subjects, such as Microsoft or engineering or classical music. But it would be good if you were looking for a page about Seji Ozawa or Arctic seals. Only use the search engines for things that are specific enough and obscure enough that you won't get buried. Sometimes they are useful in searching for combinations of things, such as \"wolf and Yellowstone.\" You probably won't find a Web site under that definition in an index, but there's a good chance of finding a page with those words on them somewhere on the Web. If you are mainly interested in finding an expert or the subject is even too obscure for the search engines, try searching the usenet discussion groups. Searching the discussion groups is also useful for finding more current topics, since it generally takes a while for information to get posted into Web sites. In general, start with indexes because they're fast and efficient. If what you want isn't there, try the search engines. Finally, turn to discussion group searches. Dan Keating can be reached at (954) 985-4571, or send e-mail to dtkeats@ibm.net\n\ndesign or reproductive rights, you'll almost certainly heading on that subject in the index. You'll find the related sites. If you are looking for something that isn't major and whose creator wouldn't register it with the indexes, then try the search engines. Don't do this with general subjects, such as Microsoft or engineering or classical music. But it would be good if you were looking for a page about Seji Ozawa or Arctic seals. Only use the search engines for things that are specific enough and obscure enough that you won't get buried. Sometimes they are useful in searching for combinations of things, such as \"wolf and Yellowstone.\" You probably won't find a Web site under that definition in an index, but there's a good chance of finding a page with those words on them somewhere on the Web. If you are mainly interested in finding an expert or the subject is even too obscure for the search engines, try searching the usenet discussion groups. Searching the discussion groups is also useful for finding more current topics, since it generally takes a while for information to get posted into Web sites. In general, start with indexes because they're fast and efficient. If what you want isn't there, try the search engines. Finally, turn to discussion group searches. Dan Keating can be reached at (954) 985-4571, or send e-mail to dtkeats@ibm.net\n\ndesign or reproductive rights, you'll almost certainly heading on that subject in the index. You'll find the related sites. If you are looking for something that isn't major and whose creator wouldn't register it with the indexes, then try the search engines. Don't do this with general subjects, such as Microsoft or engineering or classical music. But it would be good if you were looking for a page about Seji Ozawa or Arctic seals. Only use the search engines for things that are specific enough and obscure enough that you won't get buried. Sometimes they are useful in searching for combinations of things, such as \"wolf and Yellowstone.\" You probably won't find a Web site under that definition in an index, but there's a good chance of finding a page with those words on them somewhere on the Web. If you are mainly interested in finding an expert or the subject is even too obscure for the search engines, try searching the usenet discussion groups. Searching the discussion groups is also useful for finding more current topics, since it generally takes a while for information to get posted into Web sites. In general, start with indexes because they're fast and efficient. If what you want isn't there, try the search engines. Finally, turn to discussion group searches. Dan Keating can be reached at (954) 985-4571, or send e-mail to dtkeats@ibm.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-on-the-internet-awards-honors-and-prizes","slug":"1996-may-on-the-internet-awards-honors-and-prizes","url":"https://dwillis.github.io/uplink/article/1996-may-on-the-internet-awards-honors-and-prizes.html","year":1996,"month":"May","monthNum":5,"headline":"On the Internet Awards, honors and prizes","kicker":"","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute","summary":"A journalism newsletter column highlights various prestigious journalism awards and competitions, providing resources and website links where readers can find information about submission guidelines and view award-winning entries across multiple disciplines including photojournalism, broadcast journalism, science journalism, and newspaper design.","keywords":["awards","journalism","prizes"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Yes, of course, we write and produce great journalism for the benefit of our readers and viewers. But it sure is nice when those stories win prizes.\n\nThis issue's Web column focuses on journalism awards and on where you can find information on types of honors and prizes. Check out the sites with the winning entries available and learn about great newswriting from the best in the industry.\n\nfor this is the site for information on prize and information about how to submit an\n\nThe prize winning photos and cartoons are the individual prize announcement pages. The able on the site in June.\n\nhttp://www.inform.umd.edu:8080/News/AJR/\n\nInformation about a variety of journalism Review's annual special advertising section.\n\nthe Year Contest: http://www.missouri.edu/\n\nPictures of the Year is one of the world's oldest and largest photojournalism competitions. This site shows the annual winners and information about the judges and judging of the award.\n\nRTNDA/Radio and Television News Directors Association: http://www.rtnda.org/rtnda/awards.html\nThis comprehensive listing of major electronic journalism awards, grants and fellowships provides information on more than 70 awards or programs for journalists.\n\nAmerican Association for the Advancement of Science: Science Journalism Awards: http://www.aaas.org/communications/awards.htm\nThis award for excellence in science journalism (print or broadcast) is sponsored by the Whitaker Foundation. The site tells about about applying and links to the 1995 winners.\n\nAPME (Associated Press Managing Editors) Awards: http://www.apme.com/apmewin.htm\nThis listing of APME awards links to a page with information about the winners and finalists. Entire winning entries are available.\n\nSND (Society for Newspaper Design) Awards: http://www.medill.nwu.edu/snd/contest17/winners.html\nThis is a listing of awards for excellence in newspaper design.\n\nNora Paul can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-heres-a-tip-if-youre-trying-to-find-a-listserv","slug":"1996-may-heres-a-tip-if-youre-trying-to-find-a-listserv","url":"https://dwillis.github.io/uplink/article/1996-may-heres-a-tip-if-youre-trying-to-find-a-listserv.html","year":1996,"month":"May","monthNum":5,"headline":"Here's a tip if you're trying to find a listserv","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article provides guidance on locating listserv mailing lists and usenet discussion groups through several online resources including Tile.Net, Nova University's Inter-Links, and MIT's RTFM repository.","keywords":["listserv","discussion","resources"],"topics":["Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","FTP","Listserv"],"full_text":"Here's a tip if you're trying to find a listserv mailing list or usenet discussion group: Try Tile.Net at http://tile.net. It has lists of discussion groups and listservs. Nova University also has a good site at http://www.nova.edu/Inter-Links/listserv.html. Another place where you can find those lists, and tons and tons of other information about the online world, is at RTFM (for \"Read the Friendly Manual\") at MIT. It's at ftp://rtfm.mit.edu. It's not a pretty interface (it's general ftp navigation), but it has mountains of great information.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-from-page-one-finding-fertility-fraud","slug":"1996-may-from-page-one-finding-fertility-fraud","url":"https://dwillis.github.io/uplink/article/1996-may-from-page-one-finding-fertility-fraud.html","year":1996,"month":"May","monthNum":5,"headline":"From page one: Finding fertility fraud","kicker":"","author_name":"Ernie Slone","author_title":"Reporter","affiliation":"Orange County Register","summary":"Orange County Register reporters uncovered illegal egg transfers at a fertility clinic without patients' consent, using creative investigative techniques including database construction and public records searches to identify victims and locate a fugitive doctor.","keywords":["fertility","investigation","records"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Access","Paradox","World Wide Web","askSam"],"full_text":"ferring these eggs to other barren women. The catch: The eggs were being transferred without the patients' consent, and some of the illegal transfers had resulted in live births.\n\nHospital roadblocks\n\nRegister reporters Kim Christensen, Michelle Nicolosi, Ernie Slone and Kelleher had to go to great lengths to obtain the medical records of the allegedly criminal procedures that the doctors were performing.\n\nSome sources returned records anonymously by way of regular mail. Others used what investigations editor Jim Mulvaney called a \"modern age Transom system,\" returning records to members of the team by way of self-addressed Federal Express envelopes. Kelleher even worked out a system where sources left records in the unlocked trunk of her car.\n\nOnce validated, the records were turned over to Slone, who, using Access and Paradox, built a database of patients who had donated or received eggs through procedures performed by the accused doctors. Some records clearly showed corruption, while others took more investigation. But without Slone's database, the information would have been unwieldy.\n\nUse local resources\n\nSlone's work was particularly handy when it came to identifying patients. Using local resources such as voter registration records, motor vehicle records and Social Security numbers, Slone was able to find matches for a good number of the patients in his database.\n\n\"When we went to these women, we were going to be changing their lives,\" said Mulvaney. \"Most of the people didn't even know they were being victimized.\"\n\nThat meant the Register staff would have to make absolutely sure they were contacting the right women.\n\nThe team used commercial databases, such as CDB Infotek and Information America to further identify patients. This allowed them access to local census and advertising databases.\n\nSlone was then able to target certain women, such as those between the ages of 25 and 45 and those in higher income neighborhoods, to find additional matches among them. These women were far more likely candidates for in vitro fertilization and could possibly afford the $10,000 price tag. The extra step helped save the reporters from the potential disaster of contacting the wrong patient. \"It's the ultimate invasion of privacy,\" added Mulvaney. \"You have to be relatively certain.\"\n\nCaught in the web\n\nAs the team continued to follow the case, the reporters began posting their articles on the Register's World Wide Web site (http://www.ocregister.com).\n\nThey expected it to be an excellent resource for anyone who had heard about the case and wanted to know more. What they didn't expect was for it to help them locate Ricardo Asch, the primary doctor under investigation.\n\nAsch and one of his partners had fled the country under the pressure of the Register articles and five federal probes into the allegations. The health minister of Mexico called saying he had seen the Register's Web site and knew that Asch was trying to apply for a license to practice in Mexico.\n\nThe team subsequently sent an undercover reporter to check out the story.\n\nNot just for numbers\n\nAfter eight months, the Register's project produced some startling numbers: 61 women were identified as fraud victims, 10 children were born from stolen eggs, and 32 patients had filed lawsuits against the doctors.\n\nOn Jan. 19, in Tijuana, Asch gave a four-day deposition in response to those lawsuits, and Slone drew on his computer-assisted instincts once again: He electronically transferred transcripts from the depositions to the Register, where they were entered into a text database program called askSam.\n\nReporters queried the database for specific information from the deposition. This afforded the Register a major advantage over the competition.\n\n\"We were able to get straight to the exact facts of the day,\" said Mulvaney, who helped edit the 300 pages of transcripts per day. \"Meanwhile, the Los Angeles Times' reporters were still writing off of the lawyers' claims.\"\n\nErnie Slone can be reached at (714) 835-1234, or send e-mail to eslone@link.freedom.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-growing-collection-of-federal-databases-from-the-nicar-library","slug":"1996-may-growing-collection-of-federal-databases-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-may-growing-collection-of-federal-databases-from-the-nicar-library.html","year":1996,"month":"May","monthNum":5,"headline":"Growing collection of federal databases From the NICAR library","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers a comprehensive collection of federal government databases covering campaign finance, healthcare, transportation, crime, loans, and other areas from various agencies, available through CD subscription or direct contact.","keywords":["databases","federal","government"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"Growing collection of federal databases\n\nFrom the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW. A monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\n\nNEW The Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\n\nCoast Guard boating accidents, 1969-1994.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\n\nFederal procurement data, 1992-1994, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records.\n\nNational Bridge Inventory System data, includes inspection grades.\n\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\n\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1994.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nNational Endowment for the Arts, grants, 1989-1993.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-bits-bytes-and-barks","slug":"1996-may-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1996-may-bits-bytes-and-barks.html","year":1996,"month":"May","monthNum":5,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This newsletter highlights IRE journalism contest winners, available government census data for investigative reporting, staff promotions at NICAR, IRS criminal prosecution disparities across regions, and online resources for journalists interested in computer-assisted reporting.","keywords":["journalism","data","reporting"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["CD-ROM","Internet","World Wide Web","CompuServe"],"full_text":"Check out the IRE contest home page\n\nRead the abstracts of the 1995 IRE contest winners and finalists. The site also includes winners going back to 1979, information on how to enter, and background on the judges for the 1995 contest. The site is at http://www.ire.org/contest\n\nData to think about\n\nYou can find more than 85,000 governmental bodies in the census of governments. In addition to states, counties, and cities, the census includes every taxing entity from sewer districts to school district to library districts.\n\nThe census looks at debt, property values, revenue and expenditures. The data includes the source of the revenue and purpose of the expenditure. There is also a directory file identifying all U.S. governmental units.\n\nThe census is conducted every five years in years ending in \"2\" and \"7.\" The most recent year for which all the data is available is 1987. However, the preliminary finance file for 1992 is available on CD-ROM from the Census bureau.\n\nTo order the data, call the Census Bureau's customer service line at (301) 457-4100 Your state data center should also offer the data, probably at a cheaper rate.\n\nNICARIANS moving on\n\nGlaston Ford, a NICAR graduate researcher, has accepted a job as a computer-assisted reporter at the Corpus Christi Caller-Times.\n\nAlso, staff member Natalya Shulyakovskaya has accepted a position with the St. Louis Post-Dispatch's computer-assisted reporting program.\n\nTaxed to the limit?\n\nTaxpayers in some American communities are at least 10 times more likely to face criminal charges from the Internal Revenue Service than those living in others, according to a new analysis of internal government records.\n\nDetailed data obtained by Syracuse University's Transactional Records Access Clearinghouse (TRAC), also show that the chances the IRS will recommend criminal charges against anyone are extraordinarily remote: only 17 per million. (The FBI says you're five times more likely to be murdered.)\n\nTRAC's data is available via the web for reporters who want to do a CAR story on what's happening in their area. Reporters can download the data for individual federal districts.\n\nThe Transactional Records Access Clearinghouse is a data research organization connected with Syracuse University. The co-directors of TRAC are David Burnham, a former investigative reporter with the New York Times, and Susan Long, a professor in Syracuse's School of Management who has specialized in IRS issues for more than 25\n\nKeep up with NICAR online\n\nSubscribe to our listserv and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURI.EDU In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.\n\nIRE and NICAR are also accessible through CompuServe's Journalism Forum. Go to the JForum, Section 19. Also look into the IRE/NICAR files in Library 19.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-check-out-the-ire-contest-home-page","slug":"1996-may-check-out-the-ire-contest-home-page","url":"https://dwillis.github.io/uplink/article/1996-may-check-out-the-ire-contest-home-page.html","year":1996,"month":"May","monthNum":5,"headline":"Check out the IRE contest home page","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The IRE website provides information on the 1995 investigative reporting contest winners and finalists, along with historical contest data dating back to 1979, entry guidelines, and judge backgrounds.","keywords":["investigative","contest","reporting"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Read the abstracts of the 1995 IRE contest winners and finalists. The site also includes winners going back to 1979, information on how to enter, and background on the judges for the 1995 contest. The site is at http://www.ire.org/contest","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-data-to-think-about","slug":"1996-may-data-to-think-about","url":"https://dwillis.github.io/uplink/article/1996-may-data-to-think-about.html","year":1996,"month":"May","monthNum":5,"headline":"Data to think about","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The U.S. Census of Governments surveys over 85,000 governmental bodies including states, counties, cities, and various taxing districts, collecting data on debt, property values, revenue, and expenditures every five years. Complete data is available through 1987 with 1992 preliminary data on CD-ROM, obtainable from the Census Bureau or state data centers.","keywords":["census","governments","revenue"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"You can find more than 85,000 governmental bodies in the census of governments. In addition to states, counties, and cities, the census includes every taxing entity from sewer districts to school district to library districts.\n\nThe census looks at debt, property values, revenue and expenditures. The data includes the source of the revenue and purpose of the expenditure. There is also a directory file identifying all U.S. governmental units.\n\nThe census is conducted every five years in years ending in \"2\" and \"7.\" The most recent year for which all the data is available is 1987. However, the preliminary finance file for 1992 is available on CD-ROM from the Census bureau.\n\nTo order the data, call the Census Bureau's customer service line at (301) 457-4100 Your state data center should also offer the data, probably at a cheaper rate.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-nicarians-moving-on","slug":"1996-may-nicarians-moving-on","url":"https://dwillis.github.io/uplink/article/1996-may-nicarians-moving-on.html","year":1996,"month":"May","monthNum":5,"headline":"NICARIANS moving on","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"Two NICAR-affiliated journalists have accepted positions at major newspapers: Glaston Ford joins the Corpus Christi Caller-Times and Natalya Shulyakovskaya joins the St. Louis Post-Dispatch's computer-assisted reporting program.","keywords":["journalists","positions","NICAR"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"NICARIANS moving on\n\nGlaston Ford, a NICAR graduate researcher, has accepted a job as a computer-assisted reporter at the Corpus Christi Caller-Times.\n\nAlso, staff member Natalya Shulyakovskaya has accepted a position with the St. Louis Post-Dispatch's computer-assisted reporting program.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-taxed-to-the-limit","slug":"1996-may-taxed-to-the-limit","url":"https://dwillis.github.io/uplink/article/1996-may-taxed-to-the-limit.html","year":1996,"month":"May","monthNum":5,"headline":"Taxed to the limit?","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A Syracuse University research group analyzed IRS criminal prosecution data and found that taxpayers in some American communities face charges at dramatically different rates, though the overall likelihood of criminal charges remains extremely low at 17 per million.","keywords":["IRS","prosecution","disparities"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Internet","World Wide Web"],"full_text":"Taxpayers in some American communities are at least 10 times more likely to face criminal charges from the Internal Revenue Service than those living in others, according to a new analysis of internal government records.\n\nDetailed data obtained by Syracuse University's Transactional Records Access Clearinghouse (TRAC), also show that the chances the IRS will recommend criminal charges against anyone are extraordinarily remote: only 17 per million. (The FBI says you're five times more likely to be murdered.)\n\nTRAC's data is available via the web for reporters who want to do a CAR story on what's happening in their area. Reporters can download the data for individual federal districts.\n\nThe Transactional Records Access Clearinghouse is a data research organization connected with Syracuse University. The co-directors of TRAC are David Burnham, a former investigative reporter with the New York Times, and Susan Long, a professor in Syracuse's School of Management who has specialized in IRS issues for more than 25","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-may-keep-up-with-nicar-online","slug":"1996-may-keep-up-with-nicar-online","url":"https://dwillis.github.io/uplink/article/1996-may-keep-up-with-nicar-online.html","year":1996,"month":"May","monthNum":5,"headline":"Keep up with NICAR online","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Instructions for subscribing to journalism discussion listservs through email and accessing IRE/NICAR resources via CompuServe's Journalism Forum.","keywords":["journalism","listserv","subscribe"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","modem/BBS","CompuServe"],"full_text":"Subscribe to our listserv and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURI.EDU In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on first line, write: subscribe ire-1 your name.\n\nIRE and NICAR are also accessible through CompuServe's Journalism Forum. Go to the JForum, Section 19. Also look into the IRE/NICAR files in Library 19.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-oops-uplink-update","slug":"1996-june-oops-uplink-update","url":"https://dwillis.github.io/uplink/article/1996-june-oops-uplink-update.html","year":1996,"month":"June","monthNum":6,"headline":"Oops! Uplink update","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Computer-assisted reporting enables journalists to produce detailed, fact-rich stories but also increases the speed at which errors can be made; this issue explores common CAR mistakes and strategies for data integrity checks.","keywords":["reporting","data","errors"],"topics":["Investigative Methods","Public Safety","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Oops!\n\nUplink update\n\nTo err is human; to forgive, divine; and to print a correction, humiliating.\n\nWhile computer-assisted reporting allows us to write industrial-strength stories loaded with facts and details, it also lets us make eye-popping errors at light speed.\n\nIn this issue of Uplink, reporters share their own CAR mistakes in hopes of preventing the rest of us from repeating their errors. Plus, NICAR's Brant Houston and the AP's Drew Sullivan share tips on performing integrity checks that may head off data catastrophes.\n\nAlso in this issue, Penny Loeb describes how U.S. News and World Report used train data to examine railroad safety problems around the country, NICAR's Andy Lehren reveals how reporters exploited FAA data after the ValuJet DC-9 crash, and David Milliron of Gannett News Service shares some handy Web sites.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-errors-can-wreck-stories","slug":"1996-june-errors-can-wreck-stories","url":"https://dwillis.github.io/uplink/article/1996-june-errors-can-wreck-stories.html","year":1996,"month":"June","monthNum":6,"headline":"Errors can wreck stories","kicker":"The perils of computer-assisted reporting","author_name":"Brant Houston","author_title":"managing director","affiliation":"NICAR","summary":"A correction by the Atlanta Constitution Journal highlights the risks of computer-assisted reporting, as the newspaper incorrectly ranked Atlanta's violent crime rate first when it actually ranks in the top 5 due to missing data entry. The article discusses common data errors in journalism and provides examples of mistakes made during CAR investigations.","keywords":["data","errors","reporting"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"The recent correction of a page-one story by the Atlanta Constitution Journal was a nasty reminder of how easy it is to make a critical error when doing computer-assisted reporting.\n\nThe newspaper reported that Atlanta, on the eve of the Olympics, has the highest violent crime rate among United States cities. In fact, Atlanta is in the Top 5, but it is not No. 1. The newspaper accidentally left out several major cities when typing in data.\n\nThis certainly is not the first mistake made while doing a CAR story (or any news story). Mistakes have been made since journalists began using software to collect and analyze data. Some mistakes have made it to print or on air. Many did not because the journalists were aware where things could go awry and because they followed rudimentary guidelines for preventing errors.\n\nData snafus\n\nTo get an idea of some of the data perils out there, NICAR recently asked journalists for examples of errors and snafus. Here are a few of them:\n\nThe Miami Herald was examining the sentencing patterns for drunken drivers, using court files. The data showed a category for jail time and a category for a fine. In looking at the data, the Herald deduced that a small percent of drunken drivers received no punishment.\n\nIn fact, judges always gave some kind of punishment, including community service. But there was no field in the data for community service. The clerks using the data knew \"0\" for jail time and \"0\" for fine meant community service, but no one put the information in the code book for the database, and no one told the Herald about the lack of codes. In another story, the Herald examined problems in local schools and in-\n\nBrant Houston can be reached at (573) 882-1984, or send e-mail to brant@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-checking-data-integrity","slug":"1996-june-checking-data-integrity","url":"https://dwillis.github.io/uplink/article/1996-june-checking-data-integrity.html","year":1996,"month":"June","monthNum":6,"headline":"Checking data integrity","kicker":"Tech Tip","author_name":"Drew Sullivan","author_title":"","affiliation":"Associated Press","summary":"NICAR instructor Richard Mullins emphasizes that all databases contain flaws and errors, such as data entry mistakes by government workers and incomplete information that even skilled designers cannot fully prevent.","keywords":["databases","data","journalism"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"A pair of mantras that NICAR's Richard Mullins teaches his students at the Missouri School of Journalism is: \"All databases are bad,\" and \"All data is dirty.\" If you've worked with many databases, you soon realize the truth in these statements. Underpaid and overworked government data-entry clerks will forever confuse similar postal codes, such as Missouri (MO) and Montana (MT) or Connecticut (CT) and Colorado (CO). And even the most clever database designers cannot account for incomplete data.\n\nDrew Sullivan can be reached at (800) 845-8450, ext. 7639, or send e-mail to drew@ap.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-crash-course-on-plane-crashes","slug":"1996-june-crash-course-on-plane-crashes","url":"https://dwillis.github.io/uplink/article/1996-june-crash-course-on-plane-crashes.html","year":1996,"month":"June","monthNum":6,"headline":"Crash course on plane crashes","kicker":"","author_name":"Andy Lehren","author_title":"NICAR staff","affiliation":"NICAR","summary":"Journalists covering the ValuJet plane crash used FAA databases and aviation records to investigate the accident, but encountered challenges including data quality issues, inconsistent reporting practices by airlines, and the need to cross-reference multiple sources for accurate information.","keywords":["aviation","databases","investigation"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":[],"full_text":"Crash course on plane crashes\n\nBy Andy Lehren\nNICAR staff\n\nWhen the ValuJet DC-9 crashed near Miami, many reporters poured through airline data to find out what went wrong.\n\nA common step was taking the airplane's tail number and running it through key Federal Aviation Administration databases, such as Service Difficulty Reports and Accidents and Incidents. Some went further, using the airplane's serial number and studying its history before ValuJet began using the jet.\n\nHere are some lessons from reporters who mined data for covering the crash:\n\nprepared. The Cleveland Plain Dealer's Beth Marchak had been reporting about ValuJet's problems before the crash. She began by sorting SDR data by date, by tail number, by serial number and by the troubled part of the plane. She checked for structural work. And she used her search results as a springboard for additional reporting. SDRs helped prepare her for interviewing passengers and airline workers. They also helped her craft Freedom of Information requests for FAA documents. She said paper records were vital for showing the depth of ValuJet's problems.\n\nWatch out for dirty data. For instance, in the FAA Accidents and Incidents database, Delta had mistakenly reported the fatal DC-9's serial number as its tail number. Severity codes in the SDR database are inconsistent. The FAA violates its own rules for the key fields in the SDR table. Operator codes in the Accidents database are often missing.\n\nBeware of how information is entered into the database. The SDR database allows the airline to report one character for the precautionary procedure taken if the plane was in the air. The St. Paul Pioneer Press' Dan Browning found Northwest often did not use the code for dumping fuel. But the airline routinely dumped fuel when facing a problem in the air. The Seattle Times's Bryan Acohido reported about rudder problems in Boeing 737s and found those difficulties were not always coded the same way (see Uplink, February 1995).\n\nLook how often an airline submits reports. They are supposed to report any mechanical difficulty. They often don't. Airlines report at different rates. Browning said he tries to adjust for reporting rates. He also looks at what the airline owns by using the FAA's Annual Inventory of Airframes and Aircraft Engines by Carrier.\n\nBut the problem raises questions about how the agency works. \"Why does the FAA tolerate the disparity?\" Browning asked. The University of North Carolina's Phil Meyer said that, with this problem, a lack of reports may well tip off a reporter to probe into an airline.\n\nLook at other databases. The ValuJet crash was linked with a hazardous chemical. The U.S. Department of Transportation's Hazardous Materials database includes airline accidents that involve hazardous chemicals. The Miami Herald used this information to put the crash into context, and show how rarely such accidents happen. It also gave readers a chronology of earlier accidents involving such substances.\n\nDateline NBC's David Hinchman used the FAA's Enforcement Information System to examine lax enforcement in commuter airline safety (see Uplink, April 1996).\n\nCBS's Roberta Baskin recommended using NASA's Aviation Safety Reporting System, where pilots make anonymous reports on problems. By using dates, she has been able in other cases to strip away that anonymity and learn more about an incident.\n\nThe FAA's collection of advisories showed the agency was already concerned about bad wiring in the cockpits of DC-9s. National Transportation Safety Board records are often valuable for understanding airline safety, said the New York Times' Steve Engleberg.\n\nKeep in mind that human errors play a big role in crashes. The Minneapolis Star Tribune's Greg Gordon recommended that, along with other databases, include a look at FAA records for runway incursions (when mistakes happen on landings and takeoffs), pilot diversions (when pilots make mistakes), and near mid-air collisions. These can provide great anecdotal information.\n\nLook overseas. Newsday's Ford Fessenden used Aviation Information Services Ltd. and Flight International, both in the United Kingdom, to probe airline accidents worldwide. For tracking who owns planes outside the United States, he uses a book called the Jet Airliner Production List, published in Great Britain, and confirms information with the airplane's makers. Track ownership and major changes to an aircraft. The AP's Drew Sullivan checked FAA ownership and airworthiness certification records in Oklahoma City. Major repair records also include information for developing sources, including the name of the engineer who signed off on the work.\n\nAndy Lehren can be reached at (573) 882-0684, or send e-mail to andy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-playing-by-the-numbers","slug":"1996-june-playing-by-the-numbers","url":"https://dwillis.github.io/uplink/article/1996-june-playing-by-the-numbers.html","year":1996,"month":"June","monthNum":6,"headline":"Playing by the numbers","kicker":"Indices illuminate data","author_name":"Andy Lehren","author_title":"NICAR staff","affiliation":"NICAR","summary":"Journalists can develop airline safety indices using FAA data to help predict crashes and measure relative safety across carriers, moving beyond simple rankings to create meaningful barometers for readers. The approach involves analyzing historical accident patterns, pilot errors, maintenance issues, and safety culture to identify which factors best predict future aviation accidents.","keywords":["aviation","indices","safety"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Playing by the numbers\n\nBy Andy Lehren, NICAR staff\n\nThe holy grail in aviation data is figuring out what predicts accidents.\n\nWhat causes plane crashes: The airline? The type of aircraft? Pilot mistakes? Poor maintenance? Violent weather? How can reporters sift through patterns when so much changes over time - airlines change management, the aircraft they fly, and even owners.\n\nAfter the ValuJet crash, a challenge remains in trying to develop an airline safety index. And even if this is not successful, University of North Carolina Professor Phil Meyer, author of \"The New Precision Journalism,\" said this problem sheds light on how to think about developing an index.\n\nIndices are no stranger to news pages. Look at the Dow Jones Industrial Average or the Standard & Poor's 500. Both are used daily to reflect the stock market's health. The Philadelphia Inquirer's Neill Borowski wrote in April's Uplink how he indexed various local economic segments against the nation. That helped him determine the Philadelphia area's strengths and weaknesses. Baseball fans track how many games back a team is from the leaders.\n\nMore than ranking\n\nThe idea is that these numbers do more than rank things. They are barometers. They show relative strength or weakness. A baseball fan knows more than whether a team is in second place; games back tell whether the team is close or far from the division leader. The S&P 500 indicates how much better or worse the stock market is doing. The index used by Borowski showed how much the local economy's makeup is different.\n\nAt Newsday, Ford Fessenden had previously looked at accidents worldwide to show a history of airline crashes, including predecessors in the numbers for the airlines that took them over, and included information such as the odds of a plane crashing. That, he said, showed readers that foreign carriers in certain countries have troubling accident rates.\n\nThe stories also underscored how human mistakes - and companies with a culture that did not emphasize safety and training play a great role in crashes.\n\nMeyer suggested reporters can try to go one step further. The goal would be to create a safety index for airlines, gauging the likelihood that someone would get killed riding a particular airline. It would include an analysis looking at whether factors in older crashes predict more recent accidents.\n\nDrawing from the data\n\nAn index would use one or more factors for measuring safety. Journalists look to Federal Aviation Administration data such as Service Difficulty Reports, which include equipment problems, engine flame outs and emergency landings. They also look at the FAA's Accidents and Incidents reports, which include when people are hurt or killed, or the airplane suffers major damage.\n\nThe FAA also keeps information on accident rates, and one recent report, separating start-up carriers from major airlines, claimed there is a difference between the two groups. The report ranks airlines by accident rates, but does not include a statistical analysis.\n\nReporters have already documented airlines failing to report SDRs. Instead of representing the whole universe of U.S. airline problems, or even a random sample, the database is skewed because of contrasting reporting rates. Airlines are required to file SDRs when they repair aircraft, but reporters have found many don't comply with FAA rules, and the agency lacks the teeth for enforcing regulations.\n\nRichard Newman of U.S. News & World Report is among those reporters who believe SDRs could never be part of an index on airline safety. In an unpublished analysis, St. Paul Pioneer Press' Dan Browning attempted to adjust SDR data by looking at reporting rates. The Cleveland Plain Dealer reported nine of 10 repairs were unreported in SDRs (See Uplink, February 1995). Even when reports are made, key information may be missing. Details may not be coded consistently. Accidents and Incidents data also suffer from missing information, though many reporters agree that accident data appears more complete.\n\nWhat if a reporter did not face these and other problems, Meyer said they could begin the quest by looking at whether fatal air crashes and major repairs during the 1980s predict accidents during this decade. \"It may be that the past doesn't predict the future,\" Meyer said. \"If it doesn't, there is no reason to look at an index.\" But if it did, reporters could also look at how well several factors such as aircraft age and the number of most severe SDRs - fit together. One test for reliability is called Chronbach's alpha, which looks at whether factors move together consistently. (In SPSS 6.1, with professional statistics, go to statistics scale reliability analysis). Those who attended the 1996 Advanced Computer-Assisted Reporting Bootcamp in North Carolina used it for studying polling data. The most robust indices, Meyer said, are those that still predict even when some elements are stripped away.\n\nBecause the FAA's inspector general raised so many concerns about ValuJet before the crash, Meyer said the quest for developing a safety index is worth pursuing. But one problem - for data hounds, not passengers - is the few number of crashes in U.S. aviation. That means an analysis trying to predict accidents is fragile. To get an idea of what that means, if you changed an airline's accident total just by one crash, it could drastically alter results. The FAA noted that in its recent report on start-up carriers. One way to shore up that problem is to also look at severe mishaps and other incidents. But those records - along with those on pilot mistakes and other human errors - are less complete. A recent New York Times story noted these problems for developing a safety predictor. Trying to use airline data to do this kind of analysis, said the Minneapolis Star Tribune's Greg Gordon, \"is a lot like squeezing molasses and watching it go in all directions.\"\n\nAndy Lehren can be reached at (573) 882-0684, or send e-mail to andy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-erratic-enforcement-found","slug":"1996-june-erratic-enforcement-found","url":"https://dwillis.github.io/uplink/article/1996-june-erratic-enforcement-found.html","year":1996,"month":"June","monthNum":6,"headline":"Erratic enforcement found","kicker":"IRS prosecution checked","author_name":"John Sullivan","author_title":"NICAR staff","affiliation":"NICAR","summary":"An investigation into IRS prosecution data reveals significant geographic disparities in tax fraud prosecutions, with some regions like Roanoke, Virginia experiencing dramatically higher prosecution rates than others, raising questions about the consistency and fairness of IRS enforcement practices.","keywords":["IRS","prosecution","disparities"],"topics":["Business & Economy","Investigative Methods","Government & Politics"],"technologies":["Internet"],"full_text":"Sleepy Roanoke Virginia has never been associated with rampant tax fraud. But if you happen to live there, you are 57 times as likely to be recommended for prosecution by the IRS than if you lived in New Mexico. The likelihood of prosecution in Pittsburgh is 29 times greater than that of all of Idaho, according to \"Where the IRS Aims the Prosecution Ax,\" a recent New York Times article by David Cay Johnston. Johnston documented capricious IRS prosecution after examining Justice Department data on 4,542 cases in which the IRS sought prosecution in 1994. The data, which was stripped of taxpayers' names, was made available to reporters on the Internet by Susan Long and David Burnham, co-directors of the Transactional Records Access Clearinghouse (TRAC) at Syracuse University.\n\nJohnston says the data raises some serious questions about the disparity in the way the IRS pursues tax cases, such as the case of Albert Taggi of Scarsdale, N.Y., who took a \"voluntary\" buyout from AT&T Corp. in 1985 along with 11 other engineers and middle managers.\n\nAccording to Johnston, the 12 involved, all residents of New York, filed for refunds of income taxes on the buyout and got varying results: Some paid no taxes. Some paid partial taxes. And two, including Taggi, were taxed on the full amount. The federal appeals court that heard the case said it was irrelevant that the 12 were taxed differently.\n\nAnalysis of the data indicates Taggi's case may be closer to the rule than the exception, although IRS and Justice Department officials Johnston interviewed disagreed.\n\n\"Officials of the IRS and the Justice Department tried to disown the data after TRAC had spent five years in a protracted lawsuit trying to gain access to it. If the data truly was flawed, the Justice Department had five years to deal with that issue,\" Johnston said. Long and Burnham argue that the data is useful in showing how the IRS carries out enforcement.\n\nJohn Sullivan can be reached at (573)882-0684, or send e-mail to john@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-whats-on-tracs-irs-web-site","slug":"1996-june-whats-on-tracs-irs-web-site","url":"https://dwillis.github.io/uplink/article/1996-june-whats-on-tracs-irs-web-site.html","year":1996,"month":"June","monthNum":6,"headline":"What's on TRAC's IRS Web site","kicker":"","author_name":"","author_title":"","affiliation":"TRAC","summary":"TRAC's IRS criminal enforcement website offers four main navigation options including roadmaps, data graphics, downloadable datasets, and topic-based, district-based, or comparative ranking views to analyze criminal referrals, convictions, and prosecution data across U.S. federal districts.","keywords":["IRS","enforcement","data"],"topics":["Data & Databases","Crime & Justice","Technology"],"technologies":["Internet","World Wide Web","MapInfo"],"full_text":"TRAC's IRS Web sight for criminal enforcement allows you to choose from four main menu choices. The first is a roadmap to the sight, which includes information about TRAC.\n\nThe second, Findings and Data Graphics on IRS, includes graphs, rankings and tables on the United States and each district, including a ranking by district.\n\nThe third is Data Sets on IRS for downloading information on each criminal referal recommending prosecution in a district, or summary information on all districts, including comparative counts, percents, rates and ranks for districts and the nation.\n\nFinally, you can also navigate by topic, district or by comparative rankings.\n\nIf you choose to navigate by topic you may view a series of graphs and tables that give an overview of IRS referrals for criminal prosecution. It also provides information about conviction and prison terms, length of time to prosecute or decline a referral, and odds of referral.\n\nNavigating by district provides essentially the same information, but allows you to choose from 90 separate federal districts and view analyses from those districts.\n\nNavigating by comparative rankings provides a series of maps displaying comparative district rankings and tables used to tabulate the rankings.\n\nTRAC's IRS Web site is located at http://trac.syr.edu/tracirs/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-tracking-rail-safety","slug":"1996-june-tracking-rail-safety","url":"https://dwillis.github.io/uplink/article/1996-june-tracking-rail-safety.html","year":1996,"month":"June","monthNum":6,"headline":"Tracking rail safety","kicker":"Answers found in the data","author_name":"Penny Loeb","author_title":"Reporter","affiliation":"U.S. News and World Report","summary":"The Federal Railroad Administration provides downloadable databases on train accidents, casualties, and grade-crossing incidents that journalists can use to investigate railroad safety records, though the data has significant limitations including a high damage threshold that omits over 90% of accidents.","keywords":["railroad","safety","database"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["MapInfo","9-track tape","World Wide Web","DBF"],"full_text":"Tracking rail safety\n\nBy Penny Loeb\n\nU.S. News and World Report\n\nA train crashes in your city. What's the accident record of that railroad? Several people in a car are killed at a crossing. How many people have been killed at crossings in your state? Some of the answers are in the databases distributed by the Federal Railroad Administration at its Web site, and also by NICAR.\n\nIn May, U.S. News and World Report and ABC's PrimeTime Live released a joint investigation of safety problems with railroads. It revealed that accidents have remained pretty steady for a decade while the FRA and railroads ignored calls for better braking systems, tracks, passenger safety and working conditions from Congress and the National Traffic Safety Board.\n\nThe FRA's user-friendly Web site has .dbf files ready for download, as well as record layouts and code lists. There are four files. One is accidents, which are available from 1991 through February 1996. Reports provide rich detail, including railroad, time, weather, place, cause, type of track, number killed, number injured, number evacuated and number of cars with hazardous materials. The one catch is that there is a record for each railroad and track owner involved in the accident, making for as many as three records for one accident. The FAQ file explains that the unique identifier is year3, month3, rr3 and incdno3. The data also has a field with the same state-county code used by MapInfo. mapped the serious accidents for December 1995, which made a compelling graphic.\n\nThere are also separate tables on casualties, grade-crossing accidents and number of miles traveled by each railroad. Casualties contains injuries to employees, trespassers, passengers and others. Grade crossings has accidents at railroad crossings. A few of these will be reported in the accident file, if damages to the train were more than $6,300. The grade crossing database also has multiple entries for some accidents, so you need to group by the same four fields.\n\nThe FRA has put the entire, very detailed 1994 Accidents/Incidents bulletin on the Web site for downloading. You can spot check your calculations against those in the report.\n\nThere are several weaknesses in the data. First, it is self-reported by railroads. Second, the accidents database only has those with damages over $6,300. The damage threshold omits more than 90 percent of the accidents. CSX, one of the largest railroads, had 104 reportable accidents in 1995, but the total number was 1,755. found a number of small spills of hazardous materials on the Hazmat database, but missing from FRA.\n\nWe got total numbers of inspections and violations from the FRA, but did not get the data. It is available on nine-track tape, but it is difficult to use because the inspection system changed in the past two years, as did some of the data. The FRA database administrator, Robert Finkelstein, is very helpful, though.\n\nIf you are contemplating a story on railroads in your area, check NTSB accident reports and recent Congressional testimony for detailed background. You will need to develop sources among rail workers, who can guide you to the worst problems. Smaller railroads often have higher accident rates.\n\nHere are some useful Web sites:\n\nRailroad Definitions: http://pavel.physics.sunysb.edu/rr/railroaddefinitions.html\nThis includes a glossary of railroad terms, such as FREDs, blue flags, bottling air.\nStone Railroad Switching: http://www.fortnet.org/%7fdk/uprrhome.htm\nThis links to many railroad sites.\nTrack Warrants: http://www.aimnet.com/~steves/n/twar/twmenu.htm\n\nThis is a personal Internet publication with information on some crashes.\n\nPenny Loeb can be reached at (202) 955-2640, or send e-mail to ploeb@usnews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-data-you-can-surf","slug":"1996-june-data-you-can-surf","url":"https://dwillis.github.io/uplink/article/1996-june-data-you-can-surf.html","year":1996,"month":"June","monthNum":6,"headline":"Data you can surf","kicker":"On the Internet","author_name":"David A. Milliron","author_title":"","affiliation":"Gannett News Service","summary":"The article examines how journalists are increasingly using the Internet as a reporting tool, highlighting both successful examples and cautionary tales of misinformation, while providing a list of free databases available to newsrooms.","keywords":["Internet","journalism","verification"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","FTP","gopher","WordPerfect"],"full_text":"When the federal building was bombed in downtown Oklahoma City, Detroit News reporter Dave Farrell turned to the Internet to show readers how bomb books are just a phone call away.\n\nFarrell had found one of the most complete encyclopedias on destruction - \"The Big Book of Mischief,\" a 40,000-word electronic tome on bomb-building, explosives and weapons of terror.\n\nWhen ValuJet Flight 592 crashed in the Florida Everglades, killing all 110 people aboard, Tennessean reporter Heather Newman turned to the Internet for a listing of the aircraft's Service Difficulty Reports. Newman was able to document specific problems experienced by the downed plane in the months prior to the crash.\n\nThese are just two examples of how the Internet can help reporters cover major news events.\n\nComputer-assisted reporting is quickly becoming as valuable to reporters as notepads and the telephone. However, although the Internet can be a valuable reporting tool, the information gained is no more accurate than it is from other sources.\n\nSome reporters and their news operations were embarrassed in the aftermath of the Oklahoma City bombing when they reported as factual two online hoaxes. Someone on America Online identifying himself as \"Timothy McVeigh\" posted several messages that tabloid TV shows - and later some newspapers - picked up and used without confirmation. McVeigh had been behind bars two days prior to the postings. On an Internet news group, a man identifying himself as a militia member from Montana posted inflammatory messages that the U.S. government was behind the bombing. The man later admitted he had posted the messages as a joke, but not before some newspapers already had reported the remarks as factual. When surfing the Internet, always be leery of dirty data. Almost every piece of data you will work with is certain to have some quirks. So, bottom line online: Check it out!\n\nHere's a list of free searchable databases any newsroom with Internet access can access:\n\nFACSNET is a service developed by the Foundation for American Communications for print and broadcast journalists. Not just another homepage with lists of Internet links, FACSNET goes deep into economics, science, business, technology and other areas that journalists cover every day. Databases include reporting tools, sources online, internet resources for journalists and top issues. http://www.facsnet.org\n\nThe Poynter Institute for Media Studies' \"Hot News/Hot Research\" homepage will feature research sources that will help you understand and cover top stories. They will be looking at a number of links, but only list the ones with some useful content. http://www.poynter.org/poynter/hrintro.html\n\nU.S. Federal and State Courts Finder is a service of the Emory School of Law. This point-and-click map to federal courts provides hypertext links to all on-line federal court sites.\nhttp://www.law.emory.edu/FEDCTS/ http://www.law.cornell.edu/opinions.html\n\nSupreme Court opinions and related documents in WordPerfect 5.1 or html format. From the HERMES project, these documents are transmitted by the court upon release and uploaded to The Federal Bulletin Board daily. http://fedbbs.access.gpo.gov/court01.htm\nhttp://www.law.cornell.edu/supct/ http://www.law.cornell.edu/syllabi ftp://ftp.cwru.edu/hermes/\n\nOne of the most complete all-in-one sites with access to several dozen state constitutions, statutes, codes, legislation, is found at http://www.law.cornell.edu/statutes.html\n\nUnited U.S. Post Office Database that actually corrects addresses and gives you ZIP+4.\nhttp://www.cedar.buffalo.edu/adserv.html http://www.ualberta.ca/~slis/guides/direct/phone.htm\nhttp://www.usps.gov/ncsc/aq-zip.html gopher://odie.niaid.nih.gov:70/77/deskref/zipcodes/index\n\nAnd, you can even look up ZIP+4 Codes for United States vessels at http://www.semaphorecorp.com/cgi/navy.html\n\nSearch college telephone books from around the world through a vast collection of searchable telephone books and other directories. gopher://gopher.nd.edu:70/11 Click on \"Non-Notre Dame Information Sources\"\nClick on \"Phone Books-Other Institutions\"\n\nReporters and editors can calculate everything from foreign exchange rates, local tides, distances between cities, taxes, loan and mortgage rates, the amount of seed needed to plant an acre of different crops and even a quarterback's passer rating on the Calculators On-line Web page. http://www-sci.lib.uci.edu/HSG/RefCalculators.html\n\nWe'll look at some more sites next month.\n\nDavid Milliron can be reached at (703) 276-5805, or send e-mail to dmilliro@gcil.gannett.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-from-the-nicar-library","slug":"1996-june-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-june-from-the-nicar-library.html","year":1996,"month":"June","monthNum":6,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering topics including campaign finance, healthcare, transportation safety, crime statistics, and small business loans, available through subscription or purchase.","keywords":["databases","government","federal"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW A monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\nNEW The Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1994, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records.\nNational Bridge Inventory System data, includes inspection grades.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1994.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nNational Endowment for the Arts, grants, 1989-1993.\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-bits-bytes-and-barks","slug":"1996-june-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1996-june-bits-bytes-and-barks.html","year":1996,"month":"June","monthNum":6,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The redesigned reporter.org website is now live, hosting professional journalism organizations and mailing lists including IRE, NICAR, and various diversity-focused journalist associations. Users can subscribe to announcement updates and provide feedback to the webmaster.","keywords":["website","journalism","organizations"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"New and Improved reporter.org\n\nThe new version of the www.reporter.org web site is up. It's located at http://www.reporter.org/index.html Among the professional journalism organizations and related mailing lists hosted by the site are:\n\nwww.ire.org (Investigative Reporters and Editors)\nwww.nicar.org (National Institute for Computer-Assisted Reporting)\nwww.aaja.org (Asian-American Journalists Association)\nwww.nabj.org (National Association of Black Journalists)\nwww.nyabj.org (New York Association of Black Journalists)\nwww.ewa.org (Education Writer's Association)\nwww.jaws.org (Journalism and Women Symposium)\n\nTo subscribe to a mailing list for announcements regarding the www.reporter.org web site, send mail to: majordomo@reporter.org\n\nIn the body of the message, type:\n\nsubscribe reporter-announce\n\nYou'll receive a confirmation of your subscription shortly thereafter. This information, as well as the archived postings to the list, is available at: http://www.reporter.org/lists/reporter-announce\n\nIf you have any comments regarding the new site or would like to become part of the reporter.org web project, send e-mail to Web Master Wallace Winfrey at wally@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-producers-wanted","slug":"1996-june-producers-wanted","url":"https://dwillis.github.io/uplink/article/1996-june-producers-wanted.html","year":1996,"month":"June","monthNum":6,"headline":"Producers wanted","kicker":"","author_name":"","author_title":"","affiliation":"WDIV-TV","summary":"WDIV-TV in Detroit is hiring three producers for its I-Team, Consumer Unit, and Medical team, requiring television news experience and investigative reporting skills for the I-Team position.","keywords":["hiring","producers","Detroit"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet"],"full_text":"WDIV-TV, the Post-Newsweek-owned NBC affiliate in Detroit, is looking for three producers, including one each for its I-Team, Consumer Unit and Medical team. All should have several years television news experience in writing and producing. For the I-Team, the candidate must have done investigative reporting in the past. Ideally, the person would also be familiar with CAR techniques and the Internet. Send tapes and resumes News Director Carol Rueppel, WDIV-TV, 550 W. Lafayette Blvd., WDIV-TV, Detroit, Mich., 48226. For questions, call Mike Wendland at (313) 222-0532, or send e-mail to mikew@wdiv.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-miami-herald-changes-job-opening","slug":"1996-june-miami-herald-changes-job-opening","url":"https://dwillis.github.io/uplink/article/1996-june-miami-herald-changes-job-opening.html","year":1996,"month":"June","monthNum":6,"headline":"Miami Herald changes, job opening","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Steve Doig, longtime Associate Editor at The Miami Herald, is departing after 19 years to join Arizona State University as a Knight Professor of Journalism. The Herald is reorganizing its staff with Dan Keating moving to replace Doig and seeking a CAR-skilled reporter for the Broward position.","keywords":["journalism","promotion","staffing"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"After more than 19 years at The Miami Herald, Associate Editor and Researcher Steve Doig is leaving to become Knight Professor of Journalism at Arizona State University, Tempe. Dan Keating will move from the Herald's Broward office to replace Doig, and the Herald is searching for a reporter with CAR skills to fill the Broward opening. Send resume and clips to: Rick Hirsch, Broward City Editor, The Miami Herald, 3325 Hollywood Blvd, Suite #102, Hollywood, Fla., 33021.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-june-new-ire-l-and-nicar-l-addresses","slug":"1996-june-new-ire-l-and-nicar-l-addresses","url":"https://dwillis.github.io/uplink/article/1996-june-new-ire-l-and-nicar-l-addresses.html","year":1996,"month":"June","monthNum":6,"headline":"New IRE-L and NICAR-L addresses","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE-L and NICAR-L discussion lists have moved to a new server at lists.missouri.edu, with updated email addresses for posts and subscription management.","keywords":["lists","migration","email"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"Effective immediately, IRE-L and NICAR-L have changed addresses. They are located on lists.missouri.edu, a dedicated Unix system designed specifically to service the over 200 discussion groups run out of the University of Missouri. All posts to the lists should now be sent to: ire-l@lists.missouri.edu or to nicar-l@lists.missouri.edu\n\nAll commands for subscription changes should be sent to: distproc@lists.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-wish-you-were-here","slug":"1996-july-wish-you-were-here","url":"https://dwillis.github.io/uplink/article/1996-july-wish-you-were-here.html","year":1996,"month":"July","monthNum":7,"headline":"Wish you were here","kicker":"Uplink update","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR dedicate this newsletter issue to international journalism resources, featuring experiences from journalists in the Netherlands, Canada, and Denmark, plus reviews of international reporting websites and an update on establishing an investigative journalism organization in Mexico.","keywords":["international","journalists","data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["World Wide Web"],"full_text":"Ola, bonjour, NPUBET, goddag and hallo. In other words, hello.\n\nIn IRE and NICAR's ongoing effort to reach international journalists, we dedicate this issue of Uplink to resources useful to reporters at home and abroad. Marjan Agerbeek of The Netherlands shares her first CAR experience. Mike Gordon of Canada tells us how he won a battle for data. Freelance reporter Gwen Carleton interviews Danish journalists about their battle for data access.\n\nNatalya Shulyakovskaya reviews the Russian Sources for Reporters homepage, part of the Hillman Project, along with other Web sites that have useful international information.\n\nAnd IRE Executive Director Rosemary Armao brings us up to date on Periodistas de Investigacion, a project made possible by the McCormick Foundation to start an IRE-like organization in Mexico.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-mexico-institute-revs-up-ique-pasa","slug":"1996-july-mexico-institute-revs-up-ique-pasa","url":"https://dwillis.github.io/uplink/article/1996-july-mexico-institute-revs-up-ique-pasa.html","year":1996,"month":"July","monthNum":7,"headline":"Mexico institute revs up iQué pasa?","kicker":"","author_name":"Rosemary Armao","author_title":"Executive Director","affiliation":"IRE","summary":"Periodistas de Investigacion aims to establish an IRE-like organization in Mexico to promote investigative journalism and CAR training.","keywords":["Mexico","journalism","IRE"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"Periodistas de Investigacion means \"Investigative Journalists\" in Spanish. It also means IRE is going international for the first time in a big and systematic way. It's risky, we know. An IRE/NICAR organization may not work for our Mexican colleagues. Our members could object to organization time spent on international networking when reporters here have so many problems. Such arguments seem familiar. Three years ago, IRE, with a good idea, a potful of money from the Freedom Forum, and a great deal of trepidation, set out to do something it had never done in its then 17-year history. The result of that venture, as we all know now, is NICAR, Uplink, thousands of journalists trained, heightened attention to the need for computer-assisted reporting at all media outlets and mounting pressure for laws that open electronic records. Here we go again, with NICAR as our model. As you read this, Lise Olsen, formerly a database specialist for The Virginian-Pilot in Norfolk, Va., a frequent IRE/NICAR speaker and a trilingual reporter, is in Mexico City renting an office and hiring staff for Periodistas de Investigacion. Olsen will be the managing director, and her major hire will be a Mexican journalist who will work with her as training director of the new organization. In three years, if this works as we expect, the training director will take over as the boss, Periodistas will be a wholly Mexican organization, and IRE could be working with colleagues in another country. Olsen, 31, is a graduate of the University of Nebraska at Lincoln. She's been at The Virginian-Pilot since 1988, except for a year she took off in 1993-94 to travel to Argentina, studying Spanish, and writing freelance articles. Since 1994, she has concentrated on securing, analyzing, and developing stories from databases. The Mexico project has been made possible by a three-year $540,000 grant from the McCormick Tribune Foundation. With that money, IRE/NICAR hopes to offer low-cost or free training in newsrooms and at seminars across Mexico and to work at setting up some of the same components that have made IRE invaluable to its 3,500 members. That includes a library of investigative projects, a newsletter and other publications, a collection of databases, and a homepage on the World Wide Web that reporters can turn to when they need help or advice on a story. Computer-assisted reporting, particularly developing databases and online research, will be a highlight of the curriculum, which is still being put together for Periodistas de Investigacion, by Olsen with help from NICAR. We foresee that Periodistas de Investigacion will widen the international network of journalists that IRE has helped nurture across the border. We expect the organization to foster bilateral investigations on such topics as immigration, drug trading, and the environment. Student and intern exchanges also have been discussed. It's an exciting time as the possibilities begin to unfold. We welcome your ideas, your advice, your warnings, and your experiences reporting in and on Mexico. In the meantime, we are learning about whole other systems of taxation, banking, news gathering, and ethics. We'll keep you informed.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-july-fishing-in-international-waters-reeling-in-web-data","slug":"1996-july-fishing-in-international-waters-reeling-in-web-data","url":"https://dwillis.github.io/uplink/article/1996-july-fishing-in-international-waters-reeling-in-web-data.html","year":1996,"month":"July","monthNum":7,"headline":"Fishing in international waters Reeling in Web data","kicker":"","author_name":"Natalya Shulyakovskaya","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"A journalism resource guide explaining challenges in accessing international data electronically and recommending key U.S. government sources including the Census Bureau, State Department, and CIA for reporters seeking global information.","keywords":["international","data","reporters"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["Internet","World Wide Web","gopher"],"full_text":"There are a few problems with getting international data electronically: It's spotty and not too many countries collect it themselves let alone put it on the Web. So, we have to rely on either international organizations or U.S. sources to collect our beloved numbers.\n\nMy attempt to put together information on some international data and sources of information resulted in Russian Sources for Reporters, part of the Hillman Project, at www.reporter.com/hillman/russian/ The pages are the seed of a plan to develop an international tip sheet and library of resources about Russia for English-speaking reporters. The second half of the project is to develop a collection of resources in the Russian language for Russian journalists writing about the United States. A third goal is to provide information about investigative and computer-assisted reporting in Russian. Since most international data cover all foreign countries, these pages are not limited to information about the new states of Eastern Europe. I couldn't possibly find and list all available information, but there are listservs and other human resources that you can use to dig further.\n\nIn the meantime, here are some useful sites: If you are on a fishing expedition, the good old Census Bureau is the first place to check. Its International Program Center (www.census.gov/ipc/www/) lists several databases covering foreign countries.\n\nMy favorite is the International Database (IDB), which covers a full bouquet of demographic and socio-economic data. It goes into such details as prevalence of contraceptives used in different countries and number of babies born to women of different age groups (look out for those teen-agers).\n\nA word of warning: Be careful when talking to bureaucrats about international data: One well-meaning woman at the Department of Commerce tried to sell me IDB for $40 while the data was sitting nicely on the department's site.\n\nThe Census Foreign Trade Division (www.census.giv/foreign-trade/www/) is a good place to pick up statistics on international trade and U.S. trade surpluses and deficits with its top trading partners. Census also has data on immigration and numbers of foreign-born U.S. population.\n\nThe State department dishes out some really good reports (skip the travel advisories) (http://www.state.gov/www/about_state/dos_welcome.html, bottom of the page). One is on terrorism patterns in 1995, another one is on country background and international organizations reports, and another on U.S. Assistance to the New Independent States for FY 1996.\n\nThe CIA's World Fact Book is another good source. The CIA compiles this every year. It describes each country's political system, major players, land area, environmental laws, life expectancy and other crucial bits of information in flat text. The CIA has it in a database, but does not release it as such. Find it and other CIA publications at its homepage (http://www.odci.gov/cia).\n\nAnother great source for international data is the World Health Organization (www.who.ch or gopher://gopher.who.ch:70/11/). A list of the WHO's databases and links to them are at http://www.who.ch/whosis/whosis.htm#databases. Most of the data come in fixed-width text format. World Health Organization - Epidemic Info-Incidents is a great source of data on outbreaks of tuberculosis, measles, etc., in different countries. Data are in text format, tabular, fixed length, with some text mixed in.\n\nUN Criminal Justice Country Profiles (gopher://uacsc2.albany.edu:70/11/newman/crjdoc) is another valuable source, especially with the crime and prison explosions that nations around the world are experiencing. It describes nations and their criminal justice and prison systems. See the State Department's reports on human rights practices in different countries, too.\n\nAgricultural/Trade Databases from USDA (gopher://mann77.mannlib.cornell.edu:70/11/data-sets/international) is a good source for agro-economic information. Data comes from the USDA's Economic Research Service. You can download it, but the documentation also tells how to order data on a disk. Tables are readable in Excel. Russian commodity prices are there, as well as a bunch of European and African data.\n\nMost of the pages containing data description (some systems refer to it as metadata) also list names, phone numbers and e-mail addresses of researchers who collect the information. I didn't have much luck e-mailing them, though.\n\nNatalya Shulyakovskaya can be reached at (800) 365-0820 ext. 8183, or send e-mail to natalya@pd.stinet.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-danes-battle-for-access","slug":"1996-july-danes-battle-for-access","url":"https://dwillis.github.io/uplink/article/1996-july-danes-battle-for-access.html","year":1996,"month":"July","monthNum":7,"headline":"Danes battle for access","kicker":"Fighting for data","author_name":"Gwen Carleton","author_title":"","affiliation":"Freelancer","summary":"Danish journalists Nils Mulvad and Flemming Svith are working to increase access to government electronic databases in Denmark, where strict data access laws currently restrict information requests to those who already know what they're looking for, though they achieved a breakthrough agreement in February to open electronic indices to journalists by fall.","keywords":["Denmark","databases","journalism"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Excel","Access","Paradox","SPSS","Internet"],"full_text":"Does it ever seem all of modern bureaucracy is standing between you and the database you desire?\n\nTake heart. At least you don't live in Denmark.\n\nThe country known for Hans Christian Andersen, big government and interesting furniture has another claim to fame: It has some of the most restrictive data access laws in northern Europe.\n\nIn Denmark, obtaining government data in print form works like this: Journalists interested in a particular issue approach the relevant government agency, describe the issue and, hopefully, receive information, explained Nils Mulvad and Flemming Svith, two Danish journalists who attended NICAR's March bootcamp.\n\nNo fishing allowed\n\n\"In Denmark, you can only get access to information if you know quite a bit about the issue already,\" Mulvad said. In other words, no fishing expeditions allowed.\n\nOnce the journalist and bureaucrat agree upon the issue under discussion, the journalist must be concise. In the register of deeds' office, for example, \"you may only ask three questions, then you must come back the next day,\" Mulvad said.\n\nIt is not surprising then that electronic information is strictly off limits. But Mulvad and Svith, reporters for Jyllands-Posten, Denmark's largest national daily, are working to open up the country's trove of public data.\n\nThe task has not been easy. Mulvad and Svith are part of a group of journalists, editors and journalism educators that has been negotiating with government officials for months. But in February, they reached a breakthrough: The government agreed to consider opening electronic indices of files, mail and other official documents to journalists.\n\nSuch access - which includes only lists of documents, not the documents themselves - is likely to begin this fall, Mulvad said. \"It was really exciting after that meeting. Before then, it was like the conversation was taking place in two different rooms.\"\n\nThe prospect of improved openness is exciting for Danish journalists because of the sheer volume of information gathered and archived by the government. The educational system, health care and the national transportation system are just a few of the powerful industries operated by the Danish government.\n\nThe country's generous social welfare system requires further government record keeping. Some citizens appear in government databases because of what they receive, such as unemployment insurance or student stipends. Others appear because of what they owe, such as taxes or time in the national penal system. But nearly all Danish citizens appear in at least one database, and most appear in several.\n\nPublic opposition\n\nDanes are acutely aware of how much information the government holds about them. As a result, public opinion is squarely against increasing access to electronic databases.\n\nDanish journalists have a few helpful online resources, including Profile, a service similar to Dialog, and Pol Info, which offers newspaper articles, a database of female sources and other sources. Mulvad and Svith said the Internet is a promising source; however, with few Danish sites and almost no government sites, its uses are still limited.\n\nStill, there have been successes. Since Jyllands-Posten installed PCs last year, Mulvad and Svith have purchased Excel, Access, Paradox and SPSS. They created a database of sources for the newsroom after a big story left the staff scrambling for names and telephone numbers. Spreadsheets helped out with a story about debt and price-gouging in the national telephone industry. When Danish physicians began encouraging young women to take large doses of folic acid, Mulvad and Svith used e-mail and listservs to investigate other countries' policies.\n\nAs for their biggest challenge, gaining access to government data, Mulvad said he is cautiously optimistic.\n\n\"We're testing the borders to see which files we can get out,\" Mulvad said. \"At least we'll get something.\"\n\nGwen Carleton can be reached at (608) 245-0607, or send e-mail to gwen@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-jyllands-posten-website-notice","slug":"1996-july-jyllands-posten-website-notice","url":"https://dwillis.github.io/uplink/article/1996-july-jyllands-posten-website-notice.html","year":1996,"month":"July","monthNum":7,"headline":"Jyllands-Posten website notice","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Jyllands-Posten, a major Danish newspaper, offers daily news coverage across multiple categories with both free summaries and paid full article access on its website.","keywords":["newspaper","Danish","news"],"topics":["Technology"],"technologies":["World Wide Web"],"full_text":"For a taste of Danish, visit the Jyllands-Posten at www.jp.dk/ The site, which is updated daily, presents a thorough cross-section of the day's news, from domestic and international events to business, sports and features. Visitors can read story summaries for free, or pay a fee and gain access to the full articles.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-dueling-for-canadian-data","slug":"1996-july-dueling-for-canadian-data","url":"https://dwillis.github.io/uplink/article/1996-july-dueling-for-canadian-data.html","year":1996,"month":"July","monthNum":7,"headline":"Dueling for Canadian data","kicker":"Journalist wins","author_name":"Mike Gordon","author_title":"intern reporter","affiliation":"TVO","summary":"A Montreal Gazette intern reporter won an access to information case against the City of Montreal in 1995, forcing them to release a construction permits database by arguing that extracting existing data is not creating a new document. The Quebec Commission d'accès à l'information ruled in the reporter's favor, establishing a potential precedent for future journalism access cases in Canada.","keywords":["database","access","information"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"In August 1994, I was an intern reporter with the Montreal Gazette when I requested a database of construction permits from the City of Montreal. I was strung along for a few months before filing a formal access to information request in the first week of December 1995. The final word from the city's access commissioner came in February 1995: \"No way, Jose.\"\n\nSo, off I went to the Commission d'accès l'information (du Quebec). Backlogs in the commission's system meant that I was not offered a hearing until mid-October.\n\nIn the meantime, I had moved from Montreal to Toronto to start my current job. The commission didn't find me at my old address, and I did not hear about my date until the day before. I couldn't go, so my date was rescheduled to Dec. 8, 1995.\n\nThe decision\n\nI waited and waited for the decision. Finally, on Feb. 7, I received a certified mail envelope with the decision. Nervously, I flipped to the end of the four-page decision. Here, I read the wonderful words (for those of you who read French):\n\n\"POUR CES MOTIFS: La Commission ACCUEILLE la demande de revision et ORDONNE l'organisme de communiquer au demandeur la base des données qui contient les demandes pour les permis de construction sous forme sortie informatique.\"\n\nThat means I won, and the city was ordered to give me the database.\n\nThe issues\n\nFor those of you who are interested in the legal issues:\n\nThe city argued that while they accepted the information I wanted was public information, the structure and organization of the database was such that protected information that I did not seek was mixed up in the same file matrix. They argued that to extract the information I sought would require them to create a new document, a software program. Article 15 of the Access to Information Act says no organization has to create a new document. I argued that in no way was the programming required to manipulate a database included in the sense of \"a new document\" as mentioned in the Act. The judge ruled in my favor.\n\nShe wrote:\n\n\"Avec égard, je ne crois pas qu'il s'agisse d'un cas de l'application de l'article 15 de la loi. Les renseignements que desire le demandeur existent. Aucun calcul ni comparaison l'est necessaire au sens de l'article 15 de la loi. Il ne s'agit pas de la creation d'un nouveau document mais d'une simple manipulation d'un programme informatique.\"\n\nBasically, what she said is that the information exists; no calculation or comparison of data is required to create the information I wanted. This was not a case of creating a new document, but the simple manipulation of a computer program.\n\nSo, I won a case. Maybe it'll be a useful precedent for other reporters in Canada.\n\nIn preparing for my case, I was helped by the CAR Caucus of the Canadian Association of Journalists, especially Robert Washburn of the Cobourg Daily Star. They helped me get some pro bono legal advice from the LaFleur, Brown law firm in Montreal. Charles Bury of The Sherbrooke Record attended my hearing on behalf of the CAJ. Hope this bucks up the spirits of those of you engaged in similar legal battles. It takes a long time, but it is possible to win.\n\nMike Gordon can be reached at (416) 484-2600, ext. 2534, or send e-mail to mike_gordon@tvo.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-quebec-launch-pad-for-journalists","slug":"1996-july-quebec-launch-pad-for-journalists","url":"https://dwillis.github.io/uplink/article/1996-july-quebec-launch-pad-for-journalists.html","year":1996,"month":"July","monthNum":7,"headline":"Quebec Launch Pad for Journalists","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Quebec Launch Pad for Journalists is a comprehensive online resource providing journalists with links to government departments, news sources, and other useful information about Quebec in both English and French.","keywords":["Quebec","journalists","resources"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"The Quebec Launch Pad for Journalists at http://www.CAM.ORGI ~andymon features links to government departments, sources, and up-to-the-minute news from Quebec, as well as dozens of other useful links for journalists who cover or have an interest in the Canadian province. Most links are in English and French, but some are in French only.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-canada-explores-car","slug":"1996-july-canada-explores-car","url":"https://dwillis.github.io/uplink/article/1996-july-canada-explores-car.html","year":1996,"month":"July","monthNum":7,"headline":"Canada explores CAR","kicker":"Database of Databases","author_name":"Robin Rowland","author_title":"","affiliation":"Ryerson School of Journalism","summary":"The Database of Databases project aims to help Canadian journalists and academics access government and organizational databases for computer-assisted reporting, addressing challenges like government secrecy and lack of public data access compared to the United States.","keywords":["databases","journalism","Canada"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["World Wide Web","dBase","AskSam","CSV"],"full_text":"The Database of Databases project (http://www.ryerson.ca/~ricarc/pchome.htm) is aimed at jump-starting computer-assisted reporting in Canada. Visitors to the site will find lists that may be searched by agency, database or city. Searches reveal, among other things, an agency's address, telephone number, whether the agency maintains a database, and information on access.\n\nThis ongoing project aims to:\n\nProvide lists of government, association, foundation and private sector databases not only to journalists but to the academic community as well.\nComplete the \"Canadianization\" of CAR exercises for Canadian students.\nBegin the liaison between the journalistic community and the people who run the databases, so they will understand the needs of journalists and other researchers.\n\nIt's estimated that 80 percent to 90 percent of government information will soon be on computer.\n\nIn the United States, much of this material is available to the public and the news media through open records laws. Other material is often pried from government through Freedom of Information requests.\n\nIn Canada, we are facing a number of problems.\n\nA tradition of greater government secrecy.\nCrown Copyright - information belongs to \"the crown\" rather than \"the people.\"\nHigher costs. The government sees data, collected at taxpayers' expense, as a profit center in an age of cost-cutting.\nThere have been only a few requests, so far, for data.\n\nCAR reporters in the United States say you must start with a good story idea, then find the database. The Database of Databases will make this easier.\n\nIf a reporter has a story idea, he or she can then look up the Database of Databases and see if the data is available.\n\nIn addition, the simple knowledge that those databases exist could suggest story ideas to reporters as well as new, publishable research projects to academics.\n\nHow it started\n\nFor the past two years, students at Ryerson School of Journalism have completed an assignment on finding what's \"out there.\"\n\nAlmost always, the bureaucrats have been astonished that anyone would call and ask for this information. Often they have refused to give more than bare-bones information to the students. And Canadian bureaucrats have been equally surprised that the United States generally makes this information easily available.\n\nSo far, the Database of Databases project has identified 60 databases used by federal, provincial and municipal governments, police forces, fire departments, industry and professional associations, research foundations and the private sector.\n\nThis is just scratching the surface. There is a lot more out there.\n\nObjective\n\nThe objective will be to create a foundation for an ongoing Database of Databases project created by the Ryerson School of Journalism as a service to journalism and academia. Among other things, the project would like to:\n\nIdentify as many databases across Canada as possible.\nIdentify U.S. databases that contain Canadian data.\nCreate a Database of Databases in the industry standard .dbf format, as well as comma separated values and AskSam formats.\nIdentify and obtain databases that can could be used for student training and research.\nCreate a foundation for the project to continue using students as researchers. The database could then be updated on a regular basis.\n\nRobin Rowland can be reached by e-mail at eridani@inforamp.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-public-information-dataset-manifest-notice","slug":"1996-july-public-information-dataset-manifest-notice","url":"https://dwillis.github.io/uplink/article/1996-july-public-information-dataset-manifest-notice.html","year":1996,"month":"July","monthNum":7,"headline":"Public Information Dataset Manifest notice","kicker":"","author_name":"Robin Rowland","author_title":"","affiliation":"Ryerson Polytechnic University School of Journalism","summary":"Robin Rowland recommends a Canadian database that tracks hazardous waste generators, carriers, and receivers across the Canada-U.S. border, noting data quality issues with U.S. entries.","keywords":["waste","database","Canada"],"topics":["Health & Environment","Data & Databases"],"technologies":[],"full_text":"Robin Rowland of the Ryerson Polytechnic University School of Journalism in Canada, recommends a database called the \"Public Information Dataset Manifest and Generator Registration Information,\" which tracks all the hazardous waste generators, carriers and receivers in Canada. \"If a U.S. company on the border trucks its waste across the border to Canada or vice versa, it is listed,\" Rowland says. \"U.S. data is dirty. It lists 'USA' under province in some conditions.\" To get the free database, send three high-density PC disks to: Ontario Ministry of Environment and Energy, Environmental Monitoring and Reporting Branch, Environment Information and Systems Section, 40 St. Clair Ave. West Floor 12, Toronto M4V IM2","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-excel-goes-dutch","slug":"1996-july-excel-goes-dutch","url":"https://dwillis.github.io/uplink/article/1996-july-excel-goes-dutch.html","year":1996,"month":"July","monthNum":7,"headline":"Excel goes Dutch","kicker":"CAR Trip: First ventures","author_name":"Marjan Agerbeek","author_title":"education reporter","affiliation":"Trouw","summary":"An education reporter conducts her first CAR project analyzing employment data from Dutch universities, discovering that while cutting expenses, university boards ironically increased spending on professor salaries while reducing overall teaching staff.","keywords":["universities","employment","analysis"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["Excel","Word","WordPerfect","Microsoft Office Professional"],"full_text":"Last January, when I came back from the United States, I was eager to start my first CAR project.\n\nAs an education reporter, I decided that the project should be about education. Another quickly made decision was that the project had to be about universities, because there are only 13 in the whole country. If necessary, 13 records could be imported by hand. And calculations with 13 columns could be checked with an old-fashioned calculator. I felt safe, knowing I could defeat bad luck, such as data that didn't want to be imported into my spreadsheet, or other dirty tricks software plays on clumsy beginners.\n\nFinding the data\n\nEach year, the Dutch education department publishes a book with data about universities. I looked it through and found three uncomplicated tables with information about the staff at every university, including how many employees have temporary contracts, how many have steady ones, the number of employees in the distinctive job levels - that kind of stuff.\n\nThese tables were potentially interesting because of political circumstances. At a national level, it was decided that the universities had to become more autonomous in their staff policy. And the university boards had to cut expenses. I wondered whether the universities could shrink without mass layoffs. I also wanted to know what kind of employment policy they had four years ago, when they also had to cut expenses.\n\nMy colleagues at the education desk agreed that these questions were worthwhile, so I started the project. I asked the information official at the department of education to send me the book with the university data on floppy disk. The older data I found in another book. These were not available in an electronic version, so I had to type those in.\n\nHitting paydirt\n\nWhen the floppy arrived, the data appeared to be in WP 6.1 format. As there is no Windows-PC at the newsroom, I had to work at home, where I have installed Microsoft's Office Professional. I translated the data into Word, which was a piece of cake. And it turned out Excel had no problems with the tables. I was stunned about my luck and the intelligence of Excel.\n\nI analyzed the employees' records of public universities with Excel and found the number of professors, who get the highest salaries, had been increasing, while the total number of teachers had been declining. I also found that this had nothing to do with selective layoffs. The boards had ignorantly hired more professors. So, while the boards were trying to deal with lack of money, they were spending an increasing amount of money on salaries.\n\nI visited three universities to find explanations for the data. I knew which university faced mass layoffs. And I found universities that had cut expenses by using Ph.D. students as teachers, which is only allowed on a small scale.\n\nThree weeks and counting\n\nThe project took three weeks. My colleagues found this ridiculously long and were pressing me to publish during the last two weeks. At Dutch newspapers, working three days on one story is considered long.\n\nThe stories were hard to read, because they mainly explained the data; I had no time to do some reporting. Nevertheless I'm satisfied. The project was a little CAR example in Dutch, which I needed to show the newsroom what CAR is capable of. The management of Trouw appears to be interested. There is a lack of money, but hopefully, by the end of this year, there will be a Windows-PC in the newsroom.\n\nMarjan Agerbeek can be reached by e-mail at agerbeek@knoware.nl","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-kyodo-news-agency-homepage-notice","slug":"1996-july-kyodo-news-agency-homepage-notice","url":"https://dwillis.github.io/uplink/article/1996-july-kyodo-news-agency-homepage-notice.html","year":1996,"month":"July","monthNum":7,"headline":"Kyodo news agency homepage notice","kicker":"","author_name":"","author_title":"","affiliation":"Kyodo","summary":"Kyodo News Agency's foreign news editors maintain an online homepage where readers can submit comments to editor Tsukasa Yokoyama via email.","keywords":["Kyodo","homepage","journalism"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web","CompuServe","Email"],"full_text":"Foreign news editors of the japanese news agency Kyodo maintain a homepage at http://ourworld.compuserve.com/homepages/tyoko/ E-mail comments on the page to Tsukasa Yokoyama at tyoko@po.iijinet.or.jp","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-bootcamps-fill-up-fast","slug":"1996-july-bootcamps-fill-up-fast","url":"https://dwillis.github.io/uplink/article/1996-july-bootcamps-fill-up-fast.html","year":1996,"month":"July","monthNum":7,"headline":"Bootcamps fill up fast","kicker":"Popular training","author_name":"Brant Houston","author_title":"managing director","affiliation":"NICAR","summary":"NICAR has expanded its computer-assisted reporting bootcamp programs from six-day immersive courses to include road seminars and specialized training, with over 225 journalists completing bootcamps since 1994 and plans for intermediate and advanced level courses. The institute continues to evolve its curriculum to incorporate new technologies like the Internet while training journalists at all skill levels, including international participants.","keywords":["NICAR","bootcamp","journalists"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet","9-track tape","Internet","World Wide Web"],"full_text":"When the institute for computer-assisted reporting began at Missouri in 1989 (then named MICAR), the basic offering was a six-day immersion \"bootcamp.\"\n\nUnder the direction of computer guru Elliot Jaspin (and later former executive IRE director Andrew Scott), the four bootcamps a year turned out dozens of reporters who were pioneers in their newsrooms in the early '90s.\n\nIn early 1994, when the institute became NICAR, it began an ambitious outreach program of road training that has led to about 135 seminars in two years throughout the United States.\n\nAt the same time, the bootcamps have continued to thrive. With the increase in NICAR staff, we have been able to expand the average number of bootcamp participants from 12 to 20. This year, because of the demand, we added a fifth bootcamp and are trying to find a way to add specialized seminars in the coming year.\n\nThe curriculum at the bootcamps also has changed as the technology has changed. While we still teach database managers, spreadsheets, using 9-Track tapes, and getting and cleaning data, we have incorporated the Internet into the curriculum, showing journalists how to gather and analyze data from the Web sites.\n\nAs result of the expansion, more than 225 print and broadcast journalists have completed the bootcamp since January 1994.\n\nWe also have been working to make our curriculum more relevant to international journalists. In the past year, journalists from Denmark, Finland, Australia and England have signed up.\n\nIn the coming year, we are faced with the challenge of teaching various skill levels. We intend to keep the basic bootcamp, but we are already developing the materials for an intermediate-level bootcamp. In addition, we will again offer our advanced seminar at the University of North Carolina at Chapel Hill May 4-9, 1997, with CAR pioneer Philip Meyer.\n\nIf you are interested in attending bootcamp, the next one scheduled with remaining openings is for Jan. 5-10. For more information, call NICAR at (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-car-workshop-for-minority-journalists-notice","slug":"1996-july-car-workshop-for-minority-journalists-notice","url":"https://dwillis.github.io/uplink/article/1996-july-car-workshop-for-minority-journalists-notice.html","year":1996,"month":"July","monthNum":7,"headline":"CAR Workshop for Minority Journalists notice","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE, NICAR, and The Richmond Times-Dispatch are offering a Computer-Assisted Reporting Workshop for minority journalists in Richmond, Virginia from November 15-17, featuring hands-on training in internet navigation, spreadsheets, databases, and data negotiation.","keywords":["workshop","journalism","training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"IRE, NICAR and The Richmond Times-Dispatch will offer the Computer-Assisted Reporting Workshop for Minority Journalists in Richmond, Va., Nov. 15-17. The hands-on training will include navigating the Internet, spreadsheets, database managers, negotiating for data and more. Forty slots are available. For more information, call (573) 882-0684, or send e-mail to Wendy Charron at wendy@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-ire-l-and-nicar-l-list-archives-notice","slug":"1996-july-ire-l-and-nicar-l-list-archives-notice","url":"https://dwillis.github.io/uplink/article/1996-july-ire-l-and-nicar-l-list-archives-notice.html","year":1996,"month":"July","monthNum":7,"headline":"IRE-L and NICAR-L list archives notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The article announces that archives for IRE-L and NICAR-L mailing lists are available on the IRE and NICAR websites, organized by thread, author, and date in both HTML and plain text formats.","keywords":["archives","mailing","organized"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Miss something? Check out the IRE-L and NICAR-L mailing list archives on our websites at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-untangling-problem-layouts","slug":"1996-july-untangling-problem-layouts","url":"https://dwillis.github.io/uplink/article/1996-july-untangling-problem-layouts.html","year":1996,"month":"July","monthNum":7,"headline":"Untangling problem layouts","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"NICAR staff","affiliation":"NICAR","summary":"The article provides practical tips for efficiently loading tape data into a PC database program, emphasizing the importance of thorough testing to catch formatting errors before processing. It recommends using CSV format for testing and Nine Track Express's Preview feature to verify record lengths and field descriptions.","keywords":["data","testing","formatting"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Nine Track Express","Excel","FoxPro","Paradox","Word"],"full_text":"Getting new data is like getting a present. You want to get the data loaded into your PC database program as quickly as possible so you can start playing with it.\n\nBut this unwrapping process can be delayed by simple typing mistakes on our part, or, what's worse, there can be small or frustratingly substantial differences between the record layout we have on paper and the data we were given on tape.\n\nJust like alarm clocks going off at 6:15 a.m., this kind of thing happens more often than we'd like. Here are some of the procedures I've come up with to deal with the problems that sometimes hold us up from the fun part: getting to interview data.\n\nFor the tape processing, I'm using Nine Track Express and assuming familiarity with its Create File Parameters and Create Field Descriptions steps. Additionally, these tips take advantage of Microsoft Excel's marvelous Text Import Wizard; a graphical word processor such as Microsoft Word; and a file viewer called List, a shareware program I find indispensable.\n\nTesting format\n\nUse .CSV as the output format while tesing. The Preview feature in Nine Track Express can save you a lot of time. I never process any tape without using it. While the Transfer File procedure is counting off the records moved from the tape to the PC, pressing the \"P\" key pauses the process and shows you exactly what Nine Track is reading as a single record. If the data at the beginning and at the end looks OK, then you know the record length you are using is correct. I always use the Preview several times to make sure the one record I saw wasn't coincidentally correct and most of the others were wrong.\n\nMistakes in field lengths are more frustrating. If you are outputting the data to the FoxPro or Paradox format, the preview will not show you how the data is being split fields. To find out if right, you have to quit Nine Track and look at the results in the database program. If it's wrong, that means quitting FoxPro or Paradox and going back to Nine Track. And if you're using the Windows version of these database managers, then you have the extra step of loading and unloading Windows.\n\nTime saver\n\nSave yourself some time and use .CSV as the output format until you know that the data fields are being split in the right places. Then, when the field description is right, change the output format to the database format you want.\n\nThe commas and quotes will reveal simple field length problems such as: every ZIP code field beginning with a letter (some column before the ZIP is one byte too long. Of course, this example is too easy. Finding mistakes in data layouts when most of the information is readily recognizable stuff such as names, addresses, amounts and dates is easy compared to data sets that are hundreds of bytes long, with nothing but key fields, and lots of small fields with codes like \"EA\" and \"J1.\" When scanning the preview output, go to the end of the line, and work back to the beginning. Load a small sample of the file (1,000 records or less) in fixed-position format.\n\nSaving samples\n\nI often do this step first with a dataset I haven't processed before. I save the sample file on the floppy I keep for every data project. All finished and tested Nine Track profile (PR2) files go here too. (I know I wouldn't want to have to do all that typing and testing all over again.) I also save small .dbf samples of all the tables in the data set. I often print a small portion of this file, especially if it's obvious that some record layout debugging is going to be needed. Most data records are longer than 80 characters, so this is where a graphical word processor comes in handy. First, for the printout to be useful, you have to set the text to a monospace font such as Courier or Courier New. Proportional fonts such as Times Roman will display the text with less width for the i's and l's and more space for M's and W's. The result is text that doesn't look like a fixed-position file. If necessary, I set the point size to eight, use Print Setup to shrink the left and right margins to the minimum and print sideways on the paper (landscape mode), or even choose legal size paper over letter.\n\nDisplaying this file with the List program also helps, since one of the features of this program displays a ruler or counter line at the top of the screen to make it easy to see, for example, what characters occur in positions 149 through 156.\n\nSince fixed-position output only divides the data records, but doesn't split the fields, you can quickly fill out the Field Description part of the PR2 file by making a single field with a length equal to the data record length. Use character as the data type.\n\nNine Track Express has a limit on how big a single field can be, so if the record length is greater than 500 bytes, you have to enter descriptions for 500-byte fields until what's left is less than 500 bytes. Then you make the final field length equal to the remainder. For example, if the record length is 1637 bytes, add field descriptions for three fields of 500 bytes, and then one field of 137 bytes.\n\nOther tools\n\nUse Excel on the sample fixed position file. I think everyone who sees and uses the Excel 5 Text Import Wizard immediately likes it because it's so easy. I used it recently on a non-problem data set I had just received because it was the fastest way to get the data imported without a lot of typing of field lengths.\n\nI also like the Text Import Wizard because it's a great teaching tool for the concept of importing data. It presents the perfect picture of what has to occur when fixed-position files are imported into any spreadsheet or database, regardless of the software method.\n\nThis picture is also a great help in figuring out problem record layouts. You can use Excel's wizard to scan the records (as with a file viewer) and at the same time, designate where the fields should be split. When the wizard asks for the data type for each column, choose \"Text\" for every column. By default, the Excel Wizard designates every column as \"General,\" but in the program's intent to be all-purpose and helpful, data such as Social Security numbers or other ID data composed of all numerals and no alphabetic or punctuation characters will be converted to scientific notation like 2.35E7, which is my Social Security number with the confidential parts rounded off.\n\nSpreadsheet limits\n\nSpreadsheets have a limit of about 16,000 rows, but you only need to test a few hundred rows. This is another reason for doing that fixed-position sample output of the data.\n\nAfter you figure out the correct layout, you have several options. You can import the spreadsheet into FoxPro or Paradox and print out the table structure for reference in correcting the Nine Track field description. It temporarily takes up more disk space, but you could also load the entire tape file as a fixed-position file; empty all the sample rows from the FoxPro or Paradox table and then import the fixed-position file into the empty table.\n\nBecause of the difference in how Excel and FoxPro or Paradox store data, there is a limitation to this method. If the data has long character fields that are filled with just spaces, instead of data, those fields will show up in the database file with much smaller widths. The only way around this problem that I can suggest is to replace all spaces in the fixed-position file with some distinct character that is not used in the data. The caret character (^ or Shift+6) is a good choice.\n\nRichard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-namebase-research-tool-notice","slug":"1996-july-namebase-research-tool-notice","url":"https://dwillis.github.io/uplink/article/1996-july-namebase-research-tool-notice.html","year":1996,"month":"July","monthNum":7,"headline":"NameBase research tool notice","kicker":"","author_name":"Drew Sullivan","author_title":"","affiliation":"Associated Press","summary":"Drew Sullivan of the Associated Press highlights NameBase, a free research tool from Public Information Research that indexes names from various sources covering topics like CIA activities, intelligence operations, corporate fraud, and organized crime.","keywords":["research","intelligence","database"],"topics":["Investigative Methods","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"Drew Sullivan of the Associated Press recommends a research tool called NameBase, available for free at http://www.pir.org Namebase, run by Public Information Research, \"is an index of names pulled from books, magazines, newletters and all sorts of strange and rare sources dealing with the CIA, intelligence, corporate fraud, drug-dealing, the mob and more than a few conspiracy theories,\" Sullivan says. \"If you can't remember what Thomas Clines or Richard Secord was involved in, this is your place.\" Drew Sullivan can be reached by e-mail at drew@ap.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-ire-committees-notice","slug":"1996-july-ire-committees-notice","url":"https://dwillis.github.io/uplink/article/1996-july-ire-committees-notice.html","year":1996,"month":"July","monthNum":7,"headline":"IRE committees notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE is recruiting volunteers for several committees including the Advisory Council for Periodistas de Investigacion, the International Committee, and the Computer Committee, with contact information provided for each committee chair.","keywords":["volunteers","committees","IRE"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"IRE committees always are on the lookout for good volunteers. The Advisory Council for Periodistas de Investigacion is led by IRE Executive Committee Chair Deb Nelson of the Seattle Times. To volunteer, send e-mail to Nelson at dnelnew@seatimes.com The International Committee is chaired by IRE President John Lindsay of Oregon Public Radio. To volunteer, send e-mail to Lindsay at Lindsayopb@aol.com The Computer Committee is chaired by IRE Director Shawn McIntosh of The Dallas Morning. To volunteer, send e-mail to McIntosh at dmnproj@airmail.net For information on other IRE committees, send e-mail to IRE Executive Director Rosemary Armao at armao@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-data-you-can-access","slug":"1996-july-data-you-can-access","url":"https://dwillis.github.io/uplink/article/1996-july-data-you-can-access.html","year":1996,"month":"July","monthNum":7,"headline":"Data you can access","kicker":"On the Internet","author_name":"David A. Milliron","author_title":"","affiliation":"Gannett News Service","summary":"A comprehensive guide to free, searchable online databases available to newsrooms, including telephone directories, business listings, legal databases, mapping tools, and specialized resources for pesticide and environmental information research.","keywords":["databases","directories","newsrooms"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web","gopher"],"full_text":"Here's more of our list of free searchable databases any newsroom with Internet access can access:\nArea codes for U.S. and Canadian telephone numbers seem to change faster than the latest Top 10 songs in America. Here is a database that keeps you current. Look up area codes anywhere in the country in a flash.\nhttp://www.555-1212.com/aclookup.html gopher://odie.niaid.nih.gov/77/deskref .areacode/index\ngopher://coral.bucknell.edu:4320/ 7areacode\ngopher://riceinfo.rice.edu:1103/7areacod gopher://gopher.austin.unimelb.edu.au:70 7waissrc%3A/general/country-codes\nFind people using all the telephone directories in the United States and Canada. Toll-free numbers can be found by name and geographic location. Find business or government in the United States and Canada. Blue Pages is a category-based directory of city, state, and federal government listings. Yellow Pages allows searches by category and topic in the United States and Canada. Fax Directory is the largest single source of fax numbers available.\nhttp://www.geocities.com/SiliconValley/ 8377/ultimate.html\nhttp://www.infospaceinc.com/ http://www.switchboard.com\nThe entire USA/Canada Yellow and White Pages has been made into a database and is available for searching. There's more than 105 million listings. Search by name, phone number, occupation, etc.\nOnce you get a \"hit,\" you can click on a button to see a map of the address; search for their e-mail address; browse addresses in the same neighborhood; obtain county and ZIP+4 information; search for Usenet Newsgroup postings; and get driving directions to the matched address.\nhttp://wyp.net/Search.html http://s12.bigyellow.com/ http://www.bigbook.com/ http://www.iypn.com/ http://att.net/dir800/\nSearch by keyword the index of scholarly internet mailing lists and newsgroups.\ngopher://gopher.usask.ca/77%2F9 2Eindex%2Facad\nWest's Legal Directory (WLD) is a comprehensive directory of U.S. and Canadian law firms, government offices, corporate law offices and lawyers. Search for attorneys and offices by practice type (private, government, corporate, etc.), name, office name, city, state, county, county, etc. Provides biographical data.\nhttp://www.wld.com/\nMapQuest you locate a business, address; or city in the continental U.S. on an interactive map; zoom in on any location in the continental U.S.; add Points of Interest to your maps; and more.\nhttp://www.mapquest.com/\nFor a pesticide database, check out Pesticide Information Service, a group of non-governmental agencies collaborating for pesticide reform. It presents a searchable database of pesticides.\ngopher://gopher.igc.apc.org/11/orgs/panna/ pestis\nEnvirosearch is an information search tool that offers downloadable files on recent environmental and safety standards the U.S. and abroad. Includes the semi-monthly \"Environmental Manager's Compliance Advisor\" newsletter, Environmental acronyms list, recently issued environmental regulations and proposals, international environmental standards, safety standards, GAO environmental reports, Clean Water Act discharge limits and enforcement, etc.\nhttp://www.envirosearch.com/\nThe National Center for Missing and Exploited Children maintains a large database of missing children, with pictures and other details.\nhttp://www.missingkids.org/\nFor state-by-state listing of available sources of vital records check out http://www.medaccess.com/address/vital_toc.htm This database provides state government addresses for birth, marriage, divorce, and death records.\nGPO access: http://thorplus.lib.purdue.edu/gpo/ http://ssdc.ucsd.edu/gpo/multi.html http://www.access.gpo.gov/su_docs/aces/ aaces001.html\nAccess to the following searchable databases: Congressional Bills, Congressional Calendars, Congressional Directory, Congressional Documents, Congressional Record and Index, Congressional Reports, Economic Indicators, Economic Report of the President, Federal Register, General Accounting Office, Government Information Locator Service, U.S. Government Manual, History of Bills, House Rules Manual, Interior Reports, Privacy Act Issuances, Public Laws, Unified Agenda and U.S. Code.\nThe U.S. General Accounting Office now offers free on-line access to the GAO reports and investigations on everything from HMOs to immigrant students and government operations. http://www.gao.gov/ David Milliron can be reached at (703) 276-5805, or send e-mail to dmilliro@gcil.gannett.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-advanced-bootcamp-notice","slug":"1996-july-advanced-bootcamp-notice","url":"https://dwillis.github.io/uplink/article/1996-july-advanced-bootcamp-notice.html","year":1996,"month":"July","monthNum":7,"headline":"Advanced Bootcamp notice","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"NICAR is offering an Advanced Bootcamp led by Philip Meyer on May 4-9 in Chapel Hill, focusing on statistics and maps for journalists.","keywords":["bootcamp","journalism","statistics"],"topics":["Journalism & Newsroom","Technology"],"technologies":["ArcView"],"full_text":"Don't forget that the Advanced Bootcamp offered by NICAR and Philip Meyer, author of \"The New Precision Journalism\" and journalism professor at the University of North Carolina-Chapel Hill, will be coming up May 4-9. The seminar, which is held in Chapel Hill, will concentrate on stats and maps. For registration information, send e-mail to Wendy Charron at wendy@nicar.org, or call NICAR at (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-growing-collection-of-federal-databases","slug":"1996-july-growing-collection-of-federal-databases","url":"https://dwillis.github.io/uplink/article/1996-july-growing-collection-of-federal-databases.html","year":1996,"month":"July","monthNum":7,"headline":"Growing collection of federal databases","kicker":"From the NICAR library","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering topics including election contributions, healthcare, transportation safety, crime statistics, and loan programs, available for journalistic research and investigation.","keywords":["databases","federal","journalism"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"Growing collection of federal databases\n\nFrom the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW A monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\n\nNEW The Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\n\nCoast Guard boating accidents, 1969-1994.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\n\nFederal procurement data, 1992-1994, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records.\n\nNational Bridge Inventory System data, includes inspection grades.\n\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\n\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1994.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nNational Endowment for the Arts, grants, 1989-1993.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-nicar-bootcamps-notice","slug":"1996-july-nicar-bootcamps-notice","url":"https://dwillis.github.io/uplink/article/1996-july-nicar-bootcamps-notice.html","year":1996,"month":"July","monthNum":7,"headline":"NICAR bootcamps notice","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR is offering week-long bootcamps in computer-assisted reporting skills, including spreadsheet and database training, with the next session scheduled for January 5-10 in Columbia, Missouri.","keywords":["training","data","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet","9-track tape"],"full_text":"NICAR's week-long bootcamps offer hands-on training in computer-assisted reporting skills, including the use of spreadsheets and database managers, accessing data in various media, such as nine-track tapes, and negotiating for data. The next open session is Jan. 5-10 in Columbia, Mo. For more information, call NICAR (573) 882-0684, or send e-mail to nicar@muccmail.misssouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-bits-bytes-and-barks","slug":"1996-july-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1996-july-bits-bytes-and-barks.html","year":1996,"month":"July","monthNum":7,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A journalism newsletter covering job transitions and opportunities in newsrooms, with a focus on computer-assisted reporting skills and proposed NICAR training seminars for intermediate data analysis, front-end development, and mapping.","keywords":["journalism","reporting","training"],"topics":["Journalism & Newsroom","Technology"],"technologies":["SQL","Access","Visual Basic","ArcView","Internet","World Wide Web","spreadsheet"],"full_text":"Bits, Bytes and Barks\n\nMoving on\n\nPontiac, Mich., Oakland Press reporter Jeff Green is leaving his job as environment/demographic/science reporter after seven years for a similar position as associate editor at Wards Automotive Reports in Southfield, Mich., a Wards Communication Publication. Green will continue to use CAR skills, adapting them to covering the automotive industry.\n\nJobs available\n\nThe Intelligencer Record, a Philadelphia suburban daily, seeks an aggressive, hard-working reporter. Internet, spreadsheet and database skills a big plus. Send letter, resume and clips to Lanny Morgnanesi, executive editor, The Intelligencer Record, Box 858, Doylestown, PA 18901.\n\nThe San Francisco Bay Guardian, an alternative weekly, seeks a city editor to manage newsroom and plan coverage of Bay Area news and progressive politics. Minimum three years management, editing, and investigative journalism experience required. Experience with computer-assisted reporting a plus.\n\nSend cover letter, resume, three recent clips, and reference to News Editor Search, S.F. Bay Guardian, 520 Hampshire, San Francisco, CA 94110. No phone calls please.\n\nGive us your opinion\n\nIn response to popular demand, NICAR is considering scheduling one or more three-day intermediate seminars in the coming year in Missouri and possibly at other sites. We would like your suggestions and thoughts.\n\nHere are the proposed intermediate seminars:\n\nAdvanced data analysis: Covering advanced SQL queries, data cleaning, lookup tables and more. Also, problem data from miscoding to messy record layouts. Should we also include problems that government computer tapes offer?\n\nFront-ends: Making data easier to search and use in the newsroom. This would involve using Access and Visual Basic, plus training in putting data up on internal Web sites.\n\nMapping: While the annual University of North Carolina-Chapel Hill workshop links statistics and mapping, this workshop would focus on the nuts and bolts of mapping. We probably would use ArcView, but also teach principles that could be applied to all mapping.\n\nLast, we are looking at updating our Internet training again and would appreciate suggestions on that.\n\nPlease send messages to Brant Houston, NICAR managing director, at brant@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-get-them-while-they-last","slug":"1996-july-get-them-while-they-last","url":"https://dwillis.github.io/uplink/article/1996-july-get-them-while-they-last.html","year":1996,"month":"July","monthNum":7,"headline":"Get them while they last","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A bound edition collection of Uplink 1995 issues is available for purchase at $10 plus shipping costs.","keywords":["publication","archive","ordering"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Uplink 1995 issues are available in a bound edition for $10 plus shipping. To order, call (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-july-new-ire-l-and-nicar-l-addresses","slug":"1996-july-new-ire-l-and-nicar-l-addresses","url":"https://dwillis.github.io/uplink/article/1996-july-new-ire-l-and-nicar-l-addresses.html","year":1996,"month":"July","monthNum":7,"headline":"New IRE-L and NICAR-L addresses","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE-L and NICAR-L mailing lists have relocated to a new dedicated Unix server at the University of Missouri with updated email addresses for posting and subscription management.","keywords":["mailing","lists","Missouri"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"IRE-L and NICAR-L have changed addresses. They are located on lists.missouri.edu, a dedicated Unix system designed to service discussion groups run out of the University of Missouri. All posts to the lists should now be sent to: ire-l@lists.missouri.edu nicar-l@lists.missouri.edu\n\nAll commands for subscription changes should be sent to: listproc@lists.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-august-getting-down-to-business","slug":"1996-august-getting-down-to-business","url":"https://dwillis.github.io/uplink/article/1996-august-getting-down-to-business.html","year":1996,"month":"August","monthNum":8,"headline":"Getting down to business","kicker":"","author_name":"Neil Reisner","author_title":"NICAR staff","affiliation":"NICAR","summary":"This article provides a comprehensive list of story ideas and methodologies for business reporters to use computer-assisted reporting (CAR). It covers the use of spreadsheets, databases, and mapping software to analyze financial trends, bank lending, and corporate behavior.","keywords":["Business","Spreadsheets","Mapping"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":["spreadsheet","Excel","GIS"],"full_text":"Most business stories involve numbers, and a large portion involve many, many numbers. More business writers than ever can't imagine even beginning such a story without first entering the numbers on a spreadsheet or using a database manager to look for trends. \"A business writer who believes he/she is getting everything from the companies is just plain wrong,\" says one veteran reporter. \"It's like saying that sports reporters get all they need from the teams or political reporters get all they need from politicians.\" It's too easy to say that no one knows the balance sheets better than management, and therefore we should let them do the analysis. Just substitute \"Congress\" or \"the governor\" or \"the mayor\" for \"management\" in the last sentence, and the point will be clear: Management knows the financials - but that doesn't mean they tell you everything that shareholders or the public or employees need to know. Here are some ways business reporters can use computer-assisted reporting: Use the annual Home Mortgage Disclosure Act to examine whether banks give mortgages in all the neighborhoods they serve. HMDA summarizes how lenders responded to home-loan applications by Census tract and by applicants' race, sex, income and requested loan amount. In many cases, reporters find that banks disproportionately deny loans to minorities. Search the patents database, available in part on-line or from the Patent Trademark Office, for patents in your area and get totals from the U.S. Patent Office. What sorts of patents are most common? Quick hit: Write a spot profile of new patents in your area. Use a spreadsheet to build basic graphics that make it easier to see a company's sales/profits patterns. You can, for example, find out whether a company's newspapers or television stations make a bigger contribution to earnings. Are your airlines on time? Check out the on-time flight database from the FAA and find out which cities are the best and worst routes for your location (in terms of getting there or coming back on time), and which airlines are the best and worst in your city. Track the health of your local banks. Take banking data (it comes out quarterly) and look at (Liabilities + Loans-on-the-Brink - those that may go bad in 90 days) / assets. The ratio is a quick way to look at the health of banks. USA Today was able to get a great jump on troubles at Chase Manhattan using this method. Use a spreadsheet to analyze tariffs. Mike Fabey did this to look at beer tariffs for the Journal of Commerce. Use mapping software to examine bank locations. Are banks and S&Ls where most people can conveniently reach them? You may find that most banks are located in upscale office parks or high-rent neighborhoods. Buy the necessary addresses from Sheshunoff, the bank research firm. Or scour the Yellow Pages and enter the data yourself. Once you have a map, and see just how large the unserved area is, go visit it and find out how people get banking services. You may find they depend on check-cashing \"services\" and rarely, if ever, can borrow money on reasonable terms. Are your businesses paying back the community? Check the SBA database for the promises businesses made about how many employees they'll add if they get a loan. Did they follow through? Writing about the Orange County bankruptcy, Ron Campbell of The Register used Excel to track the professional fees in the mammoth case - $50 million and counting. He also used Excel to keep track of dozens of bond issues, each with its own interest rate, tax status, underwriter and financial adviser, and seemingly hundreds of recovery plans. Use data from the Bureau of the Census Web site to see how the age/race composition of your state or region will change over the next 25 years, and report on how that will affect state agencies, the economy, etc. How are your local or regional businesses doing on the Net? Find out whether this Web thing is hype or whether people are really making money. What are their concerns? Companies usually report several years' results in their annual report or SEC Form 10-K. Put those numbers on a spreadsheet and do some charts. What sources of income have changed in the past few years? Is the company taking on more debt? What about its inventory? In many industries, inventory is a ticking time bomb. Many companies have adopted \"just-in-time inventory.\" The reason for this strategy is purely financial: to keep from wasting valuable capital on storage for goods you might not be able to sell at a profit. Compile figures from a number of company annual reports or 10-Ks to take a look at an industry segment, using the same sort of calculations. Use the OSHA database to determine the most common accidents and causes for your area. Use the Internet and other on-line services to research small or obscure companies. Rob Gebelof at The Record in Hackensack, N.J., did that when it researched a company owned by a scientist who had been murdered. Little was known about the company. But by punching it into Altavista, a Net search engine, a reporter found a reference to it on a page maintained by another company. A call to that firm yielded an interview with someone who knew both the victim and his company well. See how many workers in your area are filing for NAFTA-related benefits compared to the rest of the country. Use a spreadsheet to look at company balance sheets. Most companies do not provide information such as debt ratios, which are vital to understanding a business. Crain's New York set up a system so reporters can type in the basic numbers and get computations on key ratios. Andy Lehren of NICAR, a former Philadelphia business writer, did the same thing as a way to find publicly traded companies worth probing. Use spreadsheets to analyze Commerce data on the fly. By using stat-USA, you can get U.S. economic data as soon as it's released by Commerce. You can quickly get that information into spreadsheets and go beyond the press releases. Business reporters can turn around quick merger stories using desktop mapping. When one company says its going to swallow another, business reporters should type company locations into a database and analyze the result with a mapping program. Dave Herzog at The Morning Call in Allentown, Pa., did that when CoreStates announced it was taking over Meridian Bank. The paper highlighted branches within a stone's throw of each other and speculated which might be shuttered, ending up close to the mark. The St. Louis Post-Dispatch used the same technique when one grocery store chain bought another.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-august-the-rich-get-richer","slug":"1996-august-the-rich-get-richer","url":"https://dwillis.github.io/uplink/article/1996-august-the-rich-get-richer.html","year":1996,"month":"August","monthNum":8,"headline":"The rich get richer","kicker":"What SBA data shows","author_name":"Andy Lehren","author_title":"NICAR staff","affiliation":"NICAR","summary":"An investigation in Philadelphia revealed that SBA loans were disproportionately awarded to businesses in wealthier neighborhoods rather than struggling communities. The reporter used national SBA loan data and Census tables to prove significant disparities in the distribution of federal aid.","keywords":["SBA","Philadelphia","Loans"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["9-track tape","SPSS","Excel"],"full_text":"In Philadelphia - a city recuperating from a severe budget crisis - officials talk a lot about economic development. But the city's poorer neighborhoods complain that too much public aid goes to Center City businesses and other well-off areas, while struggling communities get little assistance. Officials have admitted that certain programs, such as Urban Development Action Grants, were funneled downtown to ensure its economic health. But they countered that other government assistance, such as loans backed by the U.S. Small Business Administration, did not work that way. An October 1995 analysis by the Philadelphia Daily News showed that's not true. The two-part series showed how the agency did a poor job of getting money into the city and, when it did, the support went more to neighborhoods that are already better-off. The stories mined the SBA's national database for 7a loans. That is the SBA's key program for helping small businesses. The name refers to the section of law that created the program. The program is supposed to encourage bank lending to companies that would otherwise not qualify. It is designed to work this way: A company is seeking a bank loan, but the bank is reluctant. Maybe the company owners have little credit history to help the bank make a decision. Perhaps the deal is a bit chancy. But if the company qualifies, it could get the federal government to guarantee a big chunk of the loan. The bank is happy because it gets to do the deal with less risk. The company is pleased because it gets the cash. The SBA is content because it helps a small business. There are rules. However, in the decentralized federal agency, a clear picture emerged: Some district offices and banks were better at churning out loans than others. Philadelphia was among those cities with a smaller piece of the pie. The data was a key part of the story. Obtained under the Freedom of Information Act, the nine-track tapes showed where the money was going. The database was a little dirty; developing look-up tables for cities and ZIP codes helped alleviate problems caused by sloppy data entry. After that, the numbers showed that, despite being the nation's fifth-largest city, it ranked 17th when it came to getting SBA aid. In addition, when agency officials countered that they were improving, a year-by-year breakdown showed that, while Philadelphia's numbers were a bit higher as time went on, its relative position fell because other places were doing even better. I spoke to those in places getting more money. They described how they worked the system to do more deals. Those methods underscored the problems back in Philadelphia. When it came to particular loans, the database was handy for seeking paper records. I submitted an FOIA request to the SBA district office, attaching a spreadsheet that listed the names, addresses and loan dates. The district office complied with documents that showed much more than what is in the database - including details like the equipment or land used as collateral. This was also a great way to check the data's integrity. I linked the SBA data with U.S. Census Bureau poverty and population tables to see whether SBA officials were right that the money was evenly distributed among rich and poor neighborhoods. Using SPSS, I ran a series of independent t-tests and analysis of variance, and found a significant gap. All this led to much better interviews. One City Hall official pointed out that, adding insult to injury, Philadelphia was also the biggest city without its district office located within its borders. A check of SBA district offices, published on the agency's web site, showed that was true. I tracked down the Reagan administration official who moved the office out to the suburbs. He conceded that the long commute, among other reasons, caused him to pull out.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-august-like-money-in-the-bank","slug":"1996-august-like-money-in-the-bank","url":"https://dwillis.github.io/uplink/article/1996-august-like-money-in-the-bank.html","year":1996,"month":"August","monthNum":8,"headline":"Like money in the bank","kicker":"HMDA data pays off.","author_name":"John Hill","author_title":"","affiliation":"Sarasota Herald-Tribune","summary":"The Sarasota Herald-Tribune used HMDA data to investigate racial disparities in home lending, debunking bank claims of improved performance. The study showed that despite aggressive marketing to minorities, loan approvals still favored white applicants and mixed neighborhoods remained underserved.","keywords":["HMDA","Redlining","Lending"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["HMDA data"],"full_text":"Wiggling out of one problem can make you look worse. Just ask bankers in Sarasota, Fla. In July 1995, the Sarasota Herald-Tribune was preparing a story that showed racial disparities in home lending. The story was based on Home Mortgage Disclosure Act data from 1993, the most recent available at the time. Several bankers, in interviews before the story was published, insisted their performance in 1994 had improved. They credited themselves with offering a variety of innovative schemes that effectively loosened mortgage credit to blacks and to minority communities. There was only one problem: It wasn't true. After those early interviews, the Herald-Tribune decided to hold the story. When the 1994 numbers were available, an analysis showed that while blacks saw their approval rate inch up slightly - from 61 percent in 1993 to 62 percent the following year - those gains did not keep pace with other racial groups. Moreover, while the denial rate for blacks dropped in 1994, these gains were offset by the large numbers of blacks who grew frustrated with the lending process and withdrew their applications, or who had their files closed by the banks for being \"incomplete.\" As a result, while these \"new\" and \"aggressive\" schemes lured more blacks to the door, the spike in loan activity only further concentrated housing credit into the hands of whites, doing little to redistribute mortgages along racial lines or to buttress housing values in mixed communities. The result was a far stronger story, particularly given the demographics of southwest Florida, where a standard analysis of HMDA data along racial lines is complicated by the small number of black and minority applicants. HMDA numbers are problematic enough. Ask anyone who's based a story on them. You can't account for credit histories, or analyze in any quantifiable way the margins between debt and income by an applicant's race. (This is how lenders typically dismiss the charge of mortgage \"redlining\" or discrimination.) Here are ways to work around problems with the data: Analyze whether mortgage credit is flowing to minority communities. Using Census data, we analyzed the approval rates, and amount loaned, to areas which were two and three times Sarasota's nonwhite average. As it turned out, large banks generally moved away from mixed neighborhoods, leaving higher-priced mortgage and finance companies to pick up the slack. Examine the amount of loans made to individual blacks, by income. Are loans to blacks going to those with exclusive incomes, thereby making any claim of racial equalization moot? Compare denials in your area with those from similarly composed counties or metropolitan areas in your state.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-august-roll-out-the-barrel","slug":"1996-august-roll-out-the-barrel","url":"https://dwillis.github.io/uplink/article/1996-august-roll-out-the-barrel.html","year":1996,"month":"August","monthNum":8,"headline":"Roll out the barrel","kicker":"Checking a cartel rollback","author_name":"Mike Fabey","author_title":"","affiliation":"The Journal of Commerce","summary":"A journalist investigates the Federal Maritime Commission's claim that a shipping cartel agreement would save importers and exporters $65 million, discovering the agency based this figure on unverified verbal information from the cartel rather than documented data.","keywords":["shipping","investigation","verification"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":["Epson ActionNote 650","modem/BBS","PIERS","FOCUS","ProComm Plus for Windows","Transax Systems","Claris Works for windows"],"full_text":"It's never easy to check a government agency's claim that it's saving money for its constituents. It gets even trickier when the agency's the Federal Maritime Commission, and it talking about saving money on shipping contracts from exporters and importers all around the world, which vary according to the cargo, the ports and the carrier.\n\nThe commission, or FMC, was investigating the Trans-Atlantic Conference Agreement, the cartel of major ocean shipping lines that carry goods between North America and Europe.\n\nImporters and exporters said the cartel was forcing the shippers to pay too much money for the service because the ocean liners knew the shippers had few alternatives.\n\nThe FMC agreed to drop its investigation when the cartel said it would roll back its rates to the previous year's prices. The agency and the lines said the rollback would save importers and exporters about $65 million, a nice chunk of change.\n\nThe Journal of Commerce, a daily business newspaper in New York, likes nothing better than financial breakdowns. They make for nice graphics. I asked the agency and cartel for a breakdown of the $65 million. They were unable or unwilling to do so. I asked both to quantify where the $65 million would come from. The agency admitted it got its number from the cartel - verbally, with no written documentation to back it up. The cartel would only say that it had based its figures on numbers of containers moving across the Atlantic multiplied by $100, which, it said, was the average increase for each container before the rollback.\n\nThe executives in the cartel probably figured no one would check. The shipping lines transport about 1.4 million containers across the Atlantic each year, carrying beer, furniture and just about anything anyone can think of. The lines file more than 400 contracts each year for all the different commodities, and each contract can have any number of different rates, depending on all kinds of shipping variables, such as whether the cargo needs to be refrigerated.\n\nThe contracts themselves are mostly written documents, with separate rates, surcharges and other costs scattered throughout. As most of the folks at NICAR told me, there was no way to write a program to capture all the relevant data easily to format it for a spreadsheet or database manager.\n\nSo, I opted for a small piece of the pie. I figured I'd pinpoint the largest shippers in the trade and find out the rollback those companies got. They controlled most of the cargo, and their prices would serve as a good barometer for the rest of the bunch.\n\nI use an Epson ActionNote 650 laptop for most of my computer work. To find the biggest shippers in the trade, I used my Epson and my Motorola Lifestyle 14.4 Data/Fax modem to hook into a special database offered by Port Import/Export Reporting Service, or PIERS, which tracks these kinds of things. PIERS provides FOCUS software to search its data.\n\nTapping the beer trade\n\nBeer is the biggest commodity shipped in the trade. Heineken is the biggest shipper, followed by Becks and Guiness.\n\nNow came the hard part. I had to get the shipping contracts for these beer barons. Shipping line cartels have to file their contracts with the FMC, but the deals don't name the shippers. You have to look for things like port of origin to look for specific companies. For example, Heineken comes from Holland, so Rotterdam would be the biggest close port.\n\nIn times B.C. - before computers - reporters would have to check through the thousands of contracts at the FMC in Washington by hand. I never left my office in the New York World Trade Center.\n\nI used ProComm Plus for Windows to dial into another commercial database provider called Transax Systems. The database includes all shipping contracts filed with the FMC. You can search them by line, by type of cargo, by port, by date and other parameters. I retrieved the beer contracts and saved them.\n\nThen, using Claris Works for windows, I put together a spreadsheet that compared the rates for the two years. Heineken's rollback was about $12 per container - a little lower than the $100 per container the cartel said had been saved.\n\nThe cartel complained that the Heineken accounts did not truly represent the amounts and savings of the contracts on the Atlantic. Of course, they said, the biggest shipper would have the best deal, the lowest increase, and thus the lowest rollback, especially an importer whose goods, beer, was one of the cheapest goods to ship. To sketch a fairer picture of the contracts and rollbacks, they said, I would have to compare contracts for goods that cost more to ship, which would have a greater rollback.\n\nUsing PIERS data, I picked three of the top commodities being shipped in the trade - all the beer contracts and those for other alcoholic beverages and chemicals. Some contracts included rates for all three in the same deal.\n\nI used ProComm Plus to retrieve the contracts. But, because of the limitations of my equipment, I had to copy the information from the contracts by hand. I filled two notebooks with the columns. But I needed time to transfer the information into a spreadsheet, time my daily beat did not allow me.\n\nUntil, that is, the paper sent me on a cargo ship to Brazil. The three weeks on the ship gave me enough time to put together an Excel spreadsheet of more than 400 contract variations. There was little else to do at night aboard a ship off the coast of South America.\n\nWhen I finished the spreadsheet, I made a copy and saved it into a dbf file, which I opened with FoxPro. I queried the database on the average rollback costs for the commodities.\n\nI found that the average rollback was about $50 for each box. More than what was offered in the Heineken deal. But still only half of what the cartel said was the average rollback discount. I wrote a column about how the $65 million discount just didn't add up.\n\nMike Fabey can be reached by sent e-mail to MJF625775@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-august-power-lunches","slug":"1996-august-power-lunches","url":"https://dwillis.github.io/uplink/article/1996-august-power-lunches.html","year":1996,"month":"August","monthNum":8,"headline":"Power lunches","kicker":"CAR Trips: First ventures","author_name":"David M. Poole","author_title":"reporter","affiliation":"The Roanoke Times/Virginian Pilot","summary":"Journalists in Virginia used a FoxPro database to correlate lobbyist-funded meals with legislative voting schedules. The investigation highlighted a significant loophole in state disclosure laws that allows lawmakers to accept expensive meals without revealing their identities.","keywords":["Lobbyists","Virginia","Disclosure"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["FoxPro"],"full_text":"For the Virginia state capitol press corps, lobbyists' expense reports are a one-day story. The Roanoke Times/Virginian Pilot decided to delve deeper into the reports, looking for specific ways in which lobbyists influence the legislative process. The story, which ran July 12, began with a hunch: One sure way to a legislator's heart is through his or her stomach. To get the story, I built a database using FoxPro. The Virginia disclosure law requires lobbyists to report certain details of any meal or entertainment costing $100 or more. The lobbyist must indicate the number of lawmakers present, but need not name names. I created six fields: Name of company, name of individual lobbyist, date of expense, location of meal, number of legislators present, amount spent. The database contained 149 records, representing 42 companies. The next step was to look for any correlation between the dates a company took lawmakers to dinner and the dates when lawmakers voted on bills affecting the company. We logged onto the Legislative Information System to confirm the dates each vote reached a key committee or floor vote. The data revealed some interesting patterns. Virginia Power Co. entertained five lawmakers on Feb. 22, a day when the electric utility's centerpiece bill was unexpectedly stalled. The lack of lawmakers' names made it virtually impossible to establish a connection between lobbyists' generosity and lawmakers' votes. The loophole shifted the focus of the story to an all-too-common fact in Virginia politics: Disclosure laws that lead to less-than-meaningful disclosure.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-august-managing-car-reporting","slug":"1996-august-managing-car-reporting","url":"https://dwillis.github.io/uplink/article/1996-august-managing-car-reporting.html","year":1996,"month":"August","monthNum":8,"headline":"Managing CAR reporting","kicker":"What's up at NICAR","author_name":"Brant Houston","author_title":"executive director","affiliation":"NICAR","summary":"Brant Houston discusses the various models newsrooms use to manage computer-assisted reporting and provides updates on NICAR initiatives, including the NashCAR conference and training for minority journalists.","keywords":["Management","Training","NICAR"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Over the past year, many newsroom managers have been struggling with how to set up and oversee computer-assisted reporting. In response, NICAR is collecting the different models that news organizations have used for CAR. We plan to use the information to develop written materials and a curriculum for seminars. The working name of the national conference is NashCAR, to be held in Nashville March 6-9. It will offer panels and hands-on training for all levels of users. With the help of the Knight Foundation and Freedom Forum, NICAR has also increased opportunities for minority journalists. By January, more than 20 minority journalists will have gone through NICAR's week-long bootcamp seminars. NICAR's on-the-road trainer, Neil Reisner, has traveled across the country teaching at TV stations, newspapers, and universities. In January, Reisner plans to handle on-line training and database work, and we will hire a new on-the-road trainer.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-august-mexico-institute-hires-training-director","slug":"1996-august-mexico-institute-hires-training-director","url":"https://dwillis.github.io/uplink/article/1996-august-mexico-institute-hires-training-director.html","year":1996,"month":"August","monthNum":8,"headline":"Mexico institute hires training director","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Pedro Enrique Armendares has been appointed as the training director for a new investigative reporting project in Mexico, supported by a grant from the McCormick Tribune Foundation.","keywords":["Mexico","Journalism","Training"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Pedro Enrique Armendares has joined Periodistas de Investigacion - Investigative Reporters and Editors in Mexico as training director, effective Sept. 1. Through the project, made possible by a $540,000 grant from the McCormick Tribune Foundation, IRE and NICAR hope to offer low-cost or free training in newsrooms and at seminars across Mexico. Armendares' duties will include developing training programs in computer-assisted and investigative reporting. Armendares has worked since 1989 as an investigative/special projects reporter at La Jornada, a major Mexico daily. He will join Lise Olsen, managing director of Periodistas de Investigacion, in the Mexico City office.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-august-switching-from-foxpro-to-access","slug":"1996-august-switching-from-foxpro-to-access","url":"https://dwillis.github.io/uplink/article/1996-august-switching-from-foxpro-to-access.html","year":1996,"month":"August","monthNum":8,"headline":"Switching from FoxPro to Access","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"NICAR","affiliation":"University of Missouri","summary":"This technical guide outlines the key differences between FoxPro and Microsoft Access for journalists. It explains database object management, syntax variations in SQL, and the benefits of using Access for data analysis.","keywords":["FoxPro","Access","SQL"],"topics":["Technology","Data & Databases"],"technologies":["FoxPro","Access","SQL"],"full_text":"Some journalists love FoxPro; others just use it. But no one believes it's pretty or the easiest to learn. If you're making the switch, or starting to use Access for much of your work, here's a guide to the conversion. In FoxPro, every table, query, and index is a separate file. Access uses one Database file (.mdb) to hold objects like tables, queries, forms, and reports. This presents objects with a file cabinet metaphor. Character field length storage is another difference; long field lengths do not waste space in Access. The file size of an Access database increases as you add to it, but does not contract if you delete rows; you must use the Compact Database command. Making queries in Access is easier as you can type SQL with line breaks without the semicolons FoxPro requires. Syntax differences include dates, which use the pound sign (#) in Access, and wildcard characters like '*' and '?' which match DOS standards. Access defaults to being case-insensitive for criteria. You can use 'Attached tables' to query FoxPro files in their native format without duplicating them. You can also use long names with spaces for tables and columns in Access, whereas FoxPro limits you to 8 characters for table names.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-august-doing-business-on-the-web","slug":"1996-august-doing-business-on-the-web","url":"https://dwillis.github.io/uplink/article/1996-august-doing-business-on-the-web.html","year":1996,"month":"August","monthNum":8,"headline":"Doing business on the Web","kicker":"On the Internet","author_name":"Neil Reisner","author_title":"NICAR staff","affiliation":"NICAR","summary":"Neil Reisner provides a curated list of essential online resources and websites for business reporters. The list includes financial data providers, government agencies, and useful online calculators.","keywords":["Internet","Business","Websites"],"topics":["Business & Economy","Technology","Data & Databases"],"technologies":["World Wide Web","Internet"],"full_text":"The World Wide Web and other on-line resources can provide fertile ground for business reporters to mine. Here's a list of Web sites that might prove useful: American Stock Exchange (http://www.amex.com), Bizweb (http://www.bizweb.com), Business Web 100 (http://fox.nstn.ca/~at_info/w100_intro.html), Hoovers (http://www.hoovers.com), Investor Web (http://www.investorweb.com/company.htm), and the SEC/EDGAR database (http://www.sec.gov). Calculators are also available online, such as the Relocation Salary Calculator (http://www.homefair.com/homefair/cmr/salcalc.html). Government resources include the Census Bureau (http://www.census.gov), CIA World Factbook, Department of Commerce, and the Bureau of Labor Statistics (http://stats.bls.gov/dolbls.htm). Other useful sites include the Patent & Trademark Office (http://www.uspto.gov), GPO Access for GAO reports, and Thomas for Congressional data.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-august-from-the-nicar-library","slug":"1996-august-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-august-from-the-nicar-library.html","year":1996,"month":"August","monthNum":8,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering topics including campaign finance, healthcare, transportation safety, crime statistics, and loan guarantees for investigative journalism purposes.","keywords":["databases","government","journalism"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["CD-ROM"],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW A monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1995, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records. 1993, 1995.\nNEW National Bridge Inventory System data, includes inspection grades.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1994. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nNEW U.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-august-bits-bytes-and-barks","slug":"1996-august-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1996-august-bits-bytes-and-barks.html","year":1996,"month":"August","monthNum":8,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"ProfNet, an academic expert research tool, has faced reporter concerns after being sold to PR Newswire, which sells queries to paying clients and potentially compromises confidentiality. ProfNet management has responded by introducing query segmentation by institution type and launching an experts database to address privacy and targeting concerns.","keywords":["ProfNet","confidentiality","experts"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nProfNet response\n\nReporters who have used ProfNet, a valuable on-line research tool for locating academic experts on numerous subjects, have been concerned about ProfNet's recent sale to PR Newswire. That commercial outfit sells ProfNet queries to other paying clients, compromising the confidentiality of reporters' queries and sometimes resulting in a deluge of unwanted commercial appeals mixed in with sought after academic responses. PR Newswire has little incentive to keep queries quiet because the greater their distribution, the more money the company makes.\n\nHere's what ProfNet sysop Dan Forbush has had to say about the issue:\n\n\"Two recent developments at ProfNet are relevant.\n\n\"First, we'll soon be able to give reporters the ability to segment queries by type of institution. The more narrowly we can pinpoint the delivery of queries, the smaller the opportunity for off-target responses.\n\n\"By the end of this month, you'll be able send a query to the entire list or to any combination of the following:\n\nacademe\nnonprofit organizations\ngovernment\ncorporations\nPR agencies\n\n\"For example, if you want leads only on experts who can discuss the educational and First Amendment implications of filtering Internet content in the schools, we'll suggest you direct your query only to our academic list.\n\n\"If, on the other hand, your story is focused on the latest filtering products, we'll recommend a distribution that goes only to corporate and agency members. The larger and more diverse the ProfNet membership, the more powerful the resource. Segmentation is key, though, which is why we've spent the last six months developing this new distribution system.\n\n\"The second piece of relevant news is that we've created an on-line database providing bios and contact information for 2,000 leading experts across all disciplines - a utility that is proving particularly helpful to reporters either who are in immediate need of experts or who, for whatever reason, prefer not to conduct a sweep via ProfNet Search. Capable of full-text searches and 'relevance-ranking' of sources, ProfNet's Experts Database is up and running at www.vyne.com/profnet/ped\"\n\nDespite Forbush's reassurances, some reporters remain concerned about the changes in ProfNet, and wonder whether widely broadcasted queries could tip off competitors or subjects of investigations, or even whether they could be used in libel suits.\n\nTo raise these issues with ProfNet, call (800) PROFNET, or send e-mail to profnet@vyne.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-august-new-ire-l-and-nicar-l-addresses","slug":"1996-august-new-ire-l-and-nicar-l-addresses","url":"https://dwillis.github.io/uplink/article/1996-august-new-ire-l-and-nicar-l-addresses.html","year":1996,"month":"August","monthNum":8,"headline":"New IRE-L and NICAR-L addresses","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE-L and NICAR-L discussion lists have relocated to a new server at the University of Missouri with updated email addresses for posts and subscription management.","keywords":["lists","migration","subscription"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"IRE-L and NICAR-L have changed addresses. They are located on lists.missouri.edu, a dedicated Unix system designed to service discussion groups run out of the University of Missouri. All posts to the lists should now be sent to: ire-l@lists.missouri.edu or to nicar-l@lists.missouri.edu\n\nAll commands for subscription changes should be sent to: listproc@lists.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-we-did-it-our-way","slug":"1996-september-we-did-it-our-way","url":"https://dwillis.github.io/uplink/article/1996-september-we-did-it-our-way.html","year":1996,"month":"September","monthNum":9,"headline":"We did it our way","kicker":"Newsrooms share CAR models","author_name":"Angie Gaddy","author_title":"NICAR staff","affiliation":"NICAR","summary":"This article explores different newsroom models for implementing computer-assisted reporting at the Minneapolis Star Tribune, Boston Globe, Baltimore Sun, and Seattle Times.","keywords":["newsroom models","database editor","CAR training"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["FoxPro","Paradox","Quattro Pro","Access","Pentium","PC","9-track tape","CD-ROM","Excel","Atex","Internet"],"full_text":"Who does your data? That's what a lot of CAR newbies want to know from those using CAR in the newsroom. Whether run by a database editor, the library staff or a computer-savvy reporter, newspapers are structuring a CAR hierarchy. Four large dailies share their CAR models: It's been almost seven years since CAR began at the Minneapolis Star Tribune. At first, CAR was used in fits and starts, but in the past two years, a new push led by CAR director Bill Loving has gathered momentum. Loving's goal is to have more than a dozen reporters skilled and using CAR on a semi-regular basis by the end of 1996. Loving is accomplishing this goal by training and coaching the paper's newsroom staff of 350, half of whom are reporters. He has trained at least 100 reporters to use the Internet in his own workshops and plans to set up workshops for tackling database managers. He has sent six staffers to NICAR bootcamps. Loving has a CAR center in the newsroom equipped with two computers, including a Pentium and a 86PC, a nine-track tape reader, a zip drive and some external hard drives. He plans to get a scanner and CD-ROM writer. CAR reporters build queries using FoxPro, Paradox and Quatro Pro, but most stick to Access' query by example to build their own searches. While Loving waits for reporters to build skills, he is focusing on acquiring databases. Loving acquires databases on nine-track tapes, via FTP, CD-ROMs and diskettes. Most of the databases are stored on CD-ROM in the library. He is just starting to put databases on the newspapers network server. 'We've made strides in the last year,' he says. 'We have an enthusiastic backing from the managing editor and executive editor. That's an absolute necessity.' One way to incorporate CAR into the newsroom is by using an internal newsroom homepage, says Boston Globe computer-assisted reporter David Armstrong. The page - maintained by Matt Carroll, a part-time database editor, and by the paper's information services department is accessible only in the newsroom, from the reporter's homes and from bureaus. It is loaded on newsroom Pentium PCs, which are all equipped with CD-ROM drives. While the paper converts its newsroom Atex system to PCs, a process that is half-finished, reporters can access the Internet and the paper's databases at two computers at a public station in the newsroom. Eventually, all reporters will have a PC at their desks linked to the paper's internal homepage. Reporters can search databases of Massachusetts Motor Vehicle registration and driver's licenses; campaign contributions; and state, city and university system employees. The site also links to other homepages and newsroom chat groups. To make the system easier to use, there are front ends for accessing both the Internet and databases. Front ends are user-friendly interfaces that allow reporters to fill in templates, rather than build their own queries to maneuver around databases or the 'Net. Armstrong says there's no push for a nine-track tape reader in the newsroom because he can download databases more simply and with fewer data errors through FTP. Now that more data comes on diskettes or CD-ROMS, nine-track tape readers, which are clumsier, are going by the wayside in data transfer. Access is loaded on all PCs, but most Globe CAR reporters prefer FoxPro. Excel is the most popular software program for those doing CAR-approximately 12 reporters use spreadsheets for stories. The paper has brought in NICAR twice for training and has an in-house trainer from information services to help reporters. Carroll works one day a week updating databases, acquiring data, answering reporters' questions about data and working on queries with reporters. 'It's not an ideal situation, because on the other four days a week he's unavailable to reporters working with him. It's a handicap,' Armstrong says. Armstrong, who is lobbying for a full-time database editor, says what the paper needs is 'almost a cheerleader' in the newsroom, who encourages and sells the idea to reporters. With his desk equipped with a Pentium 90 with 1.6 gigs and DELL behind him, Mike Himowitz is literally situated between the newsroom and the library. His job as electronic news editor at The Baltimore Sun bridges library research with newsroom reporting. Since CAR is just starting at the paper, Himowitz says the paper does not have a CAR model. Its's evolving, and Himowitz is discovering what works and what doesn't at his paper. What works is creating front ends, or templates linked to database queries, to the paper's databases. Although it would make his job easier if reporters learned more CAR skills, his front ends makes it easier for reporters to find information without running to him with every question. 'Reporters don't have to become mavens (at CAR),' he says. 'Some disagree. Some want to make reporters computer-attack dogs, but you can't expect every reporter to do it. Most will learn the skills that apply to their particular beat.' Reporters only need to type in the name or address they are looking for in the Baltimore city property tax records database, or the school name or county in the Maryland school scores database. They can access the databases, all stored on a server linked to the PCs, at a computer station outside of the newsroom library, equipped with four Pentium 90s, all with CD-ROM drives and hooked to the Internet. One computer is hooked to a scanner and an Overland nine-track tape reader. There are two ends to CAR, Himowitz says: One is using it as an investigative tool. The other is using it to gather research for reporters. 'We will see more response when more stories come out,' he says. 'The big payoff hasn't come just yet.' Overseeing five news researchers, Tom Boyer began as database editor at the Seattle Times six months ago. The researchers look for news services, acquire new databases, coach, train and do higher end CAR work. Boyer's job is to coach researchers and reporters in CAR techniques, 'as well as be 'stopper' on technical issues that no one else can deal with.' Networked Pentiums are scattered across the Atex system newsroom, all loaded with Excel and running on a TCP/IP network so the research staff can build World Wide Web front ends. The 'hard-core' CAR computer is in the library. The paper's information systems department runs nine-track tape downloads, and they hire a local computer-consulting company to handle tougher tape downloads. Boyer says they are buying a high-end database server, most likely an MS SQL server, to handle large files. Presently, reporters come to researchers for help. But Boyer adds, 'We do not search the Times database for staffers generally. If they need help, we schedule them for training.' As the Globe and the Sun have done, Boyer plans to set up a front end so reporters can run their own database queries. And, as at the Globe, Boyer has created an internal Web site with a file of 3,000 source names, Times text archives and front ends to public records databases. 'We have to handle the basics first and organize things properly before Ican really start CAR work in earnest,' Boyer says. After that, he plans to start training reporters and editors on query techniques. 'But ultimately, it's not the responsibility of the database editor to make good CAR stories happen,' he says. 'If we're going to do this right, CAR has to come organically out of the work of the beat reporter.'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-september-picture-this","slug":"1996-september-picture-this","url":"https://dwillis.github.io/uplink/article/1996-september-picture-this.html","year":1996,"month":"September","monthNum":9,"headline":"Picture this","kicker":"Broadcasters vie for support","author_name":"John Sullivan","author_title":"NICAR staff","affiliation":"NICAR","summary":"Broadcast journalists struggling to get management support for computer-assisted reporting programs can start by investing in their own equipment and beginning with small projects to demonstrate value, according to experienced CAR reporters Mike Wendland and Alan Cox who offer practical advice for building newsroom computer programs.","keywords":["reporting","journalism","computers"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Lotus 1-2-3","Macintosh","9-track tape","Iomega Jazz drive","zip drive","T-1"],"full_text":"If you're a broadcast journalist looking to start a computer-assisted reporting program at your station but can't get support from the higher-ups, you're not alone.\n\n\"I get an e-mail every day from reporters looking for advice on how to start programs at their stations,\" says Mike Wendland, author of \"Wired Journalist\" and CAR reporter at WDIV-TV in Detroit.\n\nWendland's own experience may benefit those trying to convince news directors of the benefits of computers in the newsroom. Wendland started with equipment he purchased on his own. \"If you want to get something going, you just have to make the investment and buy the hardware to start at home. If you wait for someone to allocate the money, you may wait a long time,\" Wendland says.\n\nBack in 1985 when Wendland started using computers to do stories, it was much harder to convince management to invest in hardware. In recent years, some producers have recognized the benefits of computers and made the top-down push to move number-savvy journalists into the high-tech fray.\n\nAlan Cox at WCCO-TV in Minnesota admits his is an anomaly among broadcast programs. Cox says then news director John Lansing went to a 1993 IRE conference in Raleigh, N.C., and came back convinced computers were going to play a big role in the medium. Cox had done a county-by-county breakdown of state lottery funds in 1990 with Lotus 1-2-3 for DOS and a one-hour Macintosh training course at Kinko's. Lansing sent Cox for training, bought him some hardware, and gave him the luxury of two months free time to play around and learn what he could.\n\nBut Cox notes there was a precedent at WCCO. The station had done some of the earlier CAR stories under the direction of Pat Wyland. Cox produced stories right out of the gate and \"Computer Center 4\" was born. Cox and Wendland offer some specific advice for starting programs: If management is \"strongly encouraging\" you to embrace CAR, which may mean learn it or leave, Cox recommends getting professional training and as much material as possible. Cox says he came home from conferences with a briefcase full of IRE/NICAR handouts. Both Cox and Wendland agree that if management does not push you onto the dance floor, you need to start moving and shaking on your own. Cox calls it the guerrilla approach: \"Do whatever you can to get your hands on the equipment, even if it means going over to accounting and using what they have.\"\n\nWendland says a mistake many new CAR reporters make is biting off more than they can chew. He suggests starting small and producing stories to show news directors that investing in computers is in the station's best interest. Cox started out with Internet stories that showed consumers the kinds of services that were available on the 'Net. The hard data stories came later. Wendland and Cox now do complex investigative stories using a variety of databases and software.\n\nHardware\n\nDespite what many may think, CAR hardware necessary to do most stories can be bought on a reasonable budget. With memory as cheap as it is right now and ultra-fast processors making 486's hard to sell and therefore a potential CAR bargain - a system is definitely within reach. In fact, Cox still uses a 486DX2-66 with 32 mb RAM and two hard drives of a gigabyte each. He says the memory is especially helpful for situations like election night, when he runs a graphics program along with mapping and spreadsheets.\n\nWendland's shop uses four computers and several laptops for their four-person CAR team. The computers are all Micron units ranging from a Pentium 200 to several 120s. The 200 has two gigs of memory, and Wendland uses a one-gig Iomega Jazz drive and a zip drive for larger jobs. They are all networked to a T-1.\n\nSome hardware purchases are more specialized, such as a nine-track tape machine. However, new operations can get by without it since organizations like NICAR can process nine-track data.\n\nCox does have his own nine-track machine and some other useful tools, including an SVGA-to-NTSC converter that allows computer video to be put directly on air. Cox also says that after he lost a hard drive and found out how difficult it is to restore backups from nine-track, he got a Travan-style tape unit, which also comes in handy for moving large files from machine to machine.\n\nCox says his station uses an old 486 outfitted with an extra hard drive on the assignment desk so managers can search the station's databases.\n\nWendland has seen his CAR program grow to include a reporter, producer, and an associate-producer. Cox says he hopes his program expands with the purchase of some new hardware. Both programs prove that whether your resources are limited or unlimited or whether the initiative comes from you or someone else, you can do quality CAR stories for the broadcast medium.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-car-in-any-newsroom","slug":"1996-september-car-in-any-newsroom","url":"https://dwillis.github.io/uplink/article/1996-september-car-in-any-newsroom.html","year":1996,"month":"September","monthNum":9,"headline":"CAR in any newsroom","kicker":"Making it work","author_name":"Penny Loeb","author_title":"computer-assisted reporting specialist","affiliation":"U.S. News & World Report","summary":"Penny Loeb from U.S. News & World Report shares lessons learned from implementing computer-assisted reporting (CAR) in a newsroom, including the importance of mission statements, training, data organization, and developing a data-driven mindset among journalists.","keywords":["CAR","data","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Visual Basic"],"full_text":"CAR in any newsroom\n\nMaking it work\n\nBy Penny Loeb\nU.S. News & World Report\n\nIn the three-and-a-half years I've been at U.S. News, we've done about 70 computer-assisted stories. They range from major projects on special education, bad blood, redlining and air safety to which Senator pays the most to fly home.\n\nWe've taught about 20 people at some level. They have been through 10 weeks of classes, gone to IRE conferences, learned on projects. We have one of the best computer setups anywhere, with a great amount of software, Web access and e-mail for everyone.\n\nBut there's a lot more to do. Here's what I've learned:\n\nA mission statement is essential. Everyone from the top editor to the person who runs tapes for data processing must endorse CAR.\n\nYou need to let everyone know the possibilities of CAR. But you need to meet them at their level. Some people will go through the classes and do projects. Some will need someone to download and clean data.\n\nYou need to organize and prioritize. If you have a library, librarians should order and catalog data sets that are used routinely, such as contracts. You need to maintain lists of all data.\n\nYou need to make the data user friendly. Guides for use help. You can also write front-end search menus in Visual Basic.\n\nYou need a backup person who knows just about everything the CAR expert knows.\n\nThe hardest thing to teach is a data mindset: the ability to integrate the possibilities for data into every step of a story and the perseverance to survive the inevitable snags.\n\nPenny Loeb can be reached at (202) 955 2640, or send e-mail to ploeb@usnews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-ire-conference-handouts-notice","slug":"1996-september-ire-conference-handouts-notice","url":"https://dwillis.github.io/uplink/article/1996-september-ire-conference-handouts-notice.html","year":1996,"month":"September","monthNum":9,"headline":"IRE conference handouts notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE's June National Conference in Providence has made handout abstracts and materials available online and for purchase, with complete sets of 1,075 pages from 80 panels available for $150 or individual panel packets at 10 cents per page.","keywords":["conference","materials","ordering"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"A complete list of handout abstracts, many dealing with CAR, from IRE's June National Conference held in Providence is available on-line at http://www.ire.org/provfin2.html, or from the IRE front page at http://www.ire.org, where you'll find a Resource Center link. Each file includes material distributed and an audio tape reference number for every session. A complete set of handouts includes material from 80 different panels - a total of 1,075 pages. Cost for a complete set is $150 and includes shipping. Packets from individual panels are also available at the cost of 10 cents per page plus shipping. The minimum order is $5. Nonmembers must pay an additional $10 access fee. To order, send e-mail to Dawn Hobbs at dawn@ire.org, call (573) 882-3364, or fax (573) 882-5431.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-car-workshop-for-minority-journalists-notice","slug":"1996-september-car-workshop-for-minority-journalists-notice","url":"https://dwillis.github.io/uplink/article/1996-september-car-workshop-for-minority-journalists-notice.html","year":1996,"month":"September","monthNum":9,"headline":"CAR Workshop for Minority Journalists notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE, NICAR and The Richmond Times-Dispatch are hosting a Computer-Assisted Reporting Workshop for Minority Journalists in Richmond, Virginia on November 15-17, offering hands-on training in internet navigation, spreadsheets, databases, and data negotiation with only 10 spots remaining.","keywords":["journalism","training","data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"IRE, NICAR and The Richmond Times-Dispatch will offer the Computer-Assisted Reporting Workshop for Minority Journalists in Richmond, Va., Nov. 15-17. The hands-on training will include navigating the Internet, spreadsheets, database managers, negotiating for data and more. Only 10 of 40 slots remain available. To register, call Lisa Barnes at (573) 882-8969, or send e-mail to lisa@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-setting-net-policy","slug":"1996-september-setting-net-policy","url":"https://dwillis.github.io/uplink/article/1996-september-setting-net-policy.html","year":1996,"month":"September","monthNum":9,"headline":"Setting 'Net policy","kicker":"Wired","author_name":"Bill Dedman","author_title":"","affiliation":"The Associated Press","summary":"The Associated Press outlines ten essential guidelines for journalists to ensure responsible and professional use of the Internet and electronic services.","keywords":["Internet policy","journalism ethics","Associated Press"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","Email"],"full_text":"After collecting samples of newsroom policies on Internet use, The Associated Press devised the following policy to guide staff use of electronic resources. Some of the material is unique to AP, but may be useful to other news organizations interested in creating similar guidelines. AP Guidelines for Responsible Use of Electronic Services: 1. Accounts on AP's electronic services are intended to aid the business and professional activities of AP staff. The accounts are for business use only... 2. Each account is assigned for use only by the AP staffer. Sharing accounts is not permitted... 3. Conduct business on electronic services as if you are appearing at a public meeting representing the AP... 4. AP has long-standing rules against news employees participating in political activities or taking sides on matters of public debate. These rules apply to electronic communication as well... 5. To do their work, AP staff need to participate in electronic discussion groups on professional or technical topics... 6. When participating in discussion groups, be sure the reader knows that you are not stating AP policy... 7. Act as if the laws on libel and privacy apply to electronic communications... 8. Apply the strictest standards of accuracy to anything you find on electronic services. The Internet is not an authority; authorities may use it, but SO do quacks. Make certain a communication is genuine before relying on it as a source for a news story. More than one person may share an e-mail address, and e-mail addresses and Web page sponsorship can easily be faked. Ask yourself, 'Could this be a hoax?' Do not publish on the wire any electronic address without testing to see that it's a working address, and satisfying yourself that it is genuine. Apply, in other words, your usual news judgment. 9. Respect the copyrights of individuals and organizations, including the AP. Do not forward or post anyone's material without permission. Do not post or send to individuals any proprietary AP material, including news stories, photos, graphics, audio, video, data, or any internal communication. 10. Abide by the courtesies of the electronic community. Courtesy requires basic technical competence. For example, be careful not to send a message to a mailing list that was intended for only one user. Don't type in all caps; people will think you're shouting. Avoid the 'flame wars' that easily erupt when conversations are conducted on-line. And, because AP's Internet server has limited capacity, clean out your mailbox and home directory routinely, and log off when you're not using the system.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-september-on-the-internet","slug":"1996-september-on-the-internet","url":"https://dwillis.github.io/uplink/article/1996-september-on-the-internet.html","year":1996,"month":"September","monthNum":9,"headline":"On the Internet","kicker":"","author_name":"Nora Paul","author_title":"Internet trainer","affiliation":"Poynter Institute","summary":"A guide for journalists on designing and evaluating research web pages, including planning questions, features to include, and criteria for assessing website quality and usefulness.","keywords":["journalism","research","websites"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"On the Internet\n\nFor an in-depth look at problems with databases, check out \"Computer-Assisted Reporting: A Practical Guide\" by Brant Houston. It can be ordered from NICAR or Investigative Reporters & Editors for $26 plus shipping. Call (573) 882-2042.\n\nlaws, and an excellent listing of sources on journalism ethics.\n\nwww Virtual Library: Journalism http://www.cais.com/makulow/vlj.htm) John Makulowich, Internet trainer and early compiler of resources for journalists created this compilation of journalism related links. Section 8. Other Resources and Services, lists a number of research web pages.\n\nCreating research pages\n\nThink about the following when you go to design and compile lists of links. The answers to these questions will help give the page focus and clarity.\n\nPlanning and design:\n\nWHAT is the purpose of the web site?\nWHO is the audience for the site, who will be seeing the site?\nWHAT types of resources do you want to make available?\nWHAT kind of description will be made about the sites?\nWHO will evaluate recommended sites?\nHOW can sites be recommended?\nHOW will the sites be evaluated?\nWHO will maintain links to sites? What is the process for checking and maintaining?\nHOW will the site be organized? (around beats, topic areas, needs?)\nHOW will the site be searchable, how will be it navigated?\nSHOULD \"non-Internet\" resources be included? If so, which ones and how can they be?\n\nResearch Web features:\nAnnotation: Why this site is useful, what the site contains, specific news applications.\nInstruction: Information about how to use the link (hints about searching).\nSearchability: How can you get to the exact site you need?\nControlled vocabulary?: Is there a set of terms being used by the newsroom; if so, use them again!\nLinks to non-Internetted resources: Pointers to internal databases, records\n\nGuidelines for evaluating sites:\nWHO put the site/page together? What is their agenda? Why was it put together?\nWHO was the site designed for?\nWHAT does the site/page contain?\nWHAT perspective/side does the site take?\nWHEN was the information last updated?\nWHERE did the information come from?\nHOW is the information updated?\nHOW can this site help the journalist; what need is filled by this site?\nHOW easy is it to use (or how frustrating?)\n\nNora Paul can be reached at (813) 821-0583, or send e-mail to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-count-me-in","slug":"1996-september-count-me-in","url":"https://dwillis.github.io/uplink/article/1996-september-count-me-in.html","year":1996,"month":"September","monthNum":9,"headline":"Count me in","kicker":"CAR Trips: First ventures","author_name":"Maureen Smith","author_title":"Reporter","affiliation":"The Minneapolis Star Tribune","summary":"The Minneapolis Star Tribune education team successfully used spreadsheets on a tight deadline to analyze and compare eighth-grade test scores across 300 school districts.","keywords":["test scores","Quattro Pro","deadline reporting"],"topics":["Investigative Methods","Education","Data & Databases"],"technologies":["Quattro Pro","spreadsheet","Email"],"full_text":"The next time you think about crunching numbers with a calculator, think again. The education team at the Minneapolis Star Tribune walked into the land of computer-assisted reporting much like a nomad would wander into the forest: equipped with survival skills and resourceful minds, but little technical know-how. And we were more comfortable with calculators than computers. But we got over it. On deadline. It this summer. Our five-reporter-and-one-editor-team was strategizing around noon about the news to come: a breaking test score story. All we knew was that later in the afternoon, we'd have the test scores of scads of Minnesota eighth graders on new state-wide tests in math and reading. Comparing school districts: We already knew that fewer students passed the basic skills tests than hoped or expected. What grabbed our curiosity was how districts compared. By 3 p.m., a reporter came back armed with stacks of blue sheets of paper- one for each of about 300 school districts. Each blue sheet had the name of the district, the number of students who took the math and reading tests. But instead of telling us what percentage passed and failed, each sheet broke the students into six score categories that didn't neatly add up. The bottom line was that we had to do the math ourselves. Then began the debate: Should we use calculators, which we all knew how to use, or those unfathomable computer spreadsheets? Our education editor, Maureen McCarthy, bargained on the risk. Fortunately, Bill Loving, since named CAR director, happened to be within earshot. 'Hey Bill, do you think we could do this on a spreadsheet?' Loving and I sat down and I learned how to use Quattro Pro while we set up the spreadsheet. Another reporter prepared four others for data entry. It was going on 3:30 p.m. as I agonized over figuring out the formula for percentage and finding the formatting command for a percentage column. Meanwhile, three teams of two people each entered a stack of raw data: One read it, the other typed it in the computer. Each group also proofread their data entry list once. By about 5 p.m., we had three lists of school districts and merged them together into a common spreadsheet that summarized the percentage of students who passed and failed at each district. Despite some technical obstacles between reporter's PCs and graphics Macintosh layout system, we found a way to send the whole spreadsheet through e-mail. Graphics artist Mark Boswell cleaned up the data and made a chart. And we learned that obvious lesson in CAR-land: Computers work and think faster, better, and more efficiently than calculators.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-september-irenicar-student-regional-conference-notice","slug":"1996-september-irenicar-student-regional-conference-notice","url":"https://dwillis.github.io/uplink/article/1996-september-irenicar-student-regional-conference-notice.html","year":1996,"month":"September","monthNum":9,"headline":"IRE/NICAR Student Regional Conference notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are hosting a regional student conference at Syracuse University on November 9-10, featuring sessions on records research, interviewing, FOI, writing, and investigations for print and broadcast journalists. The conference offers affordable memberships bundled with registration, with additional hands-on computer training available on Sunday.","keywords":["conference","journalism","training"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"IRE and NICAR will hold a regional/student conference Nov. 9 and 10 at Syracuse University. The conference will have sessions on paper and on-line records; interviewing, FOI, writing, and investigations on a budget for both print and broadcast. Cost is $10 for members, $50 for non-members, which includes a year membership worth $40. The student cost is $35, which also includes a year's membership worth $25. Student members can also attend for $10. Panels will take place on Saturday. Hands-on computer training will be available on Sunday for an extra $25. For more information, e-mail Rose Ciotta at ciotta@buffnews.com or Rosemary Armao at armao@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-todays-top-10-list","slug":"1996-september-todays-top-10-list","url":"https://dwillis.github.io/uplink/article/1996-september-todays-top-10-list.html","year":1996,"month":"September","monthNum":9,"headline":"Today's Top 10 list","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"","affiliation":"University of Missouri/NICAR","summary":"Richard Mullins provides technical instructions on how to use SQL keywords and functions in Microsoft Access and FoxPro to retrieve top-ranked data results.","keywords":["SQL query","Microsoft Access","FoxPro"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Access","FoxPro","SQL"],"full_text":"Is there a way to write a query that just gives you the top 10 (or 20 or 101) rows? I'll cover Access first, then FoxPro. In Access, this query, which orders candidates according to their fundraising, doesn't limit the result to any rank: SELECT DISTINCTROW CandName, Receipts FROM CandRept ORDER BY Receipts DESC. Adding one keyword and a number gives the same results, but with a specified cutoff in rank: SELECT TOP 10 CandName, Receipts FROM CandRept ORDER BY Receipts DESC. There is no opposite keyword for TOP to get the bottom of the list. Instead, you do that by specifying whether the ranked columns should be ordered ascending or descending. You can also add the keyword PERCENT to make the query return only a certain portion of the ranked data: SELECT TOP 10 PERCENT CandName, Receipts FROM CandRept ORDER BY Receipts DESC. In FoxPro, this SQL will return the top 10 fund-raisers: SELECT CandName, Receipts FROM CandRept ; WHERE RECNO () < 11 ; ORDER BY Receipts DESC. The trick here is in the WHERE clause and the built-in function, RECNO(). For every table and query result, FoxPro can figure out the total record count and the record number of every row. Changing the expression in the WHERE clause lets you limit the results to a certain percent of the full set of rows: SELECT CandName, Receipts ; FROM CandRept ; WHERE RECNO () < 20 * (RECCOUNT () / 100) ; ORDER BY Receipts DESC. The built-in function RECCOUNT() determines how many rows are in the complete query result.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-september-advanced-bootcamp-notice","slug":"1996-september-advanced-bootcamp-notice","url":"https://dwillis.github.io/uplink/article/1996-september-advanced-bootcamp-notice.html","year":1996,"month":"September","monthNum":9,"headline":"Advanced Bootcamp notice","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"NICAR and Philip Meyer are offering an Advanced Bootcamp May 4-9, 1997 in Chapel Hill focused on statistics and maps for journalists.","keywords":["bootcamp","journalism","statistics"],"topics":["Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Don't forget that the Advanced Bootcamp offered by NICAR and Philip Meyer, author of \"The New Precision Journalism\" and journalism professor at the University of North Carolina-Chapel Hill, will be coming up May 4-9, 1997. The seminar, which is held in Chapel Hill, will concentrate on stats and maps. For registration information, send e-mail to Wendy Charron at wendy@nicar.org, or call NICAR at (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-the-heat-is-on","slug":"1996-september-the-heat-is-on","url":"https://dwillis.github.io/uplink/article/1996-september-the-heat-is-on.html","year":1996,"month":"September","monthNum":9,"headline":"The heat is on","kicker":"Fire data is hot","author_name":"David Migoya","author_title":"Reporter","affiliation":"Detroit Free Press","summary":"David Migoya explains how he used the National Fire Information Reporting System (NFIRS) to investigate fire statistics, land baron ownership, and the effectiveness of smoke detector programs.","keywords":["NFIRS","fire statistics","Devil's Night"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["NFIRS","9-track tape","Atlas GIS"],"full_text":"The Detroit phenomenon known as Devil's Night was only weeks away, and familiar newsroom rumblings of City Hall's dubious fire count accuracy began anew. Each year, the day before Halloween becomes the prime focus of pyromaniacs intent on destroying the scant homes that remain in Detroit's more impoverished neighborhoods. Similarly, each year, the Detroit Free Press puts a team of reporters on the streets to assess the latest fire damage and patiently awaits the release of the fire department's official fire count. Enter the National Fire Information Reporting System (NFIRS), a little-known database that nearly every state keeps in nearly identical form. I first learned of the NFIRS tapes while working with the Belleville (III.) News-Democrat in 1992. Every fire company, no matter if paid or volunteer, is required to fill out the reports that make up the database and send them to the marshal. For a mere $20, I received five years worth of nine-track tapes for the southwestern Illinois region. In East St. Louis, I was able to determine there was a higher propensity for fires and a greater ratio for burning houses owned by the land barons than in the rest of East St. Louis. For the Devil's Night analysis, I obtained five years of data for the Detroit metropolitan regions from the Michigan fire marshal. Summarily, the database contains: time, date, address, time call received, time dispatched, time arrived, type of fire, cause, and damage assessment. It's a CAR reporter's dream, but can be a nightmare because entries are entered by hand and keypunch accuracy varies. Last year, we analyzed the effectiveness of the Detroit Fire Department's two-year-old smoke detector giveaway program. By isolating actual house fires and combining fields for smoke detector presence and functionality, we were able to tell which areas had a greater number of detectorless fires and greatest need. We learned that 'exposure fires'—where a fire spreads from one building to another—were often reported as a single incident by the department, leading to an undercount of actual homes destroyed. Using Atlas GIS mapping software, we found the Devil's Night 'hot spots.' The NFIRS system is becoming more universal, although some departments still fail to file reports. If you run into a problem, call the data analyst for the state fire marshal.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-september-national-fire-information-reporting-system-availability-notice","slug":"1996-september-national-fire-information-reporting-system-availability-notice","url":"https://dwillis.github.io/uplink/article/1996-september-national-fire-information-reporting-system-availability-notice.html","year":1996,"month":"September","monthNum":9,"headline":"National Fire Information Reporting System availability notice","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"The National Fire Information Reporting System is available through the National Technical Information Service for $1,265, covering data through 1994. NICAR may consider acquiring the database if multiple journalists express interest despite the high cost.","keywords":["database","fire","NICAR"],"topics":["Public Safety","Data & Databases","Journalism & Newsroom"],"technologies":["National Fire Information Reporting System"],"full_text":"The National Fire Information Reporting System is available from the National Technical Information Service at (703) 487-4650. The most recent year available is 1994, and, according to NTIS, the cost is $1,265. If enough journalists are interested in this data, but find the cost prohibitive, NICAR will consider acquiring the database. Send e-mail to jo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-dont-get-burned","slug":"1996-september-dont-get-burned","url":"https://dwillis.github.io/uplink/article/1996-september-dont-get-burned.html","year":1996,"month":"September","monthNum":9,"headline":"Don't get burned","kicker":"What to look for in NFIRS","author_name":"David Migoya","author_title":"","affiliation":"Detroit Free Press","summary":"The National Fire Information Reporting System is a comprehensive database containing detailed fire incident data, but its complexity, manual data entry errors, and inconsistencies across states require careful verification by investigative reporters before use.","keywords":["database","firefighting","verification"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":[],"full_text":"So, what's in National Fire Information Reporting System, this amazing, previously unknown database?\n\nSimply put, everything.\n\nSummarily: time, date, address, time call received, time dispatched, time arrived, time left the scene, number of rigs, which station, which apparatus, type of fire, locations within the building, cause, number of injuries and deaths to civilians and/or fire personnel, even the amount of water and length of hose that was used. House fires, car fires, grass fires, trash fires, false alarms, investigative calls. If there were a line for a firefighter running in place, it would be in there.\n\nCAR dream\n\nIt's a CAR reporter's dream. It can also be his or her worst nightmare.\n\nThe data field code book is almost two inches thick and varies slightly from state to state. Some fields cancel each other out and others are completely unreliable. What at first glance appears to be a conflagration can quickly turn out to be a flaming paper bag. It has taken me four years to master the database because of its complexities. Best said, there's too much information, and in the hands of a haphazard reporter, it can easily turn into an ombudsman's career. It is extremely easy to make a mistake.\n\nAs with most databases, the information is entered by hand. Since 99 percent of the entries are numeric, keypunch accuracy quickly goes out the window. While the generic numbers the number of house fires, the number of false alarms, etc. - are normally reliable, the more specific information should always be double checked on the database and with department records if you can get them.\n\nSpecifics\n\nThe information is specific. A house fire can be further detailed to show if it's a two-story, brick, no-foundation house. Buildings are detailed by their use, such as a bowling alley, a 22-unit apartment, a nursing home, an elementary school, or a church.\n\nAddresses should always be confirmed through the department or in a city directory.\n\nFirefighters often are reliable when it comes to dousing flames, but filling out reports is another matter. For example, the address of a burning house at the corner of an intersection will either be entered accurately or as \"0000ELM\" or as \"ELMandMAIN,\" entries that make mapping difficult at best.\n\nCar fires have no address so the entry may either be the intersection, the street block number or the address of the nearest house. Assuming the latter, the information at first appears to be a house fire. The car fire information is found in a different data field, and its type whether it was an overheated engine, a smoldering seat cover or a true car fire - in yet another.\n\nSmoke detectors\n\nLast year, we analyzed the effectiveness of the Detroit Fire Department's two-year-old smoke detector giveaway program. The department didn't have the addresses to which they provided the detectors, but did have a tally by Census tract. Since the database also includes Census tract numbers, the story was still a go.\n\nBy isolating the number of actual house fires (you have to combine three data fields to assess the validity of a fire), separating the fires at abandoned houses, and adding the data fields that provide smoke detector information (there are three fields showing whether a detector was present, whether it worked and what role it had in warning occupants about a fire), we were able to tell which areas of the city had a greater number of detectorless fires and, arguably, the greatest need.\n\nThe database includes damage assessments, so we further isolated those fires that had significant damage. If there was little damage but a civilian injury, the information was included. Houses where there was more than one fire in a year were counted as one each year.\n\nOther data faults\n\nThen we grouped the information by Census tract, and to our dismay found scores of Census tracts that did not exist. Why? The department had been entering tract data based on indexes from 1980, not 1990 when several tracts were melded or separated into different tracts. We were forced to use a current city directory to fix the tract data and used a computerized tract index provided by the U.S. Census Bureau to clean the information.\n\nSince the department said it based its giveaway on need, we used Census data to profile the tracts that received the detectors and those that had the greatest number of detectorless fires and greatest need. What we found was telling: The department often played politics with a $15 smoke detector by providing them to areas that not only needed them least, but could afford them most. The areas that had the greatest need and often the most injuries received nothing.\n\nTo personalize the story, we isolated the detectorless fires that had an injury and crossed them with another database that provided the names of the injured (called a \"casualty report,\" you must ask for this separately from the NTIS or state fire marshal). We tracked the victims, and told their story.\n\nThe department has since changed how it gives away smoke detectors, many of which are donated by large corporations. The National Fire Protection Association also has adopted our method as an effective way to assess smoke detector need, and recommends it to participating departments.\n\nIn East St. Louis, we found how unreliable address ownership data can be. There are two data fields - owner and occupant - that are generally blank, wrong or interchanged. We needed to isolate the fires at houses owned by the two land baron families. We did this by hand checking every address with the county's tax rolls (the county had provided us with useless data tapes on three separate occasions, and we opted to do it by hand rather than waste any more valuable time). Once done, we were able to assess that the ratio of fires was greater for the land barons than in the rest of the city, indicating something more sinister might have been occurring.\n\nTry it out\n\nAs you can see, the database is swamped with information. Some innocuous, some vital. Most is accurate; much is not. The only way to know is to get the database and play with it. Start with a simple story, such as the number of arsons (two codes fit the bill: suspicious and deliberately set) or the number of false alarms. Danger zone: These are initial reports. Get the final cause for every apparent arson from the local fire marshal; otherwise the data will be skewed.\n\nTrying to tackle a much more difficult and complex piece, such as whether slumlords are burning their houses or whether the fire department effectively uses its resources, will do nothing but intimidate and frustrate you and cause you to fear what could be one of the newsroom's most valued resources.\n\nIf you run into a problem (which is likely), call the data analyst for the state fire marshal. They are the masters and often will be only too happy to help.\n\nDavid Migoya can be reached at (313) 222-6677, or send e-mail to migoya@det-freepress.com, or to 75613,262 (CompuServe).","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-ire-committees-volunteers-notice","slug":"1996-september-ire-committees-volunteers-notice","url":"https://dwillis.github.io/uplink/article/1996-september-ire-committees-volunteers-notice.html","year":1996,"month":"September","monthNum":9,"headline":"IRE committees volunteers notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE is seeking volunteers for its various committees, particularly the Computer Committee chaired by Shawn McIntosh, with contact information provided for those interested in volunteering.","keywords":["volunteers","committees","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"IRE committees always are on the lookout for good volunteers. Of particular interest to Uplink readers may be the Computer Committee, chaired by IRE Director Shawn McIntosh of The Dallas Morning News. To volunteer, send e-mail to McIntosh at dmnproj@airmail.net. For information on other IRE committees, send e-mail to IRE Executive Director Rosemary Armao at armao@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-100-computer-assisted-stories-notice","slug":"1996-september-100-computer-assisted-stories-notice","url":"https://dwillis.github.io/uplink/article/1996-september-100-computer-assisted-stories-notice.html","year":1996,"month":"September","monthNum":9,"headline":"100 Computer-Assisted Stories notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR have published '100 Computer-Assisted Stories,' a collection of accounts from newspaper and television reporters about their investigative projects using computer tools. The guide costs $20 and can be ordered by phone.","keywords":["reporting","computer","investigation"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"\"100 Computer-Assisted Stories\" is published by IRE and NICAR. Newspaper and television reporters recount how they got the story. The price is $20. To order, call (573) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-nicar-on-track-for-nashcar-conference","slug":"1996-september-nicar-on-track-for-nashcar-conference","url":"https://dwillis.github.io/uplink/article/1996-september-nicar-on-track-for-nashcar-conference.html","year":1996,"month":"September","monthNum":9,"headline":"NICAR on track for NashCAR conference","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR's NashCAR conference will be held March 6-9, 1997 in Nashville, Tennessee, featuring hands-on training in computer-assisted reporting for journalists of all skill levels across various beats and roles. The event will include vendor exhibitions, job opportunities, and discussions on the emerging impact of the World Wide Web on journalism.","keywords":["conference","journalism","training"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"NashCAR, NICAR's 1997 national conference, will be March 6-9 in Nashville, Tenn., with the support and help of The Tennessean and a panel of local journalists.\n\nThe conference will offer the widest range of panels and hands-on training yet. Building on the previous conferences in Santa Clara, Calif. CAR Trek), and Cleveland, Ohio (CAR Rock), NashCAR will have sessions for beginners, intermediates and advanced users.\n\nWe'll have tracks for beat reporters, editors, newsrooms managers, broadcasters, researchers and librarians, and trainers.\n\nThe conference will use of computer labs at local universities and will have a demonstration room to display equipment and software.\n\nWe'll also have vendors and numerous opportunities for journalists to find jobs.\n\nIn addition, we expect to explore the effect of the World Wide Web on journalism and how journalists are using the web for their stories.\n\nFor more information, call (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-get-your-training","slug":"1996-september-get-your-training","url":"https://dwillis.github.io/uplink/article/1996-september-get-your-training.html","year":1996,"month":"September","monthNum":9,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR and The Associated Press","summary":"NICAR and the Associated Press offer specialized training programs in computer-assisted journalism across multiple cities and conferences, teaching reporters to use databases, spreadsheets, and online tools for investigative reporting.","keywords":["training","journalism","databases"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["spreadsheet","Internet"],"full_text":"NICAR and, where indicated, The Associated Press provide specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases.\n\nCost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below.\n\nNote: In the following list, \"Open to all\" means any journalist may sign up. \"Closed\" means the session is open only to members of the host organization.\n\nRichmond, Va., AP - Oct. 3-4, 1996. Open to all. (804) 643-6646.\nKansas City, Mo., AP - Oct. 7-8, 1996. Open to all. All spots filled.\nKeene, N.H., Keene State College - Nov. 7-9, 1996. Open to all.\nKiplinger Seminar, Columbus, Ohio - Nov. 8, 1996. Closed.\nAllentown, Pa., Allentown Call - Nov. 12-13, 1996. Closed.\nNew Brunswick, N.J., AP - Nov. 15-16, 1996. Open to all. (609) 392-3622.\nWashington, D.C., Washington Post - Dec. 9-10, 1996. Closed.\nWashington, D.C. - Medill School of Journalism - Dec. 12-14, 1996. Open to all. (573) 882-0684\nMilwaukee, Wisc., Milwaukee Journal - Jan. 20-22, 1997. Closed.\nYork, Pa., York Daily Record - Jan. 27-28, 1997. Closed.\nSan Diego, Calif., San Diego Tribune and SPJ - Feb. 19-21, 1997. SPJ Day open to all. (619) 293-1261\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Wendy Charron at (573) 882-0684.\n\nRTNDA (Radio and Television News Directors Association), Los Angeles - Oct. 9-13, 1996.\nSEJ (Society for Environmental Journalists), St. Louis, Mo. - Oct. 17-20, 1996.\nMulticultural Management Program - NICAR, Columbia, Mo. - Oct. 13-18, 1996.\nIRE/NICAR Student Regional Conference, Syracuse, N.Y. - Nov. 9-10, 1996. Cost is: IRE member $10 for conference. Non IRE Member, professional rate $50 for conference (includes $40 IRE membership fee). Non IRE member, student rate $35 for conference (includes $25 student membership fee). Syracuse students, $5 to attend conference. Additional $25 fee for Sunday computer training.\nMinority Regional Conference, Richmond, Va., - Nov. 15-17, 1996. Open to minorities.\nMinority Regional Conference, Dallas, Texas - Dec. 7-10, 1996. Open to minorities.\nNashCAR, NICAR National Conference, Nashville, Tenn. - March 3-9, 1997. Costs are: IRE member, $150 for entire conference. Non IRE member, $190 for entire conference, including IRE membership. Renewal member, $190 for entire conference, including IRE renewal. Students, $125 for conference, including student membership. Late fee is $15 for registrations not postmarked by Feb. 14, 1997\nIRE National Conference, Phoenix, Ariz., June 12-15, 1997.\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo. As with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases - but you'll be drilled all day, every day for a full week. Cost varies. For information, call Wendy Charron at (573) 882-0684.\n\nJan. 5-11, 1997. Waiting list only.\nJan. Intermediate Bootcamp\nMay 4-9, 1997 - Advanced Bootcamp concentrating on stats and maps at the University of North Carolina-Chapel Hill with Philip Meyer, author of \"The New Precision Journalism\"\nMay 18-23, 1997.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-from-the-nicar-library","slug":"1996-september-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-september-from-the-nicar-library.html","year":1996,"month":"September","monthNum":9,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering topics including campaign finance, health care, transportation safety, crime statistics, and business records, available through subscription.","keywords":["databases","federal","government"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\nNEW A monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\nFederal procurement data, 1992-1995, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records. 1993, 1995.\nNEW National Bridge Inventory System data, includes inspection grades. 1994-1995\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nThe National Inventory of Dams. 1991-1995.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to hicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-bits-bytes-and-barks","slug":"1996-september-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1996-september-bits-bytes-and-barks.html","year":1996,"month":"September","monthNum":9,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"Bob Tiernan","author_title":"enterprise/database editor","affiliation":"Newsday","summary":"Newsday conducted a computer-assisted analysis of New York lottery data to reveal that gambling disproportionately affects low-income residents, while the Miami Herald seeks a leader for its new online services division.","keywords":["gambling","analysis","journalism"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["FoxPro","GIS"],"full_text":"Bits, Bytes and Barks\n\nNewsday gambling project\n\nAt Newsday in New York, we did a five-part series in December on how states had become hooked on gambling revenue and were introducing new games. We did a computer-assisted analysis of the demographics of who played the lottery, instant cash games and a new video keno game that you can play every five minutes.\n\nTo do this, we FOI'd the tape of more than a million lottery winners in New York State for the past five years and analyzed it using both FoxPro and a mapping program. We showed conclusively that the lottery was taking the most money from those least able to afford it. This contradicted the claims the state had been making to our reporters. Experts confirmed that this was a statistically valid approach, compared to just getting lottery revenue by outlet.\n\nOther parts of the series included a profile of a major vendor, GTECH, a look at how states are spending heavily to promote gambling ($40 million in New York), and a case study of Tunica, Miss., which has had problems since casinos went in. The reporters were Bill Falk, John Riley, Ford Fessenden and Stephanie Saul. - Bob Tiernan, enterprise/database editor, New York Newsday\n\nMiami Herald on-line job\n\nThe Miami Herald's On-line Services Division is seeking to fill a key leadership position; the title and job description will depend on the skills and interests of the applicants, but the person hired will help manage this new department.\n\nAmong the experience that might be helpful: World Wide Web page design, computer-assisted research and reporting, graphic artistry, marketing and promotion, computer programming, newspaper copy editing and page design. The ability to write, edit and converse in both English and Spanish is a plus.\n\nSend a resume and a note outlining why you think you're right for this job to Rich Gordon, the on-line services manager, at rgordon@herald.com, or via snailmail to him at 1 Herald Plaza, Miami, Fla. 33132. NOTE: If you send via e-mail, do not send the resume as an attachment - embed it as text in your message.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-september-profnet-update","slug":"1996-september-profnet-update","url":"https://dwillis.github.io/uplink/article/1996-september-profnet-update.html","year":1996,"month":"September","monthNum":9,"headline":"ProfNet update","kicker":"","author_name":"","author_title":"ProfNet sysop","affiliation":"","summary":"ProfNet has expanded its query targeting capabilities to allow journalists to search for expert sources across five specific institution types, including colleges, corporations, think tanks, government agencies, and non-profits.","keywords":["ProfNet","sources","institutions"],"topics":["Technology"],"technologies":[],"full_text":"ProfNet update\n\nDan Forbush, ProfNet sysop, says ProfNet queries can now target or exclude any of the following five types of institutions:\n\n1. Colleges and universities: 700 institutions in 17 countries, including the entire Ivy League and Big Ten.\n2. Corporations and PR agencies: 200 companies and agencies in all industry sectors.\n3. Extended academe: 100 think tanks, national laboratories, and other government and non-profit organizations oriented to teaching, scholarship and research. Also includes PR agencies representing academically oriented organizations.\n4. Government agencies: Includes 13 units of government at federal and state levels. (Elected officials not included.)\n5. Non-profit organizations: 100 professional and trade associations, hospitals, museums, and libraries.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-for-the-birds","slug":"1996-october-for-the-birds","url":"https://dwillis.github.io/uplink/article/1996-october-for-the-birds.html","year":1996,"month":"October","monthNum":10,"headline":"For the birds","kicker":"Data exposes avian deaths","author_name":"James Bruggers","author_title":"","affiliation":"Contra Costa Times","summary":"A journalist investigates bird deaths at California wind farms, discovering that nearly 680 birds of prey were killed over three years, with deaths increasing annually despite the clean energy benefits of wind power.","keywords":["wind","birds","investigation"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"I knew the wind farms of Northern California were killing birds in their giant blades. I knew it was enough of a problem that researchers were spending big bucks on studies designed to explain the avian deaths and find ways to reduce the mortality. What I didn't know was how many birds the turbines chewed up, nor did I have a breakdown of the kinds of species affected. I also did not know exactly how the birds died.\n\nSix months later\n\nAfter a six-month investigation and a computer analysis of mortality reports, I had my answers - and my readers did, too. On May 14, 1995, the Contra Costa Times reported that about 8,900 turbines in the hills east of San Francisco had killed or injured 679 birds over a period of three years and one month. Most were birds of prey.\n\nThe death toll included 266 red-tailed hawks, 64 golden eagles, 48 barn owls, and 34 American kestrels. None of these are protected by state or federal endangered species laws, but all are by other federal laws.\n\nThe government can fine companies and individuals thousands of dollars if they kill golden eagles or other migratory birds under the Bald Eagle Protection Act or the Migratory Bird Treaty Act.\n\nThe computer analysis also revealed that the deaths were on the increase. The wind farms killed 10 golden eagles, for example, in 1992, 23 the following year and 28 in 1994. The toll on red-tailed hawks climbed from 40 in 1992 to 164 in 1994. These numbers, however, were only a small part of the three-story package that began on the cover and filled most of two inside pages. The main story explored a deep rift in California's sizable environmental community over the issue. After all, wind energy is clean energy. It produces no greenhouse gasses, which many scientists blame for causing global warming.\n\nIt also detailed industry and government efforts to curb the deaths and revealed that there had been no prosecutions. Persuading a jury that the companies were intentionally violating the law would be difficult, a former federal prosecutor told me.\n\nThe second story documented how wind energy, once a booming affair in California, was facing an uncertain future due to land-use conflicts, the loss of subsidies and changing politics. Elsewhere in the United States, and in other countries, wind farms are expanding.\n\nThe third story was a short sidebar on a proposed new Bay Area wind farm.\n\nIt all started with a tip from a source who said he had heard that golden eagle deaths were increasing. I called the U.S. Fish and Wildlife Service and asked for mortality statistics, but was quickly rebuffed. They said the records were part of an ongoing criminal investigation, so I couldn't have them.\n\n'No' is not an answer\n\nInstead of filing a lawsuit, I followed another lead and found the records, or at least many of them. Companies operating wind farms in the region have to report mortality data to the Fish and Wildlife Service.\n\nIn Alameda County, thanks to a planning commission requirement, they must also submit the reports to the county planning department. The largest company, Kenentech, which has since filed for bankruptcy protection, also reported mortality data for neighboring Contra Costa Solano counties to Alameda County.\n\nHad Contra Costa and Solano counties also required companies to report mortality incidents, my numbers would have no doubt been higher. Those counties told me they weren't in the business of regulating wildlife.\n\nThe records came in two formats. One was a list that itemized each death or injury and included the date of the incident, species of bird, type of injury and location. The other was a sheet of paper that included the same information for each incident.\n\nNotes on the data also included written descriptions of how the birds died. \"It was these notes that impressed me, the descriptions of just how badly these birds were being mutilated, and the fact that the birds couldn't see the danger they were about to face,\" said Project Editor Gene Williams.\n\nBuilding the database\n\nContra Costa Times computer guru Morgan Cartwright used Visual Basic to design a front-end program that allowed easy data entry. To make sure we entered the data in a consistent fashion, he designed a series of scrolling list boxes. One list, for example, contained all the different kinds of birds. Another was the location of the incident. One more provided choices for different kinds of injuries.\n\nCartwright chose the database program Microsoft Access to use on our IBM-clone 486 computer to manage and manipulate the data.\n\nIt took a temp worker and myself three days to enter the data. Once the data was in, the numbers were extremely simple to calculate.\n\nAfter we crunched the numbers, I presented them to a variety of sources including the Fish and Wildlife Service, which had refused to share their information. Were we on target? No one objected.\n\nWe presented the numbers in roughly four paragraphs in the main bar and in a front-page graphic that also included the location of the turbines and energy trade-offs between wind and other sources.\n\nTips\n\nIf one government source won't give you the data you need, don't give up. Look for other agencies that might have the same information.\n\nBalance and juggle. Though I worked on this project for more than six months, I still produced regular daily stories. I'd try to devote one day a week to the birds. Once I was ready to write, I persuaded my editors to give me two weeks to complete the project.\nSee what others have written on the subject. A Nexis search turned up wind farm articles from around the globe.\nCheck your results. Run them by a variety of experts and see what they think.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-new-kid","slug":"1996-october-new-kid","url":"https://dwillis.github.io/uplink/article/1996-october-new-kid.html","year":1996,"month":"October","monthNum":10,"headline":"New kid","kicker":"SEJ homepage now located at www.reporter.org","author_name":"Russell Clemings","author_title":"SEJ board member","affiliation":"Society of Environmental Journalists","summary":"The Society of Environmental Journalists has moved its website to the IRE/NICAR server and is launching upgraded online services including a search engine, archived publications, curated resource links, and information about member conferences.","keywords":["journalists","environmental","website"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web","Internet"],"full_text":"There's a new kid in the www.reporter.org neighborhood: The Society of Environmental Journalists. SEJ moved to the IRE/NICAR server this fall as the first step in an ambitious upgrade for SEJ's online services.\n\nAs an SEJ board member and manager of its Web page, I'm excited about the move and the plans we have for new services to our members. Best of all, except as noted, all of the services we have are open to non-members as well.\n\nWe especially encourage Uplink readers and IRE members to take advantage of what we've been putting together. We hope to create a one-stop site for information about and Internet resources of use in environmental journalism. We're not quite to that point, but once we've settled into our new digs, the upgrades will be coming fast and furious.\n\nMore documents\n\nThe centerpiece is our Web page, at http://www.sej.org. Begun two years ago on space donated by the Tacoma (Wash.) News Tribune, our page has two types of resources. First, it has a rapidly growing collection of documents created by SEJ and related groups: Our quarterly SEJournal, with back issues to Spring 1995 and more being added steadily; the biweekly TipSheet, a guide to upcoming environmental news from SEJ, the National Safety Council's Environmental Health Center and the Radio and Television News Directors Association; and other publications. Our near-term plans include several new projects, such as \"Frequently Asked Questions\" files on environmental topics.\n\nPertinent hot links\n\nSecond, the SEJ Web page also contains a well-edited collection of links to other Internet sites that we believe are not just useful to environmental journalists, but also reliable. Some of these sites are featured in On the Internet on page six of this issue. Additional sites are added as they are brought to our attention. Our intent with these listings is to separate the wheat from the chaff, and focus on sites that either a) contain good, solid content, or b) have encyclopedic collections of links to other sites on the subject.\n\nSearch engine added\n\nWith our move to reporter.org we'll be adding one new feature that our members have requested many times: A search engine. Now, visitors to the Web page will be able to search by keyword for, say, an unfamiliar chemical that they have to write about on deadline, or a new subject - electromagnetic fields, for example - that they need to learn about in a hurry.\n\nThe Web page also contains information about how to join SEJ and regular updates on our national and regional conferences, many of which attract non-members as well. For example, on Sept. 7, we brought 80 people to Stanford University for an all-day conference on computer-assisted reporting and the Internet, an event co-organized by the Northern California Science Writers Association and sponsored in part by NICAR.\n\nWe've also posted frequent agenda updates for our 1996 national conference Oct. 17-20 in St. Louis, and as plans come together for our next national conference in Tucson Oct. 2-5, 1997, and the one after that in Chattanooga, Tenn., in 1998, we'll be posting updates on the Web site. The same is true for activities of our sister organization, the International Federation of Environmental Journalists. On the SEJ Web page, you'll find the IFEJ newsletter and a summary of plans for that group's annual conference in November in the Philippines.\n\nNew listservs\n\nThe Web page isn't the only thing we're doing at the sej.org domain. We're also creating several new listservs to supplement our existing, members-only list. At least one of these will be a general discussion list for environmental journalism, open to members and non-members alike. And in imitation of nicar.org, we also plan to create our own newsgroups for whatever purposes appropriate - looking for roommates at an upcoming conference, posting press releases and other announcements, and so forth.\n\nWe're grateful to NICAR for making room for us on its server, and to the Tacoma News Tribune for having given us a home during our formative period. We hope you'll visit the Web page at http://www.sej.org and let us know how you think it could be improved.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-what-toxins-are-wafting-your-way","slug":"1996-october-what-toxins-are-wafting-your-way","url":"https://dwillis.github.io/uplink/article/1996-october-what-toxins-are-wafting-your-way.html","year":1996,"month":"October","monthNum":10,"headline":"What toxins are wafting your way?","kicker":"Toxic Release Inventory","author_name":"Greg Reeves","author_title":"","affiliation":"The Kansas City Star","summary":"The EPA's Toxic Release Inventory database provides accessible, detailed information on chemical emissions from companies nationwide, making it an excellent resource for computer-assisted reporting on pollution patterns and trends in local communities.","keywords":["pollution","database","chemicals"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["FoxPro"],"full_text":"For a high-impact story using one of the most interesting databases around, take a look at the Toxic Release Inventory data from the Environmental Protection Agency. Company by company, across the nation, you can produce fascinating reports on the toxic pollutants that waft into our air, float down our rivers, are dumped in a landfill or—the way some toxins are removed—are shot a mile underground.\n\nGood for beginners\n\nIf you are a beginner at computer-assisted reporting with no more than a spreadsheet at hand, the TRI database would be an excellent entry point—cheap, clean data with lots of public interest. If you've thrown around a few gigabytes, you can generate great maps offering a detailed portrait of pollution in your community, neighborhood by neighborhood.\n\nSince 1987, federal law has required companies to report \"toxic releases\" to the EPA. They must report what chemicals were involved and what amounts (in pounds) ended up in the air, land, water, sewer or storage site. Each chemical emission is a separate report. And each report becomes a record in the TRI. The easiest way to get the data you want is probably a call to your local EPA office, state environmental office, or NICAR. Various Web sites also offer some TRI data. TRI data is not harmful to your computing environment. The files are roughly one megabyte per state per year. Missouri data, for example, contained 1,845 records for 1994 and fit neatly on one diskette. That included mid-Missouri lead smelter country, where lead emissions in some places total nearly 12 tons per child.\n\nWhat's in the data\n\nThere are 38 fields in the TRI database, and by now you are surely hungry to know them. Basically half the fields identify the \"facility,\" or company site, and the other half identify the chemical and where it went.\n\nFor each facility you get: an ID number, name, street address, city, state and zip; parent company (if any) and contact name and phone; and, for map-making purposes, the latitude and longitude of the facility.\n\nFor each chemical you get an English name (acetone, styrene, zinc, 1,2,4-trimethylbenzene, etc.), and a unique identifying code from the Chemical Abstracts Service (CAS) registry. Use the CAS number rather than the chemical name to sort and group data. For each release, you get: how much went up a smokestack (\"stack\" air); how much just drifted away (\"fugitive\" air); and how much went into the water, in the ground, to sewers or to offsite recycling plants. Finally, each record contains one or more industry codes and federal ID numbers to help identify the facility.\n\nHow to use it\n\nOne of the best uses you can make of the TRI data is to report how pollution has changed in your community over the past eight years. Have the worst companies from the 1980s cleaned themselves up? In FoxPro, you might want to find the biggest overall polluters first:\nSELECT facility, SUM (alw) FROM tri;\nGROUP BY 1;\nORDER BY 3 DESC\n\nThe first field, tri_id, is the facility ID number mentioned above. The \"alw\" field (\"air-land-water\") is the sum of stack-air, fugitive-air, land emissions and water emissions. Another handy add-up is stack-air plus fugitive-air, which equals total air pollution.\n\nThis query will give you a company name at the top and some huge number of pounds-emitted. Take the first 10 or so companies and see how they've fared since 1987:\nSELECT tri_id, year, facility, SUM(alw);\nFROM tri;\nWHERE INLIST (tri_id \"1st tri_id\", \"2nd tri_id...\");\nGROUP BY 1, 2 ORDER BY 1,2,3 DESC\n\nYou're on your way to a story that people need to know. And there's much more. Maps bring out TRI's impact in vivid clarity: per capita toxic waste, the most polluted ZIP code, the center of zinc toxicity in your community.\n\nCancer-causing chemicals\n\nNot all of these 75 known carcinogens may show up in the TRI data for your area. In the Missouri and Kansas data for 1994, for example, 48 of the chemicals appear. The following list includes the CAS number, followed by the chemical name and can be used in a look-up table.\n\n000140885 - Ethyl Acrylate\n000151564 - Ethyleneimine\n000075218 - Ethylene Oxide\n000096457 - Ethylene Thiourea\n000050000 - Formaldehyde\n000118741 - Hexachlorobenzene\n000302012 - Hydrazine\n010034932 - Hydrazine Sulfate\n007439921 - Lead\n000058899 - Lindane\n000101144 - 4,4'-Methylenebis (2 Chloroaniline)\n000101779 - 4,4'-Methylenedianiline\n000090948 - Michler's Ketone\n000134327 - Alpha-Naphthylamine\n007440020 - Nickel\n000139139 - Nitrilotriacetic Acid\n000079469 - 2-Nitropropane\n001336363 - Polychlorinated Biphenyls\n001120714 - Propane Sultone\n000075558 - Propyleneimine\n000075569 - Propylene Oxide\n000081072 - Saccharin (Manufacturing)\n000100425 - Styrene\n000096093 - Styrene Oxide\n000127184 - Tetrachloroethylene\n000062566 - Thiourea\n000584849 - Toluene-2,4-Diisocyanate\n000091087 - Toluene-2,6-Diisocyanate\n026471625 - Toluene Diisocyanate (Mixed Isomers)\n000095534 - O-Toluidine\n000088062 - 2,4,6-Trichlorophenol\n000051796 - Urethane\n000593602 - Vinyl Bromide\n000075014 - Vinyl Chloride\n000075070 - Acetaldehyde\n000060355 - Acetamide\n000079061 - Acrylamide\n000107131 - Acrylonitrile\n000060093 - 4-Aminoazobenzene\n000092671 - 4-Aminobiphenyl\n000090040 - O-Anisidine\n007440417 - Beryllium\n000542881 - Bis(Chloromethyl) Ether\n000106990 - 1,3-Butadiene\n007440439 - Cadmium\n000056235 - Carbon Tetrachloride\n000067663 - Chloroform\n000107302 - Chloromethyl Methyl Ether\n007440473 - Chromium\n008001589 - Creosote\n000120718 - P-Cresidine\n000135206 - Cupferron\n000615054 - 2,4-Diamino Anisole\n000101804 - 4,4'-Diaminodiphenyl Ether\n025376458 - Diaminotoluene (Mixed Isomers)\n000095807 - 2,4-Diaminotoluene\n000106934 - 1,2-Dibromoethane\n025321226 - Dichlorobenzene (Mixed Isomers)\n000106467 - 1,4-Dichlorobenzene\n000091941 - 3,3'-Dichlorobenzidine\n000107062 - 1,2-Dichloroethane\n000075092 - Dichloromethane\n\nReproductive toxins\n\nNot all of these chemicals, which are known to damage reproductive organs, may show up in the TRI data for your area. The following list includes the CAS number, followed by the chemical name, useful for look-up tables.\n\n000071556 - 1,1,1-Trichloroethane\n000071432 - Benzene\n007440439 - Cadmium\n000020042 - Cadmium Compounds\n000075150 - Carbon Disulfide\n000106898 - Epichlorohydrin\n000107211 - Ethylene Glycol\n000075218 - Ethylene Oxide\n000020100 - Glycol Ethers\n007439921 - Lead\n000108383 - M-Xylene\n007439965 - Manganese\n007439976 - Mercury\n000095476 - O-Xylene\n000106423 - P-Xylene\n000100425 - Styrene\n000127184 - Tetrachloroethylene\n000108883 - Toluene\n000079016 - Trichloroethylene\n001330207 - Xylene (Mixed Isomers)\n\nThen do the same thing with reproductive toxins, making a FoxPro table out of a list of their CAS numbers. See tables on page 4.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-cancer-causing-chemicals","slug":"1996-october-cancer-causing-chemicals","url":"https://dwillis.github.io/uplink/article/1996-october-cancer-causing-chemicals.html","year":1996,"month":"October","monthNum":10,"headline":"Cancer-causing chemicals","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This article lists 48 cancer-causing chemicals reported in the Toxic Release Inventory (TRI) data for Missouri and Kansas in 1994, providing CAS numbers and chemical names for reference and lookup purposes.","keywords":["chemicals","carcinogens","toxins"],"topics":["Health & Environment","Data & Databases"],"technologies":["Toxic Release Inventory"],"full_text":"Cancer-causing chemicals\n\nshow up in the TRI data for your area. In the Missouri and Kansas data for 1994, for example, 48 of the chemicals appear. The following list includes the CAS number, followed by the chemical name and can be used in a look-up table.\n\n000140885 - Ethyl Acrylate 000151564 - Ethyleneimine 000075218 - Ethylene Oxide 000096457 - Ethylene Thiourea 000050000 - Formaldehyde 000118741 Hexachlorobenzene 000302012 Hydrazine 010034932 - Hydrazine Sulfate 007439921 - Lead 000058899 Lindane 000101144 - 4,4'-Methylenebis (2 Chloroaniline) 000101779 - 4,4'-Methylenedianiline 000090948 - Michler's Ketone 000134327 Alpha-Naphthylamine 007440020 Nickel 000139139 Nitrilotriacetic Acid 000079469 - 2-Nitropropane 001336363 - Polychlorinated Biphenyls 001120714 - Propane Sultone 000075558 - Propyleneimine 000075569 - Propylene Oxide 000081072 - Saccharin (Manufacturing) 000100425 Styrene 000096093 Styrene Oxide 000127184 Tetrachloroethylene 000062566 Thiourea 000584849 - Toluene-2,4-Diisocyanate 000091087 - Toluene-2,6-Diisocyanate\n\nNot all of these 75 known carcinogens may 026471625 Toluene Diisocyanate (Mixed Isomers) 000095534 - O-Toluidine 000088062 - 2,4,6-Trichlorophenol 000051796 - Urethane 000593602 - Vinyl Bromide 000075014 - Vinyl Chloride 000075070 - Acetaldehyde 000060355 - Acetamide 000079061 - Acrylamide 000107131 - Acrylonitrile 000060093 - 4-Aminoazobenzene 000092671 - 4-Aminobipheny 000090040 - O-Anisidine 007440417 - Beryllium 000542881 - Bis(Chloromethyl) Ether 000106990 - 1,3-Butadiene 007440439 - Cadmium 000056235 - Carbon Tetrachloride 000067663 - Chloroform 000107302 - Chloromethyl Methyl Ether 007440473 - Chromium 008001589 - Creosote 000120718 - P-Cresidine 000135206 - Cupferron 000615054 - 2,4-Diamino Anisole 000101804 - 4,4'-Diaminodiphenyl Ether 025376458 - Diaminotoluene (Mixed Isomers) 000095807 - 2,4-Diaminotoluene 000106934 - 1,2-Dibromoethane 025321226 - Dichlorobenzene (Mixed Isomers) 000106467 - 1,4-Dichlorobenzene 000091941 - 3,3'-Dichlorobenzidine 000107062 - 1,2-Dichloroethane 000075092 - Dichloromethane\n\n000542756 - 1,3-Dichloropropylene 000123911 1,4-Dioxane 000117817 Di(2-Ethylhexyl) Phthalate 000106898 Epichlorohydrin (Dehp) 007440382 Arsenic 000064675 - Diethyl Sulfate 001332214 Asbestos (Friable) 000119904 - 3,3'-Dimethoxybenzidine 000071432 Benzene 000057147 - 1,1-Dimethyl Sulfate 000092875 Benzidine 000077781 - Dimethyl Sulfate 000098077 - Benzoic Trichloride","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-reproductive-toxins","slug":"1996-october-reproductive-toxins","url":"https://dwillis.github.io/uplink/article/1996-october-reproductive-toxins.html","year":1996,"month":"October","monthNum":10,"headline":"Reproductive toxins","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A reference list of chemical compounds known to damage reproductive organs, with their CAS numbers and names, that may not be fully tracked in Toxic Release Inventory data.","keywords":["chemicals","reproductive","toxins"],"topics":["Health & Environment","Data & Databases"],"technologies":["Toxic Release Inventory"],"full_text":"Reproductive toxins\n\nNot all of these chemicals, which are known to damage reproductive organs, may show up in the TRI data for your area. The following list includes the CAS number, followed by the chemical name, useful for look-up tables. 000071556 - 1,1,1-Trichloroethane 000071432 - Benzene 007440439 - Cadmium 000020042 - Cadmium Compounds 000075150 - Carbon Disulfide 000106898 Epichlorohydrin 000107211 - Ethylene Glycol 000075218 - Ethylene Oxide 000020100 - Glycol Ethers 007439921 Lead 000108383 M-Xylene 007439965 Manganese 007439976 - Mercury 000095476 - O-Xylene 000106423 - P-Xylene 000100425 - Styrene 000127184 - Tetrachloroethylene 000108883 Toluene 00108883 Toluene 000079016 - Trichloroethylene 001330207 - Xylene (Mixed Isomers)","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-car-helps-catch-big-fish-story","slug":"1996-october-car-helps-catch-big-fish-story","url":"https://dwillis.github.io/uplink/article/1996-october-car-helps-catch-big-fish-story.html","year":1996,"month":"October","monthNum":10,"headline":"CAR helps catch big fish story","kicker":"Trawling the 'Net","author_name":"Dan Kim","author_title":"NICAR staff","affiliation":"NICAR","summary":"In 1994, Times-Picayune reporters Mark Schleifstein, John McQuaid, and Bob Marshall investigated troubled Gulf of Mexico fisheries using computer-assisted reporting and internet research, uncovering a global story about overfishing, wetland destruction, and economic impacts on fishing communities worldwide.","keywords":["fishing","investigation","technology"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["World Wide Web","Internet","Excel"],"full_text":"In 1994, Mark Schleifstein of The Times-Picayune asked a simple question: \"Is the Gulf of Mexico fishing industry in trouble?\" Then, Schleifstein and reporting partners John McQuaid and Bob Marshall used CAR and the 'Net to catch a big fish. Gulf of Mexico Fisheries Management Council staff members told Schleifstein there would be plenty of fish - more than enough for both sport fishermen and commercial fishing - for the next 20 years. Staffers also said they had learned a lesson from the near-demise of the commercially valuable redfish, which was nearly wiped out in the 1980s by over-fishing, caused in part by the rise in popularity of Cajun blackened redfish. Yet, when Schleifstein asked the council how the continued rapid loss of wetlands along the Louisiana coast would affect the Gulf fishery, the council replied that if the wetlands continued to disappear at their present rate of destruction, most commercially-important species of fish would disappear, too. That's when Schleifstein realized there was a bigger story.\n\nWorld scope\n\nSchleifstein, McQuaid and Marshall soon found their investigation of fishing issues had expanded into a world-wide story explaining the commercial, environmental and social forces affecting the world's fishing industry. They examined the giant Japanese fish markets, which have driven much of the growth in the Gulf's fishing industry and endangered many Gulf fish species. They looked at environmental damage in Ecuador and Thailand resulting from aquaculture. They probed how overfishing has affected communities in Massachusetts, Maryland and Alaska. And they examined how the destruction of wetlands affects fishing and fishers in Louisiana and abroad.\n\nThe 15-month investigation resulted in \"Oceans of Trouble: Are the world's fisheries doomed?\" - an eight-part series published in March.\n\nTrawling the 'Net\n\nMuch of Schleifstein's computer work involved using the World Wide Web and Internet e-mail. He netted four listservs that were key to finding contacts among scientists and the fishing industry and to becoming quickly acquainted with the issues. He searched Web sites to find scientific papers on fisheries issues, as well as more fisheries contacts.\n\nWhile Schleifstein trawled the listservs, McQuaid handled most of the database work. The data, supplied by the National Marine Fisheries Service, included historical catches and market quantities of species for various regions and times. Some of the older data was provided on paper and had to be entered by hand. McQuaid entered and analyzed the data with Excel, generating summary data on fish catches by species and region.\n\nMost agencies he went to had little data on the economic impact of changes in the fishing industry or the quality of the fisheries, but economic problems caused by over-fishing became a primary focus for the series. To get at this issue, McQuaid ran market-value analyses for the fisheries catch data, using Consumer Price indices to normalize prices. McQuaid reporters to look for the agenda of the agency supplying data they use.\n\nProblems\n\nMcQuaid said the real difficulty in doing this story was the lack of statistics and data concerning the economic effects and trends in the fishing industry. Often, the number of people employed by the fishing industry was not well documented, nor was the actual economic impact of increased fish catch amounts and changing fish prices. McQuaid also emphasized that the lack of data was as important to the story as the data he was able to get: The lack of data, he said, limited the ability of agencies whose job is to protect the environment and the fisheries to effectively do that job. Most of the data he was able to get was only collected for species of fish that were already seriously threatened by over-fishing.\n\nThe agencies seemed to do little to gather data on existing fish stocks before problems arose.\n\nResults\n\nSchleifstein said he hopes the series may have helped shape the revisions to the Magnusson Act recently approved by Congress. The act is the major federal regulatory system governing commercial fishing. Items focused on in the series that are included in the revision are:\n\nIndividual transferable quotas, a way of assigning fish species-specific property rights to individual fishers aimed at protecting the resource while limiting the economic impact on fishers; and requirements that fish nets include devices to allow the escape of endangered sea turtles and fish bycatch - unwanted or too-small fish that often are thrown away when caught in existing nets.\n\nSchleifstein mentioned that the series' discussion of the link between the annual summer \"dead zone,\" or area of very low oxygen in the Gulf of Mexico, and the use of agricultural fertilizers in the Midwest, has been used by policymakers working with Midwest farmers to reduce their use of fertilizers.\n\nThe importance of wetlands, as the story points out, has led the Gulf of Mexico Fisheries Management Council to look at protecting the wetlands from further deterioration.\n\nYou can reach Mark Schleifstein at (504) 826-3327, or send e-mail to mersmia@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-environmental-sites-that-cook","slug":"1996-october-environmental-sites-that-cook","url":"https://dwillis.github.io/uplink/article/1996-october-environmental-sites-that-cook.html","year":1996,"month":"October","monthNum":10,"headline":"Environmental sites that cook","kicker":"On the Internet","author_name":"Russell Clemings","author_title":"","affiliation":"The Fresno Bee","summary":"This article provides a comprehensive guide to online environmental resources and databases, including government agencies like the EPA, nonprofit organizations like CNIE, and specialized databases for environmental journalists and researchers seeking information on environmental issues and regulatory documents.","keywords":["environment","resources","databases"],"topics":["Health & Environment","Technology","Data & Databases"],"technologies":["Internet","World Wide Web","Telnet"],"full_text":"National Library for the Environment: A project of the nonprofit Committee for the National Institute for the Environment (CNIE). At present, the library includes more than 100 reports from the Congressional Research Service (a branch of the Library of Congress) on the environment, agriculture, and natural resources. Eventually, it is supposed to include two other on-line components: \"An Encyclopedia of the Environment,\" and \"In Depth Issues in the Environment.\"\n\nhttp://www.cnie.org/nle/index.shtml\n\nEnvironmental Health Center: Part of the National Safety Council. Including a number of publications aimed specifically at environmental journalists, including the Environment Writer newsletter, \"Chemical backgrounders,\" and TipSheet, an alert to upcoming environmental news (produced in cooperation with the Radio and TV News Directors Foundation and the Society of Environmental Journalists).\n\nhttp://www.cais.com/nsc/ehc/ehc.html\n\nConsortium for International Earth Science Information Network: A nonprofit established in 1989 by universities and other research organizations in response to a directive by Congress. Includes thematic guides to key environmental issues (including agriculture, human health, land use, and ozone depletion) and the full text of more than 140 international environmental agreements, plus a searchable catalog of other Internet environmental data.\n\nhttp://www.ciesin.org/\n\nEnvironmental Protection Agency: On the whole, EPA's server is a mess. It's disorganized, and the search engine tends to return lots of trivial regulatory documents with little value. But at this URL, labeled \"Finding EPA Information,\" there are links to the catalog for EPA's library system, the full text of the agency information guide \"Access EPA,\" and other useful resources.\n\nhttp://www.epa.gov/epahome/finding.htm\n\nEnvirolink: A Yahoo for the environment. Contains hundreds of links to sites operated by environmental groups, with a brief summary of each group's purpose. Also has links to environmental news briefs and a large collection of links to environmental sites of varying worth.\n\nhttp://envirolink.org/orgs/\n\nGPO Access: While strictly speaking it's not an environmental site, this Government Printing Office service is extremely useful for retrieving documents generated by the environmental bureaucracy, including Federal Register notices, congressional bills and General Accounting Office reports.\n\nhttp://www.access.gpo.gov/su_docs/aces/aaces004.html\n\nSecurities and Exchange Commission EDGAR: The same is true of this site; it's not just environmental, but it contains plenty of data on publicly traded corporations that may be involved in environmental stories.\n\nhttp://www.sec.gov/edgarhp.htm\n\nOn-line environmental databases\n\nenvirofacts: EPA's effort to put five of its major environmental databases, including the Toxic Release Inventory, on-line in searchable form. Difficult to use, but improving.\n\nhttp://www.epa.gov/enviro/html/ef_query.html\n\nRTK-Net: A telnet site where an environmental group has mounted the Toxic Release Inventory and other databases (including some non-environmental ones, such as the Home Mortgage Disclosure Act) in searchable form.\n\ntelnet://rtk.net\n\nChemicals and toxicology\n\nExtension Toxicology Network: Peer-reviewed information about toxic chemicals, including pesticides. Includes contributions from the University of California-Davis, Oregon State University, Michigan State University, and Cornell University. Searchable.\n\nhttp://ace.ace.orst.edu/info/extoxnet/\n\nAgency for Toxic Substances and Disease Registry: From the U.S. Centers for Disease Control and Prevention. Includes HazDat (a database of contaminants found at Superfund sites, searchable by geographical area), plus toxicological profiles of hazardous substances, public health statements concerning hazardous substances, and Frequently Asked Questions about chemicals and pesticides.\n\nhttp://atsdr1.atsdr.cdc.gov:8080/atsdrhome.htm\n\nNational Toxicology Program: From the National Institute of Environmental Health Sciences. This agency produces toxicological data for use by other federal agencies. The site includes searchable lists of known and anticipated carcinogens and processes, and other health and safety data on chemicals.\n\nhttp://ntp-server.niehs.nih.gov/\n\nOther topics\n\nUniversities Water Information Network: Includes a database of U.S. Geological Survey water research, an experts directory, a searchable index of water-related web sites and a calendar of water events.\n\nhttp://www.uwin.siu.edu/\n\nEPA's stratospheric ozone depletion page: Contains information about the science of ozone depletion, federal regulations to protect the ozone layer, information on methyl bromide, information for the general public, links to the Montreal Protocol, and other topics.\n\nhttp://www.epa.gov/docs/ozone/index.html\n\nNuclear Regulatory Commission: Includes a variety of data on U.S. commercial nuclear power plants, including basic details about each plant and its design, SALP (for Systematic Assessment of Licensee Performance) reports, and the commission's \"watch list\" of troubled plants.\n\nhttp://www.nrc.gov/\n\nGlobal Change Master Directory: From NASA, a good starting point for research on global warming. Includes highly technical data sets, but also a searchable collection of links to other servers with more easily understandable content.\n\nhttp://gcmd.gsfc.nasa.gov/\n\nEndangered Species Homepage: From the U.S. Fish and Wildlife Service, includes the current U.S. endangered species list, plus many candidate species, news media fact sheets, the text of the Endangered Species Act as amended, and selected articles from the official Endangered Species Bulletin.\n\nhttp://www.fws.gov/~r9endspp/endspp.html\n\nRussell Clemings can be reached at (209) 441-6371, or send e-mail to clemings@cris.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-web-sites-for-job-listings","slug":"1996-october-web-sites-for-job-listings","url":"https://dwillis.github.io/uplink/article/1996-october-web-sites-for-job-listings.html","year":1996,"month":"October","monthNum":10,"headline":"Web sites for job listings","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A guide to online job listing resources for journalism and media professionals, including TV stations, Gannett positions, and IDG publishing careers.","keywords":["jobs","listings","careers"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Here are some Web sites for job listings recently posted in NICAR-L For TV jobs, try http://www.tvjobs.com, with links to a lot of individual station's Web sites. Gannett has updated its homepage to include job postings from all departments, including advertising, circulation, finance, human resources, marketing, news, online, production and systems. Check the \"Job Opportunities\" listing on the company's homepage at: http://www.gannett.com. IDG publishes computer-related publications from news-focused weeklies such as Network World to more feature-oriented publications such as PC World and Macworld. You can find job listings at: http://www.idg.com/idg/careers/listing/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-data-makes-ins-sweat","slug":"1996-october-data-makes-ins-sweat","url":"https://dwillis.github.io/uplink/article/1996-october-data-makes-ins-sweat.html","year":1996,"month":"October","monthNum":10,"headline":"Data makes INS sweat","kicker":"CAR Trips: First ventures","author_name":"April Lynch","author_title":"editor","affiliation":"The San Francisco Chronicle","summary":"A San Francisco Chronicle investigation using database analysis revealed that the INS rarely punishes employers who hire undocumented immigrants, with over 7,000 cases resulting in no fines, and even found that garments bearing actress Jaclyn Smith's label were made in such sweatshops.","keywords":["investigation","immigration","database"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["Paradox"],"full_text":"When Congress held hearings this summer on sweatshops and the garment industry, actress Jaclyn Smith turned down an invitation to stop by. The \"Charlie's Angels\" icon, who lends her name to a clothing line sold by K-Mart, said none of her apparel comes from sweatshops.\n\nApparently, she never saw a series from The San Francisco Chronicle from a few months before.\n\n\"Crime Without Punishment,\" investigating how the INS lets employers who hire undocumented immigrants off the hook, showed a shop full of women hunched over sewing machines in New York, stitching garments with the Jaclyn Smith label.\n\nParadox at work\n\nWith the help of Paradox, the series uncovered a long-standing pattern of employers flouting immigration laws while the feds looked the other way. Of about 12,000 cases filed by the INS against employers over five years, more than 7,000 resulted in no fines or punishment of any kind. The average fine was about $1,600 - small change for most companies.\n\nBig companies routinely negotiated fines down to zero, and then got the official records sealed. Agricultural growers, one of biggest hirers of undocumented workers, were hardly ever visited by INS inspectors at all. That was especially true in California, where farm interests are big contributors to Gov. Pete Wilson.\n\nPulling the three-day series together took almost two years. The project began when Chronicle reporter Aurelio Rojas decided to examine the INS emphasis on controlling U.S. borders rather than workplaces.\n\nHe first FOIA'ed the INS in early 1994, asking for a record of every case the INS had pursued against employers for the last five years. He got stonewalled for months. But in the summer of that year, INS Commissioner Doris Meissner visited The Chronicle. Rojas slipped her a letter and explained his plight.\n\nCovered in paper\n\nA few months later, an envelope three inches thick landed in his mailbox. It was the INS records, and they appeared almost completely useless.\n\nThere were more than 12,000 records listing businesses, addresses, fines assessed and fines that were actually paid. The records were in alphabetical order by business name. There was no way to sort all the paper.\n\nI had worked with Paradox and other database programs, and knew they could help make sense of the information. As the editor on the project, I suggested going back to the INS and asking for the disk or tapes that had generated the records. The INS refused, saying the computer information didn't exist.\n\nSo, we fed all 12,000 records into a scanner and transferred them into Paradox for Windows. That was a major improvement, until the realization hit that the scanner didn't always see numbers and symbols the way we did. The machine, for example, had sometimes read \"$\" as \"5,\" turning a fine of \"$10,000\" into one of \"$510,000.\" Cleaning up and double-checking all the data was a slow, painful task.\n\nIrrefutable facts\n\nOnce the data was in good shape, however, it revealed more than the INS could refute. Rojas traveled California, the Southwest and New York, visiting farms and factories. When he visited INS district offices armed with our findings, regional directors could not believe he had obtained the information he was showing them.\n\nEven after being confronted with The Chronicle's findings, the INS has fought every step of the way. Without computer analysis, getting through their records would have been almost impossible.\n\n\"I'm still waiting to get individual case files for some employers, which I requested in May 1995,\" Rojas said.\n\n\"Crime Without Punishment\" ran in March, shortly before Congress opened hearings on immigration law and sweatshop employers. Meissner promised quick reforms of how the INS allocates its resources. And closer to home, the series brought in $10,000 from readers for the families of three Mexican men electrocuted to death in a farm accident in California's San Joaquin Valley.\n\nApril Lynch can be reached at (415) 777-8432, or send e-mail to lyncha@sfgate.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-students-beat-odds","slug":"1996-october-students-beat-odds","url":"https://dwillis.github.io/uplink/article/1996-october-students-beat-odds.html","year":1996,"month":"October","monthNum":10,"headline":"Students beat odds","kicker":"CLAS scores","author_name":"Susan Lessmeister","author_title":"","affiliation":"Missouri School of Journalism","summary":"Fresno Bee reporters analyzed California test scores against socioeconomic factors, finding that some schools with disadvantaged students outperformed wealthier districts, challenging assumptions about the relationship between poverty and academic achievement.","keywords":["education","analysis","California"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["FoxPro","Microsoft Graph","Internet"],"full_text":"One in five students at Fowler High School near Fresno, Calif., cannot speak English and comes from a family on welfare. Yet, by at least one measure, such students outscored their peers at more affluent schools on the California Learning Assessment System tests.\n\nAnastasia Hendrix and Russell Clemings of The Fresno Bee compared the socioeconomic status of students in Fresno County schools with students' performance on the California Learning Assessment System (CLAS) tests.\n\nThrough their analysis, Clemings and Hendrix found that some schools did better than would be predicted given their economic backgrounds, which has been shown to have significant impact on academic performance.\n\nFive of the schools in the Central Unified School District scored significantly lower than their statewide peers. Central High School in this district is located in a suburban area west of Fresno.\n\nCLAS tests were given to over 1 million students in spring 1994. The test measured students' ability in reading, writing, and math. Students received a score for each subject area; the Bee averaged these scores for analysis, Clemings said.\n\n\"Generally, we published the data by ranking raw scores,\" Clemings said. But this time, the Bee included a comprehensive graph ranking the schools' performance against similar schools in California, measured by parent education levels and occupation.\n\nClemings said the data was readily available on the California Department of Education bulletin board system on the Internet. He converted the raw text files to .dbf files for manipulation.\n\nInterpreting the data was relatively easy, Clemings said, because the state department of education calculated the socioeconomic scores for each school.\n\n\"We had no major problems other than logistical,\" he said. In order to create the graph, Clemings downloaded the numbers into FoxPro and graphed them using Microsoft Graph.\n\nClemings' and Hendrix's efforts did not go unnoticed by parents and teachers.\n\n\"They were generally pleased with where their school came out,\" Clemings said. \"A lot of people called asking more questions about what the data meant.\"\n\nAdvice\n\nClemings had one piece of advice to offer reporters wanting to try a similar story: Use some other form of data to confirm that socioeconomic status is the most significant predictor of test scores. Clemings used Limited English Proficiency and percentage of minority enrollment to double-check his assumption.\n\n\"The idea that socioeconomic factors influence academic performance is not controversial, but it assures you don't go to the trouble to find data that it's the most important predictor and then learn it's something entirely different.\"\n\nRussell Clemings can be reached at (209) 441-6371, or send e-mail to clemings@cris.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-ire-committees-seeking-volunteers","slug":"1996-october-ire-committees-seeking-volunteers","url":"https://dwillis.github.io/uplink/article/1996-october-ire-committees-seeking-volunteers.html","year":1996,"month":"October","monthNum":10,"headline":"IRE committees seeking volunteers","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE is seeking volunteers for its committees, particularly the Computer Committee chaired by Shawn McIntosh, with contact information provided for interested volunteers.","keywords":["volunteers","committees","IRE"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"IRE committees always are on the lookout for good volunteers. Of particular interest to Uplink readers may be the Computer Committee, chaired by IRE Director Shawn McIntosh of The Dallas Morning News. To volunteer, send e-mail to McIntosh at dmnproj@airmail.net For information on other IRE committees, send e-mail to IRE Executive Director Rosemary Armao at armao@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-how-long-did-that-take","slug":"1996-october-how-long-did-that-take","url":"https://dwillis.github.io/uplink/article/1996-october-how-long-did-that-take.html","year":1996,"month":"October","monthNum":10,"headline":"How long did that take?","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"","affiliation":"University of Missouri/NICAR","summary":"This article explains how to use custom FoxPro functions (DateTime, DateTimeDiff, and Hours) to calculate elapsed time between two date-time combinations, which is useful for analyzing database records like court cases, inspections, and emergency response times.","keywords":["database","calculations","FoxPro"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["FoxPro"],"full_text":"Many databases include dates that define significant periods of time and deserve analysis. Court data can have arrest date, trial date and sentence date, for example. Inspection data can have date reported, date inspected and date enforcement action was taken.\n\nDatabase software has the capability to do calendar arithmetic to answer questions such as, \"Which case went the longest between arrest and trial?\" or \"Which inspection cases took the longest to get around to?\"\n\nSometimes, however, hours and minutes count. For example, you might want to calculate response times for emergency calls, phone call duration, or how long someone was in jail. Comparing the elapsed time with this information is a little more involved, since the beginning time may start at 2:03 p.m. on one day and the ending time may be at 5 a.m. the next morning or the next week.\n\nThis month's tip is a custom FoxPro function that you can use to do these time calculations. You can download the file TimeDat.zip from http://www.reporter.org\n\nThe file contains the source code for the functions, a sample table to test with, instructions on how to use the file and the functions.\n\nThe functions are: DateTime(), DateTimeDiff(), and Hours(). You can use these functions as you would the regular built-in FoxPro functions: in a query, or in a FoxPro command such as REPLACE ALL. Copy the file TimeDat.prg to your FoxPro directory and then in the command window, type:\n\nset procedure to TimeDat.prg\n\nThis makes the functions available. If you exit FoxPro or issue the CLOSE ALL command, you will have to type the command again to use the functions. For more information, see the documentation file included in TimeDat.zip\n\nFor illustration, I'll assume a table called TEST (included in TimeDat.zip) with five columns: ID, BeginDate, BeginTime, EndDate and EndTime. The question we're trying to answer is how much time passed between one starting date and time and the ending date and time.\n\nThe function DateTime takes two arguments: a date and a clock time. The clock time must be in the 24-hour format and indicated in hh:mm:ss or in hh:mm. The function converts the date and clock time to a single unit to define a point in time: the Julian day and the decimal proportion of the time of that day. If you want to think of Julian date as \"star date,\" that's OK, since it is used in astronomy to sidestep the quirks of our calendar.\n\nThe function DateTimeDiff compares two DateTime results and returns the time interval, expressed in days. To convert the interval to hours, divide the result by 24.\n\nHere are some query examples. This one uses DateTime:\nSELECT id, ;\nDateTime (begindate, begintime) ;\nDateTime (enddate, endtime) ;\nFROM test\n\nThis next query adds one more output column, using to calculate the days elapsed between beginning time/date and the ending time/date. Note that the functions are nested: the values returned from DateTime are passed to DateTimeDiff:\n\nSELECT id, ;\nDateTime (begindate, begintime) ;\nDateTime (enddate, endtime) ;\nDateTimeDiff (DateTime (begindate, begintime) ;\nDateTime (enddate, endtime) ) ;\nFROM test\n\nHere is how the DateTime function works, applying it to June 16, 1996, at 15:46:31.\n\nThe Julian date for 06/16/96 is 2450251. A Julian date is just a really big number: 2,450,251. It doesn't break down into anything else. At 15:46:31 on the clock, the day is 15.7753 hours old. Dividing that by 24 shows what proportion of the day has passed: 0.6573. Adding the Julian day and the proportion of the incomplete day gives the absolute reference we need to measure time periods that span a calendar day. With this kind of measure we can do time/date arithmetic on the mayor's cell phone calls. Here is what the FoxPro command window would look like to get these answers. The SYS(11) function is a built-in FoxPro function that converts a calendar date to a Julian date. FoxPro has more than 50 different SYS() functions (In the commands below, the lines in bold would show up on the screen, not in the command window).\n\nRichard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-excel-tip-flipping-data","slug":"1996-october-excel-tip-flipping-data","url":"https://dwillis.github.io/uplink/article/1996-october-excel-tip-flipping-data.html","year":1996,"month":"October","monthNum":10,"headline":"Excel tip: flipping data","kicker":"","author_name":"Tim Henderson","author_title":"","affiliation":"Gannett Suburban Newspapers","summary":"Tim Henderson shares a tip for transposing data in Excel by using Copy and Paste Special with the Transpose option, noting that the Cut function does not work for this operation.","keywords":["Excel","transpose","data"],"topics":["Investigative Methods","Technology"],"technologies":["Excel"],"full_text":"Tim Henderson of Gannett Suburban Newspapers in White Plans, N.Y., recently posted the following tip on NICAR-L: The way to \"flip\" data in Excel is this: Select the area you want to flip, copy it onto the clipboard, then choose \"Paste Special,\" and check \"transpose\" in the resulting dialog. It doesn't work with \"cut,\" only \"copy\" (at least in Excel 4.0), so you have to paste the transposed data in another worksheet, or another part of the same worksheet and then delete the original selection if you want to get rid of it.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-math-for-journalists","slug":"1996-october-math-for-journalists","url":"https://dwillis.github.io/uplink/article/1996-october-math-for-journalists.html","year":1996,"month":"October","monthNum":10,"headline":"Math for journalists","kicker":"Handout of the month","author_name":"Scott Thurm","author_title":"","affiliation":"San Jose Mercury News","summary":"A guide for journalists on using basic mathematics, particularly percentages and percent change, to accurately report numerical information in news stories, with practical examples including the biblical flood narrative and salary calculations.","keywords":["percentages","journalism","mathematics"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["Excel"],"full_text":"Math for journalists\n\nFor more information on how to successfully use numbers in news stories, consult: John Allen Paulos, \"Innumeracy,\" Vintage Books, 1988 John Allen Paulos, \"A Mathematician Reads the Newspapers,\" Basic Books, 1995 Victor Cohn, \"News and Numbers,\" Iowa State University Press, 1989 Philip Meyer, \"The New Precision Journalism,\" Indiana university Press, 1991\n\nBefore you can use a spreadsheet, you need to understand the math behind the program. Here is one reporter's instructions, as presented at the 1996 IRE National Conference in Providence, R.I. Excel uses cell addresses to instantly perform the calculations below, so once you understand the math, you can make a spreadsheet do the heavy lifting.\n\nThe Great Flood: Did it happen?\n\nIt's simple arithmetic, and context, that will save your butt, not complex statistical analysis or calculus. We'll cover percent, percent change, percentiles, compound percentages, mean vs. median, context, and resources. The Bible says, \"All high hills that were under the whole heaven were covered.\" Let's assume the highest peak was around 5,000 feet. Rain: 40 days + 40 nights = 960 hours. Rate: Two to three inches per hour is considered heavy rain. So, let's assume one foot per hour. Amount: 960 feet, not accounting for any runoff. How hard would it have had to rain? 5,000 feet / 960 hours = 5.21 feet per hour, not accounting for any runoff.\n\nPercents\n\nPercents are a standardized way of expressing a fraction, where the denominator (the bottom number) equals 100. So, 1/4 is 25 percent, 1/3 is 33 percent, etc. Percent Change = \"change\"/\"old\" Example: 1996 budget = $500,000 1997 budget = $400,000 Change = $100,000 Percent change = $100,000 / $500,000 0.2 = minus 20 percent. (We frequently say something declined by 20 percent.) Note that if the budget were cut by $100,000 this year and restored next year, that would be a 20 percent decrease, followed by a 25 percent increase. Lesson: You can't just add percentages to compute an overall percentage change. In general, quantities cannot decline by more than 100 percent, because there's nothing left.\n\nHowever, there is no upper limit on how much something can increase, in percentage terms. Example: Michael Jordan's 1986 salary = $1 million Michael Jordan's 1996 salary = $8 million Change = $7 million Percent change: $7,000,000/$1,000,000 = 7 = 700 percent\n\nPercentage points\n\nThese are useful when comparing two numbers already expressed as percentages. March unemployment = 5 percent April unemployment = 6 percent Change = 1 percentage point Percent change = 1 (the change) / 5 (the old) = 20 percent\n\nCompound percentages\n\nA school district signs a contract giving teachers 8 percent raises in each of the next three years. What's the percentage increase in the contract? Note that the percent increase is applied to a bigger base each year. Assume the average teacher made $30,000 under the old contract. First year: $30,000 * 0.08 = $2,400, so new salary is $32,400. Second year: $32,400 * 0.08 = $2,592, so new salary is $34,992. Third year: $34,992 * 0.08 = $2,799, so new salary is $37,791. Total percent change: $37,991 - $30,000 = $7,991 (change)/$30,000 (old) = 0.2597 = 25.97 percent\n\nMean vs. median\n\nMean is an arithmetic average. Add up the values and divide by the number of values. Median is the middle value in a group, where the values have been ranked from top to bottom. (If there are an even number of members in the group, the median is the average of the middle two.) How they can differ: Assume there are 100 workers at a company, and 95 make $30,000 a year, while five make $300,000. Mean = (95 * 30,000) + (5 * 300,000))/100 = $4,350,000/100 = $43,500 Median = $30,000 (the average of the 50th and 51st highest salaries) Which is more reflective?","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-math-for-journalists-resources","slug":"1996-october-math-for-journalists-resources","url":"https://dwillis.github.io/uplink/article/1996-october-math-for-journalists-resources.html","year":1996,"month":"October","monthNum":10,"headline":"Math for journalists resources","kicker":"","author_name":"Scott Thurm","author_title":"","affiliation":"San Jose Mercury News","summary":"A curated list of essential books for journalists seeking to improve their numerical literacy and data reporting skills, featuring works by mathematicians and journalism experts on handling statistics and numbers in news stories.","keywords":["mathematics","journalism","numeracy"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Math for journalists resources\n\nFor more information on how to successfully use numbers in news stories, consult: John Allen Paulos, \"Innumeracy,\" Vintage Books, 1988 John Allen Paulos, \"A Mathematician Reads the Newspapers,\" Basic Books, 1995 Victor Cohn, \"News and Numbers,\" Iowa State University Press, 1989 Philip Meyer, \"The New Precision Journalism,\" Indiana university Press, 1991","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-weve-got-plans-for-you","slug":"1996-october-weve-got-plans-for-you","url":"https://dwillis.github.io/uplink/article/1996-october-weve-got-plans-for-you.html","year":1996,"month":"October","monthNum":10,"headline":"We've got plans for you","kicker":"NICAR notes","author_name":"Brant Houston","author_title":"managing director","affiliation":"NICAR","summary":"NICAR announces plans to expand its computer-assisted reporting programs and services, including increasing publication frequency, adding new seminars, hosting a national conference in Nashville, and offering more training opportunities for journalists of all levels and backgrounds.","keywords":["expansion","training","journalism"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Lotus Notes"],"full_text":"We've got plans for you\n\nBy Brant Houston\nNICAR managing director\n\nIt's the time to accelerate again at NICAR. Over the coming months, we will start a series of new efforts and expand on the many already underway. Here's a partial list. We will:\n\nIncrease Uplink from 12 to 16 pages in order to accommodate the interests and requests of its readers. We will include \"a handout of the month\" feature, bring you more news of programs at NICAR, Investigative Reporters & Editors and the Missouri School of Journalism, and expand our coverage of CAR in broadcast and print and at universities.\nAdd an intermediate level seminar (coming this January) on data cleaning, good database design, and advanced data analysis.\nAdd a seminar in the spring on creating searchable databases on the Web.\nBecome a broader repository for common and often used data, particularly state and local campaign data.\nContinue our programs to make CAR available to minority journalists by offering fellowships and working with the journalism school's Multicultural Management Program. We are giving three regional seminars for minority journalists in the next few months, one in Richmond, one in Dallas, and one in San Francisco.\nOffer management seminars for broadcast and print journalists on how to bring and keep a good CAR program at their news organizations.\nContinue our basic Bootcamps. Our January Bootcamp is full, and our May Bootcamp is filling fast. We'll have two more in the summer.\nGive an expanded national conference in Nashville in March. This national conference, dubbed NashCAR, will not only cover the usual computer-assisted reporting panels and training, but also will have advanced classes in mapping, statistics, programming, Web development, and Lotus Notes.\nDevelop more joint programs with other universities. We already work with Northwestern University, the University of North Carolina, the University of Maryland and Columbia University.\nContinue our on-the-road programs to better help small- to medium-size news organizations, but also add on-line tutorials as follow-up training.\n\nBrant Houston can be reached at (573) 882-0684, or send e-mail to brant@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-contest-publications-and-more","slug":"1996-october-contest-publications-and-more","url":"https://dwillis.github.io/uplink/article/1996-october-contest-publications-and-more.html","year":1996,"month":"October","monthNum":10,"headline":"Contest, publications and more","kicker":"IRE notes","author_name":"Bruce Moores","author_title":"staff","affiliation":"IRE","summary":"IRE offers campaign coverage resources and CAR techniques for election reporting, and invites journalists to submit computer-assisted reporting stories for their idea book and annual investigative reporting contest.","keywords":["campaigns","reporting","data"],"topics":["Investigative Methods","Government & Politics","Journalism & Newsroom"],"technologies":["Internet"],"full_text":"Looking for last-minute, substantive story as the candidates begin the homestretch to the November elections? We can help. The IRE Resource Center has produced a campaign coverage booklet that includes resources and tips for covering campaigns on-line, analyzing FEC data through CAR techniques, backgrounding members of Congress, individual contributors or businesses. A supplement to the booklet includes handouts from IRE's National Conference this June in Providence, such as an FEC guide to contributions and the law, among other enlightening nuggets.\n\nYou've spread out the data, polished the numbers, mined the Internet, backgrounded and double-checked your facts and figures, written sharply and nailed a story. You're proud of it, so why not share it? We are looking for stories that used computer-assisted reporting. We're developing our newest \"idea\" book, where we take not the story, but the information on how the story was done and publish it so that other reporters don't have to start from scratch when embarking on a similar story. Look for fresh ideas, new perspectives on old problems and, more importantly, practical stuff including a story synopsis, list of sources, databases, spreadsheets, and Internet sources. Read about therefore avoid the pitfalls that tripped up others.\n\nSubmit stories to consider to Brant Houston, c/o NICAR, 138 Neff Hall Annex, Missouri School of Journalism, Columbia, Mo., 65211. Along this same thread, consider submitting your best work for the upcoming IRE Annual Contest for the best in investigative reporting. The deadline is Jan. 15. We'll send out entry forms soon. Remember to check out the contest information, and a list of previous winners, on the IRE homepage at http://ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-get-your-training","slug":"1996-october-get-your-training","url":"https://dwillis.github.io/uplink/article/1996-october-get-your-training.html","year":1996,"month":"October","monthNum":10,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers specialized training in data journalism and computer-assisted reporting through seminars, conferences, and intensive bootcamps at various locations throughout 1996-1997. The training covers transferring government data, building spreadsheets, database management, and online research skills for journalists.","keywords":["training","journalism","databases"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Excel","Internet","World Wide Web"],"full_text":"NICAR and, where indicated, The Associated Press provide specialized training in your newsroom. Learn to transfer data from government newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases.\n\nCost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below.\n\nNote: In the following list, \"Open to all\" means any journalist may sign up. \"Closed\" means the session is open only to members of the host organization.\n\nKiplinger Seminar, Columbus, Ohio Nov. 8, 1996. Closed.\nAllentown, Pa., Allentown. Call Nov. 12-13, 1996. Closed.\nNew Brunswick, N.J., Nov. 15-16, 1996. Open to all. (609) 392-3622.\nWashington, D.C., Washington Post Dec. 9-10, 1996. Closed.\nWashington, D.C. - Medill School of Journalism - Dec. 12-14, 1996. Open to all. (573) 882-0684\nMilwaukee, Wisc., Milwaukee Journal- Jan. 20-22, 1997. Closed.\nYork, Pa., York Daily Record - Jan. 27-28, 1997. Closed.\nSan Diego, Calif., San Diego Tribune and SPJ - Feb. 19-21, 1997. Open to all. (619) 293-1261\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Wendy Charron at (573) 882-0684.\n\nRTNDA (Radio and Television News Directors Association), Los Angeles Oct. 9-13, 1996.\nSEJ (Society for Environmental Journalists), St. Louis, Mo. - Oct. 17-20, 1996.\nMulticultural Management Program-NICAR, Columbia, Mo. - Oct. 13-18, 1996.\nIRE/NICAR Student Regional Conference, Syracuse, N.Y.-Nov.9-10, 1996. Cost is: IRE member $10 for conference. Non IRE Member, professional rate $50 for conference (includes $40 IRE membership fee). Non IRE member, student rate $35 for conference (includes $25 student membership fee. Syracuse students, $5 to attend conference. Additional $25 fee for Sunday computer training.\nMinority Regional Conference, Richmond, Va. - Nov. 15-17, 1996. Open to minorities.\nMinority Regional Conference, Dallas, Texas Dec. 7-10, 1996. Open to minorities.\nNashCAR, NICAR National Conference, Nashville, Tenn. - March 3-9, 1997. Costs are: IRE member, $150 for entire conference. Non IRE member, $190 for entire conference, including IRE membership. Renewal member, $190 for entire conference, including IRE renewal. Students, $125 for conference, including student membership. Late fee is $15 for registrations not postmarked by Feb. 14, 1997\nIRE National Conference, Phoenix, Ariz.,- June 12-15, 1997.\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo.\n\nAs with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size.\n\nFor information, call Wendy Charron at (573) 882-0684.\n\nJan. 5-11, 1997. Waiting list only.\nJan. 24-26, 1997 - Intermediate Bootcamp concentrating on data clean-up and more. For details, call (573) 882-1984.\nMay 4-9, 1997 - Advanced Bootcamp concentrating on stats and maps at the University of North Carolina-Chapel Hill with Philip Meyer.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-from-the-nicar-library","slug":"1996-october-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-october-from-the-nicar-library.html","year":1996,"month":"October","monthNum":10,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering topics including campaign finance, healthcare, transportation safety, crime statistics, and business records, with pricing and contact information available.","keywords":["databases","government","records"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW A monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\n\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\n\nCoast Guard boating accidents, 1969-1994.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns.\n\nFederal procurement data, 1992-1995, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records. 1993, 1995.\n\nNEW National Bridge Inventory System data, includes inspection grades. 1994-1995\n\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1994 data.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nNEW The National Inventory of Dams. 1991-1995.\n\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\n\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-nashcar-on-track","slug":"1996-october-nashcar-on-track","url":"https://dwillis.github.io/uplink/article/1996-october-nashcar-on-track.html","year":1996,"month":"October","monthNum":10,"headline":"NashCAR on track","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"","summary":"NashCAR, NICAR's 1997 national conference in Nashville, will offer comprehensive computer-assisted reporting training for journalists of all skill levels, with specialized tracks for different roles and exploration of web journalism.","keywords":["conference","training","journalism"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"NashCAR, NICAR's 1997 national conference, will be March 6-9 in Nashville, Tenn., with the support and help of The Tennessean and a panel of local journalists.\n\nThe conference will offer the widest range of panels and hands-on training yet. Building on the previous conferences in Santa Clara, Calif. (CAR Trek), and Cleveland, Ohio (CAR Rock), NashCAR will have sessions for beginners, intermediates and advanced users.\n\nWe'll have tracks for beat reporters, editors, newsrooms managers, broadcasters, researchers and librarians, and trainers.\n\nThe conference will use of computer labs at local universities and will have a demonstration room to display equipment and software. We'll also have vendors and numerous opportunities for journalists to find jobs. In addition, we expect to explore the effect of the World Wide Web on journalism and how journalists are using the web for their stories. For more information, call (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-jobs","slug":"1996-october-jobs","url":"https://dwillis.github.io/uplink/article/1996-october-jobs.html","year":1996,"month":"October","monthNum":10,"headline":"Jobs","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Two newspapers are hiring journalists: The Daily Press in Newport News, Virginia seeks a sports editor to oversee coverage of regional teams, while the York Daily Record in Pennsylvania is recruiting a prep sports writer and metro reporter with digital reporting skills.","keywords":["hiring","journalism","sports"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"The Daily Press of Newport News, Va., is looking for a sports editor who is at home in cyberspace, with television and in the locker room to lead a group of writers who cover sports ranging from the Washington Redskins and minor league hockey to AAA baseball.\n\nSend resume, cover letter and supporting information to: Felicia Mason, staff development editor, Daily Press, 7505 Warwick Blvd., Newport News, Va. 23607, or send e-mail inquiries to fmason@tribune.com\n\nThe Daily Press is part of Tribune Co.\n\nThe York Daily Record in York, Pa., is looking for a prep sports writer and metro reporter. Familiarity with the Internet and computer-assisted reporting is a plus for each of the jobs. Send resume and clips to Sports Editor, York Daily Record, P.O. Box 15122, York, Pa. 17405-7122\n\nThe prep sports writer is wanted immediately to \"continue a tradition of putting a pro slant on prep and community athletics.\"\n\nThe metro general assignment reporter is wanted for Jan. 1. Those with police reporting experience are especially encouraged to apply. Send resume and clips to Metro Editor, York Daily Record, P.O. Box 15122, York, Pa 17405-7122.\n\nThe York Daily Record is a 42,500 circulation, Monday-Saturday morning paper. For more information about the paper, see its Web site at http://www.ydr.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-october-ire-l-and-nicar-l-addresses","slug":"1996-october-ire-l-and-nicar-l-addresses","url":"https://dwillis.github.io/uplink/article/1996-october-ire-l-and-nicar-l-addresses.html","year":1996,"month":"October","monthNum":10,"headline":"IRE-L and NICAR-L addresses","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Instructions for posting to and managing subscriptions for IRE-L and NICAR-L discussion lists hosted on the University of Missouri's lists.missouri.edu server.","keywords":["lists","subscription","Missouri"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"IRE-L and NICAR-L are located on lists.missouri.edu, a dedicated Unix system designed to service discussion groups run out of the University of Missouri. All posts to the lists should be sent to: ire-1@lists.missouri.edu or to nicar-1@lists.missouri.edu\n\nAll commands for subscription changes should be sent to: listproc@lists.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-uplink-update","slug":"1996-november-uplink-update","url":"https://dwillis.github.io/uplink/article/1996-november-uplink-update.html","year":1996,"month":"November","monthNum":11,"headline":"Uplink update","kicker":"FOI","author_name":"","author_title":"","affiliation":"","summary":"The newsletter introduction outlines the issue's focus on FOIA battles, leadership changes at IRE and NICAR, and electronic record access issues.","keywords":["FOIA","NICAR","Staffing"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"This issue of Uplink details journalists' ongoing battles to access information. The new Electronic FOIA Amendments say the government should provide electronic data - but the law is not without its critics. Bob Anderson of the Freedom of Information Center tells the story.\n\nMeanwhile, Jennifer LaFleur of the San Jose (Calif.) Mercury News talks about access problems journalists face in California and other states, and tells us how to get involved in the fight. NICAR's John Sullivan and Jo Craven detail recent electronic battles and the flimsy excuses bureaucrats offer in data dodges.\n\nIn other news, IRE Executive Director Rosemary Arma resigns to become Anne Arundel County bureau chief for the Baltimore Sun. And NICAR Training Director Neil Reisner leaves for the Miami Herald. The jobs are open","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-ready-aim-foia","slug":"1996-november-ready-aim-foia","url":"https://dwillis.github.io/uplink/article/1996-november-ready-aim-foia.html","year":1996,"month":"November","monthNum":11,"headline":"Ready, aim, FOIA","kicker":"","author_name":"Bob Anderson","author_title":"","affiliation":"Freedom of Information Center","summary":"The Freedom of Information Center has adapted to the digital age by addressing new questions about electronic government records access, while maintaining its core mission as a research library on information controls. The center now helps navigate complex state and federal regulations regarding e-mail, databases, and electronic FOIA requests.","keywords":["FOIA","electronic","government"],"topics":["Investigative Methods","Government & Politics","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Ready, aim, FOIA\n\nKeeping up with the digital revolution\n\nFreedom of Information Center The Freedom of Information Center has donned electronic flak jackets and entered the digital revolution. After responding to questions about accessing government secrets for almost 40 years, the center is now firing back answers to a bundle of queries its founders never could have imagined.\n\nSome examples:\nWhich states have the best open records laws for access to electronic information? Answer: One of the best is North Carolina with passage of recent amendments.\nIs Federal government e-mail considered a public document? Answer: Sometimes yes, sometimes no, depending on whether the information is covered by one of the nine exemptions to the Freedom of Information Act.\nHow much can a state charge for copies of an electronic database? Can a state charge for redacting electronic information? Answer: Depends on the state. In Kansas in 1995, the State Court of Appeals ruled that the Department of Revenue could force newspapers to pay, at a minimum, hundreds of thousands of dollars for access to the state driver's license database. But the same year, Kentucky's state attorney general said that a $240 fee to redact juvenile delinquents' names from a database was unallowable.\n\nAs anyone can see, some information about electronic access particularly on the state level is as difficult to find as data on a fragmented hard drive.\n\nBut some other answers about electronic access are getting easier to find, particularly with the passage this fall of the Electronic Freedom of Information Act Amendments law. Those amendments ease getting electronic information from the federal government. Throughout these shifts, the role of the FOI Center, located at the Missouri School of Journalism, remains the same: It is a research and reference library focusing on controls placed by government and society on the free flow of information. The center's staff surveys articles from almost 150 publications and journals to add to its files collection.\n\nThose files now contain more than 1 million articles and documents on the controls of information. That is why the center regularly gets requests from such news organizations as the Kansas City Star, the Arizona Republic, and the Associated Press. The center receives about 3,000 requests annually from students, attorneys, business people and journalists.\n\nSome of the most recent additions to our collection add to the understanding of how government controls electronic access. They include BNA's Electronic Information Policy & Law Report, which surveys court and government developments on electronic information. The center also subscribes to \"Government Technology,\" a publication geared toward government managers who set up information systems. The FOI Center has put its own digital footprint on the Internet with its Web site at: http://www.missouri.edu/~foiwww. The site has the text of federal and some state public information laws, as well as links to a variety of sources about access to information.\n\nGot a question about what information you can get from the government? Call the FOI Center at (573) 882-4856 or, send e-mail to jourke@muccmail.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-electronic-foia-amendments-get-mixed-review","slug":"1996-november-electronic-foia-amendments-get-mixed-review","url":"https://dwillis.github.io/uplink/article/1996-november-electronic-foia-amendments-get-mixed-review.html","year":1996,"month":"November","monthNum":11,"headline":"Electronic FOIA Amendments get mixed review","kicker":"","author_name":"Bob Anderson","author_title":"","affiliation":"Freedom of Information Center","summary":"The article discusses the impact and criticisms of the 1996 Electronic Freedom of Information Act Amendments, which formalize public access to federal electronic records.","keywords":["EFOIA","Federal Records","Public Access"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["CD-ROM"],"full_text":"The Electronic Freedom of Information Act Amendments of 1996 are bringing the FOIA into the digital age. The amendments were signed into law by President Clinton on Oct. 2. One of the key points of the amendments is that government should \"provide for public access to information in an electronic format,\" such as databases stored on computer tapes and CD-ROM disks. That whether electronic information is accessible - has been acknowledged by several federal agencies for years. The new law \"makes explicit\" that such records are open to the public. The law also: Allows a requester to get information in \"any form or format\" requested, including computer format. Requires agencies to put a substantial amount of information on-line. That requirement applies to documents created after Nov. 1 and include \"agency opinions and policy statements,\" information released under FOIA and likely to be requested again, and other information that the agencies now are required to release. Agencies have until Nov. 1, 1997 to make these records available on-line. Requires creation by agencies of electronic on-line indices of the records previously released under FOIA. The indices must be on-line by Dec. 31, 1999. Requires agencies to create a \"multi-track\" system for the release of records. In essence, simpler requests would be put on a different \"track\" than more detailed requests. Persons asking detailed requests would be given the option of limiting their request to get onto a faster track. Changes the time agencies have to respond to FOIA requests from 10 days to 20 days. How worthwhile the changes will be is being debated. The National Security Archives, a private nongovernmental research library, states that some of the changes are flawed including the multitrack provision, which, the Archives says, is \"permissive\" and old-hat, because many agencies already have a multitrack provision. For a look at the Archives' analysis of the EFOIA amendments, check its homepage at \"http:/ www.seas.gwu.edu/nsarchive.\" A more harsh view of the amendments is offered by Bob Gellman, a Washington, D.C., privacy and information consultant. He calls the provisions \"mediocre and limited.\" For his viewpoints, see the Oct. 9 issue of Access Reports. Despite its shortcomings, the EFOIA does at least codify access to databases, the mainstay of reporters who specialize in computer-assisted reporting, said Paul McMasters, the First Amendment ombudsmen for the Freedom Forum and a nationally known expert on FOIA issues. \"Journalists should be especially happy about this,\" said McMasters, who recently was inducted into the National Freedom of Information Act Hall of Fame.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-november-electronic-battles","slug":"1996-november-electronic-battles","url":"https://dwillis.github.io/uplink/article/1996-november-electronic-battles.html","year":1996,"month":"November","monthNum":11,"headline":"Electronic battles","kicker":"Media, agencies wrangle over fees, access","author_name":"John Sullivan","author_title":"NICAR staff","affiliation":"NICAR","summary":"Journalists face significant obstacles when attempting to obtain electronic records and databases from government agencies, which often cite prohibitive fees, privacy concerns, and claims that data redaction constitutes creating new records. State-level records access is particularly challenging, with varying laws across jurisdictions and some states using residency requirements to deny access to non-residents.","keywords":["records","access","database"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"And your total is $300,000 - will that be cash or charge? That's what the AP's Special Assignment team heard when it requested the Immigration and Naturalization Services database on non-immigrant status, a system that tracks foreigners who enter the country for business or pleasure.\n\n\"This is a completely bogus denial,\" says Drew Sullivan, national news data editor for the AP. \"I have been fighting this for two years, first at NICAR and now here.\"\n\nSullivan's experience mirrors that of many reporters seeking to obtain electronic records, says Rebecca Daugherty, director of the Freedom of Information Act Service Center at The Reporters Committee for Freedom of the Press in Arlington, Va.\n\n\"A large number of (our) calls deal with fee provisions for electronic data,\" Daugherty says.\n\nThe AP's case is doubly complicated. In its wranglings with the INS, a privacy issue arose. The AP agreed to the redaction of the questionable field from the database; however, the INS said the redaction constituted the creation of a new database - which is not required under federal law.\n\n\"We have seen agencies give the line that they don't have to create a record. In some cases, it involves only pushing a couple of buttons, but this is not a reasonable explanation,\" Daugherty says. She also says the EFOIA seems clear: Redacting a field does not equate to the creation of a new record.\n\nBattle for state records\n\nThe EFOIA, effective in the coming year, will help reporters get public records, Daugherty says. However, federal bills such as the Boxer Bill may limit access to Department of Motor Vehicle records, requiring states to allow residents to \"opt-out\" having their driving records open to public scrutiny. This will exacerbate what is already the most difficult battle for reporters: accessing state records.\n\nLeslie Reis, an attorney for the Reporters Committee, says there is little precedent or case law in many states, and records access laws vary widely when you cross a state line.\n\nResidency rule\n\nThat's what Geoff Davidian found when he sought to start a small investigative newspaper in Tennessee. Davidian tried to get some records through the state and was denied. The attorney general said that Tennessee law provides access to public records for state residents only. Since Davidian had just moved to Tennessee and was not yet a declared resident, he couldn't access the information. He got the information through his son who is a resident, but not without a fight.\n\nDavidian has filed a federal suit claiming that the state is violating his right to privileges and immunities granted by the Constitution.\n\nReis says there is not much a reporter can do in such a situation.\n\n\"Many papers have argued that since they represent a considerable public interest in the state, they should be allowed to get information, but this approach hasn't been particularly successful,\" Reis says. \"We have seen an increasing number of cases where residency is being used to deny access to records.\"\n\nSome cases are not a question of fees or residency. When a male nurse was fingered in the suspicious death of 165 patients at Vermillion County Hospital, the Indianapolis Tribune-Star began to request death certificates from the state. They discovered that many records were kept at the county level and were open. They also found that some counties opt to forward death certificates to the state. But at the state level, the same records are no longer considered open, says Tribune-Star City Editor Jim Lewers. The Star-Tribune sued.\n\nThousands of dollars later, the state settled out of court and the paper got the records.\n\nLewers says it is important to pursue these matters legally. \"These are complex suits that are breaking new ground in the state and establishing some precedent,\" he says.\n\nThe Reporters Committee agrees and offers some recourse for reporters.\n\n\"We will give advice on cases where reporters have made requests for public records and have been denied,\" says Daugherty.\n\nJohn Sullivan can be reached at (573) 882-0684, or send e-mail to john@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-journalists-can-get-help-with-foi-issues-by","slug":"1996-november-journalists-can-get-help-with-foi-issues-by","url":"https://dwillis.github.io/uplink/article/1996-november-journalists-can-get-help-with-foi-issues-by.html","year":1996,"month":"November","monthNum":11,"headline":"Journalists can get help with FOI issues by:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Journalists facing freedom of information obstacles can seek legal assistance through the Reporters Committee for Freedom of the Press hotline, email, or website, or contact the Justice Department's FOIA ombudsman hotline.","keywords":["FOIA","journalists","access"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","Email"],"full_text":"Journalists can get help with FOI issues by: Calling the Reporters Committee for Freedom of the Press Hotline for legal advice at (800) 336-4243. The committee receives 3,000 calls annually, 60 percent of which come from reporters whose access to public records has been thwarted in some way. E-mailing the Committee at RCFP@CAIS.COM. Calling the Justice Department Hotline, which acts as an ombudsman for FOIA fights, at (202) 514-FOIA. Also, check the Reporters Committee for Freedom of the Press Web site at http://www.rcft.org/rcft","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-excuses-excuses","slug":"1996-november-excuses-excuses","url":"https://dwillis.github.io/uplink/article/1996-november-excuses-excuses.html","year":1996,"month":"November","monthNum":11,"headline":"Excuses, excuses","kicker":"No data? No dice","author_name":"Jo Craven","author_title":"NICAR Staff","affiliation":"NICAR","summary":"NICAR compiles humorous and frustrating excuses that government agencies have used to deny or delay public records requests, featuring war stories from journalists across the country who have encountered absurd refusals to release data.","keywords":["government","records","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Government & Politics"],"technologies":["mainframe"],"full_text":"NICAR has been updating its list of the best (or worst, depending on your perspective) excuses that government agencies have given for refusing to fill data requests. Here are the war stories that reporters around the country have shared with us:\n\nAnde Engleman, former executive director of the Nevada Press Association:\n\"My favorite came from a county clerk in reference to a law requiring public access to 'all claims paid by the county' (meaning all checks written to whom and what for): 'That's not an important law to follow,' the clerk said.\n\"My query of, 'Who says which laws are important and which ones are not?' was answered by a copy of a directive from the Nevada Department of Taxation, which said they would not be audited for following that law. We uncovered a real mess, which we took to the governor. The directive got changed very quickly, and all of a sudden this information became available in all counties.\"\n\nBryant Jordan, Air Force Times:\n\"My editor recently asked the Air Force for some statistics relating to promotions. The official he asked responded by saying something to the effect that, 'Oh, those are irrelevant. You don't need those.'\n\"My editor's response went something like, 'With all due respect, I decide what's relevant.'\"\n\nDrew Sullivan, The Associated Press:\n\"Here's one I got at NICAR: 'We can't give you the record layout because then you can hack into our system and change the data.'\"\n\nMichelle Moon, WAVE-TV:\n\"How about this excuse I got: 'Well, let me talk to your data guy.' My response: 'I am my data guy.'\"\n\nBill Kelly, Nebraska ETV Network:\n\"I don't think that's even in the computer.\"\n\"On records of purchases of tax delinquencies from a county treasurer: 'We only sell those to private businesses.' A protest to county board members eventually 'clarified' this policy, and the records were released, free of charge.\"\n\"In the midst of assisting in the instruction of an advanced reporting class with CAR skills training at the University of Nebraska, nearly everyone in the class was met with the following from a variety of agencies:\n'We couldn't do that for a student.'\"\n\nPaul D'Ambrosio, Asbury Park Press:\n\"New Jersey has no FOIA law, so we take routine denials in stride (can you say, masocists?) However, a few do rise to the 'you got to be kidding' level:\"\n\nME, to a N.J. state department: I would like to have your line item budget for the last five years, either on disk or on paper.\nTHEM: I'm not sure we have that.\nME: How do you run your department if you don't know how much money you have?\nTHEM: Well, we have a budget, but we don't keep it in the format you want.\nME: I'll take whatever you have.\nTHEM: It'll take an employee two full weeks to do because we have to compile the information from several computers.\nME: I'll wait. (Two weeks later)\nTHEM: It may take longer. This is a busy year for us. We're putting next year's budget together.\nME: Muffled screams\n\nMike Ravnitzky:\n\"It's still a classified national security record, but we can't locate it.\"\n\"We can't release it because the agency responsible for that document is no longer in existence, and no other agency has taken over its responsibilities.\"\n\"Now, why would you want a copy of that, huh?\"\n\"That was illegal, and we can't release information on illegal activities.\"\n\nMatthew David Waite, The Daily Nebraskan:\n\"I spent a half an hour listening to a university bureaucrat tell me that they did not have the power to pull salary data information off their mainframe system, yet they publish a book of salary data every year that is printed straight off a database.\n\"My response: 'Give me a half an hour of mainframe time, and I'll show you how to do it.'\"\n\nJo Craven can be reached at (573) 882-0684, or send e-mail to jo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-breaking-barriers","slug":"1996-november-breaking-barriers","url":"https://dwillis.github.io/uplink/article/1996-november-breaking-barriers.html","year":1996,"month":"November","monthNum":11,"headline":"Breaking barriers","kicker":"Groups work to open government records","author_name":"Rosemary Armao","author_title":"executive director","affiliation":"IRE","summary":"Journalism organizations met to address barriers preventing the Freedom of Information Act from effectively opening government records, identifying problems with FOIA implementation, access limitations, and underutilization while proposing solutions including improved compliance penalties, expanded access beyond current scope, increased journalist training, and public education about information access value.","keywords":["FOIA","journalism","government"],"topics":["Journalism & Newsroom","Government & Politics","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"In September, the American Society of Newspaper Editors marked the 30th anniversary of passage of the federal FOIA by bringing the Freedom Forum in Washington representatives from the nation's major journalism organizations, including IRE. It wasn't a celebration.\n\nThe point of the two-day session was that a whole lot more of us need to be doing more to see that anti-secrecy laws get used effectively and are expanded.\n\nSeven groups brainstormed solutions for the biggest barriers now keeping the FOIA from fully opening up government to public scrutiny. They have agreed to continue working as an informal consortium to make those solutions happen.\n\nHere are highlights of the two days' work:\n\nImproving FOIA\n\nThe problem: FOIA requests can take years because of staffing shortages in some federal agencies, bureaucratic resistance, prohibitive fees and the government's eagerness to exempt large amounts of information.\n\nSome solutions: The recent passage of the Electronic FOIA is the first step toward strengthening the act. Penalties for the lack of compliance and hiding and withholding records would help. Journalists also should support laws that would make government records held by private firms subject to all FOIA rules.\n\nExpanding access\n\nThe problem: FOIA does not apply to Congress, the federal judiciary branch or to certain offices connected with the presidency, including the Council of Economic Advisers and the National Security Council. Moreover, Congress seems willing to consider changes to the FOIA only about once a decade.\n\nSome solutions: Journalists should forget about expanding FOIA to Congress. Access to the administrative offices of the judiciary and to the presidential agencies would be more fruitful. Journalists need to reactivate old networks in which they watched over the disparate pieces of legislation that together make up federal information policy. The Presidential Records Act, for example should be reviewed with an eye toward pressing for more openness. Journalists should take advantage of Vice President Al Gore's interest in information exchange over the Internet.\n\nUsing FOIA more\n\nThe problem: It takes too long for FOIA requests to be processed. Journalists don't know how to file requests. Publishers and journalism educators are uninterested in supporting or promoting FOIA.\n\nSome solutions: FOIA response time can be improved by promoting dialogue with public affairs officers before requests are filed, by working with public interest groups on requests and by urging more funding for FOIA offices. We should increase newsroom training on the FOIA and enlist publishers, editors and legal departments in a battle for better use of a law important to democracy.\n\nSupporting access.\n\nThe problem: The public sees the media as arrogant about their right to see all records, and there is a public indifference to FOI issues.\n\nSome solutions: Emphasize the value to citizens of freely available information. We need to work with citizens groups and teach the public how to use FOIA and other records laws. We could list cyberspace sources of information and show the public its stakes in openness.\n\nBalancing security concerns\n\nThe problem: National security exemptions to the FOIA are overused, and there is still far too much classification of records.\n\nSome solutions: Reporters need to educate themselves about FOIA issues and recent presidential decisions on secrecy and classification then demand compliance.\n\nTechnology\n\nThe problem: Agencies resist compliance to requests for electronic records.\n\nSome solutions: Journalists need to be vigilant that government data enters the public domain. Contractors should not be permitted to obtain copyrights or other property rights in public information. Agencies should be prevented from charging high prices for data and encouraged to redefine their mission to include dissemination of information to the public.\n\nRosemary Armao can be reached at (573) 882-1982, or send e-mail to armao@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-check-out-the-ire-l-and-nicar-l-mailing-list-archives","slug":"1996-november-check-out-the-ire-l-and-nicar-l-mailing-list-archives","url":"https://dwillis.github.io/uplink/article/1996-november-check-out-the-ire-l-and-nicar-l-mailing-list-archives.html","year":1996,"month":"November","monthNum":11,"headline":"Check out the IRE-L and NICAR-L mailing list archives","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE and NICAR maintain searchable mailing list archives on their websites organized by thread, author, and date, available in HTML and plain text formats.","keywords":["archives","mailing","lists"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Check out the IRE-L and NICAR-L mailing list archives on our websites at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-california-dreaming","slug":"1996-november-california-dreaming","url":"https://dwillis.github.io/uplink/article/1996-november-california-dreaming.html","year":1996,"month":"November","monthNum":11,"headline":"California dreaming","kicker":"Reporters detail electronic nightmare","author_name":"Jennifer LaFleur","author_title":"","affiliation":"San Jose Mercury News","summary":"California journalists face significant challenges accessing government data due to inconsistent policies across agencies, unclear laws regarding electronic records, and privacy concerns that vary by jurisdiction. The lack of uniform application of public records laws creates unpredictable outcomes for reporters seeking databases on everything from pet licenses to personnel files.","keywords":["access","databases","California"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":[],"full_text":"We may never know what the most common name for a dog is in San Jose, Calif., or who really sponsors roads in the California adopt-a-highway program. These examples may not be critical data for California news organizations, but when government agencies limit access to pet licenses and highway cleanup programs, imagine how they react to requests for personnel files and police data. That's because getting databases about individuals is just one of the many access problems faced by California journalists. Another problem is that each agency chooses the format in which it provides records and a murky law allows for inconsistencies. \"California, being the largest state, might seem a goldmine for electronic information,\" said Chris Schmitt, Mercury News telecom editor and longtime CAR geek. \"But the truth is, it's a jumbled, chaotic mess, with no clear guidelines, and decisions about releasing information seemingly based as much on whim as intelligent policy.\"\n\n\"Those things are really the problem areas,\" said Terry Francke, executive director of the California First Amendment Coalition, a Sacramento-based organization developed to advance freedom of information. \"[One problem is the] overall anxiety about privacy concerns in general, including addresses in particular. The state legislature has just passed a bill that creates a joint committee that deals with computer and personal information and privacy, which will spend all of 1997 looking into both public and private uses of personal information in automated systems and probably will be making some legislative recommendations in the spring of 1998.\"\n\n'No middle ground'\n\n\"It seems to me that with public records requests in the Golden State paper and electronic either turn out very good, or they turn out very bad. There is no middle ground,\" said Rebecca Raney of The San Bernardino County Sun. \"I've found that the agencies most willing to abide by the law are those with the most savvy attorneys. There's nothing uniform about application of public records laws here. It's a big free-for-all, with each agency responding as it sees fit.\"\n\nWithin certain types of records little-known inconsistencies. \"We had an experience with marriage licenses, which are public - right?\" said Russell Clemings of the Fresno Bee. \"But there were some records that were confidential marriage licenses. If you get married in the jail, it's automatically a confidential marriage license.\"\n\nWhen it comes to getting access to electronic information, some of the barriers are greater because California's Open Records Act is not specific about access to electronic records; therefore, many agencies will only provide paper documents.\n\nSteve Rees, publisher for Publishing 20/20, a San Francisco-based publisher of regional guidebooks to public schools, fought with the San Francisco school district for electronic data. \"The law is somewhat ambiguous as far as electronic records whether reports or the .dbf files were the published material,\" he said. \"We ended up with hard copy only after many weeks of wrestling.\"\n\nMurky law\n\n\"Meanwhile, the public records itself is in the same stage it was in the '70s in a different universe in what is practical in the realm of computers,\" Francke said. \"It's up to each agency as to what form or format or medium they need to give you.\"\n\nThe fuzziness regarding electronic records in the Open Records Act results in inconsistencies statewide - certain data is easy to get in one county but not in another. Clemings successfully gained access to police call data in his area while other reporters across the state have been denied similar access. While several cities provide their pet license database to journalists, the city of San Jose refuses, arguing that abusive spouses will seek out their mates-in-hiding or that others may use the data to locate police officers.\n\n\"There's also the issue about segregation of disclosable from nondisclosable information and whether an agency has the obligation to do that, especially if doing that requires a new program,\" said Judy Alexander of GenesisLaw, a San Jose media law firm. \"It's that kind of manipulation and whether it's the creation of a new record that is really murky under California law.\"\n\nIn addition, agencies are all over the board when it comes to fees charged for data. Journalists can help each other by not paying high prices for data because it ends up setting a precedent and hurting others.\n\n\"It's also pretty clear if you listen to the lobbyists for state and local government,\" said Francke. \"One thing they want to get out of all this is money.\"\n\nBut the picture of access in California is not completely grim, and experts are hopeful that revisions in FOIA at the federal level may spur states to include electronic information in their laws. In addition, we do have access to some databases not available in all states:\n\nVoter Registration data: This database is not a public record, but the press has access through a special exemption. This puts a lot of pressure on news organizations to limit use of such data to just the editorial side of the business. This data is available from counties and the Secretary of State. In some cases, the data is less costly from the Secretary of State's Office.\n\nLate contributions. The California Voter Foundation provided electronic access to large late contributions in the state through a listserv. This was an invaluable resource for reporters looking into campaign finance in the last days before the election.\n\nIn addition, more agencies are making data available on the Internet. Here are some:\n\nA complete list of California agencies online is available at: http://www.ganymede.org/agencies.html\nDepartment of Education: http://goldmine.cde.ca.gov/ This site provides a lot of detailed education data.\nDepartment of Insurance: http://www.insurance.ca.gov This site provides a database regarding companies, their rates and complaints against them.\nCalifornia Environmental Protection Agency's Department of Pesticide Regulation provides pesticide use reports on-line at http://www.cdpr.ca.gov/docs/archives/document/docmenu.htm\nCalifornia Department of Transportation provides some report and several of their software engineering models at http://www.dot.ca.gov\nEmployment Development Department http://www.edd.cahwnet.gov/ This site provides a bounty of employment and economic data for California.\nThe Department of Finance's Statistical Abstract is now on-line at http://www.dof.ca.gov/html/fs_data/stat-abs/toc.htm\nCalifornia UCC filings are available during beta test from Merlin Information Services at http://www.merlindata.com They have several other databases available on CD-ROM for a fee. A price list is available at this site.\nState Campaign Finance data is available for a fee through of Sacramento. (916) 447-1886.\n\nListservs:\n\nCalifornia Freedom of Information Listserv: CAL-FOI. This list, sponsored by the Northern California Chapter of SPJ's Freedom of Information Committee, is a forum for discussion of freedom of information, public records and First Amendment issues. Subscribe to maiser@rosebud.berkeley.edu with the message: subscribe cal-foi [your name]\nCalgovinfo This list from Computer Professionals for Social Responsibility provides a forum for discussion of access and computerization of government records in California. Subscribe to calgovinfo@cpsr.gov\n\nJennifer LaFleur can be reached at jenster@sjmercury.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-join-the-fight","slug":"1996-november-join-the-fight","url":"https://dwillis.github.io/uplink/article/1996-november-join-the-fight.html","year":1996,"month":"November","monthNum":11,"headline":"Join the fight","kicker":"IRE Access Committee","author_name":"Jennifer LaFleur","author_title":"","affiliation":"IRE and NICAR","summary":"IRE has formed an Access Committee to help reporters fight for public records access across the country, developing resources like FOIA templates and advocacy letters to combat growing restrictions on documents needed for in-depth investigations.","keywords":["access","records","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Government & Politics"],"technologies":[],"full_text":"Twenty years ago, a group of reporters came together to share resources on important investigations, and IRE was formed. Today, groups of reporters are joining to maintain and open access to public records.\n\nSome of those examples include a group of Arizona reporters fighting potential changes in their state's Open Records Act; reporters in New York who put in a joint request for speeding tickets in order to review the impact of the 65 mph speed limit enacted last year - they were refused access to the database; and in several states, reporters are fighting for DMV records.\n\nIn response to those access issues, IRE recently formed a committee dedicated to helping reporters fight these battles. If we are to continue working toward in-depth reporting, we must continue to have access to the documentation necessary for such reporting. The Access Committee has set the following objectives so far:\n\nNearly every day, the IRE and NICAR listservs see postings about more problems with access to records.\n\nDevelop a set of letters to be sent on behalf of news organizations fighting for access to particular records. This puts an organization of thousands of journalists behind such news organizations.\nDevelop templates for FOIA letters and contacts for use at news organizations around the country.\nDevelop a list of story examples where using particular types of data has been in the public's interest. This includes providing slides or videos of stories for presentations.\nDevelop a list of resources for reporters fighting for access.\nTrack access issues around the country at federal, state and local levels.\nDevelop a list of examples where citizens themselves have used particular records.\nSeek out access problems of interest to IRE.\nA large part of the committee's work is access to electronic records across the country and to help members gather ammunition for records fights. The work of this committee doesn't just help that one news organization; it helps everyone by setting precedents to open records.\n\nJennifer LaFleur can be reached at (408) 920-5728, or send e-mail to jenster@sjmercury.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-access-these-foi-sites","slug":"1996-november-access-these-foi-sites","url":"https://dwillis.github.io/uplink/article/1996-november-access-these-foi-sites.html","year":1996,"month":"November","monthNum":11,"headline":"Access these FOI sites","kicker":"On the Internet","author_name":"Nora Paul","author_title":"","affiliation":"Poynter","summary":"A comprehensive guide to online resources and tools for using the Freedom of Information Act to obtain information from government agencies, including reference lists, discussion groups, filing guides, and contact information.","keywords":["FOIA","transparency","resources"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["World Wide Web","gopher","Internet","Adobe Acrobat"],"full_text":"Needing information on how to pry information from reluctant agencies? There is a wealth of resources compiled and available on the World Wide Web.\n\nResource Lists\n\nResources on Freedom of Information Issues: http://town.hall.org/places/spj/foi-resources.html\n\nCompiled by Syracuse University professor Barbara Croll Fought, the list points out Organizations and Resource Centers, Hotlines, Periodicals, Books and On-Line Resources. Although it is now a few years old, the references are all mostly still valid.\n\nParascope's FOIA Help Center Home Page: http://www.parascope.com/foia.html\n\nOK, yes, Parascope's mission is to uncover alien conspiracies and paranormal occurrences, but they have compiled a nice list of links to other FOIA information sites.\n\nState FOI Laws-Titles and Statutory Citations: http://www.missouri.edu/~foiwww/citelist.html\n\nNeed to recite the chapter and verse of your state's Public Records laws? This listing, by state, gives you the citations - have those recalcitrant officials read the rules.\n\nDiscussion Lists\n\nFOI-L: Listserv discussion group on FOI http://www.reporters.net/nfoic/foi-1.html Find out how to join the listserv, sponsored by the National Freedom of Information Coalition and Barbara Fought.\n\nOrganizations\n\nNational Freedom of Information Coalition Home Page: http://www.reporters.net/nfoic/index.html\n\nLinks to information about the NFOIC.\n\nGuides to Filing\n\nDepartment of Justice FOIA Guide gopher://justice2.usdoj.gov:70/11/oip/foia2\n\nThis gopher site listings and links to the chapters of DOJ's FOIA guide. They are in PDF format, which requires availability of an Adobe Acrobat viewer (download for free from http://www.adobe.com/acrobat/readstep.html)\n\nFOIA Tips and Tricks: http://www.parascope.com/tips.html\n\nAlthough the source of this site, Parascope, focuses on paranormal phenomenon, they have a useful, reasonable listing of the seven steps you should take in filing a FOIA request:\n\nDo your research first. Write your request clearly and be specific. Target your request. Establish and maintain contact with the agency. Stay administratively correct. Delays in processing requests while unfortunate are normal. Be reasonable.\n\nSTEP-BY-STEP GUIDE to using the Freedom of Information Act: http://www.aclu.org/library/foiafaq.html\n\nA FAQ on FOI.\n\nPoynter's List of the Top 38 Excuses Government Agencies Give for Not Being Able to Fulfill Your Data Request (And Suggestions on What You Should Say or Do)\nhttp://www.poynter.org/prof/poynter/excuse.html\n\nCompiled by participants at Poynter's Computer-Assisted Reporting seminar, 1994.\n\nContacts\n\nFOIA Administrative and Legal Contacts at Federal Agencies: http://www.soc.american.edu/~wc9923a/foia.html\n\nListing, by agency name, of the contacts for questions about FOIA request filing in that agency.\n\nNora Paul can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-moving-on-up","slug":"1996-november-moving-on-up","url":"https://dwillis.github.io/uplink/article/1996-november-moving-on-up.html","year":1996,"month":"November","monthNum":11,"headline":"Moving on up","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"Uplink newsletter is increasing its subscription price in January after doubling in size this year, with new pricing of $40 for IRE members and $60 for nonmembers.","keywords":["pricing","newsletter","subscription"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"The price of Uplink will increase in January. The newsletter, which has grown from eight pages to 16 pages this year, will cost $40 for one year for IRE members and $60 for nonmembers. Annual institutional subscriptions are now available for $250. To beat the rate increase, subscribe before year's end at the current price of $35.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-using-report-writer","slug":"1996-november-using-report-writer","url":"https://dwillis.github.io/uplink/article/1996-november-using-report-writer.html","year":1996,"month":"November","monthNum":11,"headline":"Using Report Writer","kicker":"Tech Tip","author_name":"Tim Henderson","author_title":"","affiliation":"NICAR","summary":"This article provides an introduction to FoxPro's Report Writer tool, demonstrating how to create formatted reports from database queries using a night burglary data example and step-by-step instructions for customizing reports with titles, headers, and formatting.","keywords":["FoxPro","Report","Database"],"topics":["Technology","Investigative Methods"],"technologies":["FoxPro"],"full_text":"If you're like me, and I know I am, you were horrified the first time you realized there's no \"print query\" button in FoxPro.\n\nSince then, you've probably copied your queries to a spreadsheet, then to a word processor, or found some other way to get your results on paper. If you found your way into FoxPro's Report Writer, there was no way to guess how it worked, and you gave up.\n\nTruth is, once you learn a little about Report Writer, you may forgive its daunting first impression.\n\nNot only can you print the same query you see in the browse window, you can improve on it by adding titles, changing your column headers to English, setting off groups with summary information, even calculating new fields and using functions to determine what's printed under what circumstances.\n\nAs an introduction to Report Writer, let's make a report on night home burglaries in the Phoenix area using the retas.dbf file in the NICAR ftp site (ftp://nicar.org/pub/data/retas.dbf). Run this query to get the population and the number of night, day, and total home burglaries for the year:\n\nSELECT agency, pop, burgrnt + burgrd + burgru; AS homeburg, burgrnt AS night,; burgrd AS day; FROM Retas; WHERE msa = \"662\"\n\nTo make this query work in a full Return A Supplement file, you'd have to sum the fields and group by agency because it would be broken down by month. This sample data includes only one month.\n\nNow, choose Run Wizards Report, and pick Report Wizard from the Wizards menu.\n\n(Experiment with the other Report Wizards also when you have appropriate data: multi-column reports will help you fit long, skinny queries on a single page; Group/Total is good for multiple groupings where you want to set off and summarize the groups)\n\nIn the dialog boxes that follow, pick \"Query\" as the table to use, Ledger style, add all fields, and horizontal layout. Enter \"Night burglaries\" as the title. Hit the \"Preview\" button, click on the preview page to zoom in, and you'll see that without much effort you've recreated the look of your query browse window, and you even have a nifty title at the top.\n\nIf you're satisfied, you could just close the preview window and \"save and print\" the report. To sample some of the real power of Report Writer, choose \"Modify with Design Tool\" and \"Finish.\"\n\nIf you've ever used a graphics program or FoxPro's screen builder, you'll have an easier time with the report design screen. Everything you see is an \"object\" that can be created, dragged, resized or manipulated with the tools in the toolbar.\n\nTo see what I mean, click on the date field just under the title to select it - you'll see sizing handles appear around it. Hit delete and it disappears. Click the text tool - the capital A - and then click in the same area, and type \"Phoenix metropolitan area, January 1993.\" Now, click on the arrow tool - the \"object picker\" - and select the text you just typed. Choose Object Font. Experiment with different font sizes and types for the title.\n\nIf it's not lined up right, drag it into place. Use the text tool to change the column headings to English and change the font if you wish. Note that everything in the top Title Bar prints once, at the top of the first page, while the Page Header band prints under the title and atop any further pages. The Detail band prints for every record in your query. To make more room in a band, drag the buttons to the left of the band names. Notice that space is always added at the bottom - can't directly add space at the top. To work around this, add room at the bottom, then double-click on the bar under the band, which will select all the objects inside. You can then drag them en masse to the bottom, giving you room at the top.\n\nTo cut down on white space or save room, you can shorten the detail fields by dragging the sizing handles. Try shortening the Agency field by about half.\n\nTo make sure you're not truncating anything, double-click the field box to bring up the Report Expression dialog. Make sure the \"Relative to Top-Field can stretch\" option is checked (the Report Wizard should have automatically done this for you), and preview again. You should see the agency field shortened, and the few long names are wrapped to another line.\n\nIf you want some practice in dragging objects, follow up by resizing and dragging the other fields and headings to the left to take advantage of the space. The Report Snap to Grid option should be checked to help you line things up - you may also use the arrow keys to move the selected object in small increments.\n\nThe Ledger format we chose puts a grid around the fields: you can drag these grid bars to accommodate the new sizes - or delete them. To draw these yourself, use the Rectangle tool to put a box around all the detail fields, then the Line tool (it looks like two intersecting lines) to draw vertical cell lines between fields.\n\nThe Field tool - a little box with ab inside - can be used to add a whole new field to the report. Let's try adding one that calculates the percentage of night burglaries among those in which the time is known.\n\nClick on the Field tool, then in the Detail band where you want to place the new field. That brings up the Report Expression dialog box. Type this into the Expression box: night/(night+day)*100\n\nClick OK to close the dialog, and you'll see your new field. Use the Text tool to type a percent sign just to the right of it. Choose Report Page Preview to see the results.\n\nNow, to print the report, you have to run the query again, with this additional tag at the bottom\n\nINTO CURSOR Query REPORT FORM query.frx TO PRINTER NOCONSOLE\n\nsubstituting the name of the frx file where you saved the report. If you used RQBE to make the original query, you set the output to Report/Label and check Options to bring up a dialog box where you can browse to the frx file you made and print the report.\n\nYou might instinctively try File Print from the design screen, but if you do that, you'll print the code rather than the report. Remember, you're not designing a printout, you're designing a template for a printout. I guess this is meant for businesses who want a standard report form for an ever-changing database.\n\nAs you experiment with Report Writer's capabilities, here are a few of pitfalls to watch out for:\n\nYou can make reports from joined database files rather than starting from a query. But watch out - the Report functions are not as sophisticated as SQL SELECT. You have to index and set the order of the linked (child) table to the joined field or you could get incorrect data.\n\nWhen you use functions to define a print field, as we did in the exercise, you can't mix data types. The whole field has to be character or numeric, or you get an error. So if you wanted to add the percent sign directly to the field above with +\"%\", it wouldn't work unless you first converted the number to a parsed character string with STR(ALLTRIM(numeric_function)\n\nIf your data won't fit onto one page horizontally, and the space-gaining tricks above don't make enough room, you may pine for Excel's \"Fit to Page\" print option. Alternatively, you can set a smaller font size for your detail and header bands by double-clicking the bands to select everything, then choosing Object Font.\n\nTim Henderson can be reached by e-mail at thenders@gannett.infi.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-weve-just-made-it-easier-to-become-an-ire-member","slug":"1996-november-weve-just-made-it-easier-to-become-an-ire-member","url":"https://dwillis.github.io/uplink/article/1996-november-weve-just-made-it-easier-to-become-an-ire-member.html","year":1996,"month":"November","monthNum":11,"headline":"We've just made it easier to become an IRE member","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE has simplified its membership process by offering online registration through their secure web server and providing downloadable membership forms in Adobe Acrobat format for fax submission.","keywords":["membership","registration","online"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web","Adobe Acrobat"],"full_text":"We've just made it easier to become an IRE member. Now, you can join IRE via our secure Web server, or you can download a copy of our membership form in Adobe Acrobat format from the Web site and submit it by fax to (573) 884-5544. For more information, check out http://www.ire.org/membership.html If you have any questions, e-mail Web Master Wally Winfrey at wally@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-reporter-iso-love-story","slug":"1996-november-reporter-iso-love-story","url":"https://dwillis.github.io/uplink/article/1996-november-reporter-iso-love-story.html","year":1996,"month":"November","monthNum":11,"headline":"Reporter ISO love story","kicker":"CAR Trips: First Ventures","author_name":"Dave Sheingold","author_title":"Reporter","affiliation":"Gannett Newspapers","summary":"A Gannett Newspapers analysis of 2,175 personal ads from northern New York City suburbs reveals that men and women continue to approach dating with fundamentally different priorities: men emphasize physical appearance and interests while women prioritize personality and emotional qualities, despite both listing long-term commitment as their top goal.","keywords":["personals","gender","dating"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Paradox","WordPerfect","ASCII"],"full_text":"In the 1990s, age of enlightened thinking, it has been suggested that men and women are full of new insight into the ways of love and what makes each other tick.\n\nEnter the computer to make Cupid's arrow land in the dirt.\n\nVeering off course from the usual heavy duty CAR story, Gannett Newspapers in New York recently looked at an issue we knew everyone would read about - the personal ads - and put a new twist on it. We ran three month's worth of personal ads that appeared in our papers, which cover the northern New York City suburbs, to see what men and women said about themselves in the ads, what they said about their ideal mate and what it all meant, if anything.\n\nThe results? Not too encouraging. Contrary to what we would like to believe, men and women remain on opposite sides of love's Grand Canyon. Neither side, it appears, is doing a whole lot of clear thinking about how to find love everlasting.\n\nAccording to the most common words they use in their personal ads, men remain in search of Eve. The woman of their dreams can be a blast at parties and quick with a joke, but she better have a sweet smile and body to match or else she might get passed over. When it comes to luring a partner, men highlight a mix of their looks and their eclectic tastes, be they a love of fine food, good movies or the sporting life. Their inner self somehow gets second billing. Women, meanwhile, were different. They saw the ideal male more for his personality than his looks. While a handsome gent is pretty cool, they said they'd prefer someone caring, romantic, witty and secure. But to bait him, they switched gears and played into the old stereotypes. They talked most about their looks before they mentioned that they were worldly, bright, eclectic and successful.\n\nAbout the only hope that the differences may be narrowing was that both sexes listed long-term commitment as the No. 1 or 2 thing they were looking for.\n\nAs one social planning consultant who gives seminars on dating put it: \"Men are still judging with their eyes, women with their hearts.\" The bottom line, according to the experts, was that personals can work, but require patience, kind of like any other mating ritual.\n\nLove match\n\nThis also could have been said for the computer process of getting from the raw text of the ads to the final results.\n\nFirst, we got a list 2,175 personals as ASCII text from the company that runs them in our papers. Three months was as far back as they kept them, so that's what we took.\n\nThe information imported into the Paradox database manager as one text field, so the next thing we needed to do was create a second field and retype the acronym for each ad placed (SWM for single white male, GWM for gay white male, SBPF for single black professional female, etc.) This sounds time consuming, but only took a few hours. We needed to do this so we could run queries on how many of the ad-placers were men or women, Jewish or Christian, gay or straight, professional or nonprofessional.\n\nThen we separated the ads into different databases for males and females.\n\nAt that point, we needed to split the text of the male- and the female-placed ads into two fields one with the section of the ad describing what the ad-placer was like and the second describing what he or she was seeking in a mate.\n\nThere was no clear way of doing this. But we knew that almost all the ads began with the person's self-analysis while the second part was about their potential mate. That second part almost always began with ISO (in search of) or with words like 'desires,' 'seeks,' 'wants,' 'looking for' and similar types of verbs indicating a longing for that special person. This was where we wanted to split the text into two fields.\n\nSo, we separately exported the new male and female databases out as text and reopened them in Word Perfect. Using the search-and-replace function, we stuck an @ sign in front of every word we could think of that would split the ads into self-descriptions and desires.\n\nConverting the edited text back to ASCIIDOS, we re-imported each database into Paradox using the @ sign as a delimiter. Now we had our two fields.\n\nFrom there, we began running queries to count the number of times key words appeared in the ads. We figured out what words to look for simply by perusing a few dozen ads in the paper. We essentially divided up the results into categories of physical features, personality traits and other characteristics. Key words involving looks included handsome, pretty, fit, tall, blond, and attractive. Personality traits included honest, loving, caring and sincere. Other characteristics were most often things like athletic, intelligent, financially secure and an interest in movies, sports, dining and travel. You might think that these words were just stock, superficial things people would say or that they were exaggerated. But that didn't matter. We were most interested in the types of things people highlighted. That really told the story of what people think will attract a mate and the kinds of partners they seek. Note that the results were not exact because not every single ad had the same format. A handful had to be discarded, so the results are somewhat general, open to a bit of interpretation. But, hey, that's love, isn't it? Dave Sheingold can be reached at or send e-mail to 102503.520@CompuServe.COM","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-trucking-trucking-trucked","slug":"1996-november-trucking-trucking-trucked","url":"https://dwillis.github.io/uplink/article/1996-november-trucking-trucking-trucked.html","year":1996,"month":"November","monthNum":11,"headline":"Trucking, trucking, trucked","kicker":"NICAR notes","author_name":"Neil Reisner","author_title":"Training Director","affiliation":"NICAR","summary":"Johnny Dataseed reflects on his year as NICAR's training director, traveling the country teaching computer-assisted reporting techniques to journalists while balancing the challenges of constant travel and administrative work. He shares his positive experience working with passionate newsrooms and talented colleagues while announcing the search for his successor.","keywords":["journalism","training","travel"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet"],"full_text":"Call me Johnny Dataseed. As NICAR's training director since January, I've spent my time wandering the country, scattering data, spreadsheets, and database managers wherever I've been.\n\nIt has (and few who know me will be surprised to find a Grateful Dead reference here) been a long, strange, and completely wonderful trip. In fact, I can't think of a better job.\n\nWhat could be better than standing at the cutting edge of journalism and teaching excited and willing audiences how to do their jobs better? What could be better than getting on-site insight into newsrooms around the country? (And, given that I telecommuted to NICAR Central in Missouri from my home in New Jersey, what could be better than being more than 1,000 miles away from your boss?)\n\nIt's been a year of travel, averaging two weeks out of every month. The travel is both enjoyable and draining. It's hard to leave home and hearth that often. (It must be said, though, that the travel hasn't interfered that much. By the time this reflection sees print, my wife and I should be the proud parents of our first child.)\n\nIt's been a year of hanging out with journalists, all of whom were passionately interested in journalism and how to add computer-assisted reporting techniques to their arsenal of professional weapons. It's been a year of learning my craft better and learning how to teach it.\n\nIn some ways it's been a year of show biz - it's impossible to hold an audience for a two- or three- or five-day seminar without injecting a good deal of entertainment, stand-up comedy and bad puns.\n\nAnd, most of all, it's been a year of working, even at long-distance, with the NICAR and IRE staffs, some of the most collegial, supportive, talented, committed and just plain decent people in journalism. (Although trying to fill the shoes of my predecessor, the extraordinarily talented and energetic Jennifer LaFleur was, at best, intimidating.) It's been challenging in other ways, as well. Administration, organization and, most of all, patience, is not something that comes naturally to many reporters. And certainly not something that comes naturally to me. But administration and organization is critical when you're on the road doing one seminar, planning the seminar you're going to lead next week, making contacts for the seminars you're going to do next month, answering e-mailed questions from the folks who attended the seminars you did last month, and overseeing seminars being led by other NICAR staff members and friends.\n\nThere have been other benefits. Meeting reporters, editors and news executives all over the country hasn't hurt professionally, as Jennifer Lafleur's move from this post to the San Jose Mercury News and mine to the Miami Herald clearly shows.\n\nIn all, it's been a blast. I recommend it to anyone.\n\nNICAR is searching for a new training director. Those interested should e-mail NICAR Managing Director Brant Houston at brant@nicar.org, or call (573) 882-1984.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-executive-director-armao-resigns","slug":"1996-november-executive-director-armao-resigns","url":"https://dwillis.github.io/uplink/article/1996-november-executive-director-armao-resigns.html","year":1996,"month":"November","monthNum":11,"headline":"Executive Director Armao resigns","kicker":"IRE Notes","author_name":"","author_title":"","affiliation":"","summary":"IRE Executive Director Rosemary Armao has announced her resignation to return to active journalism at the Baltimore Sun, prompting a search for her replacement.","keywords":["Rosemary Armao","IRE","Baltimore Sun"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":[],"full_text":"After three years, Rosemary Armao has resigned as IRE executive director to become Anne Arundel County bureau chief for the Baltimore Sun. IRE is opening a national search to replace Armao, who resigned Nov. 1 in a letter to the group's board of directors and the dean of the Missouri School of Journalism. Explaining why she is leaving IRE, Armao said she can no longer ignore her strong desire to return to the practice of journalism. She feels, she said, \"an increasing homesickness for the newsroom.\" After nearly three years as executive director, a time \"spent working with great reporters, reading wonderful journalism, and advocating in-depth journalism around the world,\" Armao said she is inspired. \"Now its my turn to try to produce my own great journalism.\" IRE, based at the University of Missouri- Columbia, is the leading educational organization for journalists who want to learn to do more thorough, better organized, and important stories. Armao's tenure marks a dramatic increase in the programs and reach of IRE. The National Institute for Computer-Assisted Reporting (NICAR), a joint program of IRE and the Missouri School of Journalism, has held seminars and conferences training more than 8,000 journalists. Thanks to a grant from the Robert McCormick Tribune Foundation, IRE created Periodistas de Investigacion which will serve as an IRE-like organization providing investigative and computer training for Mexican journalists. Armao will be leaving her position in mid-December.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-november-ire-committees-always-are-on-the-lookout-for-good-volunteers","slug":"1996-november-ire-committees-always-are-on-the-lookout-for-good-volunteers","url":"https://dwillis.github.io/uplink/article/1996-november-ire-committees-always-are-on-the-lookout-for-good-volunteers.html","year":1996,"month":"November","monthNum":11,"headline":"IRE committees always are on the lookout for good volunteers","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE is seeking volunteers for its committees, particularly the Computer Committee chaired by Shawn McIntosh, with contact information provided for those interested in volunteering.","keywords":["volunteers","committees","IRE"],"topics":["Journalism & Newsroom"],"technologies":["Internet","Email"],"full_text":"IRE committees always are on the lookout for good volunteers. Of particular interest to Uplink readers may be the Computer Committee, chaired by IRE Director Shawn McIntosh of The Dallas Morning News. To volunteer, send e-mail to McIntosh at dmnproj@airmail.net For information on other IRE committees, send e-mail to IRE Executive Director Rosemary Armao at armao@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-get-your-training","slug":"1996-november-get-your-training","url":"https://dwillis.github.io/uplink/article/1996-november-get-your-training.html","year":1996,"month":"November","monthNum":11,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR and The Associated Press offer specialized journalism training through on-the-road sessions, conferences, and intensive bootcamps covering data analysis, spreadsheets, databases, and online research skills.","keywords":["training","journalism","data"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["spreadsheet","Internet"],"full_text":"NICAR and, where indicated, The Associated Press provide specialized training in your newsroom. Learn to transfer data: from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases.\n\nCost varies. For information, call Lisa Barnes at (573) 882-8969; or to register, call the numbers below.\n\nNote: In the following list, \"Open to all\" means any journalist may sign up. \"Closed\" means the session is open only to members of the host organization.\n\nOn-the-road training\n\nWashington, D.C., Washington Post Dec. 9-10, 1996. Closed.\nWashington, D.C. - Medill School of Journalism - Dec. 12-14, 1996. Open to all. (573) 882-0684\nMilwaukee, Wisc., Milwaukee Journal - Jan. 21-23, 1997. Closed.\nYork, Pa., York Daily Record - Jan. 27-28, 1997. Closed.\nSan Diego, Calif., San Diego Tribune and SPJ - Feb. 19-21, 1997. Open to all. (619) 293-1261\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969.\n\nMinority Regional Conference, Dallas, Texas - Dec. 7-10, 1996. Open to minorities.\n\nNashCAR, NICAR National Conference, Nashville, Tenn. - March 3-9, 1997.\n\nCosts are: IRE member, $150 for entire conference. Non IRE member, $190 for entire conference, including IRE membership. Renewal member, $190 for entire conference, including IRE renewal. Students, $125 for conference, including student membership. Late fee is $15 for registrations not postmarked by Feb. 14, 1997\n\nIRE National Conference, Phoenix, Ariz., - June 12-15, 1997.\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo.\n\nAs with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the Net and on-line databases but you'll be drilled all day, every day for a full week.\n\nTuition ranges from $500-$1,000 depending on circulation or market size. For information, call Wendy Charron at (573) 882-0684.\n\nJan. 5-11, 1997. Waiting list only.\nJan. 24-26, 1997 - Intermediate Bootcamp concentrating on data clean-up and more. For details, call (573) 882-1984.\nMay 4-9, 1997 - Advanced Bootcamp concentrating on stats and maps at the University of North Carolina-Chapel Hill with Philip Meyer, author of \"The New Precision Journalism.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-from-the-nicar-library","slug":"1996-november-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-november-from-the-nicar-library.html","year":1996,"month":"November","monthNum":11,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers a comprehensive collection of federal government databases covering transportation safety, finance, health care, crime statistics, and regulatory compliance, with datasets spanning from 1937 to 1996. The resources include FAA aviation records, FEC campaign contributions, Medicare data, crime reports, and various accident and violation databases useful for investigative journalism.","keywords":["databases","government","records"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["CD-ROM"],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW Federal Aviation Administration's accidents and incidents, including major plane crashes since 1971.\nNEW NASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\nA monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\nFederal procurement data, 1992-1995, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1995.\nNational Bridge Inventory System data, includes inspection grades. 1994-1995\nNEW FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1995 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nThe National Inventory of Dams. 1991-1995.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-gearing-up-for-nashcar","slug":"1996-november-gearing-up-for-nashcar","url":"https://dwillis.github.io/uplink/article/1996-november-gearing-up-for-nashcar.html","year":1996,"month":"November","monthNum":11,"headline":"Gearing up for NashCAR","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"","summary":"NashCAR, NICAR's 1997 national conference in Nashville (March 6-9), offers comprehensive computer-assisted reporting training for journalists at all skill levels with sessions for various professional roles and exploration of Web journalism.","keywords":["conference","journalism","training"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"NashCAR, NICAR's 1997 national conference, will be March 6-9 in Nashville, Tenn., and will offer the widest range of panels and hands-on training yet. Building on the previous conferences in Santa Clara, Calif. (CAR Trek) and Cleveland, Ohio. (CAR Rock), NashCAR will have sessions for beginners, intermediates and advanced users. We'll have tracks for beat reporters, editors, newsrooms managers, broadcasters, researchers and librarians, and trainers both in newsrooms and universities.\n\nThe conference will make use of computer labs at local universities and will have a demonstration room that will display equipment and software. We'll also have vendors and numerous opportunities for journalists to find jobs.\n\nIn addition, we expect to explore the effect of the World Wide Web on journalism and how journalists are using the Web for their stories.\n\nFor more information, call (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-moving-on","slug":"1996-november-moving-on","url":"https://dwillis.github.io/uplink/article/1996-november-moving-on.html","year":1996,"month":"November","monthNum":11,"headline":"Moving on","kicker":"","author_name":"","author_title":"editor for computer-assisted reporting","affiliation":"Allentown Morning Call","summary":"David Herzog, the computer-assisted reporting editor at the Allentown Morning Call, is departing in December to join the Providence Journal-Bulletin as a CAR specialist on their investigative team starting January 2.","keywords":["journalism","investigative","technology"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Allentown (Pa.) Morning Call editor for computer-assisted reporting David Herzog is leaving the paper in December. On Jan. 2, he will become the CAR specialist on the Providence (R.I.) Journal-Bulletin investigative team","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-jobs","slug":"1996-november-jobs","url":"https://dwillis.github.io/uplink/article/1996-november-jobs.html","year":1996,"month":"November","monthNum":11,"headline":"Jobs","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Two journalism positions are available: a business reporter at The Buffalo News covering labor and trade issues, and a computer-assisted reporting editor or reporter at The Allentown Morning Call to lead their CAR program.","keywords":["reporting","journalism","positions"],"topics":["Journalism & Newsroom","Technology"],"technologies":[],"full_text":"The Buffalo (N.Y.) News has an opening for an experienced business reporter to handle labor, the auto industry, trade issues and small business.\n\nThe News has an active CAR program, and there is an opportunity to do computer-assisted reporting. If you are already trained in CAR, that would be a plus. Applicants should have at least three years of experience at a midsized daily; i.e., 80,000 circulation and up.\n\nThis is a Guild paper, so salary is set by years of experience ranging from $37,000 for three years of experience to the current $48,000 at the top of the scale.\n\nSend clips and resume to Bill Flynn, financial editor, One News Plaza, Buffalo, N.Y. 14240\n\nThe Allentown (Pa.) Morning Call has an opening for an editor with advanced computer skills or reporter with strong CAR experience to grow into leadership role on the newspaper's CAR team. Willingness to coach required.\n\nSend cover letter with ideas you would bring to the program, resume and clips to David Erdman, city editor, The Morning Call, 101 6th St., Allentown, Pa. 18105, or call (610) 778-7929, or send e-mail to david.erdman@mcall.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-ire-l-and-nicar-l-addresses","slug":"1996-november-ire-l-and-nicar-l-addresses","url":"https://dwillis.github.io/uplink/article/1996-november-ire-l-and-nicar-l-addresses.html","year":1996,"month":"November","monthNum":11,"headline":"IRE-L and NICAR-L addresses","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Instructions for posting to and managing subscriptions for IRE-L and NICAR-L discussion lists hosted on the University of Missouri's dedicated server.","keywords":["listserv","discussion","subscription"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"IRE-L and NICAR-L are located on lists.missouri.edu, a dedicated Unix system designed to service discussion groups run out of the University of Missouri. All posts to the lists should be sent to: ire-l@lists.missouri.edu or to nicar-l@lists.missouri.edu\n\nAll commands for subscription changes should be sent to: listproc@lists.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-intermediate-car-seminar","slug":"1996-november-intermediate-car-seminar","url":"https://dwillis.github.io/uplink/article/1996-november-intermediate-car-seminar.html","year":1996,"month":"November","monthNum":11,"headline":"Intermediate CAR seminar","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A new hands-on seminar scheduled for January 24-26 will teach advanced database skills including data cleanup, database design, and front-end construction for experienced database professionals.","keywords":["database","seminar","training"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"This first-time-ever seminar, set for Jan. 24-26, will cover data clean-up, database design and construction of front-ends. The intense, hands-on seminar for those who have mastered database managers and have some knowledge of building databases. For registration information, call (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-november-dont-forget-that-the-advanced-bootcamp","slug":"1996-november-dont-forget-that-the-advanced-bootcamp","url":"https://dwillis.github.io/uplink/article/1996-november-dont-forget-that-the-advanced-bootcamp.html","year":1996,"month":"November","monthNum":11,"headline":"Don't forget that the Advanced Bootcamp","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"NICAR and Philip Meyer are offering an Advanced Bootcamp in Chapel Hill from May 4-9, 1997, focusing on statistics and mapping for journalists. Registration is available through Wendy Charron via email or by calling NICAR.","keywords":["bootcamp","statistics","journalism"],"topics":["Journalism & Newsroom","Technology"],"technologies":["ArcView"],"full_text":"Don't forget that the Advanced Bootcamp offered by NICAR and Philip Meyer, author of \"The New Precision Journalism\" and journalism professor at the University of North Carolina-Chapel Hill, will be coming up May 4-9, 1997. The seminar, which is held in Chapel Hill, will concentrate on stats and maps. For registration information, send e-mail to Wendy Charron at wendy@nicar.org, or call NICAR at (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-diagnosing-hospital-health","slug":"1996-december-diagnosing-hospital-health","url":"https://dwillis.github.io/uplink/article/1996-december-diagnosing-hospital-health.html","year":1996,"month":"December","monthNum":12,"headline":"Diagnosing hospital health","kicker":"Vital Signs'","author_name":"Rick Linsk and Paul D'Ambrosio","author_title":"projects reporter","affiliation":"Asbury Park Press","summary":"A series titled 'Vital Signs' investigates the health of New Jersey hospitals, spurring significant discussion and action within the state's medical establishment.","keywords":["hospitals","health","New Jersey"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["UB","ICD","CD-ROM","SPSS"],"full_text":"How healthy are New Jersey hospitals? A year ago, we asked ourselves that question at the Asbury Park Press. The answers turned into \"Vital Signs,\" a series that ran from July to December and rocked the state's medical establishment. The project's roots go back two years, to the day someone slipped projects reporter Rick Linsk several pages of statistics on all New Jersey hospitals, including mortality rates, mortality rates after cardiac bypass surgery, and Cesarean section rates. Thinking these might be internal reports from the state Department of Health, Linsk called there. Not us, officials said; the state publishes no such performance measures on hospitals. But, they said, if you have $1,600 handy, we can sell you the data and you can do it. We were struck by this irony. In the very area where the stakes were so high - literally life and death - it is virtually impossible for consumers in New Jersey to get any meaningful information about their hospital or doctor. Across in the border in New York and Pennsylvania, by contrast, state agencies routinely publish bypass-surgery mortality rates and other figures for the public. We took the plunge. The first step was buying New Jersey's hospital uniform billing or \"UB\" data. About 30 states sell these inpatient discharge records, which contain demographic information such as age and gender plus myriad fields for conditions diagnosed and procedures performed. The price varies by state. To find out whether your state sells the data, contact the National Association of Health Data Organizations (703-532-3282). Diagnoses and procedures are represented by codes under the International Classification of Diseases (ICD short). You can decipher them with a $17 CD-ROM sold by the National Center for Health Statistics. Several medical publishers put out coding manuals that are good to have handy. Using UB data for medical research is not without controversy. Pitfalls include inconsistency in coding between hospitals, especially when it comes to areas with fuzzy definitions like various complications of labor. Experts also question whether hospitals reliably code their mistakes. And experts point out that the ICD codes do not yet distinguish between ailments that patients came in to the hospital with and those they acquire during their stay (pneumonia and infections being prime examples). Despite all this, a growing number of health researchers are using UB data because it's the best and only database available. New Jersey's public UB records identify the hospitals, but not doctors' or patients' identities. Date fields are also scrubbed. Some states reportedly withhold even the hospital names. If you get this far, the next hurdle is risk adjustment. Hospitals argue, and experts agree, that it's unfair to compare hospitals that have different kinds of patients. The place with a higher mortality rate may have older or sicker patients. To compare apples and apples, you have to adjust for the patient case mix. We briefly considered trying risk adjustment in-house using SPSS's logistic regression, but quickly became convinced it would not be a wise move. The major concern was credibility. We were about to launch the most ambitious review of hospitals in the state's history. We were going to tell 7.7 million residents which hospitals were great and which ones were killing too many patients. Our information had to be solid, and we had to present evidence that was acceptable to the medical and regulatory communities. No one would take us seriously unless we used a well-known vendor who did risk-adjustment for hospitals and insurance companies. Fortunately, we were able to forge a relationship with Inforum, Inc., a Nashville, Tenn., company whose parent firm, MedStat, essentially invented the medical risk-adjustment methodology 15 years ago. Inforum was looking to break into the New Jersey market and we were looking for attack-proof data. Inforum provided us risk-adjusted data in three areas: cardiac bypass surgery, stroke treatment and pneumonia/influenza care. We got the adjusted death rates, adjusted lengths of stays, and complication rates in each area and compared it to hospitals in eight other states. This was the basis for the report cards we published. We went a step further and extracted insurance, demographic and geographic data from the UB tapes. This helped us show how minorities of all ages were less likely than their white counterparts to undergo bypass surgery or pneumonia treatment. We also found women with insurance were more likely to have C-sections than poor women insured by Medicaid. Rates varied too between cities and suburbs.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-december-rocking-the-rules","slug":"1996-december-rocking-the-rules","url":"https://dwillis.github.io/uplink/article/1996-december-rocking-the-rules.html","year":1996,"month":"December","monthNum":12,"headline":"Rocking the rules","kicker":"Baby business","author_name":"Laurie Snyder","author_title":"projects reporter","affiliation":"The Cleveland Plain Dealer","summary":"Cleveland Plain Dealer reporters discovered that Level II hospitals were illegally accepting severely premature babies to increase profits, resulting in significantly higher death rates compared to specialized Level III hospitals. The investigation used birth and death records to expose how financial incentives were overriding patient safety protocols in neonatal care.","keywords":["hospitals","premature","deaths"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["9-track tape","PC","FoxPro"],"full_text":"On a tip, The Cleveland Plain Dealer's Joan Mazzolini, medical reporter, and Dave Davis, projects reporter, investigated why several hospitals were lobbying against tighter restrictions on neonatal care units.\n\nBy examining two years of birth and death records, the reporters found that many hospitals were accepting transfers of extremely premature babies, even though a state system was supposed to route these sick babies to the most sophisticated medical centers. These centers, which are certified as Level III hospitals, had cutting edge technology and a staff of specialists.\n\nBut in a time of managed care and shrinking profits, many hospitals are searching for ways to generate funds - and obstetrics, especially neonatal intensive care, is a good way to do it. In Ohio, the average cost of treating a premature baby, in 1994, was about $40,000 and could hit upwards of $400,000.\n\nSo many Level II hospitals, in an effort to \"get a bigger piece of the baby market,\" Davis says, were squeezing by state restrictions and presenting themselves to parents as able to provide the most advanced neonatal care.\n\nHowever, when it comes to the treatment of premature babies, all hospitals are not equal. The Plain Dealer series, which ran on Nov. 27 and 28, 1994, shows that the death rate of premature babies was significantly higher at the Level II hospitals than at nearby Level IIIs. The articles highlights what Davis calls \"another example of the business side of hospitals dictating how health issues are handled.\"\n\nThe data\n\nMazzolini and Davis requested Ohio's birth and death records for 1992 and 1993, although, at the time, only eight months of the 1993 records were available.\n\nEach nine-track magnetic tape cost $100. Getting the records was relatively easy, Davis says. \"In most cases, we had the data a few days after we filed our request\" with the state health department. And, the data were clean.\n\nThe reporters transferred the data from the tapes to an IBM 486, then used FoxPro for their analysis.\n\nMazzolini and Davis discovered that birth records are a CAR gold mine, with more than 60 fields of information. The records included when and where the baby was born, as well as such details as the gestation age of the baby, the mother's ZIP code, the number of prenatal doctor visits; and, most important, whether the mother was transferred from one hospital to another, either during labor or afterwards. \"Basically, we were able to track the flow of babies,\" Davis says. The birth records allowed Mazzolini and Davis to identify the hospitals that were treating extremely premature babies, Davis says. Then, linking this information to the death records using the hospital code number, they calculated the death rates at various Level II hospitals vs. those certified as Level III.\n\nFinding the names of babies who died wasn't as easy. Birth records do not contain the names or an unique identifier, such as a Social Security number, so they roughly matched the birth records by linking the birth date and hospital code to these same fields in the death records, which do contain names.\n\n\"We had good luck; out of 20 names, 15 would be correct matches,\" Davis says.\n\nThey also downloaded data from the Census Bureau site (www.census.gov), which provided the median income level for various ZIP codes. This data did need some cleaning, Davis says. The Census database listed nine numbers in the ZIP code column, so they wrote a program in FoxPro that removed the last four digits.\n\nAfter the series ran, the state legislature issued a moratorium on hospitals adding neonatal intensive care units-and, in the years since, have closed the loopholes that allowed Level II hospitals to skirt state restrictions. But, Davis says reporters should revisit the issue. \"Hospitals routinely thumb their noses at the state health department.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-whats-the-911","slug":"1996-december-whats-the-911","url":"https://dwillis.github.io/uplink/article/1996-december-whats-the-911.html","year":1996,"month":"December","monthNum":12,"headline":"What's the 911?","kicker":"Uplink","author_name":"Chris Ford","author_title":"Senior Investigative Producer","affiliation":"WTVT-TV","summary":"A report on EMT response times in Hillsborough County, Florida, reveals the EMS system's failures in meeting national goals, influenced by computer-assisted reporting techniques.","keywords":["response time","EMT","Florida"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["Excel","AutoTrack","DataTimes"],"full_text":"When WTVT-TV in Tampa, Fla., got a tip that its county's emergency medical system was taking too long to reach people in need, reporters used CAR and vans to illustrate the problem. With CAR, the reporters used spreadsheets of response times, maps of response zones and ambulance run reports to show how long ambulances were taking to reach people who had dialed 911. In unmarked vans, the station raced ambulances to the scenes of incidents they heard on a scanner. Reporters often arrived before paramedics and used a stopwatch to determine how long paramedics took to arrive. The reporters found that Hillsborough County's EMS failed to meet national response-time goals in one out of three cases. In one such case, Rafael Gonzalez died of a heart attack. It took paramedics 11 minutes to get to his home. The house was one mile from an EMS station. In one out of every four calls, it took paramedics nine or more minutes to reach people in the most densely populated parts of the county. In the rural areas, the response times were even longer. In more than one out of every three calls, it took paramedics 11 minutes or more. Chris Ford, senior investigative producer, said this series, which was broadcast in November 1995, marked the first time the station used computer-assisted reporting in all aspects of an investigation. It paid off. Ford said the statistics proved to the county's officials and residents that these poor response times were not isolated cases. \"Unquestionably, the computer concept and ideas it exposed allowed us to do this story in a way that was comprehensive and hard to challenge,\" he said. \"The computer really enabled us to make it hard for them to get any wiggle room.\" After receiving a tip from a paramedic, reporters at the station began investigating. The county's EMS officials did not want to release time records for the whole year, so reporters accepted response-time printouts for a month and started building an Excel database of their own. After much prodding from the station's lawyer, Hillsborough's EMS eventually released response records for the year. Reporters added those figures to their database. Ford said the key was in the field section. \"We were able to sort by different variables, and that really made a difference,\" Ford said. \"We were able to show not just that it was a problem, but where it was a problem, specific neighborhoods.\" One of those neighborhoods proved to be an area where the service was needed the most—a retirement community named Sun City. One paramedic, who did not want to be named, told reporter Kevin Kalwary and photographer Phil Celli that any resident of the retirement community who had a heart attack would be in danger because of the slow response times. \"You're basically gonna die,\" he said. \"We probably should give the pre-arrival instructions to hop in your car and drive like hell to the hospital because your chances are gonna be much better.\" The station was quick to remind viewers that they were not questioning the paramedics' quality of care, only their response times. The station talked to residents who had to wait for the ambulance, and some blamed the response times for the death of a family member. But Ford said reporters did not want to show up at people's doorsteps without an idea of what had happened. They again turned to computers. The station used Autotrak, an online public records database, for background information on victims and their families. They also used the Datatimes clip service to read about the incidents with long response times.\n\ntals are searching for ways to generate funds - and obstetrics, especially neonatal intensive care, is a good way to do it. In Ohio, the average cost of treating a premature baby, in 1994, was about $40,000 and could hit upwards of $400,000.\n\nSo many Level II hospitals, in an effort to \"get a bigger piece of the baby market,\" Davis says, were squeezing by state restrictions and presenting themselves to parents as able to provide the most advanced neonatal care.\n\nHowever, when it comes to the treatment of premature babies, all hospitals are not equal. The Plain Dealer series, which ran on Nov. 27 and 28, 1994, shows that the death rate of premature babies was significantly higher at the Level II hospitals than at nearby Level IIIs. The articles highlights what Davis calls \"another example of the business side of hospitals dictating how health issues are handled.\"\n\nThe data\n\nMazzolini and Davis requested Ohio's birth and death records for 1992 and 1993, although, at the time, only eight months of the 1993 records were available.\n\nEach nine-track magnetic tape cost $100. Getting the records was relatively easy, Davis says. \"In most cases, we had the data a few days after we filed our request\" with the state health department. And, the data were clean.\n\nThe reporters transferred the data from the tapes to an IBM 486, then used FoxPro for their analysis.\n\nMazzolini and Davis discovered that birth records are a CAR gold mine, with more than 60 fields of information. The records included when and where the baby was born, as well as such details as the gestation age of the baby, the mother's ZIP code, the number of prenatal doctor visits; and, most important, whether the mother was transferred from one hospital to another, either during labor or afterwards. \"Basically, we were able to track the flow of babies,\" Davis says. The birth records allowed Mazzolini and Davis to identify the hospitals that were treating extremely premature babies, Davis says. Then, linking this information to the death records using the hospital code number, they calculated the death rates at various Level II hospitals vs. those certified as Level III.\n\nFinding the names of babies who died wasn't as easy. Birth records do not contain the names or an unique identifier, such as a Social Security number, so they roughly matched the birth records by linking the birth date and hospital code to these same fields in the death records, which do contain names.\n\n\"We had good luck; out of 20 names, 15 would be correct matches,\" Davis says.\n\nThey also downloaded data from the Census Bureau site (www.census.gov), which provided the median income level for various ZIP codes. This data did need some cleaning, Davis says. The Census database listed nine numbers in the ZIP code column, so they wrote a program in FoxPro that removed the last four digits.\n\nAfter the series ran, the state legislature issued a moratorium on hospitals adding neonatal intensive care units-and, in the years since, have closed the loopholes that allowed Level II hospitals to skirt state restrictions. But, Davis says reporters should revisit the issue. \"Hospitals routinely thumb their noses at the state health department.\"\n\nMazzolini and Davis discovered that many of the babies were from \"families who were relatively well off\" and the majority of these families had private insurance, Davis says. \"Those are the cases that the hospitals wanted.\"\n\nwww.airportnet.org/DEPTS/federal/gao/safety.htm\n\nFor a federal court opinion on another tiff between the FAA and the NTSB, check out http://www.ll.georgetown.edu/Fed-Ct/Circuit/dc/opinions/94-1428a.html at Georgetown University.\n\nNASA\n\nThe National Aeronautics and Space Administration offers several sites that will help you analyze its Air Safety Reporting System database. Begin at http://olias.arc.nasa.gov/ for the aviation operation's homepage. Check out http://olias.arc.nasa.gov/ASRS/ASRS.html for the database's homepage. The site offers information such as the monthly newsletter Callback, which excerpts some reports on studies that use the data. Also look at ASRS Directline, at http://www-afo.arc.nasa.gov/ASRS/drctlin.html, where analysts highlight key information from the database. You can order the data here, though another shameless NICAR plug - we sell the data to journalists for less.\n\nHuman factors\n\nMany air crashes are blamed not on the planes but on the pilots. Two sites that offer information on human errors are the NASA Ames Research Center at http://olias.arc.nasa.gov/zteam/fcp/FCP-home.html and the FAA's http://www.tc.faa.gov/hfl/hfl.html\n\nA group of psychologists funded by NASA published their paper - including some specific accident information - at http://ww.psy.utexas.edu/psy/helmreich/acdntlst.htm\n\nInternational sources\n\nThe aviation safety Web pages at http://Web.inter.nl.net/users/H.Ranter/index.html offer a variety of information on international and U.S. air crashes. It's an intriguing Web site including things like a movie of the Ethiopian 767 crash, pages highlighting recent disasters and a listserv that promises to tell about accidents within a day or two. Much of the information is pulled from news accounts and others' data. This is a good attempt at gathering information on international air disasters.\n\nThe United Kingdom Air Accidents Investigation Branch at http://www.open.gov.uk/aaib/aaibhome.htm reports on accidents and makes safety recommendations.\n\nThe Commercial Aviation Resource Center at http://w3.one.net/~flypba/AIRLINES/carc.html offers a variety of links to airlines and airports around the world, identifying codes that may be useful for reporting certain stories, and symbols that indicate whether the sites are official or not.\n\nBoeing offers summary information on the world commercial jet airplane fleet at http://www.bts.gov/smart/cat/461.html\n\nGo to http://WWW.CAM.ORG/~icao/sites.htm and http://www.airparts.com/ for a vast list of airports, airlines and more.\n\nAndrew Lehren can be reached at (573) 882-0684, or send e-mail to andy@nicar.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1996-december-from-page-two-station-times-emergency-responses","slug":"1996-december-from-page-two-station-times-emergency-responses","url":"https://dwillis.github.io/uplink/article/1996-december-from-page-two-station-times-emergency-responses.html","year":1996,"month":"December","monthNum":12,"headline":"From page two: Station times emergency responses","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Journalists investigating emergency medical services should examine whether paramedics are deployed on fire trucks and consult ER doctors and the American Heart Association, while those analyzing hospital data must use reliable risk-adjustment methods from established vendors rather than attempting in-house analysis to maintain credibility.","keywords":["paramedics","hospitals","data"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["CD-ROM","SPSS"],"full_text":"Warning signs: Ford said there are several places to look if a reporter believes their city's emergency medical services are lagging. \"One real danger sign is look at the EMS to see if it has dual response or paramedic equipped fire trucks,\" he said. If paramedics are not riding on fire trucks, there could be a problem. In Hillsborough County, firefighters who had no paramedic training would often arrive long before EMS. Ford also suggested talking to emergency room doctors. The American Heart Association often has a local board that monitors response times.\n\nContinued from page one: Vital signs\n\nshort). You can decipher them with a $17 CD-ROM sold by the National Center for Health Statistics. Several medical publishers put out coding manuals that are good to have handy.\n\nUsing UB data for medical research is not without controversy. Pitfalls include inconsistency in coding between hospitals, especially when it comes to areas with fuzzy definitions like various complications of labor. Experts also question whether hospitals reliably code their mistakes. And experts point out that the ICD codes do not yet distinguish between ailments that patients came in to the hospital with and those they acquire during their stay (pneumonia and infections being prime examples). Despite all this, a growing number of health researchers are using UB data because it's the best and only database available.\n\nNew Jersey's public UB records identify the hospitals, but not doctors' or patients' identities. Date fields are also scrubbed. Some states reportedly withhold even the hospital names.\n\nNext hurdle\n\nIf you get this far, the next hurdle is risk adjustment. Hospitals argue, and experts agree, that it's unfair to compare hospitals that have different kinds of patients. The place with a higher mortality rate may have older or sicker patients. To compare apples and apples, you have to adjust for the patient case mix.\n\nWe briefly considered trying risk adjustment in-house using SPSS's logistic regression, but quickly became convinced it would not be a wise move. (Logistic regression takes string variables and predicts relationships. In medical data, it will tell if there is a strong relationship between, say, diabetes and kidney failure.)\n\nThe major concern was credibility. We were about to launch the most ambitious review of hospitals in the state's history. We were going to tell 7.7 million residents which hospitals were great and which ones were killing too many patients. Our information had to be solid, and we had to present evidence that was acceptable to the medical and regulatory communities. No one would take us seriously unless we used a well-known vendor who did risk-adjustment for hospitals and insurance companies.\n\nFortunately, we were able to forge a relationship with Inforum, Inc., a Nashville, Tenn., company whose parent firm, MedStat, essentially invented the medical risk-adjustment methodology 15 years ago. Inforum was looking to break into the New Jersey market and we were looking for attack-proof data. Inforum provided us risk-adjusted data in three areas: cardiac bypass surgery, stroke treatment and pneumonia/influenza care. We got the adjusted death rates, adjusted lengths of stays, and complication rates in each area and compared it to hospitals in eight other states. This was the basis for the report cards we published. We went a step further and extracted insurance, demographic and geographic data from the UB tapes. This helped us show how minorities of all ages were less likely than their white counterparts to undergo bypass surgery or pneumonia treatment. We also found women with insurance were more likely to have C-sections than poor women insured by Medicaid. Rates varied too between cities and suburbs.\n\nWords of caution\n\nDon't trust anybody or anything. We found errors with the raw UB data (incorrect hospital ID numbers), and Inforum's information (they told us one facility did not note secondary diagnoses when it did). Send your results to your target hospitals before you run and ask them to provide any data that is inconsistent with yours.\n\nBe prepared for a heavy barrage of misinformation, lies and damn lies from all fronts.\n\nHospital executives and trade groups will talk out of both sides of their mouths. In the same breath that they disparage your consultant's work, they will tell you they use the same data for their own internal reviews. Ironically, the hospital association - which laughed at an early inquiry from us about how to examine the discharge data - came in midway through the series and asked to work with us. Our response: Pucker up and kiss our.\n\nThe impact of the series was almost immediate. The state's health commissioner promised to issue his own bypass report card in 1997, moved to standardize stroke and pneumonia care, promised to beef up his hospital inspection staff, and crack down on a Newark bypass center that was violating its license.\n\nRick Linsk and Paul D'Ambrosio can be reached at (908) 922-6000\n\nREPLACE ALL astName ; WITH left (name, at (\",\" name)\nTo do the right part of the name, you need to know how to use the substr( ) function. Here are two examples:\n? substr (name, 6) n, William Jefferson\n\n? substr (name, 6, 5) n, Wi\n\nYou could sweat this one out, or just go back to the top of this article for the answer.\n\nRichard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-handout-of-the-month","slug":"1996-december-handout-of-the-month","url":"https://dwillis.github.io/uplink/article/1996-december-handout-of-the-month.html","year":1996,"month":"December","monthNum":12,"headline":"Handout of the month","kicker":"Medicine and health resources","author_name":"Barbara Gastel","author_title":"","affiliation":"IRE National Conference","summary":"This article provides journalists with comprehensive resources for researching health and medical topics, including reference books, internet directories, medical databases, and contact information for government health agencies.","keywords":["medicine","resources","journalists"],"topics":["Health & Environment","Journalism & Newsroom","Technology"],"technologies":["World Wide Web","Internet","gopher"],"full_text":"Medicine and health resources\n\nGood reference books:\n\nDorland's Illustrated Medical Dictionary (Philadelphia: W.B. Saunders, 1994 [28th ed.])\nStedman's Medical Dictionary (Baltimore: Williams & Wilkins, 1995 [26th ed.])\nThe American Medical Association Family Medical Guide (New York: Random House, 1994)\nThe Merck Manual of Diagnosis and Therapy (Rahway, N.J.: Merck & Co., 1992 [16th ed.])\nThe Physicians' Desk Reference (Montvale, N.J.: Medical Economics, 1996 [50th ed.])\nThe Control of Communicable Diseases Manual (Washington, D.C.: American Public Health Association, 1995 [16th ed.])\n\nInternet directories\n\nMedical \"nodes\" or \"hubs\" on the World Wide Web can be good starting points for gathering background information. Such sites include MedWeb (http://www.emory.edu/WHSCL/medweb.html), Medical Matrix (http://www.kumc.edu:80/mmatrix/), and Yahoo-Health (http://www.yahoo.com/Health). Of course, medical information on the Web varies greatly in quality; thus, let the browser beware. Among books that may aid in finding Internet resources are Dr. Tom Linden's Guide to Online Medicine (New York: McGraw-Hill, 1995) and The Internet Health, Fitness, and Medicine Yellow Pages (Berkeley, Calif.: Osborne McGraw-Hill, 1995). Given the rapid growth of the Internet, such books cannot be fully up-to-date. However, they may provide at least initial direction.\n\nJournal articles\n\nArticles in medical journals also can serve as useful background. Such articles can most readily be identified through the National Library of Medicine (NLM) database MEDLINE. Sources of access to MEDLINE include medical (and some other) libraries, some commercial online services, and NLM (see http://igm.nlm.nih.gov). To identify articles providing overviews of given areas, you can search MEDLINE specifically for review articles.\n\nAgencies\n\nAmong government agencies, the National Institutes of Health (NIH) is particularly geared to providing health information to reporters. The NIH homepage, through which the homepages of various NIH institutes can be reached, may be accessed at http://www.nih.gov/home.html\n\nPhone numbers of media contacts at selected NIH components are:\n\nNational Cancer Institute, (301) 496-6641\nNational Eye Institute, (301) 496-5248\nNational Heart, Lung, and Blood Institute, (301) 496-4236\nNational Institute on Aging, (301) 496-1752\nNational Institute on Alcohol Abuse and Alcoholism, (301) 443-3860\nNational Institute of Allergy and Infectious Diseases, (301) 402-1663\nNational Institute of Arthritis and Musculoskeletal and Skin Diseases, (301) 496-8188\nNational Institute of Child Health and Human Development, (301) 496-5133\nNational Institute on Deafness and Other Communication Disorders, (301) 496-7243\nNational Institute of Dental Research, (301) 496-4261\nNational Institute of Diabetes and Digestive and Kidney Diseases, (301) 496-3583\nNational Institute on Drug Abuse, (301) 443-1124\nNational Institute of Environmental Health Sciences, (919) 541-3345\nNational Institute of Mental Health, (301) 443-4536\nNational Institute of Neurological Disorders and Stroke, (301) 496-5924\nNational Institute of Nursing Research, (301) 496-0207\nOffice of Alternative Medicine, (301) 496-1712\n\nIf unsure which NIH component(s) to consult about a topic, you can search the NIH information index at gopher://gopher.nih.gov:70/11/clin/nih-infobook or call the main NIH telephone number, (301) 496-4000.\n\nOther government information sources include the Centers for Disease Control and Prevention (http://www.cdc.gov/cdc.html, media phone number (404) 639-3286) and the Food and Drug Administration (http://fdahomepage.htm media phone number (301) 443-3285). CDC has an automated fax information service providing fact sheets and other documents on diseases and other health topics; for information, call (404) 332-4565.\n\nMedical Librarians\n\nMedical librarians can aid greatly in finding written and electronic sources of information. Such individuals are readily contacted at the libraries of medical schools.\n\nThe National Library of Medicine online database DIRLINE can help identify volunteer and medical associations and government agencies. On the Internet, DIRLINE may be accessed by typing \"telnet locator.nlm.nih.gov\" and then selecting \"Directory of Information Resources\" from the menu; it also may be reached from the NLM Locator page at (http://www.nlm.nih.gov/databases/locator.html). Institutional sources may be identified through the American Association for the Advancement of Science directory \"Science Sources\" (http://sci.aaas.org/sources/). And the standard reference work the Encyclopedia of Associations has a section listing more than 2,000 health and medical organizations.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-check-out-the-ama-web-site","slug":"1996-december-check-out-the-ama-web-site","url":"https://dwillis.github.io/uplink/article/1996-december-check-out-the-ama-web-site.html","year":1996,"month":"December","monthNum":12,"headline":"Check out the AMA Web site","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The American Medical Association website now features a searchable database where users can look up doctors' addresses and basic credentials through the AMA Health Insight section.","keywords":["doctors","database","credentials"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Check out the AMA Web site, which now has a searchable database of doctors listing addresses and basic credentials. http://www.ama-assn.org/ Navigate to \"AMA Health Insight\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-lets-split","slug":"1996-december-lets-split","url":"https://dwillis.github.io/uplink/article/1996-december-lets-split.html","year":1996,"month":"December","monthNum":12,"headline":"Let's split","kicker":"TechTip","author_name":"Richard Mullins","author_title":"Senior Contributing Editor","affiliation":"NICAR","summary":"A guide to learning FoxPro string functions through hands-on experimentation, using the practical example of splitting name fields separated by commas to help journalists master computer-assisted reporting.","keywords":["FoxPro","functions","reporting"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["FoxPro","Access"],"full_text":"One of the recurring questions on NICAR-L is some variation of: \"I have a table with a name field and I need to Those who are starting to bear the stains of exposure to computer-assisted reporting will use the term \"string functions.\"\n\nThe advice on several occasions recently has been a brief pointer at part of the FoxPro manual or the online help, accompanied by the encouragement to sweat it out on your own. This is good, sound advice. It's not motivated by laziness or a certain relish in picturing you slogging through the personal hell they were forced to go through, back in the old days, and not like you kids today, and you know the rest.\n\nHere how to slug it out on your own. Take short steps and make sure you understand each part before you go on. Here is the sort of answer you might get if you asked, \"How do I a put into a new field everything in a field that is found to the right of the comma?\" might come in handy for fields with contents like this: Smith, Robert- or, as in our example, Clinton, William Jefferson) REPLACE ALL RestName ; WITH SUBSTR (name, name) +1)\n\nIf you're thinking, \"This looks cryptic and hard to understand,\" then you should be declared legally sane. If you get the command to work, it may be the result of some luck and a few guesses in correcting the syntax errors after FoxPro tells you that you are missing argument, missing parenthesis, etc.\n\nTo illustrate the approach, assume a table called PREZ with a field called NAME. If the table is open and the record pointer (or cursor) is on the first row, you can try out the various functions and see the results on a single row, before you make any changes to your data. In the syntax examples that follow, the line in bold type is what you see on the FoxPro screen when you type in the command window. This is the convention used in the FoxPro manual. The question mark acts as a command word and means, \"print this on screen.\"\n\nTry out the left function with assorted numbers:\n\n? left (name, 3)\nCli\n? left (name, 11)\nClinton, Wi\n? left (name, 1)\nC\n\nNow, the original problem, posed for the sake of discussion and also one that comes up all the time in real life, is splitting name fields that are separated by a comma. If you understand how the FoxPro replace all command does the changes, then you only need three string functions to do the splitting:\n\nleft () at ( ) substr ()\n\nThe advice about sweating it out is related to the old \"teach someone to fish\" proverb: If you see these functions in action fixing one problem, you'll be able to figure out how to use them on new problems. After that, there is good news and more good news: There are more than three functions in FoxPro (and Access), and you know something useful about how to learn to use them.\n\nNow, back to the command window and taking small steps to solve the task at hand. We could state our instructions in English like this: \"For every row, find the location of the comma, then pick up the characters on the left side, up to, but not including the comma.\n\nWe've already seen how to print a certain number of characters, starting from the left. But names are not all the same length, which means the comma is not in the same place in every row of a table. This is not an obstacle, because of two facts about FoxPro: 1) There is a function to locate the position of a character within a string of characters, and 2) The results from one function can be submitted to another function. A short term for this, used by geeks, is nesting, an obvious metaphor when you think about it.\n\nApplying this, we try out the at(), function, then try combining it with left(), and when we're finished with that, put it all into the replace command. Using the at function on \"Clinton, William Jefferson\" gives the answer of 8 as the position of the comma. Note the syntax is:\n\nAT StringToSearchIn)\n\n? name\nClinton, William Jefferson\n\n? at (\",\", name)\n8\n\nBefore combining the two functions, we'll use the literal number 8 with the left function, and then using the At() we'll pass along the same number to the Left() function.\n\n? left (name, 8)\nClinton,\n? left (name, at (\",\" , name) )\nClinton\n\nTo fix the problem of the trailing comma, you include one less character in the left command. The math for this translates to subtracting one from the number that indicates the comma position.\n\n? left (name, 8 -\nClinton\n? left (name, at (\",\" , name) - 1 )\nClinton\n\nNow, you're ready to use this command to split the last name into its own field. Save yourself typing mistakes and paste your perfectly debugged statement into the replace all part of the command.\n\nREPLACE ALL LastName ; WITH left (name, at (\",\" name)\n\nTo do the right part of the name, you need to know how to use the substr( ) function. Here are two examples:\n\n? substr (name, 6)\nn, William Jefferson\n\n? substr (name, 6, 5)\nn, Wi\n\nYou could sweat this one out, or just go back to the top of this article for the answer.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-take-our-money-please","slug":"1996-december-take-our-money-please","url":"https://dwillis.github.io/uplink/article/1996-december-take-our-money-please.html","year":1996,"month":"December","monthNum":12,"headline":"Take our money - please","kicker":"CAR Trips: First Ventures","author_name":"Griff Palmer","author_title":"database editor","affiliation":"The Oklahoman","summary":"An investigation by The Oklahoman reveals that Oklahoma City's police fleet has significantly higher repair costs and mileage than comparable cities, with some vehicles accumulating over 200,000 miles and repair bills exceeding $50,000. Reporter Griffin Palmer explains the data analysis methodology used to benchmark the city's fleet performance against national standards.","keywords":["fleet","maintenance","benchmarking"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":[],"full_text":"Take our money - please\n\nBy Griff Palmer The Oklahoman\n\nGriff Palmer attended Advanced Bootcamp in May.\n\nUntil he was contacted by a reporter from The Oklahoman, Oklahoma City police officer Mike Williams didn't know repair and fuel bills on his police cruiser topped on a car that originally cost $12,250.\n\nThe car has been in the shop for maintenance or repairs nearly every two weeks since 1992, The Oklahoman reported. Yet Williams told the newspaper his cruiser is not the worst of the bunch.\n\nIt's not unusual among the city's fleet of 600-plus cars to see repair and operating costs topping $50,000, The Oklahoman found. The average mileage for Oklahoma City's police fleet is just under 100,000 miles; 303 police vehicles have logged more than 100,000 miles; 85 cars have more than 150,000 miles; and 12 cars have more than 200,000 miles.\n\nThis compares with eight out of 10 cities surveyed by the American Fleet Managers Association, which reported average mileage per police car as 69,000 miles or less.\n\nReporters Steve Lackmeyer, Penny Owen, and Diane Plumberg and database editor Griffin Palmer got at the heart of the story with data. Palmer tells how:\n\nWe worked from a cumulative database that reflected figures for the lifetime of each car. If I had more time, I'd have liked to have gotten the database broken out by year. The National Association of Fleet Managers publishes a booklet titled \"Benchmarking for Quality in Public Service Fleets.\" NAFA surveyed 90 public fleets, ranging in size from 250 units to 10,000-plus units. It analyzed a variety of service/performance measures, plotting distributions of fleet administrators' responses for each measure.\n\nOn many of the measures, figures for police cars are plotted separately from those for other fleet sedans.\n\nMany of those distributions are based on annual figures, so having local fleet figures broken out by year allows a reporter to compare the local fleet against NAFA survey results. In addition to plotting the distribution curve, NAFA charts the median value for each measure, along with the low 20th percentile and the high 20th percentile.\n\nAmong the many measures in the NAFA book are average number of miles vehicles are driven each year; average number of miles vehicles have been driven since they were put into service; yearly maintenance and repair costs per mile; fuel cost per mile; fuel mileage; total cost per mile; downtime rate; miles between breakdowns; miles per mechanic hour.\n\nTricky measures\n\nCapital cost is one of the trickier measures. NAFA defines capital cost as the vehicle's purchase price minus its projected disposal value, divided by the number of miles it is projected to be driven during its retention period.\n\nIn order to compare against the NAFA benchmarks, a reporter must have those figures for the local fleet.\n\nWe found the downtime rate to be problematic because of the way Oklahoma City's Equipment Services Division records its data. City officials said they record time out of service as the number of hours between the time a work order is written and the time it is closed off the books. However, they said a car is often actually put back onto the street long before the work order is cleared off the books, rendering the downtime figures meaningless.\n\nA great many municipal fleet maintenance services operate on enterprise accounting. They receive no general fund appropriations, and have to generate their operating revenue from chargebacks to the city agencies whose vehicles they service. They are constantly being scrutinized for the fairness of their charges, and to make sure the work cannot be done for less by commercial vendors. Thus, most fleet maintenance operations tend to keep good, detailed service data.\n\nOklahoma City's cumulative maintenance records for each police cruiser included VIN number, police unit number, original cost, fuel cost, fuel quantity, number of breakdowns, parts costs, accident repair costs, labor costs, labor hours, total work orders written, total mileage and total maintenance and operating costs.\n\nGriffin Palmer can be reached at palmer@qns.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-national-association-of-fleet-managers-notice","slug":"1996-december-national-association-of-fleet-managers-notice","url":"https://dwillis.github.io/uplink/article/1996-december-national-association-of-fleet-managers-notice.html","year":1996,"month":"December","monthNum":12,"headline":"National Association of Fleet Managers notice","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The National Association of Fleet Managers publishes a benchmarking guide that helps reporters evaluate the performance of public service vehicle fleets, including those used by law enforcement.","keywords":["fleets","benchmarking","performance"],"topics":["Investigative Methods","Public Safety"],"technologies":[],"full_text":"The National Association of Fleet Managers, (908) 494-8100, publishes a booklet titled \"Benchmarking for Quality in Public Service Fleets,\" which can help reporters judge the performance of local public service fleets, such as those operated by the police force.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-put-the-pedal-to-the-metal","slug":"1996-december-put-the-pedal-to-the-metal","url":"https://dwillis.github.io/uplink/article/1996-december-put-the-pedal-to-the-metal.html","year":1996,"month":"December","monthNum":12,"headline":"Put the pedal to the metal","kicker":"NICAR notes","author_name":"Brant Houston","author_title":"managing director","affiliation":"NICAR","summary":"NICAR announces an expanded program for 2000 including new Uplink columns on statistical methods, affordable databases for journalists, intermediate and advanced seminars on data analysis and web databases, and an expanded national conference in Nashville with training in mapping, statistics, and programming.","keywords":["training","databases","journalism"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Lotus Notes"],"full_text":"It's time to accelerate again at NICAR. Over the coming months we will start a series of new efforts and expand on the many already underway. Here's a partial list of what Uplink and the NICAR database library plan for the New Year:\n\nAdd an Uplink column on the use of statistical methods in reporting.\nProvide databases at low cost to journalists, and consistently add tipsheets, handouts and sample stories to the database package.\nAdd an intermediate level seminar (coming this January) on data cleaning, good database design, and advanced data analysis.\nAdd a seminar in the spring on creating searchable databases on the Web.\nBecome a broader repository for common and often used data, particularly state and local campaign data.\nContinue our programs to make CAR available to minority journalists by offering fellowships and working with the Missouri School of Journalism's Multicultural Management Program.\nOffer management seminars for broadcast and print journalists on how to bring and keep a good CAR program at their news organizations.\nContinue our basic bootcamps. Our January bootcamp is full, and our May bootcamp is filling fast. We'll have two more in the summer.\nGive an expanded national conference in Nashville, Tenn., in March. This national conference will not only cover the usual computer-assisted reporting panels and training, but also will have advanced classes in mapping, statistics, programming, Web development, and Lotus Notes.\nParticipate in and give academic symposiums and work with journalism faculties to integrate CAR in their curricula.\nDevelop more joint programs with other universities. We already work with Northwestern University, University of North Carolina, University of Maryland and Columbia University.\nContinue our on-the-road programs to better help small- to medium-size news organizations, but add on-line tutorials as follow-up training.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-wanted-executive-director","slug":"1996-december-wanted-executive-director","url":"https://dwillis.github.io/uplink/article/1996-december-wanted-executive-director.html","year":1996,"month":"December","monthNum":12,"headline":"Wanted: executive director","kicker":"IRE Notes","author_name":"Bruce Moores","author_title":"IRE staff","affiliation":"Investigative Reporters and Editors","summary":"Investigative Reporters and Editors (IRE) is seeking an executive director to lead the organization's training and professional development initiatives, requiring a journalist or experienced manager with demonstrated leadership skills and preferably computer-assisted reporting expertise.","keywords":["journalism","leadership","director"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"The leading association of investigative journalists is seeking an executive director to lead Investigative Reporters and Editors into the 21st century.\n\nIRE, the top newsroom training organization and standard-setter in our business, offers a host of services to the profession, including training journalists in advanced reporting techniques for print and broadcast, editing, and computer-assisted reporting. It is based at the University of Missouri-Columbia. If you have proven yourself as a journalist and manager, we want to hear from you.\n\nWe are seeking an energetic chief executive who will relish helping other journalists here and worldwide; oversee an $800,000 budget; manage a highly motivated staff of 20 individuals; raise funds; guide the development of educational conferences and publications; oversee various IRE programs, including the National Institute for Computer-Assisted Reporting and the Periodistas de Investigacion; and spread the word about the good work of IRE.\n\nThe executive director reports to an elected board of 13 journalists. A seasoned journalist is preferred but not required. The ideal candidate will have demonstrated managerial and administrative effectiveness; master's degree or equivalent combination of education and experience; and superior organization and communication skills. If you know computer-assisted reporting, you will have an edge. This faculty position at the Missouri School of Journalism includes teaching responsibilities and generous benefits. Salary ranges $50,000 to 60,000.\n\nCandidates should send a cover letter, resume and three reference letters to Bruce Moores, IRE Search Committee, 138 Neff Annex, Missouri School of Journalism, Columbia, Mo. 65211. A review of applications will begin in late January.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-you-get-what-you-pay-for","slug":"1996-december-you-get-what-you-pay-for","url":"https://dwillis.github.io/uplink/article/1996-december-you-get-what-you-pay-for.html","year":1996,"month":"December","monthNum":12,"headline":"You get what you pay for","kicker":"On the Internet","author_name":"Andrew Lehren","author_title":"staff","affiliation":"NICAR","summary":"Free aviation data websites like Landings.com offer FAA Service Difficulty Reports but may be incomplete compared to full databases obtained through FOIA requests, with the official FAA data also having limitations. The article provides guidance on accessing aviation data from various sources including the FAA, NTSB, and other searchable databases for journalists.","keywords":["aviation","data","FAA"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web","FTP","ASCII"],"full_text":"There's a lot of free aviation data on the Web but it may be worth what you paid for it. Consider www.landings.com. It offers the Federal Aviation Administration's Service Difficulty Reports. To the uninitiated, these maintenance reports seem pretty complete. But beware. I compared the Landings site vs. the full database obtained by NICAR under the Freedom of Information Act. To run this test, I picked TWA flight 800, which crashed this summer near Long Island, at the Landings site.\n\nThe result: Landings showed 18 SDRs since November 1992, vs. 20 in the full database. In addition, in the history of SDRs, there are more than 100 for that aircraft. The Landings data is not in a rows-and-columns format that can be quickly downloaded for further analysis, though you can download the text of the reports. Also, key fields that give fuller versions of events are not available. I e-mailed Landings to reply on these problems, but did not get a reply. The outfit is not an official FAA site, and does warn that its data may not be complete. To be fair, Landings does offer a lot of aviation information, and does seem promising. It's just not complete.\n\nTo download official data, the FAA offers week-by-week ASCII files at http://www.fedworld.gov/ftp/faa-asi/faa-asi.htm. But the agency does not include a record layout for the data, and it only posts reports for the most recent two months. It will fax you a copy of the record layout. Call the FAA's Jim Gillespie at (405) 954-6484. But a word of caution: The data on this Web site is truncated. It, too, does not include fields available under the FOIA, including the airline's ranking of the severity of the problem.\n\nShameless NICAR plug: We've been revising and expanding the way we collect aviation data, and we will do searches by request or send copies of the data to journalists.\n\nHere are other aviation sites that may help you cover aviation and airplane disasters:\n\nThe FAA\n\nAfter jumping off FAA's main page at www.faa.gov, check out the links and publications listed at http://www.faa.gov/APA/publicat/GUIDETOC.htm FAA press releases are also at http://www.dot.gov/affairs/faaind.htm The agency's A Journalist's Guide to the FAA is at http://www.faa.gov/apa/publicat/jnalist.htm For the bulk of FAA data, plus advisories and other detailed information, visit ftp://ftp.fedworld.gov/pub/faa-oai/faa-oai.htm\n\nThe NTSB\n\nLook up abbreviated National Transportation Safety Board accidents reports at http://www.ntsb.gov/Aviation/months.htm\n\nThe NTSB offers studies from its data at http://www.ntsb.gov/Publictn/Publictn.htm, and summary statistics at http://www.ntsb.gov/Aviation/aviation.htm\n\nOther searchable sites\n\nSearch ownership records using Canadian tail numbers (they begin with a C) at http://www.cavok.com/registration.html\n\nLandings, by the way, lets you search U.S. and U.K. tail numbers. I ran some sample tests and did not encounter any problems.\n\nFor a detailed listing of airports and navigational systems for those facilities, check out http://www.cc.gatech.edu/db1/fly/\n\nThis offers everything from usage figures to radio systems that pilots use to land.\n\nFor U.S. flight schools, visit http://www.aviationWeb.com/\n\nThe dirt on dirty aviation data\n\nThose who have covered airline crashes know aviation data is tricky.\n\nThe FAA's Aircraft Catastrophic Failure Prevention Research Program at http://www.tc.faa.gov/aar400/aar431/ offers expansive information on crashes, including aviation industry complaints about journalists probing FAA data, and how U.S. aviation data is too dirty and incomplete to quickly spot safety problems.\n\nAside from checking the U.S. General Accounting Office itself, you can quickly look at its report on dirty aviation data at http://\n\nTrade Groups:\n\nThe International and American Associations of Airport Executives run http://www.airportnet.org/ Information on certain lobbying efforts is off limits, but this does offer links to airports and information on airport security. National Business Aircraft Association is for those who fly corporate jets. At http://www.nbaa.org/ you can find tidbits such as positions on pending legislation. The Helicopter Association International offers some federal safety information at http://www.rotor.com/ But some reporters may be more intrigued by lobbying information, including stances against restrictions for flights over national parks.\n\nAndrew Lehren can be reached at (573) 882-0684, or send e-mail to andy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-ive-got-a-secret","slug":"1996-december-ive-got-a-secret","url":"https://dwillis.github.io/uplink/article/1996-december-ive-got-a-secret.html","year":1996,"month":"December","monthNum":12,"headline":"I've got a secret","kicker":"NASA reports anonymous tip-offs","author_name":"Andy Lehren","author_title":"NICAR staff","affiliation":"NICAR","summary":"A journalist used NASA's Air Safety Reporting System database to uncover near-miss airplane incidents at Newark Airport, demonstrating how this anonymous reporting system can reveal safety problems that traditional FAA and NTSB records may miss.","keywords":["aviation","safety","database"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["Air Safety Reporting System","FoxPro","Monarch","CD-ROM"],"full_text":"These are the stories of airplane crashes that almost happened.\n\nA Federal Express 727 landed on the wrong runway at Newark Airport in March 1993 nearly striking another Federal Express jet that was waiting for final clearance before takeoff.\n\nA Continental 727's pilot got confused and turned onto the wrong runway of the same airport. A USAir 737 was hurdling down that runway for takeoff. The air traffic controllers thought they would collide. The USAir pilot thought fast and got enough lift to leapfrog the Continental jet.\n\nKey source\n\nThese are some of the tales found by the Newark Star-Ledger's Ted Sherman. On Sept. 8, he published a package of stories that highlighted problems at the New Jersey airport. One of the key sources for his story was the Air Safety Reporting System, a database that can provide valuable leads about problems at your hometown airport.\n\nThe database is compiled by the National Aeronautic and Space Administration from anonymous reports filed mainly by pilots, air traffic controllers and others involved in aviation. The database started in 1988 as a way for those in aviation to report safety problems. Sherman is among those who have mined it to highlight near disasters.\n\nTricky data\n\nThe database is a little tricky. To protect anonymity, NASA strips out actual flight numbers and other identifying information. But it does include the month, year, location and other descriptive information. Sherman found that, in key cases, it gave enough clues so he could verify and figure out the details of those incidents. In some cases, the clues led him to stories in his newspaper's morgue. In other cases, he searched National Transportation Safety Board and Federal Aviation Administration records. But the NASA database underscored a key point - the NTSB and FAA are not a complete source for airline safety data.\n\nSherman also recommended, \"search broadly, because of coding problems.\" He found, for instance, one type of problem coded several different ways. It was important to pay attention to how data could be entered into the database.\n\nThe key\n\nOne helpful way he approached the data was to slice out just those incidents involving Newark. However, even then he paid attention to the way data was entered into the system. He found that looking for his region, including metropolitan New York, revealed problems that involved Newark Airport. He suggested other reporters begin searching the database for the geographic area that interests them, and know which regional facilities to track. The key is looking at the facilities that affect traffic at your local airport. Again, pay attention to coding. Sherman found one New York radar facility reported two different ways in the database.\n\nBiggest shortcoming\n\n\"The biggest shortcoming,\" Sherman said, \"is air traffic controllers. They don't use it so much.\" He hoped to find more reports filed by them. Instead, he found, \"pilots use it much more.\"\n\nMany of the interesting nuggets lie in the database's text fields, where the reports include a narrative description of what almost went wrong. The database includes instances where pilots complain about the way things work at an airport, including bad runway designs or other problems that raise safety concerns.\n\nSherman obtained the database through a private company for $135. But, he cautioned, the interface on the CD did not permit easy transfers into FoxPro. He used Monarch in a time-consuming effort to get those text fields into a .dbf.\n\nAndy Lehren can be reached at (573) 882-0684, or send e-mail to andy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-ads-nprms-and-service-bulletins","slug":"1996-december-ads-nprms-and-service-bulletins","url":"https://dwillis.github.io/uplink/article/1996-december-ads-nprms-and-service-bulletins.html","year":1996,"month":"December","monthNum":12,"headline":"ADs, NPRMs and service bulletins","kicker":"More plane crash sources","author_name":"Bonnie Britt","author_title":"Reporter","affiliation":"The Tribune Review","summary":"An aerospace reporter discovered a 20-year-old Boeing 747 crash near Madrid through aviation industry publications and FAA records, which revealed a potential link to the TWA Flight 800 disaster through similar fuel tank vulnerabilities. By systematically searching trade magazines, library databases, and NTSB records, the reporter obtained a detailed investigation report showing how a lightning strike ignited a fuel sensor spark, leading to catastrophic wing failure.","keywords":["crash","investigation","records"],"topics":["Investigative Methods","Public Safety","Technology"],"technologies":["Microfiche","Internet","World Wide Web"],"full_text":"ADs, NPRMs and service bulletins\n\nMore plane crash sources\n\nBy Bonnie Britt\nThe Tribune Review\n\nOn July 24, in the Seattle Times, aerospace reporter Byron Acohido wrote, \"The crash of TWA Flight 800 was not the first time witnesses on the ground watched as a Boeing 747-100 burst into flames and tumbled to earth.\"\n\nThe story went on to describe a crash 20 years ago that drew little public attention: A former TWA 747-100 converted into a freighter for the Iranian air force - a sister ship to the one that crashed off Long Island - exploded in flight near Madrid, Spain.\n\nHow did he know that?\n\nHot tip\n\nOn a tip, Acohido headed to the library where he spent half a day pressing through four years of indexes on microfiche from Aviation Week & Space Technology, the industry bible.\n\nWhen he didn't find the reference in the trade magazine's table of contents, he pushed through it again, 1975-1979, page by page. He found a brief reference in a column item and another reference to an FAA airworthiness directive (AD) triggered by the crash. The AD called for inspection for dangerous fuel leaks that could cause fire and explosion.\n\nThe tip was legitimate. Where were the details?\n\nAcohido says, \"It happened 20 years ago on Spanish soil. It was an Iranian jet. I assumed be impossible to confirm.\" Hungry for elaboration of the crash mentioned briefly in the two Av Week stories, Acohido looked up Spanish authorities. When that went no where, he called in the calvary.\n\n\"We have a great library. I asked for a universal search. The librarian found three or four references to the crash, plus a couple of remotely related clips, including excerpts from a magazine article that talked about major disasters involving large aircraft.\n\nNTSB\n\n\"The article made reference to the National Transportation Safety Board finding implicating lightning in the Madrid crash. It turned out the NTSB and a bunch of agencies had retrieved and closely examined the wing that exploded in Madrid. A light bulb went off. There had to be an NTSB report.\"\n\nAcohido checked the NTSB's Web site (www.ntsb.gov) but saw no reference to the Madrid incident or the 1976 report. He says, \"You can't trust their Web site since it is not complete.\"\n\nThen he went low tech. He called the records department at the agency and politely requested a copy of the report. Within 24 hours, Acohido had the 50-page report via Federal Express. The report - Special Investigation Report: Wing Failure of Boeing 747-131 near Madrid, Spain. May 9, 1976 - is available from the National Technical Information Service (U.S. Department of Commerce) in Springfield, Va.\n\nAcohido copied it to an aircraft wing expert. The report detailed clear and compelling evidence that a moderate lightning strike (which the aircraft is designed to withstand) most likely ignited a stray spark on a fuel sensor inside a partially-full fuel tank. A small explosion followed by an intense localized fire began a sequence that led to the wing coming apart and releasing a huge ball of burning fuel less than a minute after lighting struck the wing.\n\n\"The NTSB text gives step-by-step hard evidence in very technical prose. You have to read between the lines. My wing expert and other knowledgeable sources helped me get a layman's grasp of what the report said.\"\n\nNext stop, FAA\n\nThe next step was to obtain the Federal Aviation Administration's airworthiness directive from the AD database. An AD is an order from the FAA to the industry to fix a flaw in a particular model of aircraft. ADs come with deadlines. Acohido was certain an AD had been issued after the 1976 crash since the encyclopedic Av Week referred to it. Guessing there might be other fuel tank glitches, he pressed the FAA for all ADs relating to 747 fuel tank problems.\n\nHe says his newspaper is discussing building and maintaining its own database of airworthiness directives.\n\nThe FAA maintains indexes for their databases of notices of proposed rule-making and the airworthiness directives.\n\nWhen you are not sure what you want, ask for the AD index to the model aircraft (i.e. 747-100 or 737-100, 200 and 300) that interests you. It contains all regulations pertaining to everything on that model that may be flawed, except what didn't get past the rule-making process.\n\nWhen you know which AD you want, also ask for the electronic file on the Notice of Proposed Rule-Making (NPRM) associated with that AD. It contains the industry discussion of what went wrong, how serious the flaw is, economic factors in making a fix and logic about deadlines. An AD is a congressionally mandated FAA answer to each and every letter that makes up the NPRM file.\n\nContents of many items in the two databases - NPRMs and ADs - provide long satisfying looks at the process leading to orders for industry-wide fixes of known defects.\n\nTo be thorough, ask for copies of the original letters that make up the NPRM file. They reveal much more than the FAA's electronic summary of who said what in the rule-making discussion.\n\n\"From experience,\" Acohido says, \"I know you don't just want the ADs that say a flaw must be fixed in a certain way. You also want the NPRMs since they provide the discussion and criticism of the proposed fix.\"\n\nThe final rule or AD gives the industry the recipe to fix the aircraft part that's flawed in general terms, typically referring to Boeing service documents for specific steps.\n\nService bulletins\n\nYou guessed it: the Boeing documents, called service bulletins, are another gold mine.\n\nSays Acohido, \"Deeper behind the NPRMs are the service bulletins - the manufacturer's explanation and drawings on how to make the fix. You can get them from the FAA.\" Doing so calls for imagination. After the TWA crash, Acohido recalls, \"We physically had to go down to the FAA for the service bulletins and were stonewalled at that level.\"\n\nOften the service bulletin will contain a single profound line revealing the manufacturer's logic about why a fix must be made.\n\nADs seeking to fix weak locks on 747 cargo doors were supported by Boeing service bulletins, which noted what could happen if a door ever came open at altitude: \"It can cause rapid decompression.\"\n\nGiven the choice of inspecting for weak locks or installing stronger locks, many airlines procrastinated, until a cargo door came open on United Flight 811 off Hawaii in 1989. Nine passengers were sucked out a gaping hole in the fuselage during the resultant rapid decompression. The FAA then mandated stronger locks on all 747s within 30 days.\n\nThe Boeing service bulletins supporting ADs mandating inspections and fixes for leaking 747 fuel tanks note succinctly: \"Leaks can cause fire.\"\n\nWhere besides the reticent FAA (or the manufacturer) can you find service bulletins in a hurry?\n\nMake friends with the faculty at a college where aircraft mechanics are trained. Ask to see the data on the service bulletin database for the fixes that interest you.\n\nWhen you return with it, Acohido says \"Your artists will love you.\"\n\nOn Aug. 21, the Seattle Times ran a generous, above the fold graphic based on detailed diagrams of the 747, its wings, its fuel bays and areas prone to leaks sourced to Boeing's service bulletin.\n\nBold type asked: \"Did fuel leak cause explosion?\"\n\n\"Federal aviation and Boeing service records contain extensive documentation about where the wing and belly tanks on 747s are susceptible to leaks that can cause fire.\"\n\nBonnie Britt can be reached by e-mail at bonnieb@cris.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-products-notice","slug":"1996-december-products-notice","url":"https://dwillis.github.io/uplink/article/1996-december-products-notice.html","year":1996,"month":"December","monthNum":12,"headline":"Products notice","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"The article introduces two CD-based resources for journalists investigating aircraft: Jane's Electronic Information System, which catalogs civilian and military aircraft, and International Air-CD, which provides global aircraft ownership records.","keywords":["aircraft","databases","investigation"],"topics":["Technology"],"technologies":["CD-ROM","Internet","World Wide Web"],"full_text":"No endorsements here. NICAR has not tried these intriguing products. But you can check out a sample of the Jane's Electronic Information System CD at http://www3.btg.com/janes/eisdemo.html The CD itself promises full descriptions of every civilian and military aircraft made. International Air-CD at http://www.infomart.net/ promises ownership records for aircraft around the world, plus links to several aviation sites.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-get-your-training","slug":"1996-december-get-your-training","url":"https://dwillis.github.io/uplink/article/1996-december-get-your-training.html","year":1996,"month":"December","monthNum":12,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR and The Associated Press offer specialized journalism training in data analysis, spreadsheet building, and database management at various locations and conferences throughout 1997. Training options include on-the-road sessions, bootcamps, and seminars at professional conferences with varying costs and registration requirements.","keywords":["training","journalism","database"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","PC","spreadsheet"],"full_text":"NICAR and, where indicated, The Associated Press provide specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases.\n\nCost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below.\n\nNote: In the following list, \"Open to all\" means any journalist may sign up. \"Closed\" means the session is open only to members of the host organization.\n\nSan Francisco, CA., San Francisco Bay Guardian-Jan. 17, 1997\nAAN West Convention - Jan. 17-18, 1997\nMilwaukee, Wisc., Milwaukee Journal Jan. 21-23, 1997. Closed.\nYork, Pa., York Daily Record - Jan. 27-28, 1997. Closed.\nSan Diego, Calif., San Diego Tribune and SPJ - Feb. 19-22, 1997. Open to all. (619) 293-1261\nPalm Beach, CA., Palm Beach Post March 1997.\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969.\n\nMinority Regional Conference, Richmond, Va.- Nov. 15-17, 1996. Open to minorities.\nMinority Regional Conference, Dallas, Texas - Dec. 7-8, 1996. Open to minorities.\n\nNashCAR, NICAR National Conference, Nashville, Tenn.-March 3-9, 1997. Costs are: IRE member, $150 for entire conference. Non IRE member, $190 for entire conference, including IRE membership. Renewal member, $190 for entire conference, including IRE renewal. Students, $125 for conference, including student membership. Late fee is $15 for registrations not postmarked by Feb. 14, 1997\n\nIRE National Conference, Phoenix, Ariz., June 12-15, 1997.\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo.\n\nAs with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size.\n\nFor information, call Wendy Charron at (573) 882-0684.\n\nJan. 5-11, 1997. Waiting list only.\nJan. 24-26, 1997. Intermediate Bootcamp concentrating on data clean-up and more. For details, call (573) 882-0684.\nMay 4-9, 1997. Advanced Bootcamp concentrating on stats and maps at the University of North Carolina-Chapel Hill with Philip Meyer, author of The New Precision Journalism. The seminar is held in Chapel Hill. For details, call (573) 882-0684.\n*May 18-23, 1997. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-from-the-nicar-library","slug":"1996-december-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1996-december-from-the-nicar-library.html","year":1996,"month":"December","monthNum":12,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering aviation safety, elections, healthcare, transportation accidents, crime statistics, and financial records, useful for investigative journalism and data analysis.","keywords":["databases","government","investigations"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["CD-ROM"],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW Federal Aviation Administration's accidents and incidents, including major plane crashes since 1971.\n\nNEW NASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\nA monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\nFederal procurement data, 1992-1995, includes breakdowns by agency.\nAlcohol, Tobacco, and Firearms gun dealer records. 1993-1995.\nNational Bridge Inventory System data, includes inspection grades. 1994-1995\nNEW FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1995 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nThe National Inventory of Dams. 1991-1995.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-gearing-up-for-nashcar","slug":"1996-december-gearing-up-for-nashcar","url":"https://dwillis.github.io/uplink/article/1996-december-gearing-up-for-nashcar.html","year":1996,"month":"December","monthNum":12,"headline":"Gearing up for NashCAR","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NashCAR, NICAR's 1997 national conference in Nashville (March 6-9), will offer comprehensive computer-assisted reporting training for journalists at all levels, including sessions on Web journalism and job opportunities.","keywords":["conference","journalism","training"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["World Wide Web"],"full_text":"NashCAR, NICAR's 1997 national conference, will be March 6-9 in Nashville, Tenn., and will offer the widest range of panels and hands-on training yet. Building on the previous conferences in Santa Clara, Calif. (CAR Trek) and Cleveland, Ohio. (CAR Rock), NashCAR will have sessions for beginners, intermediates and advanced users. We'll have tracks for beat reporters, editors, newsrooms managers, broadcasters, researchers and librarians, and trainers both in newsrooms and universities.\n\nThe conference will make use of computer labs at local universities and will have a demonstration room that will display equipment and software. We'll also have vendors and numerous opportunities for journalists to find jobs.\n\nIn addition, we expect to explore the effect of the World Wide Web on journalism and how journalists are using the Web for their stories.\n\nFor more information, call (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-moving-on","slug":"1996-december-moving-on","url":"https://dwillis.github.io/uplink/article/1996-december-moving-on.html","year":1996,"month":"December","monthNum":12,"headline":"Moving on","kicker":"","author_name":"","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"George Landau, formerly of the St. Louis Post-Dispatch, has founded NewsEngin, Inc., a company offering affordable database applications and web-based tools for organizations. The startup provides database technology solutions and will collaborate with NICAR on research and development projects.","keywords":["database","startup","technology"],"topics":["Data & Databases","Technology","Journalism & Newsroom"],"technologies":["Lotus 1-2-3","Internet"],"full_text":"George Landau has left the St. Louis Post-Dispatch to start NewsEngin, Inc., \"an information engineering co-op.\" Landau aims to provide affordable database applications for organizations willing to build intranets, as well as Internet products using common Web database technology from Microsoft and Lotus.\n\nCheck out what Landau has to offer at http://www.NewsEngin.com\n\nAmong other things, you'll find a Cost-Of-Living calculator and Percent Change calculator in the Free Tools section. Landau will also be working on research and development projects with NICAR.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-science-journalists","slug":"1996-december-science-journalists","url":"https://dwillis.github.io/uplink/article/1996-december-science-journalists.html","year":1996,"month":"December","monthNum":12,"headline":"Science journalists","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Knight Science Journalism Fellowships at MIT are accepting applications until March 1 for a nine-month program designed for mid-career journalists with at least three years of experience. Six journalists will be selected to participate in the fellowship beginning in September.","keywords":["fellowship","journalism","MIT"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"The deadline to apply for the Knight Science Journalism Fellowships at MIT is March 1.\n\nThe fellowship provides nine months of study in Cambridge, Mass., for mid-career science journalists.\n\nThe 14-year-old program is open to staff and freelance journalists working radio, television, newspapers, magazines, and news agencies, with at least three years' full time experience. Six journalists from the United States or Canada are chosen each year by a selection committee of leading science journalists and MIT faculty.\n\nThe winners will be named late in April for the academic year beginning Sept. 1.\n\nFor information, contact: Victor K. McElheny, Knight Science Journalism Fellowships, E32-300, Massachusetts Institute of Technology, Cambridge, Mass. 02319-4307","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1996-december-ire-l-and-nicar-l-addresses","slug":"1996-december-ire-l-and-nicar-l-addresses","url":"https://dwillis.github.io/uplink/article/1996-december-ire-l-and-nicar-l-addresses.html","year":1996,"month":"December","monthNum":12,"headline":"IRE-L and NICAR-L addresses","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Instructions for posting to and managing subscriptions for IRE-L and NICAR-L discussion lists hosted on the University of Missouri's listserv system.","keywords":["listserv","subscription","journalism"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"IRE-L and NICAR-L are located on lists.missouri.edu, a dedicated Unix system designed to service discussion groups run out of the University of Missouri. All posts to the lists should be sent to: ire-l@lists.missouri.edu or to nicar-l@lists.missouri.edu\n\nAll commands for subscription changes should be sent to: listproc@lists.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-january-dig-for-on-line-gold","slug":"1997-january-dig-for-on-line-gold","url":"https://dwillis.github.io/uplink/article/1997-january-dig-for-on-line-gold.html","year":1997,"month":"January","monthNum":1,"headline":"Dig for on-line gold","kicker":"Backgrounding","author_name":"Neil Reisner","author_title":"The Miami (Fla.) Herald","affiliation":"The Miami Herald","summary":"This article explores how the Internet and commercial on-line services have revolutionized investigative reporting by making complex background research accessible and affordable.","keywords":["backgrounding","databases","investigative reporting"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Nexis","Dialog","DataTimes","Dow Jones News Service","Internet","World Wide Web","modem/BBS","CompuServe","America Online","Netscape","Internet Explorer","CDB Infotek","IRSC"],"full_text":"Time was when backgrounding a story subject wasn't for the fainthearted. It meant leafing through sheaves of yellowed clips, poring through dusty volumes, and reviewing sheaves of public records. Only the high-powered investigator with all the time in the world would attempt to deeply background a subject; the ordinary beat reporter working on daily deadlines didn't stand a chance. Even the privileged reporters who had the time to delve deeply into a subject's past wouldn't go on-line. Resources such as Lexis/Nexis, Dialog, Datatimes, and Dow Jones News Service were too expensive and not available to mere reporters. The explosion of the Internet and other on-line services has changed all that. Indeed, much of the information that once was available only by combing through books and dusty paper records or by paying hundreds of dollars for high-priced databases now can be obtained for no more than the $20 monthly it costs to connect with a local Internet provider or the $9.95 per month subscription to a commercial on-line service such as CompuServe or America On-line. Clips, Lexis/Nexis, Dialog and hard copy volumes such as \"Who's Who in [fill in the blank]\" are still invaluable resources that should be used if they are available. But journalists who want to mine all there is to know about their subjects - or even get basic information on deadline - do well to expand their horizons beyond the stalwarts. All it takes is a couple of on-line connections, ordinary journalistic curiosity, and a sense of adventure. Assuming you already have a computer, a modem running at a speed no slower than 28.8 kilobytes per second, a phone line and communication software to let them all talk to one another, you also need: An Internet account that provides access to the World Wide Web. This should cost about $20 monthly through either a national or local Internet Service Provider (ISP) and will let you surf the Web for as many hours as you like, using a Web browser such as Netscape Navigator or Microsoft Explorer. An account with a commercial on-line service such as America Online or CompuServe. America Online recently changed its price to $19.95 for unlimited access, mirroring most ISPs. CompuServe costs $9.95 monthly, including five hours of access, with additional hours at $2.95. CompuServe, especially, has deep research databases and is generally regarded a superior research tool. Either service also provides access to the World Wide Web, but the connections offered by ISPs are often faster and more stable. An account with a private data broker such as Database Technologies, CDB Infotek or IRSC, providing on-line access to specific information on individuals, their property holdings, etc. Some services can tell you what kind of car a prospect drives, how much his home and other property is worth, whether she owns an airplane, and more. The World Wide Web has achieved near divine status in the pop-culture consciousness. And the Web likely is the greatest (though least organized) collection of information known to humankind. It's all there wheat, chaff, and in between - in amounts that rapidly become overwhelming. The trick is to use the Web well, to learn how to focus your Web surfing so that you quickly get close to what you want. There is a myth that the World Wide Web is the Internet, but it's not. The Internet is simply a worldwide network of interconnected computer networks. Strip away all the hype and the Web is merely a means to navigate the Internet by way of an easy-to-use \"point and click\" interface. All you really need to know is that you use a \"browser\" such as Netscape or Microsoft Explorer to move around the Web and that a Web address looks something like this: http://www.nicar.org/nashcar Known as a \"Universal Resource Locator\" or \"URL,\" this address directs the Web browser to open a particular Web page; all Web pages have an address that resembles it. Knowing a little bit more about Internet addresses can help a lot. For example the \".org\" in the above address signals that the page is located on a computer connected to a nonprofit organization. The address would end in \".edu\" if the site were associated with an educational institution; \".gov\" if it's an official government site; \".mil\" if it's military; \".net,\" if the page is on a computer attached to an ISP; and \".com\" if it's associated with a business. This \"domain hierarchy\" can help you evaluate the quality of the information provided. What can you find? More than you possibly could imagine. Web sites will help you locate prospects' addresses and telephone numbers, articles written about them, and the socioeconomics of the neighborhoods where they live. In general, data that can be sold will not be available for free on the Internet. That means that Lexis/Nexises of the world are not likely to go away. Neither are the America Onlines and CompuServes of the world, where much of the same information is available at lower cost. Among them are: Dun & Bradstreet and TRW business profiles. Knowledge Index, a subset of Dialog containing more than 100 databases, including many major regional and national newspapers and many professional journals. The Biography Research Center, providing access to the full range of Marquis Who's Who volumes and more. Magazines and newspapers. The Magazine Database searches 250 popular magazines; the Newspaper Archives accesses 55 daily newspapers. Private data brokers, who electronically compile public records from around the country, can be the mother lode for researchers wanting to find out everything about a prospect. One of the most popular data brokers is Database Technologies of Pompano Beach, Fla., which offers a service known as Autotrak. While DBT does not offer information as comprehensive as some of the others, the price is right: $1.50 per minute, with no sign-up or subscription fee. Other brokers, such as CDB Infotek or IRSC, can be more expensive and charge monthly access fees. But they offer more information, up to and including information gathered from state court systems. On-line research will involve a learning curve. Plan on investing some time and money before you get proficient. Don't throw your books and other resources away. The on-line world is just one more tool researchers have at their disposal. It doesn't replace other sources, it supplements them. Remember that no single on-line resource will provide everything there is to know about everybody and everything. Remember, too, that information gathered via modem is subject to the same sort of errors as information gathered on paper. Use what you find on-line as a road map, but verify everything.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-january-backgrounding-sites","slug":"1997-january-backgrounding-sites","url":"https://dwillis.github.io/uplink/article/1997-january-backgrounding-sites.html","year":1997,"month":"January","monthNum":1,"headline":"Backgrounding sites","kicker":"On the Internet","author_name":"Neil Reisner","author_title":"","affiliation":"The Miami (Fla.) Herald","summary":"The article provides journalists with valuable online resources for background research on people and businesses, including tools maintained by news researchers and prospect researchers, along with specific websites for locating individuals and accessing public records.","keywords":["research","backgrounding","resources"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Internet","World Wide Web","Netscape","CompuServe"],"full_text":"Several sites provide useful starting points for reporters beginning to background people and businesses on-line. One of the best is maintained by Nora Paul, the Poynter Institute's ace news researcher: an on-line version of her book \"Computer-Assisted Research,\" that can function as an on-line tutorial.\n\nThe URL is: http://www.poynter.org/poynter/chome.html\n\nWhat may surprise many journalists, though, is that there is a whole other profession whose members are as interested in backgrounding people as they are. These are \"prospect researchers,\" the folks who help charities and other nonprofits figure out how much potential donors can afford to give by checking out property records, stock prices, biographies, demographics - records equally useful to journalists. Here are a few sites maintained by some of the best in that profession that are worth checking out.\n\nThe Internet Prospector, a terrific site maintained by Randy Bunny at the University of Wyoming, with contributions from ace researchers around the country. It includes tutorials: http://plains.uwyo.edu/~prospect\n\nThe Prospect Research Page, maintained by the University of Washington's David Lamb: http://weber.u.washington.edu/~dlamb/research.html\n\nButler University's Prospect Research Home Page, maintained by Martha T. Murphy, director of prospect research there: http://www.butler.edu/~mmurphy/index.htmlx\n\nFinding & backgrounding people\n\nUse both the Internet and private data vendors to find out everything from prospects' addresses and phone numbers to the vehicles they drive.\n\nYahoo People Search-http://www.yahoo.com/search/people/\n\nAn all-around good people locator, searchable by name or phone number, that can provide postal and e-mail addresses. Other useful people locators are Switchboard (http://www.switchboard.com) and Four11 (http://www.four11.com). Clicking on the button marked \"People\" while on-line with Netscape's toolbar will bring up a page with a number of other useful people-finding sites. Another useful site is Who Where (http://www.whowhere.com), which combines people-business-finding links.\n\nThe American Medical Association Physician Locator - http://www.ama-assn.org/\nFind physician addresses and credentials. From the AMA homepage, click on \"AMA Health Insight\" and then \"Physician Select.\"\n\nMartindale-Hubbell Lawyer Locator http://www.martindale.com\n\nThe on-line version of the well-thumbed, multi-volume lawyers directory. Find addresses, phone numbers, professional credentials. Westlaw provides similar information through West's Legal Directory: http://www.wld.com\n\nThe Biography Research Center This section of CompuServe (GO BIOGRAPHY), provides access to the full range of Marquis Who's Who volumes and additional similar works. Searches and downloads of full-text articles cost in the area of $5 each.\n\nData brokers\n\nData brokers compile billions of public records on individuals, including information on addresses, former addresses, property, vehicles, professional licenses, neighbors, relatives, court records, and more. Financial records, access to which are restricted under the Fair Credit Reporting Act, ordinarily are not available. The various data brokers have different offerings and different price structures. Some require special software, others use standard communications software or are available via the Internet. When opening an account with a broker, make certain that all information offered is truly public and legally acquired.\n\nThree of the better known brokers are:\nCDB Infotek, Santa Ana, Calif.: (800) 427-3747\nDatabase Technologies, Pompano Beach, Fla.: (800) 279-7710\nIRSC, Fullerton, Calif.: (800) 640-4772\n\nGo on-line to do in-depth research on both publicly held and private businesses.\n\nHoovers Online-http://www.hoovers.com\nSearch a database of more than 10,000 profiles of companies, both public and private. Free searches provide basic information on companies, with links to additional on-line sources. A $9.95 per month subscription enables access to more in-depth profiles.\n\nDun & Bradstreet, TRW business profiles\nThese profiles are a way to research thousands of businesses, including officer names, general business information, payment patterns, debts, liens, and judgments, etc. Most information is self-reported and must be considered in that context, but for privately-held companies, these sources often provide the best available information. Both are available on CompuServe. GO DUNS for D&B, GO TRW for TRW. Depending on the profile, prices range from about $30 to about $70.\n\nSecurity & Exchange Commission/EDGAR http://www.sec.gov\nThe SEC EDGAR database enables researchers to search for all SEC documents from publicly traded companies that file electronically. There is a 24-hour delay between the time documents are filed and when they are available via EDGAR. Immediate access is available through private vendors, including EDGAR Online, http://www.edgar-online.com, which charges $9.95 month.\n\nStock Quotes--http://www.stockmaster.com\nMany services provide on-line stock quotes, usually with a 15-20-minute delay. Stockmaster is just one.\n\nNews\n\nThe American Journalism Review, a journalism trade magazine, maintains a Web site that lists the on-line addresses of hundreds of newspapers, magazines and radio and television stations. The URL is http://www.ajr.org. The Yahoo Internet index also provides a good list at http://www.yahoo.com.\n\nSome journals permit full-text searching of their archives via their Web sites. For those that don't, try Knowledge Index (GO KI), the Newspaper Archive (GO NEWSARCHIVE), the Magazine Database (GO MAGDB). Downloads of full-text articles cost no more than a couple of dollars each.\n\nHere are some major Web news outlets. Some charge for complete access:\n\nThe New York Times: http://www.nytimes.com\nThe Washington Post: http://www.washingtonpost.com\nThe Chicago Tribune: http://www.chicago.tribune.com\nThe Los Angeles Times: http://www.latimes.com\nThe Wall Street Journal: http://www.wsj.com\nUSA Today: http://www.usatoday.com\nMSNBC: http://www.msnbc.com\nCNN: http://www.cnn.com/\n\nGeneral Information\n\nThe U.S. Census Bureau - http://www.census.gov\nThe official source of information on the nation, states, counties, municipalities, even neighborhoods.\n\nThe White House - http://www.whitehouse.gov\nA gateway to information on the president and all executive-branch departments. See especially \"The Interactive Citizens Handbook,\" which provides links to departments, independent agencies, and more.\n\nTHOMAS: Congress On-line http://thomas.loc.gov\nTHOMAS (named after Thomas Jefferson) provides links to both houses of Congress. Offerings include bill-tracking and the Congressional Record.\n\nThe CIA World Factbook - http://www.odci.gov/cia/publications/pubs.html\nA CIA handbook containing detailed profiles of every country in the world.\n\nThe America Online Reference Desk- http://www.aol.com/community/channels/reference\nA variety of reference links - dictionaries, thesauri, etc. - put on the Web by the folks at AOL.\n\nMapQuest - http://www.mapquest.com\nMaps of a neighborhood or a region. Pick the area you want to see and the map will display. Can be printed.\n\nMailing lists and newsgroups\n\nInternet mailing lists, sometimes known as listservs, allow people interested in a particular topic to talk among themselves via e-mail. The computer administering the list acts as a kind of relay station; any e-mail sent to the list is relayed to anybody who signed up for the list.\n\nPRSPCT-L is a listserv for prospect researchers that can yield useful tips for journalists, too.\n\nTo subscribe, send e-mail to: listserv@bucknell.edu\n\nIn the body of the message, type the following: SUBSCRIBE PRSPCT-L\n\nYou will receive a message acknowledging your subscription. Make sure to read and keep it - it describes how to post to the list and how to unsubscribe.\n\nTo post a message to PRSPCT-L, send it to: prspct-l@bucknell.edu\n\nNote that you post to the list itself, while you subscribe (and unsubscribe) by sending e-mail to the listserv address.\n\nThe Internet Prospector is a monthly newsletter developed and distributed by the folks who bring you the prospect research site of the same name. It is posted to the PRSPCT-L mailing list every month. You can also see both the current and past issues at the Internet Prospector Web site, listed above.\n\nIf the subject of a story is on-line, it can be useful to find out what they've been posting in Internet discussions. DejaNews archives the contents of thousands of Internet newsgroups","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-january-the-reporters-network","slug":"1997-january-the-reporters-network","url":"https://dwillis.github.io/uplink/article/1997-january-the-reporters-network.html","year":1997,"month":"January","monthNum":1,"headline":"The Reporters Network","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Reporters Network has launched a redesigned website featuring a Media Directory where journalists can manage their own listings, along with two bulletin boards for news reporters and freelance writers to share job openings, contests, and professional announcements.","keywords":["journalists","website","bulletin"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"The Reporters Network has launched its newly redesigned Web site, announces Bob Sablatura, a reporter at The Houston (Texas) Chronicle and network director. The site includes a Media Directory that allows journalists to control their own listings. Also included are two bulletin boards, one for news reporters and another for freelance writers. The new boards will carry announcements of job openings, contest entries, published work and other news of interest to journalists. The URL is: http://www.reporters.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-january-find-it-on-cd-rom","slug":"1997-january-find-it-on-cd-rom","url":"https://dwillis.github.io/uplink/article/1997-january-find-it-on-cd-rom.html","year":1997,"month":"January","monthNum":1,"headline":"Find it on CD-ROM","kicker":"Handout of the month","author_name":"Jennifer LaFleur","author_title":"San Jose (Calif.) Mercury News","affiliation":"San Jose Mercury News","summary":"This article lists various CD-ROM resources available to journalists for backgrounding people, researching government budgets, analyzing crime statistics, and mapping locations.","keywords":["CD-ROM","research","government data"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["CD-ROM"],"full_text":"Driving from courthouse to courthouse to dig through documents or ordering various faxes for background information can be minimized today with the many resources available on CD-ROM. Here's a wonderful reference list for CD-ROMs: DirectPhone/SelectPhone: Two/five regional discs containing both 10 million business and 90 million residential listings. SelectPhone can be searched by name, address, or telephone number. 11 Million Businesses Phone Directory: One disc containing business listings. 1.1 Million Health & Medical Industry Reference Directory: Listing everything from physicians to health clubs. The North American Facsimile Book: Contains 150,000 fax numbers. Military Personnel/Base Locator: Contains more than 128,000 service people and civilian employees. Statistical Abstract of the United States: Corresponds to the government bestseller containing a plethora of statistics. Budget of the United States Government: Contains all parts of the budget. Regional Economic Information System: Local area economic data including personal income and employment. Crime and Justice Data: Wealth of crime and justice data from the Bureau of Justice Statistics. School District Data Book on CD-ROM: Data derived from the 1990 Census data and the National Center for Education Statistics. Toxic Release Inventory for 1987-1990: Data about what plants are emitting what. Bureau of Transportation Statistics: Agency develops more than a dozen CD-ROMs a year including Traffic Safety Data. Mutual Funds on Disc: Analysis by Morningstar on 3,000 mutual funds. PharmAssist: Multimedia disc explaining drugs, legal and illegal. Marriage Records Series: Genealogical information covering marriages from various states. Geographic Names Information System: From the U.S. Geological Survey with about 2,000,000 place names. Federal News Service: Access to verbatim transcripts of Congressional hearings and briefings. U.S. Code: Laws of the United States. CIA World Fact Book: Profiles of countries. Military Contracts and Procurement Locator: Chronicles military contracts and purchasing. Defense Library on Disc: International affairs, science, defense, and military law. Street Atlas USA: Street level maps of the whole United States. Global Explorer: Great maps of the earth down to street level in cities. InfoTax CD: Nonprofit organizations summary data. Current Population Survey Utilities: Raw data from the annual survey conducted by the Census Bureau. National Election Studies: Database of basic research on voting and public opinion.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-january-new-on-the-road-trainer-hired","slug":"1997-january-new-on-the-road-trainer-hired","url":"https://dwillis.github.io/uplink/article/1997-january-new-on-the-road-trainer-hired.html","year":1997,"month":"January","monthNum":1,"headline":"New on-the-road trainer hired","kicker":"NICAR and IRE Notes","author_name":"Brant Houston","author_title":"Managing Director","affiliation":"NICAR and Investigative Reporters & Editors","summary":"NICAR announces Sarah Cohen as its new training director and previews the upcoming NashCAR conference scheduled for March 6-9 in Nashville with prominent CAR journalists and extensive hands-on training opportunities.","keywords":["training","conference","journalism"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web","Lotus Notes","SQL","modem/BBS"],"full_text":"As the new year starts, we want to share news on the latest happenings at NICAR and Investigative Reporters & Editors.\n\nNew training director\n\nSarah Cohen, recently of The St. Petersburg Times, becomes NICAR's newest training director. Sarah starts her new job this month and already is scheduled to travel to California and Pennsylvania.\n\nSarah will be telecommuting from her home in Maryland, following the lead of the previous training director, Neil Reisner, who worked out of his New Jersey home. (Neil begins working at the Miami Herald late this month.)\n\nAt The St. Petersburg Times, Sarah worked as a business reporter using computer-assisted reporting extensively. Before that she worked at the Tampa Tribune.\n\nSarah switched to journalism in 1992 after working as an economist, primarily at the Bureau of Labor Statistics. This means Sarah brings a wonderful amount of knowledge of numbers and stats to the job. We hope that Sarah will not only do training, but also will analyze some data for news organizations.\n\nSarah a journalism master's degree from the University of Maryland's journalism school and a bachelor's degree in economics from University of North Carolina in Chapel Hill, both schools at which NICAR has jointly done seminars.\n\nNashCAR\n\nThe schedule for panels at NashCAR, March 6-9, in Nashville, Tenn. (register through IRE, 573-882-2042 or on our Web site www.nicar.org/nashcar) is included in this issue. Panelists and instructors are being chosen as this is being written - look for them in the next issue of Uplink. As usual, we plan to cover a wide range of topics with some of the best CAR journalists in the country. Among the panelists and instructors will be CAR guru Philip Meyer, author of \"The New Precision Journalism\"; Shawn McIntosh, who heads CAR at the Dallas Morning News; training expert Jennifer LaFleur, now database editor at the San Jose (Calif.) Mercury News and NICAR's first training director; Paul Adrian of WBNS-TV in Columbus, Ohio; Neill Borowski of The Philadelphia (Pa.) Inquirer, David Milliron, head of CAR for Gannett News Service; Bill Casey of the Washington Post; Brad Goldstein of The St. Petersburg Times; Bob Port of Associated Press; Tom Gaumer of The Cleveland Plain Dealer, and a host of others from newspapers and TV stations, including some of the newer names in CAR.\n\nAt this point, the computer situation in NashCAR looks awesome. We will rent some computers for the hotel, but Vanderbilt University has generously loaned several computer labs containing about 140 computers with Internet connections for the conference.\n\nIn addition to panels and three classes of hands-on training, we will have a mentor program in which you can get your worked critiqued, a demonstration room for software, the Internet and Intranets, a section for vendors and rooms for impromptu meetings and brainstorming.\n\nIRE membership\n\nIf you use NICAR services, then becoming an IRE member can save you or your news organization more money than ever. Uplink will cost $40 for members, instead of $60 for nonmembers. Data from the database library will cost $10 more for nonmembers. Requests for articles from the IRE Resource Center - articles which often show you how to best use data from the database library - will cost more for nonmembers. If you become an IRE member you also will get the six issues of the IRE Journal free and numerous other IRE services.\n\nWeb activities\n\nOur Web site (homepage: www.reporter.org) continues to expand and improve. We hope to have new designs for our pages up and running in mid-February. Wallace Winfrey, our Web administrator, and other staff members are hard at working putting more handouts and other educational material on the site.\n\nWe also will have a demonstration soon of a clickable map where you can choose a state and get a list of investigative reporters in that state. We also are making better use of Lotus Notes and Microsoft's SQL server. More on this later. Brant Houston can be reached at (573) 882-0684, or send e-mail to brant@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-january-ire-membership-online-join-notice","slug":"1997-january-ire-membership-online-join-notice","url":"https://dwillis.github.io/uplink/article/1997-january-ire-membership-online-join-notice.html","year":1997,"month":"January","monthNum":1,"headline":"IRE membership online join notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE has streamlined its membership process, allowing people to join online through their website or by downloading and faxing a membership form.","keywords":["membership","IRE","online"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web","Adobe Acrobat"],"full_text":"We've just made it easier to become an IRE member. Now, you can join IRE via our Web server, or you can download a copy of our membership form in Adobe Acrobat format from the Web site and submit it by fax to (573) 884-5544. For more information, check out http://www.ire.org/membership.html If you have any questions, e-mail Web Master Wally Winfrey at wally@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-january-pushing-the-bridge-limits","slug":"1997-january-pushing-the-bridge-limits","url":"https://dwillis.github.io/uplink/article/1997-january-pushing-the-bridge-limits.html","year":1997,"month":"January","monthNum":1,"headline":"Pushing the bridge limits","kicker":"Car Trips: First ventures","author_name":"Dave Sheingold","author_title":"","affiliation":"Westchester Rockland News","summary":"A Gannett newspaper series utilized a state transportation database to reveal that heavy school buses and commercial trucks were routinely violating weight limits on aging bridges.","keywords":["bridge safety","infrastructure","database analysis"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["Paradox","ASCII","FLIMPORT"],"full_text":"It sounded like a no-brainer: 13-ton school buses crossing 3-, 4- and 5-ton bridges running over busy railroad tracks? Trucks weighing up to 40 tons specifically directed to cross an 11-ton bridge over the same railroad lines? Couldn't be? Could be. In fact, that - and other potential disasters - are just what Gannett Newspapers found in the northern New York City suburbs as part of a two-part series in November that stemmed from searches through a statewide database of bridge conditions. Among the possibilities for serious trouble found by the newspaper were these: Fully-loaded school buses, which weigh anywhere from 10 to 15 tons running every school day over at least six bridges with signs barring vehicles of more than 3, 4 or 5 tons. Large vehicles ranging from 6-ton delivery trucks to 15-ton fire trucks to 40-ton tractor-trailers using the same bridges as the buses. A sign posted by one village that directed industrial truck traffic such as cement mixers, tractor-trailers and garbage trucks weighing well over 20 tons when full over an 11-ton bridge. After the newspaper chain notified about 10 local busing and trucking companies, all vowed to immediately change their routes. But that didn't stop experts on bridge maintenance and construction from warning that the potential for crisis continued to loom across the state. The difficulties in dealing with the problem included the century-old age of some bridges, commercial traffic routes running over many of these spans, police manpower too limited to consistently enforce the weight limits and a professed lack of funds by a commuter railroad that owns many of the most endangered bridges. The story began with an idea to look at general bridge conditions contained in a complex ASCII, fixed-length database of about 100 heavily coded fields describing the latest bridge inspections and results, kept by the state transportation department. Initially, about 50 of the fields were imported into a database manager using Paradox for DOS. We used a fixed-length importer called FLIMPORT that allowed us to pick and choose which fields we wanted to look at. We began to notice that many of the bridges popping up in the weight-limit field ran through commercial areas. We pulled them out, sorted them by community and weight limit, then camped out by 15 bridges for about a half-hour each. It took only a few minutes of observation at 11 of the bridges to see that the limits were routinely ignored. Nearby residents and merchants said the situation had gone on for years and that they figured it was allowed. Meanwhile, school-bus officials in several communities, as well as businesses whose trucks were seen using the bridges, professed surprise and ignorance of the problem and immediately said they would tell their drivers to re-route themselves. A county bus official said he believed it was all right to run 15-ton buses over 10- and 11-ton bridges, but later changed his mind after a state bridge official strongly suggested otherwise. A follow-up story is planned in a few months to see what, if any, changes are really made.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-january-use-foxpro-to-document-a-table","slug":"1997-january-use-foxpro-to-document-a-table","url":"https://dwillis.github.io/uplink/article/1997-january-use-foxpro-to-document-a-table.html","year":1997,"month":"January","monthNum":1,"headline":"Use FoxPro to document a table","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"","affiliation":"The Missouri School of Journalism","summary":"A technical tip explaining how to use FoxPro's 'copy structure extended' command to document database layouts and create data dictionaries.","keywords":["FoxPro","database","documentation"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["FoxPro"],"full_text":"One of the paper essentials in computer-assisted reporting is the record layout. However, after you have created a table in FoxPro, it is very easy to document its structure, add in the data dictionary and even include narrative explanations of fields and stuff, all in one single file. You can do this by creating a structure description table from the data table and adding fields and information to it. Here's how to do it. First, with the desired data table open, use this FoxPro command to create the structure table: copy structure extended to NewTable. The table, NewTable, contains fields like Field_name, Field_len, Field_type, and Field_dec. You also can add fields to NewTable to help further document the original table- just be sure to add fields to the end. The most useful field to add might be a description field of 30 or 40 characters, since the 10-character field name limit in FoxPro can require cryptic names. You can also create a memo field for longer comments. This keeps the structure table usable for the next step - creating a new table based on the structure table: create NewTableName from StructureTableName.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-january-pawnshops-draw-criminals","slug":"1997-january-pawnshops-draw-criminals","url":"https://dwillis.github.io/uplink/article/1997-january-pawnshops-draw-criminals.html","year":1997,"month":"January","monthNum":1,"headline":"Pawnshops draw criminals","kicker":"Slips tell the story","author_name":"Scott Glover","author_title":"","affiliation":"Sun-Sentinel","summary":"The Sun-Sentinel analyzed 70,000 pawn slips from Fort Lauderdale in 1995 to investigate the link between pawnshops and property crime, finding that 39 of the top 50 frequent pawners had criminal records. The investigation revealed patterns suggesting both criminal activity and economic desperation, spurring legislative action to regulate the pawn industry.","keywords":["pawnshops","crime","investigation"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Paradox"],"full_text":"Fort Lauderdale has the third worst property crime rate in the nation and has more pawnshops than it does McDonald's. Is there a link? Police have been saying \"yes\" for years. We decided to check it out for ourselves.\n\nBeat the clock\n\nIn July, the Sun-Sentinel made a request under Florida's Sunshine Law for every pawn slip generated in the city in 1995. (Our request beat a change in state law in October that specifically exempted the forms from public view.)\n\nThere were nearly 70,000. About 32,000 had been entered into the police department's computer; another 38,000 had not.\n\nWe hired temporary workers to enter the paper records into our own PCs. Then we merged the two databases and loaded them into the computer program Paradox.\n\nThe result was a gold mine of information: the names, addresses and birth dates of everyone who had pawned anything in 1995. The slips, required to be filled out by pawnbrokers then sent to the police, also included detailed descriptions of the items pawned.\n\nCriminal dealings?\n\nThe first thing we did was rank the pawners by their number of trips to the pawnshop. Then we used local and state databases to check the criminal records of the top 50.\n\nThe result: Thirty-nine out of 50 had criminal arrest records; 19 for burglary, theft or other related offense; 14 others faced drugs charges, almost all of them for cocaine. The remaining five faced a hodgepodge of charges.\n\nBut we let the pawners' activities speak for themselves.\n\nIs something amiss when a 21-year-old convicted burglar goes to the pawnshop 66 times in one year, pawning, among many other things, 12 bikes, 14 beepers, five car stereos four portable stereos, two VCRs and eight rings?\n\nWhat about a man with a laundry list of drug and theft charges on his rap sheet who went to the pawnshop 64 times in 1995, pawning seven TVs, eight VCRs, four portable stereos and two home stereos?\n\nWe let readers decide. Judging from the hundreds of calls, letters, faxes and e-mails that we - and more importantly, state legislators received in response to our series, it seemed certain that readers were angry at what they had read. The state senator who sponsored a new law governing pawnshops has already filed a bill that will tighten restrictions on the industry.\n\nBanks for the poor\n\nFinding bad guys was not the only use for our database. We also wrote a story about frequent pawners who were not suspected of anything criminal, but were caught up in a cycle. That story, which included anecdotes about a man who made 153 trips to the pawnshop in a single year, focused on the role of pawnshops as \"banks for the poor.\"\n\nAs best we can tell, this project is unique. Nowhere, including in the IRE morgue and NICAR database library, could we find an investigative story based on pawn slips.\n\nDuplicating the project\n\nIf you're interested in conducting an investigation similar to ours, step one is to determine whether pawn slips are kept in your city and, if so, whether they are public record.\n\nSince the document is little known, chances are reporters haven't asked to inspect them. Go in assuming you have the right to. We didn't know whether our request would be granted and were prepared to get the lawyers involved if it wasn't. One argument we would have made is that pawn slips are provided by the Florida Department of Law Enforcement and by law are forwarded to local police departments, hence they are public.\n\nWe don't know if that argument would have worked, but it's probably worth trying under similar circumstances.\n\nIf your police department does keep pawn slips and agrees to turn them over, make sure to ask if they are available electronically. It'll save you a lot of work. Happy hunting. Scott Glover can be reached at (954) 356-4523.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-january-nashcar-97","slug":"1997-january-nashcar-97","url":"https://dwillis.github.io/uplink/article/1997-january-nashcar-97.html","year":1997,"month":"January","monthNum":1,"headline":"NashCAR '97","kicker":"The National Computer-Assisted Reporting Conference","author_name":"","author_title":"","affiliation":"","summary":"The schedule and panel descriptions for the 1997 National Computer-Assisted Reporting Conference, featuring hands-on training and expert-led sessions on diverse CAR topics.","keywords":["NashCAR","conference","journalism training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet","9-track tape","World Wide Web","Internet","Intranet"],"full_text":"Learn more than 50 ways to use computers in the newsroom at NashCAR '97, March 6-9 in Nashville, Tenn. This year's National Computer-Assisted Reporting Conference is sponsored by IRE, NICAR and The Tennessean. Break-out sessions on special topics will be held in siderooms and speakers will give presentations on problem-solving, software, hardware, and data analysis in a demonstration room. Training will include classes in spreadsheets, database managers, mapping, nine-track tapes, creating World Wide Web pages, and surfing the Internet. There will be up to six hours of hands-on training offered to registrants on a first-come, first-served basis and one-on-one work critiques for the first 50. Panels include themes such as: Finding people and sources with on-line techniques; Using spreadsheets to cover local government; Making data accessible through Intranets; Data nightmares and how to avoid them; and Using CAR for international reporting. Sunday is a morning of workshops for database editors, news librarians, managers, and broadcasters.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-january-get-your-training","slug":"1997-january-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-january-get-your-training.html","year":1997,"month":"January","monthNum":1,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR and the Associated Press offer specialized journalism training through on-the-road sessions, national conferences, and intensive week-long bootcamps covering data analysis, spreadsheet building, and computer-assisted reporting techniques. Training is available at various locations with costs varying by program and participant type.","keywords":["training","journalism","data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"NICAR and, where indicated, The Associated Press provide specialized training in your newsroom. Learn to transfer data from government newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases.\n\nCost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below.\n\nNote: In the following list, \"Open to all\" means any journalist may sign up. \"Closed\" means the session is open only to members of the host organization.\n\nYork, Pa., York Daily Record - 28. Closed.\nPNPA, Harrisburg, Pa. - Jan. 29.\nSan Diego, Calif., San Diego Tribune and SPJ - Feb. 19-22, 1997. Open to all. (619) 293-1261\nWashington, D.C., Newsday, Feb. 17-20. Closed.\nPalm Beach, Calif., Palm Beach Post March 25-27.\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969.\n\nNashCAR, NICAR National Conference, Nashville, Tenn.-March 6-9. Costs are: IRE member, $150 for entire conference. Non-IRE member, $190 for entire conference, including IRE membership. Renewal member, $190 for entire conference, including IRE renewal. Students, $125 for conference, including student membership. Late fee is $15 for registrations not postmarked by Feb. 14.\nIRE National Conference, Phoenix, Ariz., June 12-15.\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo. As with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size. For information, call Wendy Charron at (573) 882-0684.\n\nJan. 24-26. Intermediate Bootcamp concentrating on data clean-up and more. For details, call (573) 882-0684.\nMay 4-9. Advanced Bootcamp concentrating on stats and maps at the University of North Carolina-Chapel Hill with Philip Meyer, author of The New Precision Journalism. The seminar is held in Chapel Hill. For details, call (573) 882-0684.\n*May 18-23. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684\nNEW Aug. 10-15. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-january-from-the-nicar-library","slug":"1997-january-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-january-from-the-nicar-library.html","year":1997,"month":"January","monthNum":1,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering aviation, transportation, finance, health, crime, and safety records for investigative journalism purposes.","keywords":["databases","federal","journalism"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW Federal Aviation Administration's accidents and incidents, including major plane crashes since 1971.\nNEW NASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\nA monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\nFederal procurement data, 1992-1995, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1995.\nNational Bridge Inventory System data, includes inspection grades. 1994-1995\nNEW FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1995 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nThe National Inventory of Dams. 1991-1995.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to nicar@muccmail.missouri.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-january-bits-bytes-and-barks","slug":"1997-january-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-january-bits-bytes-and-barks.html","year":1997,"month":"January","monthNum":1,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A collection of short items including a story about an Indiana court clerk's refusal to release public records and multiple job listings for journalists with CAR skills.","keywords":["FOIA","jobs","court records"],"topics":["Investigative Methods","Crime & Justice","Journalism & Newsroom"],"technologies":[],"full_text":"After our issue on Freedom of Information battles, Joe Carroll of the Hammond (Ind.) Times shared this story: The criminal court clerk in a nearby city refused to let me look at a court file for a woman jailed on battery and resisting arrest charges. The court clerk said she is, by law, the Keeper of Records, not the Distributor of Records. Eventually, it took a brief lawsuit to straighten them out. Jobs: The Miami (Fla.) Herald needs a capital bureau reporter skilled in CAR. The San Jose (Calif.) Mercury News needs technology reporters. The Orange County (Calif.) Register needs a technology reporter and an assistant city editor. The Wilmington (N.C.) Morning Star needs a court reporter.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-january-jobs","slug":"1997-january-jobs","url":"https://dwillis.github.io/uplink/article/1997-january-jobs.html","year":1997,"month":"January","monthNum":1,"headline":"Jobs","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Multiple newspapers are hiring reporters for various positions including capital bureau, technology, court, and science/medicine beats. Applicants should submit resumes and writing samples, with some positions requiring specific experience in computer-assisted reporting and editing.","keywords":["hiring","reporters","newspapers"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Jobs\n\nThe Miami (Fla.) Herald needs a capital bureau reporter.\n\nApplicants should be skilled in computer-assisted reporting.\n\nContact Mark Washburn, state editor, at (305) 376-3765; George Haj, assistant state editor, at (305)-3766; Chris Morris, newsroom recruiting, at (305) 376-3592.; or Dan Keating at (305) 376-3476 or by e-mail at dtkeats@ibm.net\n\nThe San Jose (Calif.) Mercury News needs technology reporters someone to dig for and crunch business data. Daily newspaper experience is preferred.\n\nContact Peter Hillan, executive business editor, at phillan@sjmercury.com or Jonathan Krim, assistant managing editor at jkrim@sjmercury.com.\n\nThe Orange County (Calif.) Register needs a technology reporter and an assistant city editor for medicine and science.\n\nApplicants for the editing position should have at least three years reporting experience on a daily metro and at least two years editing experience.\n\nApplicants for either position should submit a packet of current work, a resume and a one-page statement of philosophy to: John Doussard, 625 N. Grand Ave., Santa Ana, Calif. 92701. Telephone: (714) 953-2225. Fax: 714-565-3681.\n\nThe Wilmington (N.C.) Morning Star needs a court reporter. Applicants should have at least two years' reporting experience and computer-assisted reporting skills.\n\nSend a resume, references and five to 10 clips to: Dave Ennis, city editor, Wilmington Morning Star, P.O. Box 840 (1003 S. 17th St.), Wilmington, N.C. 28402-0840","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-to-have-or-to-have-not","slug":"1997-february-to-have-or-to-have-not","url":"https://dwillis.github.io/uplink/article/1997-february-to-have-or-to-have-not.html","year":1997,"month":"February","monthNum":2,"headline":"To have or to have not","kicker":"Income investigation","author_name":"J.J. Thompson","author_title":"","affiliation":"U.S. News & World Report","summary":"J.J. Thompson explores income inequality using the Gini coefficient to analyze income data, showing the growing gap between different economic classes.","keywords":["income","inequality","Gini"],"topics":["Data & Databases","Demographics & Census","Business & Economy"],"technologies":["Excel"],"full_text":"Income inequality-the gap between the haves and have-nots- is a topic that begs the use of computer-assisted journalism. Being one of those trends that creeps slowly and unevenly into our lives, no one has the same perception of it. If you live in Washington, D.C., the gap seems to be huge and growing every day. If you live in Hot Springs, Ark., it may not seem quite so perilous.\n\nSo an analysis of income data is a must to answer questions such as:\n\n* How big is the gap?\n* Is it getting bigger?\n* How much worse is it in X than in Y?\n\nThe trick, obviously, is to measure the phenomenon in a way that readers can easily grasp its meaning. When, during the vice presidential debate, Jack Kemp mentioned USA Today's article mapping income inequality in the United States, I knew the Gini coefficient of inequality had done the job.\n\nThe Gini formula produces a single number between 0 and 1, with 0 indicating complete equality (everyone makes the same amount of money) and 1 indicating complete inequality.\n\nPhil Meyer, journalism professor and precision journalism guru at the University of North Carolina at Chapel Hill, heard a presentation using the Gini coefficient of inequality at a research conference. Since I was searching for a thesis topic that would incorporate data-analysis with my interest in social issues, he suggested that I look into it.\n\nThe first two hurdles were finding a formula and understanding it. After one laborious false start, I found a formula that would work with aggregated income data.\n\nYou may recognize (if you remember 10th-grade geometry better than I did) that the formula for the Gini coefficient is based on the one for measuring a triangle.\n\nIt is: Gini coefficient=1 - SUM(Xi-Xj) (Yi-Yj) where: X is the cumulative proportion of households Y is the cumulative proportion of income; is a particular income category; is i-1, or the preceding income category.\n\nThe easiest way to conceptualize this is to picture a graph in which the final point on both X and Y axes is 100 percent, or 1.0. Points are graphed based on cumulative numbers; therefore, in a completely \"income-equal\" county, 10 percent of the households would take in 10 percent of that county's income, 20 percent of households would take in 20 percent of income and on up to 100 percent of the households, which would account for 100 percent of income. The points in this unusual county would form a straight line from 0,0 to 1,1- think of it as the \"line of equality.\"\n\nIn reality, however, 10 percent of the overall households may account for only 2 percent of income; 20 percent of households only 5 percent of income and so on, so that the points would form a curve below the line of equality. By measuring the space below the curve (you can divide it up into a series of triangles) and subtracting it from 1, you get a number that indicates how big the inequality gap is. That's the Gini coefficient.\n\nBeing able to use aggregated income instead of individual incomes, like you might do in a smaller sample, was important because the way the Census reports income data. For example, the Census will tell you only that 256 people in County X had incomes between 0 and $2,500, 345 between $2,501 and $5,000 and on up the scale.\n\nI used the midpoint of each interval as the income for all households in that category as well as to calculate total income for each income category. This worked until I got to the final category, which was open ended. Fortunately, the 1990 Census reports total income for the group earning $150,000 or more, so it was fairly simple to divide that amount by the total number of households in that top category for its average income. I also used 1980 income data, and for that year the Census only recorded total income for the whole population. Therefore, a few more computations were needed to get an average income for the top group. The formula is pretty easy to put into a spreadsheet like Excel, and once the income midpoints and the number of households in an income category are entered, the Gini coefficient can be calculated quickly.\n\nFor my thesis, I looked at income inequality for all 100 counties in North Carolina using income data from the 1980 and 1990 U.S. Censuses. The income distribution gap increased by at least 5 percent in 50 counties during the decade; it decreased by 5 percent or more for only nine. For my thesis article, I concentrated on the county where the gap grew the most.\n\nIn addition to providing the focus for the story, the analysis with the Gini coefficient proved a powerful reporting tool. When the county manager denied that income inequality was a phenomenon in Montgomery County, I was armed with the Gini coefficient plus the proportion of money being earned by the 400 households with income of $150,000-plus each year. He then started coughing up explanations.\n\nThe why behind the phenomenon is what proves so interesting. In this particular county, two concurrent trends were adding to the widening gap. One is that the four or so major businesses there had been family-owned for years, with the family owners/managers keeping ever-larger proportions of the profits for themselves. Meanwhile, their workers' pay had remained relatively stagnant. The other factor was who was moving into the county-at one end, rich retirees who were building homes in a ritzy lake-front community and, at the other end, poorer immigrant workers.\n\nThe result was a small but fairly divided community of the \"haves (who took little interest in community life and showed minimal concern for the needs and frustrations of the poor in their county) and the \"have-nots\" (who tended to be a drain on public funds and services).","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-february-counting-the-crashes","slug":"1997-february-counting-the-crashes","url":"https://dwillis.github.io/uplink/article/1997-february-counting-the-crashes.html","year":1997,"month":"February","monthNum":2,"headline":"Counting the crashes","kicker":"Collision course","author_name":"Rick Linsk","author_title":"","affiliation":"Asbury Park Press","summary":"A journalist used the Fatal Accident Reporting System database to investigate whether fatal crashes involving older drivers were increasing disproportionately in New Jersey, comparing crash rates across age groups by analyzing miles driven to account for demographic changes.","keywords":["database","elderly","crashes"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["spreadsheet","FARS","Internet"],"full_text":"Every day, about 115 people die on the roads in the United States. Naturally, a government agency keeps track. The result is the Fatal Accident Reporting System, a database you can sift for many kinds of trend stories. Traffic analysts in every state compile the database for the National Highway Traffic Safety Administration using police accident reports, driver license and vehicle registration files, coroner's reports, death certificates and other documents. FARS includes any motor vehicle accident on a public roadway that results in a death of a person within 30 days.\n\nAt the Asbury Park Press, FARS enabled us to produce \"Collision Course,\" a two-part series on older drivers. Teaming up with a reporter writing about a special prosecutor's task force on fatal crashes, I acquired 1988-95 data from NICAR. Our circulation area includes Ocean County, N.J., a popular suburban retirement area where 1 in 4 people are 65 years of age or older, and where driving is virtually a matter of survival.\n\nThere was:\n\nThe 70-year-old man who gambled all night in an Atlantic City casino, fell asleep at the wheel on the way home and collided head-on with a young couple, killing them.\n\nThe 61-year-old man who took his eyes off the road and mowed down a school crossing guard.\n\nThe 82-year-old woman who left her car parked with the engine running and was run over by it.\n\nDetecting trends\n\nWere these just anecdotal shockers, isolated incidents, or part of a trend? That's what we wanted to know from FARS. On a national, state and local level, it turned out, the percentage of fatal crashes involving a senior driver had increased since 1988. Census figures helped confirm that the increase outpaced the overall aging of the driving population.\n\nAdvocates for the elderly argued that young drivers were just as menacing, or worse. We needed to compare all drivers on an equal footing.\n\nTo do that, I first grouped 1994 fatal crashes by driver age. Then I found travel estimates by driver age in the 1990 National Personal Transportation Survey, the latest year available. (New figures were collected last year but have not been released yet.) And from the Federal Highway Administration's Internet site, I snagged 1994 licensed driver totals by age group for each state.\n\nI plugged all the figures into a spreadsheet. The result: Fatal crash rates for New Jersey drivers, measured by miles driven, increased sharply after age 65.\n\nLast, we gathered projections by Census and transportation officials showing the implications for the future, when many more seniors will be behind the wheel. We also addressed ways that drivers and regulators are dealing with the changes.\n\nNot just numbers\n\nThe records don't give you the names of people in the accidents. But because there are fields that identify people by age, gender, \"person type\" (driver, passenger, pedestrian etc.), and which car they were in, it's easy to match the records to crash stories from your paper's archives or to public records. Reporter Bill Heine and I identified several compelling cases and used interviews as well as police reports, municipal court documents and drivers' histories to round out our vignettes.\n\nYou can get at what caused the crash by looking at the field that contains \"driver-related factors.\" Codes from 00 through 99 reveal, among other things, whether drivers were drowsy, inattentive, reckless, intoxicated, disabled, using a car phone, or fleeing police.\n\nOther fields detail the accident's date and time, the extent of injuries, the time span between the accident and the victim's death, seatbelt and airbag use, whether a school bus was involved, and whether drivers had recent prior crashes or speeding tickets.\n\nFARS doesn't do a great job telling where the accident happened. There is a field with the road name (called \"idyfer,\" in NICAR's version of the data), but no intersection field. There's a milepoint field that's useful if the crash was on a toll road or other major highway.\n\nOther limitations involve issues of timing. FARS analysts build the database from local police reports, and some issues are still unresolved when those reports are filled out, such as whether charges were brought against the driver, and how many people died.\n\nAlso, the database's crash totals can conflict with local and state figures because of the 30-day policy noted above. If someone dies 32 days later, your state may count it as fatal.\n\nNHTSA analysts were responsive and helpful when I had questions. The agency has information summarizing the database and how to order it on the Internet at www.nhtsa.dot.gov/people/ncsa/fars.html\n\nRick Linsk can be reached at (908) 922-6000, ext. 4360, or by email at linsk@app.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-wire-service-serves-car","slug":"1997-february-wire-service-serves-car","url":"https://dwillis.github.io/uplink/article/1997-february-wire-service-serves-car.html","year":1997,"month":"February","monthNum":2,"headline":"Wire service serves CAR","kicker":"CAR gets wired","author_name":"Jody Sowell","author_title":"NICAR staff","affiliation":"NICAR","summary":"The Associated Press is modernizing its approach to data-driven journalism by building a national computer lab and database distribution system to enable reporters across all bureaus to access public records and conduct more rigorous investigations. AP special assignments editor Bob Port argues that computer-assisted reporting is becoming simply standard journalism, and emphasizes that while technology enables deeper reporting, skilled skeptical journalists remain essential.","keywords":["databases","reporting","journalism"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["CD-ROM","Internet"],"full_text":"Bob Port, special assignments editor for the Associated Press, is ready to declare computer-assisted reporting dead-at least the term.\n\n\"The thing about the phrase 'computer-assisted reporting' is it is quickly losing its meaning,\" Port said. \"It's like saying 'telephone-assisted reporting.' If you're doing investigative reporting nowadays, you need computers.\"\n\nPort said the Associated Press soon will make computer-assisted reporting easier for all AP bureaus. The wire service currently distributes databases as best it can - mailing CD-ROMs to bureaus or doing the research in New York and sending printouts.\n\nBy the end of the year, however, that cumbersome system should change. AP is building a computer lab in its New York office that will include a Web server capable of distributing large databases. This will allow an AP reporter in Dallas to call up the latest FBI statistics, or a reporter in Oregon to check a politician's claim of lowering unemployment in the state.\n\n\"We want to make it possible for every single representative of the Associated Press to be able to look at public records electronically,\" Port said. \"We would like them to be able to do it on deadline as much as on a project or a weekend feature.\"\n\nPort, who heads a team of four writers, one researcher and a data editor, said his staff can look up several databases from their New York office, but Port wants all bureaus to be able to share in their research.\n\nHe said he hopes to have the first test databases working by March, have a few bureaus connected by the end of summer and have a national project started by the end of the year.\n\nPhilosophy change\n\nHaving these databases available at all bureaus, Port said, will allow for more \"hard-edge reporting.\" He said reporters will be able to more accurately check claims and will be able to produce their own findings based on reports.\n\n\"That might not sound like a lot, but that's a big change for a wire service,\" Port said.\n\n\"They weren't accustomed to doing research here. It was pretty much grab a quote, get their name and age and put it on the wire.\"\n\nComputers allow for deeper reporting, but Port said there is a big \"but.\"\n\n\"I think we're well down the road, but computers don't write stories,\" he said. \"Computers don't think like a skeptical reporter, and that's the other thing we need to work on - the people.\"\n\nPart of that work involves freeing up reporters to be trained to use the databases that are out there. AP has already begun to incorporate more reporters in computer-assisted projects. Bureau reporters are given chances to work on investigative pieces through a rotating national writer program.\n\nProjects and tips\n\nDespite the challenges that AP faces, its researchers, writers and editors have produced some noteworthy investigations. The most recent was a look at a federal horse adoption program that was leading to the slaughter of thousands of wild horses. Reporter Martha Mendoza found many of the adopters who are selling their horses for slaughter are employees of the Bureau of Land Management, the federal agency in charge of the program.\n\nPort said reporters and researchers matched employees in a federal government horse adoption title database with a database of all federal government employees to find employees who were adopting horses. They then used horse identification numbers to find which horses had been sold for slaughter. They found that 80 percent of the slaughtered horses were under 10 years old and 25 percent were under 5 years old.\n\nRumors of this problem have been around for a long time, Port said. \"This has been going on for many years, and a lot of people have tried to do it. I don't think anyone has done it as much as we did,\" he said. \"I think most reporters are at the mercy of who is willing to talk and under what terms, and in this case, it was nice to be able to use computer records to get at certain facts.\"\n\nPort said the series took 12 weeks to produce: four weeks of backgrounding, six weeks of reporting and two weeks for editing. Other AP projects have included stories about OSHA, speed ticketing practices in New York, and Kentucky police officers ignoring overloaded coal trucks.\n\nPort believes the AP will be able to do even more when its Web server is up and running. Port has some advice for those who are just starting to learn how to incorporate computers into daily and investigative work.\n\n\"I think I would recommend that there's a long learning curve to this, and you have to pace yourself,\" he said, \"and since it's going to take anyone a year or two to try to figure out databases don't get too married to a given way of doing anything, because in two years the technology will change.\"\n\nBut this shouldn't scare off newcomers. It's still a specialty, and the kind of skills and thinking that make good writers tend to be different from the ones that make good computer operators,\" Port said. \"It seems to be creating a new division of labor in the newsroom. We are opening new jobs, but it's for people who can do HTML pages. Journalists, committed ones, for their own protection, need to get a little savvy about this stuff.\"\n\nEven AP staffers, however, have had mixed reactions to Port's push for expanding the wire service's computer capabilities.\n\n\"It's hard to generalize,\" he said. \"Some are enthusiastic and some think it's a silly idea. I guess the results are what we'll have to wait for.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-internet-travel-assistance","slug":"1997-february-internet-travel-assistance","url":"https://dwillis.github.io/uplink/article/1997-february-internet-travel-assistance.html","year":1997,"month":"February","monthNum":2,"headline":"Internet travel assistance","kicker":"On the Internet","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute","summary":"A journalism resource guide listing websites and online tools for researching travel destinations, including maps, city information, flight bookings, and hotel reservations to help reporters prepare for assignments or trips.","keywords":["research","travel","destinations"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Whether you need some facts and figures to fill in your reporting about a place (a plane crashes in Mountain Home, Ark., and you've never heard of the town), or you are going somewhere and you need to scout it out before you get there (you are going to NashCAR and want to know more about the country music capital), there are some great gazetteers, groovy guides and helpful handbooks available about almost any location on earth.\n\nYou can book a flight, book a room, check the weather, chart an itinerary and print out a map.\n\nHere's where to go before you go:\n\nMaps\n\nThe U.S. Gazetteer http://www.census.gov/main/www/access.html\n\nThe Census department has put together a one-stop shopping site for maps and Census materials. You can select either the Map Stats for a variety of statistical profiles of states, counties, cities or go to the U.S. Gazetteer and zoom in on an area map with labels customized to your needs.\n\nBigBook\nwww.bigbook.com\n\nGot your hotel reservation, but need driving instructions or a map of where to go? Use Big Book to get details about your destination. The maps are zoomable, the directions easy to follow.\n\nYahoo Maps\nhttp://maps.yahoo.com/yahoo/\n\nAnother great service from the good folks at Yahoo. Plug in an address and city and a map will be drawn for you. Zoom in or out, rotate further north or south. You can get point-to-point directions, with a map of your starting and ending points.\n\nCity Info\n\nYahoo www.yahoo.com\n\nYep, Yahoo, again. When you want to check out what is going on in a city, get some background on the government there, find the local weather or television station or newspaper site - just go to Yahoo and click on United States and then select the state and city you want. Their compilation of information is well organized and you can run through sites with all that kind of background information and things specific to that area. A virtual guidebook.\n\nAnd if you want to find other travel guides, go to Yahoo's Travel section (under Recreation and Sports) and on Destination Guides. You'll get a listing of great sites with travel information for various cities in the United States and around the world.\n\nPlanes and Trains\n\nAlmost every airline you can imagine has a website (usually just type in the name of the airline for the address), but if you want to see what flights are available, check out:\n\nYahoo's Flight Information Site http://yahoo.flifo.com/\n\nI love this site. Put in where you want to fly from, where you want to fly to, when and what time of day and you'll get back a variety of options. Specify an airline you want to use, or see all the possibilities. They'll give you the prices, and you can even buy the ticket.\n\nAmtrak\nwww.amtrak.com\n\nDon't let fear of flying stop you. Take the train. Trakrouter has four different ways to calculate the best route.\n\nLodgings\n\nTravelweb\nwww.travelweb.com\n\nTell them the city, even the hotel chain, if you have a preference, and you'll get a list of the available hotels. Click on one of the hotels, and a page with everything you could possibly want to know will be displayed. Click on \"area info,\" and it will tell you how far that hotel is from some of the sights in that city. Nora Paul can be reached at (813) 821-9494, or by e-mail at npaul@poynter.org\n\nIf you have suggestions for this column, e-mail Nora Paul at npaul@poynter.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-mining-data-from-census","slug":"1997-february-mining-data-from-census","url":"https://dwillis.github.io/uplink/article/1997-february-mining-data-from-census.html","year":1997,"month":"February","monthNum":2,"headline":"Mining data from Census","kicker":"Handout of the month","author_name":"Jeff South","author_title":"","affiliation":"Austin American-Statesman","summary":"The Census Bureau is a valuable ongoing source of data for journalists beyond its decennial headcount, offering accessible information on demographics, housing, business, and social trends through multiple digital platforms. Reporters can leverage this data for breaking news, trend analysis, and investigative stories by combining Census figures with other data sources to compute meaningful rates and comparisons.","keywords":["Census","data","journalists"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":["CD-ROM","9-track tape","floppy disk","Internet","CompuServe","Dialog","modem/BBS"],"full_text":"Many people view the Census Bureau as the government agency that comes around every 10 years and tries to do a headcount of Americans, producing statistics that, while interesting, quickly get stale. But the bureau generates more than flash-in-the-pan demographic data: It is a steady source of information on various subjects, including politics, business, and social issues.\n\nComputer-savvy reporters and editors are in a unique position to take advantage of this flow of data, because the Census Bureau is a leader in distributing information by computer. It has a comprehensive Internet site, several electronic mailing lists, a dial-up computer bulletin board and a data bank accessible through CompuServe and Dialog.\n\nThe bureau also provides (usually at price) data on CD-ROM, nine-track magnetic tape and diskettes.\n\nFrom quick-hit data to long-term projects, Census data can be used in various ways:\n\nEvery year, the bureau updates population estimates for states, counties, and metropolitan areas. (The agency revises estimates for cities every other year.) These updates are fodder for breaking news and trend pieces. The updates include breakdowns that lend themselves spin-off stories: on race, ethnicity and age, for example.\n\nBeyond demographics, the Census Bureau tracks rental housing vacancies, government spending, business development and other trends. For example, the bureau does an annual housing survey that shows, for each metro area, the percentage of families who own homes.\n\nSometimes the numbers are old but they might provide the best comparison available. Other data sets are more current, like the bureau's monthly compilation of building permits.\n\nThe 1990 Census, even seven years later, is still a valuable source of numbers-and the only official source for many statistics. While you can't build a story on the numbers alone, you can use the 1990 Census to shape your report. It remains the best way to compare neighborhoods, cities or states on criteria ranging from the percentage of workers who carpool to the percentage of homes without plumbing.\n\nThe 1990 Census and updated estimates are especially valuable when combined with other data, such as the number of crimes, gun permits, pawn shops, supermarkets, public swimming pools, billboards - you name it.\n\nUse Census data to compute rates, like zoning violations per 1,000 households in each ZIP code, or the number of liquor stores for every 10,000 people in different parts of your city. Then use the Census to see if the rates are different between low-income and high-income areas or between predominantly minority and predominantly white areas.\n\nThe Census Bureau is still slicing and dicing the 1990 Census, generating data on special populations. Last fall, for example, the agency released, for each state and metro area, census details on mobile home residents and how they compare with other residents. The bureau previously dissected data for Hispanics, people with disabilities and other groups.\n\nKeeping up with Census data\n\nThe best way is to subscribe to the bureau's listserv for reporters. Do this by sending the message \"subscribe press-corps <your e-mail address>\" to majordomo@census.gov Then you'll automatically receive the press releases, usually embargoed so that you have time to develop a story, and tipsheets about data and reports on the horizon.\n\nA recent tipsheet was typical in listing several upcoming data sets with strong story possibilities: the Consolidated Federal Funds Report, a database available on CD-ROM that shows how much each federal program spends in each city, county, and state; new demographics about African-Americans; County Business Pattern reports, which detail the number and type of businesses and their employees; a count of Hispanic-owned businesses; and population numbers for every county from the first time it was included in the Census (as early as 1790) through 1990.\n\nThe bureau has several other listservs, including a monthly announcement of new products and periodic distribution of economic research. The agency's Web http://www.census.gov, tells you how to subscribe and how to reach the bureau's experts on specific subjects like population projections, education, and poverty.\n\nFinding and acquiring data\n\nVisit the Web site often-off hours if possible, because it is often busy.\n\nThe bureau's home page invites you to browse for reports and data under two general categories: \"Population and Housing,\" which includes such topics as race, migration and income; and \"Economy,\" which covers housing starts, business issues and government finances.\n\nLarge data sets, such as CD-ROMs and nine-track tapes are listed under \"Market Place.\" This list runs from the 1992 Census of Agriculture and the School District Data Book to Public Use Microdata Samples, or PUMS, which let you create your own crosstabs from the 1990 Census.\n\nA few of these collections such as the Census Transportation Planning Package are free. But most large data sets carry a price, typically $150. However, the Census Bureau is having a sale on older CD-ROMs: For $50 you can get 1990 Census details for your area down to the smallest geographic level, or a CD that breaks down occupations by sex, race and ethnicity.\n\nEven if you're not buying, it's important to know the specific name of the data you want. Here's why: You might be able to borrow the CD from a local library or university. Also, knowing the name of the data will help you do a search using the engine on the Census Bureau's home page. You may be able to find and download selected tables from a CD you covet. For instance, the agency has posted many tables from the Statistical Abstract and the County and City Data Book.\n\nOn the 'Net, you'll find data sets in various formats: occasionally dBASE, sometimes as spreadsheets, but usually as fixed-length records. The data and the record layout often are in separate files so be sure to get both.\n\nThe joy (or for some people the frustration) of using data from the Census Bureau's 'Net sight is that you must employ several computer-assisted reporting tools: your Internet software, of course; a spreadsheet like Excel to parse the data into columns; a database manager like FoxPro or Access to analyze the data; and a desktop mapping system like MapInfo or AtlasGIS to display your data.\n\nIn many cases the numbers you want won't require downloading a large data set or getting a CD-ROM. Instead, you simply may want to look up census data for one or more states, counties, cities or ZIP codes. You can do this under \"Data Access Tools\" from the Census Bureau's home page.\n\nOne tool lets you zoom in on a map, pick a county and view various factsheets. With another tool, you can type or paste into a box list of ZIP codes, select the census-data categories that you want to see and receive a tab-delimited file that flows easily into a spreadsheet.\n\nOther resources\n\nThe Census Bureau is by no means the only source of demographic data on the Internet. The bureau's page has links to other sites like, the University of Missouri (http://www.oseda.missouri.edu), census data at the Lawrence Berkeley National Laboratory (http://cedr.lbl.gov/mdocs/LBL_census.html) and the Consortium for International Earth Science Information Network (http://www.ciesin.org).\n\nAlso check your state government's Web page, as well as sites operated by publications like American Demographics (http://www.demographics.com), which often posts the data used in its articles.\n\nDevelop human resources too - not just at the Census Bureau but at its regional office nearest you and your state data center. The centers, usually affiliated with a state government agency or university, help to distribute and analyze Census information. You can find a list of the centers at http://www.census.gov/ftp/pub/sdc/www/\n\nAsk your state data center, contacts at a local university or the federal information section of your local library if they have census data you can borrow. Many libraries let you check out census data on CD-ROM.\n\nIf you need inspiration for Census stories, read Who We Are: A portrait of America based on the latest U.S. Census. This 1993 book, by Sam Roberts of the New York Times, shows how to put a face on Census data.\n\nThe Census Bureau produces several reports including the Statistical Abstract of the United States, the County and City Data Book and USA Counties. These documents include data ranging from the cost of living to health care. They're available as books, but the CD-ROM versions include more data. Many of the tables can be downloaded for free from the Internet.\n\nJeff South can be reached by e-mail at jsouth@statesman.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-nashcar-ready-to-roll","slug":"1997-february-nashcar-ready-to-roll","url":"https://dwillis.github.io/uplink/article/1997-february-nashcar-ready-to-roll.html","year":1997,"month":"February","monthNum":2,"headline":"NashCAR: Ready to roll","kicker":"NICAR and IRE Notes","author_name":"Brant Houston","author_title":"Managing Director","affiliation":"NICAR","summary":"NashCAR, a national computer-assisted reporting conference in Nashville (March 6-9), will feature new panels, speakers, and comprehensive training reflecting the dramatic growth and evolution of CAR in journalism, including topics on Internet data, visualization, math/statistics, and ethical practices.","keywords":["reporting","journalism","data"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","Lotus Notes"],"full_text":"New subjects, new panels, new faces. NashCAR, our national computer-assisted reporting conference in Nashville, Tenn., March 6-9, has been programmed by NICAR and the local committee from the Tennessean to have all of these.\n\nIn the past year and a half, CAR has grown dramatically and in different directions. The Internet, through the World Wide Web, has not only become a worthwhile and constant source of data and information, but also a way to distribute databases within a news organization.\n\nReporters have become more savvy about software programs, math and how to slice national data for local stories. Steps have been taken toward better visualization of information - so that it can be better understood.\n\nDuring this time, different levels of teaching have been required. Introductory training continues for those just getting to CAR, but NICAR has also developed new curricula for the intermediate and advanced journalists.\n\nAll these changes need to be reflected at NashCAR, and they will be. Among the topics will be:\n\nUsing the Internet for quick hits in broadcast.\nUsing Intranets and Lotus Notes to distribute data to the whole newsroom.\nFinding free data on the Internet.\nMaking federal data useful for local stories.\nUsing math and statistics without making critical errors.\nUsing CAR for international stories.\nManaging CAR projects.\n\nWe will have panelists from throughout the country and from news organizations who have never spoken before. There will be more speakers from broadcast on our panels and panels for special concerns of broadcasters, who have less time and space to work with.\n\nWe also will have few key experts from outside journalism who can tell you about hidden sources of data.\n\nAt the same time, we will make sure to cover the basics and include the knowledgeable veterans of CAR, who have been learning and teaching for the past decade.\n\nYou will be able to get the best ways to start in CAR and hundreds of good story ideas to which CAR can be applied.\n\nAs you enter this new arena, there will be panels on the legalities and ethics in the world of CAR to help guide you. And there will be panels on how to present the story so that it doesn't get lost in numbers or overloads of data.\n\nMeanwhile, our hands-on training is the most ambitious ever. We are offering 20 different classes in four days ranging from beginning searches of the Internet to using statistical software to analyze datasets.\n\nWe also will provide tutorials and specialized workshops in our demonstration rooms, have dozens of reprints and video clips of stories to share, and will offer seminar for impromptu gatherings on particular subjects.\n\nLast, NashCAR will have the first ever job fair to connect those looking for the right place and those looking for the right journalist. So don't miss this great event and the chance to get on the fast track to better journalism.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-ire-news","slug":"1997-february-ire-news","url":"https://dwillis.github.io/uplink/article/1997-february-ire-news.html","year":1997,"month":"February","monthNum":2,"headline":"IRE News","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE launched a satellite feed on February 4-5 showcasing television investigative stories that used computer-assisted reporting techniques, featuring stories on topics like pension fraud and corrupt elections. The one-hour program was made available free to broadcasters and was organized to demonstrate how TV journalists nationwide are successfully applying CAR skills.","keywords":["satellite","journalism","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"IRE launched on Feb. 4 and 5 a satellite feed featuring TV investigative stories that used computer-assisted reporting. The one-hour feed, \"New Adventures in Computer-Assisted Reporting,\" told stories of pension thieves, license-less cabbies, fraudulent elections and filthy restaurants. The feed was available for free to anyone who could configure a satellite to catch it.\n\nLong-time IRE members Mark Lagerkvist of News 12 Long Island and Stuart Watson of WRAL-TV in Raleigh, N.C., were responsible for the event. They pulled together the stories, the permissions to rebroadcast the stories and the technical requirements for the feed. The purpose was to show how television journalists around the country had successfully used computer-assisted reporting skills in producing stories.\n\nFor more information on this, contact the IRE Resource Center at (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-car-goes-on-field-trip","slug":"1997-february-car-goes-on-field-trip","url":"https://dwillis.github.io/uplink/article/1997-february-car-goes-on-field-trip.html","year":1997,"month":"February","monthNum":2,"headline":"CAR goes on field trip","kicker":"CARTrips: First ventures","author_name":"Christine Bush","author_title":"Reporter","affiliation":"KCTV - Kansas City","summary":"A local news team used computer-assisted reporting to investigate funding disparities in school field trips across six districts, finding that similar-sized districts spent varying amounts and relied differently on tax dollars versus private funding.","keywords":["investigation","education","spending"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["Excel"],"full_text":"Our news director succumbed to years of price quotes and good old-fashioned begging last year when he approved the purchase of a computer for our investigative unit.\n\nNot long after the mouse touched the pad, he wanted to know what story we could turn with the help of the computer.\n\nSo I glanced through IRE's 100 Computer-assisted stories. From that we found a story outlined on school field trips.\n\nJournalists at The Tennessean found that poor schools in poor neighborhoods were not going on as many field trips as students from wealthier schools, even though experts say poor children need field trips more because it is often their only opportunity to go to places like zoos and museums.\n\nThe synopsis showed the story only took a week and a half. It sounded perfect for our first computer-assisted story.\n\nGetting started\n\nOur market covers school districts in two states. So we chose some of the largest and smallest districts on both sides of the state line. After the first two telephone calls, I realized this was going to take much longer than a week and a half.\n\nOnly two school districts had the information loaded into a computer, and other districts didn't know if they could find specific costs for field trips.\n\nI told them if that was the case, we had another story.\n\nWe eventually narrowed the scope of our story to six school districts. We limited our examination of costs to elementary students. We chose that age group because they get the majority of the field trip funding.\n\nGathering the data took almost two months. The schools that didn't have the data compiled electronically really dragged their feet. When we finally got data, it often lacked the information we requested. Some districts didn't keep detailed information, such as how much each individual trip cost.\n\nWe used Microsoft Excel exclusively for this project. No scientific methodology behind that - it was the only one I knew how to use. I requested a list of all tax dollars spent for field trips in the current school year and the past school year.\n\nWe defined a field trip as any off-campus trip, excluding extracurricular activities. The database included about 300 records.\n\nWe listed each school individually, the trip taken and cost of trip. Then we added total cost per school, per school district and per student. We also categorized the types of trips taken and counted those.\n\nResults\n\nFrom the comparisons, we concluded same size school districts took about the same number of field trips, but spent varying amounts of money. Most schools took the same types of trips, except for the Kansas City School District.\n\nA few schools in that district spent a large chunk of the budget for out of town, overnight trips. The district also spent six times more than a comparably sized district on the Kansas side. We were also able to conclude some schools relied more on parent-teacher organizations or private funding while others relied on tax dollars.\n\nThe district's assistant superintendant said out-of-class experiences are important and will continue but said out-of-town trips may be reduced and spending may on those trips may be cut.\n\nNow that I know how to use a database manager, I see how the project could be easier. Counting the trips and categorizing them would be quicker and easier to evaluate. Linking tables would add more insight as well.\n\nReporters at The Tennessean had this tip for others who want to do the project: \"The information is there,\" they wrote in 100 Computer-assisted stories. \"You just have to go out and look for it. As long as you ask for it at the school administration building, you can get it.\"\n\nGood advice. All it takes is patience and persistence.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-tricks-with-the-delete-marker","slug":"1997-february-tricks-with-the-delete-marker","url":"https://dwillis.github.io/uplink/article/1997-february-tricks-with-the-delete-marker.html","year":1997,"month":"February","monthNum":2,"headline":"Tricks with the delete marker","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"NICAR instructor","affiliation":"NICAR/Missouri School of Journalism","summary":"FoxPro's DELETE TAG feature can be used as a marking system to identify and filter records in a table without adding columns, allowing journalists to sort and categorize data like distinguishing between individual and business names.","keywords":["FoxPro","database","filtering"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["FoxPro","SQL"],"full_text":"Tricks with the delete marker\n\nBy Richard Mullins\nNICAR/Missouri School of Journalism\n\nWhen we process and scan paper records and lists, we often use a sort of check-mark system to digest the information. It turns out that FoxPro has a feature that can be used as a check mark for a row in a table, without adding any columns to the table.\n\nWhat FoxPro calls the DELETE TAG is used to mark records to be permanently removed from the table by the PACK command. Unless that command is issued, the tag has no effect, so you can use it as a marker for all sorts of things.\n\nThe marker can be set row by row, by clicking the box at the very left of the table display, or it can be set by a command typed in the command window, with limits on the scope of the command specified by logical conditions. You can browse or select records based on this marker. You can remove the markers manually and individually, or by logical condition, you can remove them all.\n\nHere is an example. You have a table with a column called \"name.\" The problem is that it contains both individual and business names, and you want to untangle this by marking the business names.\n\nYou make a guess that business names will have certain words and characters in them that wouldn't typically be found in an individual name. You can test this theory by setting the delete tag on all the rows that have these. Then you browse only the marked records in the table and remove the marks from records that really are individuals. As you remove the marks, they disappear from the browse window, (not the table) leaving only the rows still to be examined.\n\nSeveral commands in FoxPro, such as replace, delete, and browse, can have a logical condition specified to limit the extent of the command. This logical condition works just like the WHERE clause in SQL, except in FoxPro the word FOR introduces the condition.\n\nIf you haven't used the $ before, it creates a logical, or true/false test on an expression, just like the equal sign does in State = \"WY\"\n\nAn expression using the $ roughly translates to: \"Is String1 found anywhere within String2?\"\n\n* set delete marker for names\n* that contain ampersands,\n* and business suffixes:\ndelete for \"Inc.\" $ Name\ndelete for \"Co.\" $ Name\ndelete for \"&\" $ Name\n* now browse just the delete-flagged records\nbrowse for deleted\n\nThe deleted() function gives the answer true or false for each record in the table, depending on whether the delete tag is set.\n\nWhen you browse through the records, you might find a row where the name column contains, \"Smith, Mr. & Mrs. William.\" You would remove the delete tag for this record, and it would disappear from the browse window.\n\nIf you have to start over, remove the delete tag on some or all of the records:\n\nremove delete tag for all records: recall all\nremove tag for some records:\nrecall for \"&\" $ Name\n\nRichard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-nashcar-97","slug":"1997-february-nashcar-97","url":"https://dwillis.github.io/uplink/article/1997-february-nashcar-97.html","year":1997,"month":"February","monthNum":2,"headline":"NashCAR '97","kicker":"The National Computer-Assisted Reporting Conference","author_name":"","author_title":"","affiliation":"","summary":"NashCAR '97 is a national conference in Nashville offering over 50 training sessions on computer-assisted reporting techniques, including hands-on classes in spreadsheets, databases, mapping, and web design for journalists of all skill levels.","keywords":["conference","training","journalism"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["spreadsheet","9-track tape","World Wide Web","Internet"],"full_text":"Learn more than 50 ways to use computers in the newsroom at NashCAR '97, March 6-9 in Nashville, Tenn. This year's National Computer-Assisted Reporting Conference is sponsored by IRE, NICAR and The Tennessean.\n\nBreak-out sessions on special topics will be held in siderooms and speakers will give presentations on problem-solving, software, hardware, and data analysis in a demonstration room. Training will include classes in spreadsheets, database managers, mapping, nine-track tapes, creating World Wide Web pages, and surfing the Internet. We will announce panelists in the coming weeks.\n\nIn addition, there will be up to six hours of hands-on training offered to registrants on a first-come, first-served basis and one-on-one work critiques for the first 50. Hands-on training will include beginning, intermediate, and advanced classes. Here is the tentative program of panels.\n\nPanels marked (B) are for Beginners; (I) for Intermediate; (A) for Advanced; and (E) for everyone.\n\nNashCAR '97 starting line\n\nWEDNESDAY:\n\n6 p.m. to 9 p.m. Early Registration\n\nTHURSDAY: 7 a.m. to 9 a.m. Registration\n\n9 a.m. to 10 a.m. Welcome and introduction to conference.\n\n10 a.m. to 10:50 a.m. Panel 1: Finding people and sources with online techniques (B) NORA PAUL, Poynter Institute BROOKE CAIN, News and Observer Panel 2: 50 of the newest CAR stories, for beat and deadline reporters (E) SHAWN MCINTOSH, Dallas Morning News Panel 3: Beginning from the beginning - How to get started. Hardware, software, expectations (B) BILL LOVING, Minneapolis Star-Tribune ANNE SAUL, Gannett Co.\n\n11 a.m. to 11:50 a.m.\n\nPanel 1: Getting CAR started in broadcast (B) PAUL ADRIAN, WBNX PHIL WILLIAMS, WKRN CHRIS FORD, WTVT Panel 2: Using spreadsheets to cover local government, budgets and salaries (B,I) WES WILLIAMS, KPNX STEPHANIE REITZ, The Hartford Courant Panel 3: Covering the environment with CAR (I) JAMES BRUGGERS, Contra Costa Times RUSS CLEMINGS, Fresno Bee definite\n\nNoon to 1:30 p.m. Lunch\n\n1:30 to 2:20 p.m.\n\nPanel 1: Making data accessible to the whole newsroom (Intranet, etc.) (A) GEORGE LANDAU, NewsEngin TOM BOYER, The Seattle Times Panel 2: Using Census data to profile your local community (I) PAUL OVERBERG, USA Today JOHN PERRY, The Daily Oklahoman Panel 3: Covering education with CAR (I) LISA GREEN, The Tennessean JOHN BARE, Educational consultant CHRIS SCHMITT, San Jose Mercury News\n\n2:30 p.m. to 3:20 p.m.\n\nPanel 1: Getting valuable, but free data from the Internet (B, I) NEIL REISNER, The Miami Herald RANDY REDDICK, Facsnet MIKE WEBER, Freelance Panel 2: Strategies for producing CAR stories at smaller news organizations (I) JANET ROBERTS, Wilmington Morning Star DAVID SHEINGOLD, Gannett Westchester GEOFF DOUGHERTY, Bergen Record Panel 3: Covering politics with CAR (it's not just the election) (I) JOE STEPHENS, Kansas City Star ANNA BYRD DAVIS, Commercial Appeal\n\n3:30 p.m. to 4:20 p.m.\n\nPanel 1: Aviation Safety (B,I) BYRON AKAHIDO, Seattle Times BETH MARCHAK, Cleveland Plain Dealer Panel 2: Data nightmares and how to avoid them (I,A) STEVE DOIG, Arizona State University RICHARD MULLINS, University of Missouri - NICAR Panel 3: Covering housing with CAR (I) PENNY LOEB, U.S. News & World Report MIKE HIMOWITZ, Baltimore Sun CAROL NAPOLITANO, Omaha World Herald\n\nFRIDAY:\n\n9 a.m. to 9:50 a.m. Panel 1: Using CAR to cover health issues (I) HAL STRAUS, Atlanta Journal Constitution CHRIS FORD, WTVT Panel 2: Finding and negotiating for data (B,I) JENNIFER LAFLEUR, San Mercury News DAVE ARMSTRONG, Boston Globe Panel 3: Examples of good stories based on use of math and statistical tools (I,A) NEIL BOROWSKI, The Philadelphia Inquirer\n\n10 a.m. to 10:50 a.m.\n\nPanel 1: Using CAR to cover transportation (trains, planes and cars) (I) HEATHER NEWMAN, Detroit Free Press PENNY LOEB, U.S. News & World Report Panel 2: Data nightmares and how to clean them up when you can't avoid them (A) GEORGE LANDAU, NewsEngin RICHARD MULLINS, University of Missouri - NICAR Panel 3: Basic math for journalists (B) NEIL BOROWSKI, The Philadelphia Inquirer\n\n11 a.m. to 11:50 a.m. Panel 1: Using CAR to cover state government (B,I) TOM LOFTUS, Louisville Courier Journal GRIFF PALMER, Daily Oklahoman KATHY HOFFMAN, Detroit News Panel 2: Managing your data (keep it up to date and accurate) (I,A) ANDY LEHREN, NICAR CHERYL PHILLIPS, Detroit News Panel 3: Using CAR to interpret government statistical reports (A) SARAH COHEN, NICAR\n\nNoon to 1:30 p.m. Lunch\n\n1:30 to 2:20 p.m.\n\nPanel 1: Using CAR to cover federal government (and applying it locally) (B,I) MARK BRAYKOVICH, Cincinnatti Enquirer MIKE BERENS, Columbus Dispatch Panel 2: Search strategies for the Web: quick hits for broadcast journalists (B) MIKE WENDLAND, WDIV-TV NORA PAUL, Poynter Institute MATT REAVY, Louisiana State University Panel 3: Using CAR to interpret business and annual reports (I) WENDELL COCHRAN, American University SARAH COHEN, NICAR\n\n2:30 p.m. to 3:20 p.m. Panel 1: Using CAR to cover the workplace (I) NATALYA SHULYAKOVSKAYA, St. Louis Post-Dispatch NANCI AMONS, Nashville TV DAVID HERZOG, Providence Journal Panel 2: Making data easy to understand through graphics and visuals (I,A) ROB COVEY, U.S. News & World Report Panel 3: Using CAR to do meaningful and understandable regression analysis (A) PHIL MEYER, University of North Carolina BARB HANSEN, USA Today\n\n3:30 p.m. to 4:20 p.m. Panel 1: Covering city government (I) CAROL NAPOLITANO, Omaha World Herald ROSE CIOTTA, Buffalo News MARGOT WILLIAMS, The Washington Post Panel 2: Great story ideas for broadcasters (B,I) MARK LAGERKVIST, Channel 12 MIKE WENDLAND, WDIV-TV Panel 3: Using CAR for significance tests. Does the info really mean something? (A) PHIL MEYER, University of North Carolina BARB HANSEN, USA Today\n\nSATURDAY:\n\n9 a.m. to 9:50 a.m. Panel 1: Managing CAR-managing the reporter, managing the editor (I) BILL DEDMAN, Associated Press SONNY ALBORADO, Commercial Appeal Panel 2: The latest doings on the Web - Security, new technologies (E) WALLACE WINFREY, NICAR BOB SULLIVAN, Microsoft News DREW SULLIVAN, AP Panel 3: Using CAR to cover business (I) JO CRAVEN, NICAR NAOMI AOKI, Virginian Pilot BOB WARNER, Philadelphia Daily News BILL RUBERRY, Richmond Times Dispatch\n\n10 a.m. to 10:50 a.m.\n\nPanel 1: Managing CAR projects (not for managers - for reporters) (E) LAURA FRANK, The Tennessean TOM CURRAN, Newark Star Ledger Panel 2: Putting investigative stories on the Web (I,A) MARK HULL, San Jose Mercury News BRAD GOLDSTEIN, St. Petersburg Times Panel 3: Using CAR to cover sports (I) TOM WITOSKY, Des Moines Register FORD FESSENDEN, Newsday\n\n11 a.m. to 11:50 a.m.\n\nPanel 1: Covering social issues with CAR (I) APRIL LYNCH, S.F. Chronicle JIM SPECHT, Gannett News Panel 2: Dealing with FOIA issues, including privatization of public functions (B,I) JENNIFER LAFLEUR, San Jose Mercury News Panel 3: Using CAR to cover federal government (and applying it locally) (B,I) JEFF SOUTH, Austin American-Statesmen MIKE MCGRAW, Kansas City Star DAVID BURNHAM, TRAC\n\nNoon to 1:30 p.m. Lunch\n\n1:30 to 2:20 p.m.\n\nPanel 1: Teaching CAR - new techniques for trainers and academics (I,A) DIANE WEEKS, Washington Post DEBBIE WOLFE, St. Petersburg Times STEPHEN MILLER, New York Times Panel 2: Using CAR for international reporting (I) ROBIN ROWLAND, Ryerson School of Journalism LISE OLSEN, Periodistas des Investagacion JENNIFER LAFLEUR, San Jose Mercury News Panel 3: Covering hospitals (I) RICK LINSK, Asbury Park Press ROBERT BINENCASA, Burlington Free Press TOM GAUMER, Cleveland Plain Dealer\n\n2:30 to 3:20 p.m. Panel 1: Writing and thinking clearly about databases (E) BOB SHERBORNE, Tennessean Panel 2: The latest legalities involving online data (E) DAVID SMALLMAN, Simpson, Thacher BRANT HOUSTON, NICAR Panel 3: Using CAR to cover utilities, technology and cyberspace (I) DAN GILMOOR, San Mercury News MIKE WENDLAND, WDIV-TV\n\n3:30 to 4:20 p.m.\n\nPanel 1: Putting some dazzle in CAR stories - combining writing and graphics (E) CHARLES BLOW, New York Times DAN KEATING, Miami Herald Panel 2: FEC and state campaign finance secrets (E) KENT COOPER, Center for Responsive Politics ANDY LEHREN, NICAR Panel 3: Ideas for matching databases to create new stories (E) ALAN LEVIN, The Hartford Courant\n\n4:30 to 6 p.m.\n\nPanel 1: Creating and analyzing campaign databases DAVID POOLE, Virginia papers SAMANTHA SANCHEZ, Western Coalition Panel 2: Covering social issues with CAR NANCY STANCILL, Charlotte Observer JUDY NICHOLS, Arizona Republic\n\n6 p.m. Supersession: Electronic information: Guidelines for making and keeping it public (Co-sponsored by the First Amendment Center) Speakers to be named\n\nSUNDAY:\n\nThis is a morning of workshops, both in panel rooms and training rooms in the hotel\n\nMain workshops\n\n10 a.m. to Noon\n\nRoom 1: Ethics and computer-assisted reporting WENDELL COCHRAN, American University Room 2: Database editors and news librarians MARGOT WILLIAMS, The Washington Post TOM BOYER, The Seattle Times Room 3: Editors, managers and trainers STEPHEN MILLER, The New York Times Room 4: Special issues for broadcasters PAUL ADRIAN, WBNX Room 5: Tricks of the CAR trade (How to use what you've just learned) NEIL REISNER, The Miami Herald JENNIFER LAFLEUR, San Jose Mercury News SARAH COHEN, NICAR","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-get-your-training","slug":"1997-february-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-february-get-your-training.html","year":1997,"month":"February","monthNum":2,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers specialized journalism training programs including on-the-road workshops, bootcamps, and conference seminars focused on computer-assisted reporting skills such as data analysis, spreadsheet building, and database management.","keywords":["training","journalism","bootcamp"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"NICAR and, where indicated, The Associated Press provide specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases.\n\nCost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below.\n\nNote: In the following list, \"Open to all\" means any journalist may sign up. \"Closed\" means the session is open only to members of the host organization. Palm Beach, Calif., Palm Beach Post March 25-27. Closed. Milwaukee, Wisc., Milwaukee Journal- April 14-16. Closed.\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969. NashCAR, NICAR National Conference, Nashville, Tenn.-Mar 6-9. Costs are: IRE member, $150 entire conference. Non-IRE member, $190 for entire conference, including IRE membership. Renewal member, $190 for entire conference, including IRE renewal.\n\nStudents, $125 for conference, including student membership. Late fee is $15 for registrations not postmarked by Feb. 14. IRE National Conference, Phoenix, Ariz.,- June 12-15.\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo. As with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size. For information, call Wendy Charron at (573) 882-0684.\n\nMay 4-9. Advanced Bootcamp concentrating on stats and maps at the University of North Carolina-Chapel Hill with Philip Meyer, author of The New Precision Journalism. The seminar is held in Chapel Hill. For details, call (573) 882-0684.\n\n*May 18-23. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684 10-15. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-from-the-nicar-library","slug":"1997-february-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-february-from-the-nicar-library.html","year":1997,"month":"February","monthNum":2,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers a comprehensive collection of federal government databases covering transportation safety, financial records, crime statistics, and regulatory compliance data from various agencies dating back to the 1930s.","keywords":["databases","government","journalism"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW Federal Aviation Administration's accidents and incidents, including major plane crashes since 1971.\nNEW NASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\nA monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\nFederal procurement data, 1992-1995, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1995.\nNational Bridge Inventory System data, includes inspection grades. 1994-1995\nNEW FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1995 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nThe National Inventory of Dams. 1991-1995.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-february-bits-bytes-and-barks","slug":"1997-february-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-february-bits-bytes-and-barks.html","year":1997,"month":"February","monthNum":2,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Bill Dedman is leaving the Associated Press after three years as director of computer-assisted reporting to join the New York Times, while multiple news organizations are hiring positions in reporting, editing, and online news management.","keywords":["journalism","hiring","reporting"],"topics":["Journalism & Newsroom"],"technologies":["Internet"],"full_text":"Bits, Bytes and Barks\n\nDedman leaves AP\n\nBill Dedman, who has served as the director of computer-assisted reporting for three years at the Associated Press, has announced he will be leaving the wire service. Dedman and his wife, Pam Belluck, will be moving to Chicago where she will work as chief of the Chicago bureau of the New York Times. Dedman will write national stories for the Times and says he is looking forward to his return to reporting.\n\nHelp Wanted\n\nTIME Online is looking to fill several positions. The service is looking for a news editor to set daily budgets and assign stories to writers, a writer who can write fast and tight but doesn't lack style, and a news monitor who can prioritize stories and combine breaking news wires with associated text and photo resources. To apply for any of the jobs or for more information, e-mail a text-only resume and a letter describing your news background and electronic skills to Flora Garcia, TIME Online's production manager, at fgarcia@pathfinder.com.\n\nThe Daytona Beach News-Journal is looking for an experienced, aggressive investigative reporter experienced in probing public records and doing computer-assisted research. Applicants should send a letter, resume and clips to Don Lindley, managing editor, The News-Journal, P.O. Box 2831, Daytona Beach, Fla. 32120-2831.\n\nThe Wilmington Morning Star needs a court reporter. Applicants should have at least two years' reporting experience and computer-assisted reporting skills. Send a resume, references and five to 10 clips to Dave Ennis, city editor, Wilmington Morning Star, P.O. Box 840 (1003 S. 17th St.), Wilmington, N.C. 28402-0840.\n\nThe Orange County Register needs a technology reporter and an assistant city editor for medicine and science. Applicants for the editing position should have at least three years reporting experience on a daily metro and at least two years editing experience. Applicants for either position should submit a packet of current work, a resume and a one-page statement of philosophy to: John Doussard, 625 N. Grand Ave., Santa Ana, Calif. 92701. Telephone: (714) 953-2225. Fax: 714-565-3681.\n\nNICAR Net\n\nDon't forget to keep up with NICAR on the Internet. Subscribe to our listserve and join in as reporters talk about how to do the job better. E-mail to LISTSERV@MIZZOU1.MISSOURI.EDU. In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on the first line, write: subscribe IRE-L your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-summa-car-laude","slug":"1997-march-summa-car-laude","url":"https://dwillis.github.io/uplink/article/1997-march-summa-car-laude.html","year":1997,"month":"March","monthNum":3,"headline":"Summa CAR Laude","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Brant Houston has been named IRE executive director and plans to strengthen connections between IRE and NICAR programs. The newsletter features profiles of recent and soon-to-be graduates using computer-assisted reporting in journalism, including work at major newspapers and universities.","keywords":["journalism","graduates","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Big news in NICAR world: Brant Houston, managing director of NICAR, has been named IRE executive director. As executive director, Brant hopes to enhance the connection between IRE and NICAR programs.\n\nNow, on to this month's issue. Here's what's in store: many students have turned in their cap and gown for computers and gigabytes. Others have produced hard-hitting CAR stories before the band began playing \"Pomp and Circumstance.\" This issue features some of those recent and soon-to-be graduates. Naomi Aoki is helping The Virginia-Pilot take a bite out of surface-level crime coverage, Natalya Shulyakovskaya of the St. Louis Post-Dispatch explains how she used CAR for everything from features to investigations. But you don't have to have a diploma to do data. Students at the University of Nebraska and University of Maryland are proving that. And Stan Dorsey, a NICAR staffer and a soon-to-be Missouri graduate, looks at sports and CAR.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-tracking-the-trends","slug":"1997-march-tracking-the-trends","url":"https://dwillis.github.io/uplink/article/1997-march-tracking-the-trends.html","year":1997,"month":"March","monthNum":3,"headline":"Tracking the trends","kicker":"","author_name":"Naomi Aoki","author_title":"","affiliation":"The Virginia Pilot","summary":"Naomi Aoki discusses efforts by The Virginian-Pilot to provide readers with a more accurate picture of crime trends in the South Hampton Roads region through regular reporting on crime trends using computer-assisted reporting.","keywords":["crime trends","Virginia","reporting"],"topics":["Crime & Justice","Public Safety","Investigative Methods"],"technologies":[],"full_text":"Last year was the safest year in South Hampton Roads, Va., since the dawn of this decade. But readers of The Virginian-Pilot might not have guessed it. One reader even called recently to suggest that we package all of our crime news in a special 'atrocity section.' In that reader's defense, we wrote an average of one murder story a day, not counting briefs. Some were about actual murders, others about murder trials. All this for an area where a person's chances of being murdered were about one in 100,000 last year. In The Pilot's defense, we did report that violent crime in Portsmouth, the most crime-ridden of the five cities in the South Hampton Roads area, was on the decline for the first time in 12 years. We also reported other trends, both good and bad. In the coming year, we hope to give readers a more accurate picture of crime in our region by reporting on more trends and doing it regularly. The Pilot has dedicated a weekly page, dubbed Public Safety, for this purpose. We hope the weekly reports will help to make sense of crimes that seem to occur randomly and chaotically. If patterns can be shown, it becomes clear that crimes are not so random, not so chaotic. And it is easier to combat crimes if there are ways to anticipate where they will occur, and when, and to whom. The success of the page, however, depends on being able to quickly produce computer-assisted reporting stories that give readers useful information about crimes, crime prevention and the local criminal justice system. In short, that means high-volume, high-quality CAR. Status reports on crime can help people evaluate their level of safety. Using local police data or FBI data, you can look at which crimes are going up or down and how your city compares to others. If you have information about where - blocks, census tracts or police patrol zones - crimes happen, you can calculate crime rates for them and see how the different areas within your city are doing. Use FBI incident data to keep tabs on your local police departments. Look at their clearance rates and their staffing. See how they compare to other similar cities. If the Department of Motor Vehicles in your state keeps records on crashes, you can use it to find the most dangerous stretches of road. Or if you can get your hands on trooper activity reports, you can see who the most prolific writer of traffic tickets is in your area and do a ride-along.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-march-quick-hits","slug":"1997-march-quick-hits","url":"https://dwillis.github.io/uplink/article/1997-march-quick-hits.html","year":1997,"month":"March","monthNum":3,"headline":"Quick hits","kicker":"","author_name":"Naomi Aoki","author_title":"","affiliation":"The Virginia Pilot","summary":"This article provides guidance on using publicly available crime and safety data from sources like the FBI and local police to evaluate community safety, track crime trends, and investigate law enforcement practices.","keywords":["crime","data","safety"],"topics":["Investigative Methods","Crime & Justice","Public Safety"],"technologies":[],"full_text":"Status reports on crime can help people evaluate their level of safety. Using local police data or FBI data, you can look at which crimes are going up or down and how your city compares to others.\nIf you have information about where - blocks, census tracts or police patrol zones - crimes happen, you can calculate crime rates for them and see how the different areas within your city are doing.\nUse FBI incident data to keep tabs on your local police departments. Look at their clearance rates and their staffing. See how they compare to other similar cities.\nIf the Department of Motor Vehicles in your state keeps records on crashes, you can use it to find the most dangerous stretches of road.\nOr if you can get your hands on trooper activity reports, you can see who the most prolific writer of traffic tickets is in your area and do a ride-along.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-crime-connections","slug":"1997-march-crime-connections","url":"https://dwillis.github.io/uplink/article/1997-march-crime-connections.html","year":1997,"month":"March","monthNum":3,"headline":"Crime connections","kicker":"","author_name":"Naomi Aoki","author_title":"Database journalist","affiliation":"Portsmouth newspaper","summary":"A newsroom explains how they build and maintain comprehensive crime databases from police departments and government agencies, then use data analysis to identify trends, track crime statistics, and develop investigative stories about public safety in their community.","keywords":["databases","crime","analysis"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["SQL"],"full_text":"The first step is arming ourselves with as many useful databases as possible and keeping them up-to-date. We've bought all the relevant data NICAR has to offer, such as FBI Uniform Crime Report and fatal accident reports. We've also negotiated with a number of different state agencies for court records, auto accidents data, state police trooper activity reports, concealed weapons permits and autopsy statistics on causes of accidental deaths, suicides and homicides.\n\nBut, perhaps, the most crucial data, such as incident reports, we have came from local police departments.\n\nThis is the heart and soul of what we will be reporting on the weekly Public Safety page.\n\nUsing the police incidents data, we can look at almost any kind of crime big or small in our area. We get quarterly updates allowing us to track trends in our area long before the FBI releases its incident reports.\n\nThe data includes information about when and where crimes happened, including block numbers and street addresses, Census tracts, and whether the incident happened in a home or in an alley. Some of the databases also include information about stolen property, suspects, victims and any weapons used.\n\nFrom data to news\n\nThe next step is to mine our growing body of data for stories. We look for one-hit stories in the data - a profile of the state's most prolific writer of traffic tickets, for instance.\n\nBut more importantly, we use the data to track crime rates, clearance rates and the changing faces of crime.\n\nWe look at which of the eight major crimes (murder, rape, robbery, aggravated assault, burglary, larceny, auto theft and arson) are increasing or decreasing. And we look for regional trouble spots.\n\nWe use the statistics to add context to daily stories by adding a paragraph to reports telling readers what the neighborhood's crime rate is and how it compares to the city rate. We can also include numbers on how prevalent the crime - whether it is murder, burglary or vandalism - is in this area. Yearly status reports on crime can also be turned quickly. Using both the FBI and our local incident data, we can track the frequency of major crimes in our area back to 1980. Often, these analyses lead us to bigger stories a recent series on violent crime in Portsmouth, for example.\n\nReporting crime in Portsmouth\n\nPortsmouth's rate of violent crime had climbed every year since 1993. By 1995, Portsmouth reached it highest violent crime rate ever, with 1,480 violent crimes per 100,000 residents. It had also achieved the dubious distinction of being the 14th most murderous city in the nation.\n\nLast year, the violent crime rate dropped 29 percent. Having tracked the crime rate for years, we were able to break the news. Our databases allowed us to give readers a comprehensive package about the drop and reality of violent crime in Portsmouth.\n\nHere's how we did the analysis. The Portsmouth incident reports database includes the following fields: case number, offense, report date, incident date and time, block number, street address and Census track where the incident occurred, weapons used, property stolen and the race and sex of victims.\n\nFirst, I added a field in order to categorize the crimes to fit the Uniform Crime Reporting definitions of murder/manslaughter, rape, robbery and aggravated assault. Then, grouping by that field, I did a citywide count of those four crimes.\n\nUsing the city's population, I used those counts to calculate a violent crime rate for the city.\n\nLooking at the four crimes individually allowed us to see that huge decreases in murders and robberies were responsible for the plummeting crime rate.\n\nIt also showed readers the frequency of each violent crime.\n\nNext, we wanted to see if the drop in violent crime was spread throughout the city or in specific areas of the city. So, we grouped by Census tracts and used a where clause to count only the four violent crimes. Using population statistics from the 1990 Census and those counts, we calculated a violent crime rate for each Census tract.\n\nWe did the same for 1995, and then calculated the percent change in violent crime for each Census tract.\n\nWhat we found was that the most crime-ridden areas showed the biggest decreases in crime. The few areas where crime had increased in 1996 were still relatively low-crime areas.\n\nBy mapping the data, we were able to show readers how the crime rate in their neighborhoods stacked up to other areas of the city.\n\nWith the bulk of the analysis done, we ranked the four violent crimes and found robberies to be the most common.\n\nGrouping by block number and street and using a where clause to narrow the field to only violent crimes, we located the 10 most dangerous city blocks.\n\nGrouping by the weapon used, we found that feet and hands were the most common weapons used in the commission of a violent crime.\n\nLast year's drop in violent crime in Portsmouth does not make a trend, and we clearly stated that fact in the third graph of the story.\n\nBut it may be an early signal that the city has reason to hope - a signal computer-assisted reporting helped us recognize.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-score","slug":"1997-march-score","url":"https://dwillis.github.io/uplink/article/1997-march-score.html","year":1997,"month":"March","monthNum":3,"headline":"Score!","kicker":"CAR assists sports writers","author_name":"Stan Dorsey","author_title":"NICAR staff","affiliation":"NICAR","summary":"Sports reporters are underutilizing computer-assisted reporting despite sports being data-driven, with Tom Witosky of the Des Moines Register demonstrating how CAR tools like databases can uncover trends and insights beyond standard statistics provided by sports information departments.","keywords":["sports","reporting","data"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Quattro Pro","Paradox","9-track tape"],"full_text":"Score!\n\nCAR assists sports writers\n\nBy Stan Dorsey NICAR staff\n\nWhen most people think of sports, the first thing that comes to mind is numbers.\n\nWhether it be the spiraling salaries of superstar athletes or the plummeting success rate of your favorite team, virtually all sports are mired in a swirling eddy of points, polls and percentages.\n\nWhy then, when most reporters think of sports, do they not think of CAR? It looks like a match made in heaven, but few sports writers take advantage of computer-assisted reporting.\n\nTom Witosky of the Des Moines Register knows this is true. Just a few years back, he was only an occasional practitioner of CAR, but now Witosky uses tools such as Quattro Pro and Paradox virtually every week to help him give readers insight past the ordinary numbers they are used to seeing in their daily sports pages.\n\nOne of the main obstacles, Witosky said, is technophobia. \"I think it is lack of knowledge and fear,\" he said. \"People just are a bit scared to jump into the water.\"\n\nIn addition, sports reporters over the years have grown increasingly dependent on the preselected and calculated statistics provided to them by sports information departments. While easing deadline pressure, it also may lead to complacency on the part of the reporter.\n\n\"Who is to say that all the information in the releases is accurate? Sports information departments will often just highlight numbers that they want you to see,\" said Witosky, who co-led a panel on sports and CAR at the NashCAR seminar in Nashville, Tenn. \"Besides, if you were a government reporter, would you necessarily rely on a press release from the governor's office on earnings or on some financial matters? I have learned to not necessarily trust the numbers I get.\"\n\nWitosky said he looks at the releases, but treats them with a grain of salt.\n\nInstead, Witosky often compiles his own numbers and uses a database manager to unveil relationships that aren't evident in the vanilla reports handed out by SIDs.\n\nHe does this every week with the Iowa State and University of Iowa football teams.\n\nUsing the play-by-play sheets distributed by SIDs after games, Witosky enters each play into a spreadsheet program and creates fields for the quarter, down, yards to go, possession, field position, play result, and time of game.\n\nWitosky said entering play-by-play is one of the easiest projects for beginning CAR reporters in that it involves simple data entry and requires no database requests. Plus, it can produce useful information every day.\n\nHe then plugs the numbers into a database manager such as Paradox and searches for trends he may not otherwise see: production by quarter or by down, tendencies of a certain player, or play calling habits of a coach.\n\n\"These are exactly the type of calculations coaches do every week,\" Witosky said. \"Why shouldn't the reporters who cover them do the same?\"\n\nHowever, many off-the-field aspects of sports also lend themselves to CAR.\n\nThat is what Doug Bedell of the Dallas Morning News found when he decided to investigate accusations of fraud made by a Texas A&M University basketball player.\n\nUsing phone records obtained from the university on nine-track tape, Bedell was able to look at the calling patterns of certain coaches on the A&M staff and found that an unusually high number of calls were being made to a certain number in New York. The number, it turned out, was that of a talent scout who had been working illegally with coaches on the staff to recruit a player who was playing for Syracuse at the time.\n\nWhen the player finally came forth and admitted his involvement with the agent, the Morning News had a story on the incriminating phone records in the next day's paper.\n\nObtaining the records wasn't easy, Bedell said, but when the school tried to give him the run-around, simply invoked the state's Open Records Law. The law, in existence in all states, classifies as public record anything toward which taxpayers' dollars are contributed. The calling card calls qualified, and the university relented.\n\nIt isn't always that easy, of course.\n\nJust recently, Bedell tried to obtain records from the University of Nebraska and, though officials granted him access to the files, they told him he could only see them in paper form and that he would have to travel to Lincoln, Neb., and bring his own photocopier.\n\n\"That's fully within the law,\" Bedell said. \"That's what they can do to you if they want to be nasty about it.\"\n\nStill, Bedell said, it is usually worth the trouble, even if you aren't looking for anything specific in the records. Red flags such as simultaneously used calling card numbers or a high volume of calls to a particular out-of-state number can often indicate trouble.\n\nDuring one such blind search, Bedell stumbled upon a coach's calling card that had far more calls than any other on the staff. The reason, he found, was that the card had been posted in the locker room of University of Texas basketball team for use by its players.\n\n\"I guess it was supposed to be like a perk,\" Bedell said.\n\nBut for all the corruption that goes on at the collegiate and professional levels, high schools are by no means immune from questionable activities.\n\nMark Skertic had never done a sports story before, but, as an education reporter at the Cincinnati Enquirer, he had plenty of CAR experience. That made him the perfect cross-over candidate when the paper decided to act on a tip and look into discrepancies in reported high-school enrollment figures.\n\nIn Ohio, as in most states, schools' athletic teams are placed in a division based on enrollment figures for ninth, 10th and 11th grades. In October of each year, school officials report one figure to the state to help determine funding; they also report a number to the Ohio High School Athletic Association for athletic division placement.\n\nWhat Skertic found, after using Quattro Pro to look at five years worth of enrollment figures from both the state and the OHSAA, was that the figures often did not match.\n\n\"In some cases, there are accusations schools are juicing up the number they give to the state to boost their funding subsidy,\" Skertic said. \"In others, the athletic numbers are lowballed to put a school in a smaller division where, presumably, they might be a powerhouse.\"\n\nWhile Skertic said some of the inconsistencies turned out to be data entry mistakes at the school or state level, his story disturbed state officials enough that they decided to launch several internal audits and search for a better way to collect the information.\n\nStories like these help bring accountability to a sports world that often thrives on two things: One, reporters' fear of numbers and technology and their subsequent unwillingness to probe issues. And two, the complacency spawned by SIDs who consistently produce boring, meaningless numbers just to keep reporters happy, and keep themselves free from scrutiny.\n\nStan Dorsey can be reached at (573) 882-0684, or send e-mail to stan@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-the-real-story","slug":"1997-march-the-real-story","url":"https://dwillis.github.io/uplink/article/1997-march-the-real-story.html","year":1997,"month":"March","monthNum":3,"headline":"The real story","kicker":"Project misconceptions","author_name":"Joanna Kakissis","author_title":"NICAR staff","affiliation":"NICAR","summary":"A college journalist used computerized police records, mortgage data, and census information to debunk persistent negative stereotypes about Lincoln's Malone neighborhood, revealing it is actually one of the city's safest areas despite persistent poverty and a history of discriminatory lending practices.","keywords":["investigation","misconceptions","data"],"topics":["Investigative Methods","Data & Databases","Demographics & Census"],"technologies":["Maptitude","Pentium"],"full_text":"The real story Project misconceptions\n\nBy Joanna Kakissis NICAR staff\n\nWhen Matthew Waite began his studies at the University of Nebraska at Lincoln three and a half years ago, he heard the \"freshman folklore\" about Malone, one of Lincoln's oldest neighborhoods.\n\nMalone was supposedly Lincoln's ghetto, a dangerous place characterized by crime and poverty. Only the uneducated poor lived there in miserable, run-down homes, and outsiders entered its seedy boundaries at their own risk.\n\nBut after a nine-month investigation using computerized police records, Home Mortgage Disclosure Act (HMDA) data, Census data and scores of interviews, the 21-year-old college senior gave a dramatically different presentation of Malone in a Dec. 12 story in the university's student newspaper.\n\nAccording to Waite's story, Malone is one of city's safest neighborhoods, ranking near the bottom in nine major crime categories when compared with eight other Lincoln neighborhoods. In addition, Malone is one of Lincoln's most ethnically diverse areas despite the common misconception that only African Americans lived there. Also, more than half of Malone residents have been to college.\n\nOnly one perception-- the area's poverty--remained as fact. Malone is Lincoln's poorest neighborhood, with a median individual income of $12,917. After an analysis of HMDA data, Waite discovered that Malone received the least amount of home-loan money of the nine neighborhoods surveyed for the story.\n\nWaite interviewed sociologists, law enforcement officials and civic and minority leaders who alleged the area's lingering poverty was strongly related to Malone's reputation as a ghetto. Because of the inaccurate perceptions, Malone residents saw a reluctance by businesses to invest in the neighborhood, low property values and a lack of medical services.\n\n\"After I realized that the data showed Malone was not a crime-ridden neighborhood, I got the HMDA data and realized that the Malone neighborhood was suffering from the burden of misrepresentation,\" Waite said.\n\nWaite first came upon the idea after meeting Malone resident Leola Bullock while he was covering an anti-death penalty rally in 1994. Bullock, an African American, had moved to Malone in 1950 because of Lincoln's racial segregation housing policies. Although she had moved from Malone in 1967, her description of it as a culturally vibrant neighborhood so contradicted Lincoln lore, Waite decided to explore the issue for an in-depth reporting class. He obtained the 1995 computerized crime records from the local police chief, loaded the 39,490 records on a Pentium 132 Megahertz with 64 megs of RAM and formatted them into the Maptitude program he had purchased for about $600.\n\nWaite is no novice with computers. His father has been in the data processing industry for 25 years, and Waite spent summers working with his father at home in Blair, Neb.\n\n\"But this project was a huge learning curve for me,\" he said. \"I didn't know much about data analysis or dirty data.\"\n\nInitially, the computer rejected 65 percent of the police records because the addresses for the scenes of crimes had not been recorded in a standard manner by Lincoln police.\n\n\"I had to hand-check 26,000 records. It took me four months,\" he said.\n\nWaite worked every day from 11 p.m. to midnight and from 3 to 4 in the morning before the address entries were consistent. After cleaning the address data, he categorized the data into nine high-profile crimes: larceny, vandalism, disturbances, robbery, burglary, miscellaneous weapons, car theft, assault and narcotics.\n\n\"When I saw the results, I noticed a pattern,\" he said. \"Crimes thinned out when it got to the Malone neighborhood.\"\n\nWaite put faces with the statistics by interviewing several Malone residents, community leaders, minority and civil leaders and law enforcement officials.\n\nWaite calls reaction to his article \"unbelievable.\" Last month, Nebraska's Cornhusker State Bank offered $1.5 million in low-income home guarantees to Malone residents. In addition, millions of tax revenue dollars used for community reinvestment funds are being re-evaluated.\n\n\"I didn't think a student newspaper could have this kind of influence,\" he said.\n\nJoanna Kakissis can be reached at (573) 882-0684, or send e-mail to c680409@showme.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-trading-places","slug":"1997-march-trading-places","url":"https://dwillis.github.io/uplink/article/1997-march-trading-places.html","year":1997,"month":"March","monthNum":3,"headline":"Trading places","kicker":"Voters shift to third parties","author_name":"Kara Morrison","author_title":"Staff Writer","affiliation":"NICAR","summary":"Graduate journalism students used computer-assisted reporting to analyze Maryland voter registration data from 1988-1996, discovering a dramatic shift away from the two major parties toward independent and minor party registration, with increases of over 55 percent among third-party registrants. Their findings revealed that rural areas were driving most of the growth in minor party support, challenging Maryland's longtime Democratic stronghold.","keywords":["voters","parties","registration"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Access","Excel"],"full_text":"Trading places\n\nVoters shift to third parties\n\nBy Kara Morrison\nNICAR staff\n\nAs Bill Thompson watched friends and acquaintances in Maryland become more disillusioned with Republicans and Democrats, he wondered how widespread the sentiment was becoming.\n\nThompson, a graduate student at the University of Maryland College of Journalism, decided to turn his question into a computer-assisted reporting project for the school's Capital News Service, a wire service based in Annapolis.\n\nAlong with graduate student Kristina Schurr, Thompson used Access and Excel to analyze voter registration data and actual voting totals by county from the 1988, 1992 and 1996 presidential elections.\n\nMinor explosion\n\nNot only were people turning away from the two major parties, but minor political parties in Maryland appeared to be gaining force rapidly.\n\nTheir data showed large increases from 1988 to 1996 in the number of voters registering as Independents or in a minor party.\n\nOverall, such registrations grew by more than 55 percent (104,868 voters), while registration for Democrats increased only 2.4 percent and Republican registrations increased 21.5 percent during the same period.\n\nIn addition, actual votes for minor party presidential candidates from 1988 to 1996 increased nearly 1,000 percent.\n\n\"The state of Maryland has been a Democratic stronghold as long as I can remember,\" said Thompson, a Maryland native. \"The thing I was surprised most about is the Democrats are losing their stronghold.\"\n\nBy dividing their data into Maryland's five main regions, Thompson said they were able to report that the majority of the increase in minor party support was coming from the state's rural areas.\n\nMinor frustrations\n\n\"It was an interesting story,\" said Barbara Sauers, managing editor of the 17,000-circulation Star-Democrat. Sauers, who ran the story the front page, said she knew readers in her area were interested in third parties, especially in light of the significant following Ross Perot's candidacy drew in the area during the 1992 election.\n\nIn all, the project Thompson and Schurr about two months. Three Maryland newspapers, including the Easton Star-Democrat, ran the story in December 1996.\n\nThompson and Schurr said the story did pose some challenges.\n\nOne of the biggest frustrations the pair encountered was an archaic record-keeping system at Maryland's State Administrative Board of Election Law. Thompson said the office kept only paper records. The result was a large chunk of time spent waiting for the records and then entering them by hand.\n\nSchurr said there were other frustrations, such as designing a spreadsheet in Excel to perform the calculations as they envisioned.\n\nAlthough both had taken a computer-assisted reporting course taught by U.S. News and World Report senior editor Penny Loeb, Schurr said working as a team helped them get through the technical glitches they encountered along the way.\n\n\"You have to really understand what it is you're looking for,\" Thompson said.\n\nHe added that a good editor is a great help. Their Annapolis bureau chief, Linda Fibich, reminded them to make sure the results couldn't be explained by population increases, which led them to an analysis of Census data in addition to voting records.\n\nIn the end, Thompson sent Shurr a Christmas card insisting it was a CAR reporter who said, \"These are the times that try men's souls.\"\n\n\"That's how we felt sometimes,\" Schurr admitted.\n\nNevertheless, Thompson said the project wasn't overly difficult. Schurr said she's convinced CAR is a necessity.\n\n\"The notion that computer-assisted reporting is just for the Big Boys is silly and outdated,\" she said. \"It's just something you ought to know.\"\n\nCurrently, Thompson, 34, and Schurr, 23, are interns at the Baltimore Sun. Both will graduate from the University of Maryland in May.\n\nKara Morrison can be reached at (573) 882-0684, or send e-mail to c684223@showme.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-babies-buildings-and-bureaucrats","slug":"1997-march-babies-buildings-and-bureaucrats","url":"https://dwillis.github.io/uplink/article/1997-march-babies-buildings-and-bureaucrats.html","year":1997,"month":"March","monthNum":3,"headline":"Babies, buildings and bureaucrats","kicker":"CAR serves many purposes","author_name":"Natalya Shulyakovskaya","author_title":"Newsroom Researcher","affiliation":"St. Louis Post-Dispatch","summary":"A journalism graduate describes his early career at the St. Louis Post-Dispatch as a newsroom researcher, sharing lessons learned from data-driven reporting projects on topics ranging from birth records to abandoned buildings.","keywords":["journalism","data","reporting"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["SQL"],"full_text":"I graduated in May 1996 from the University of Missouri and was soon hired by the St. Louis Post-Dispatch (I had a summer internship there a year before). The paper created a funky position called newsroom researcher - not a writing job, but a full-fledged computer-assisted reporting berth and I signed on. I came to the Post-Dispatch to work with George Landau. So I tinkered with SQL server (some), collected a lot of data and worked as a free agent on different projects with different reporters. In the months that followed, I analyzed Missouri state pensions, St. Louis city cellular phone use, medical data, abandoned and vandalized buildings, local campaign contributions and voter registration. I requested and got a lot more data than I was able to turn into stories. I didn't put nearly as much data on the SQL server as I should have. And I made my mistakes. Here is a cheat sheet that I slowly compiled.\n\nNo More Babies\n\nWhen you get a new job, it's great to start with something not-so-complicated. Choose your first data projects carefully - it is likely you will come back to these data over and over again. Besides, your colleagues will remember you as a person doing \"SUCH and SUCH.\" I didn't strategize and I ended up measuring babies. The Post-Dispatch had gotten several years worth of birth records for the state of Missouri. No names, but the rest of the data is intact and you can fish out all sorts of amazing information (medical procedures and complications, parents' educational level, prenatal care, etc.) My affair with the data started when an editor came to our geek nook and waived a press release about big babies in front of me. Scientists in Illinois studied babies, documented that as a norm the current newborns are much bigger than the charts that are used by the pediatricians to measure their progress. \"Can we look at Missouri infants?\" the editor asked.\n\nWe did - we charted baby weight, found some doctors who specialized in delivering huge newborns and talked to moms whose sons wore clothes for 3-year-olds right after their birth.\n\nSeveral weeks later, I was back working on a story about the skyrocketing number of multiple births. Now, whenever I pass our metro desk talking about stories, I get a grin from that editor: \"No more babies,\" he mutters. But now I have sources in the State Center for Health Statistics.\n\n$3 Million Question\n\nYou need to discuss your findings with your subjects, but you need to be careful - they would love to steal your show.\n\nSt. Louis has thousands of dilapidated abandoned buildings, and we decided to look at where they are and what the city is doing about them. Last summer, two kids were found dead on two different occasions in two different buildings in North St. Louis.\n\nWe got a copy of the city's demolition permits, a survey of the vacant and vandalized buildings that the building inspectors compile every year, a database of condemnations and started looking at the neighborhoods that were decaying.\n\nAs we were nearing the deadline, we talked to the building commissioner, inspectors, the grandmother of the dead girl and an occasional PR person from the City Hall who was always curious: \"So when is your story going to run?\" Then they saw our final data runs.\n\nOn Friday, before our Sunday story hit the presses, the St. Louis mayor declared he was pouring an extra $3 million into demolitions. It didn't kill the story, but made me much more careful when I worked on the next city-related project.\n\n18 Minute Gaps\n\nRead and check the data as soon as the tapes land on your lap.\n\nAfter a couple of weeks of working with a cumbersome and complicated file tracking pensions for Missouri Highway Patrol and Transportation Department, I discovered that multiple fields in the middle of the file were missing. Cold sweat was dripping down my face.\n\nIt turned out that a programmer forgot to include those fields in a read-out program. We got the tape back in two days. At that point I was glad my data clean-up was saved in a program.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-on-the-internet","slug":"1997-march-on-the-internet","url":"https://dwillis.github.io/uplink/article/1997-march-on-the-internet.html","year":1997,"month":"March","monthNum":3,"headline":"On the Internet","kicker":"Internet job search","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute","summary":"A comprehensive guide to journalism career resources and job search websites for recent graduates, including links to major job boards, professional associations, and listservs that post journalism positions.","keywords":["journalism","careers","jobs"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web","Internet","Listserv"],"full_text":"Congratulations, Graduates! You've worked on your studies for years and now you are ready to put all that knowledge to work.\n\nHere are some sites on the World Wide Web that can help you scout out jobs and get some job-seeking advice. Also listed are listservs that often post new job openings.\n\nSo, get yourself a job!\n\nCareers in Journalism\n\nhttp://www.asne.org/kiosk/careers/carerdet.htm\nThe American Society of Newspaper Editors has put together this page with information on preparing for a career in journalism and tips on finding your first job. Check the listing of 1997 Newspaper Internships, but hurry: Many of them have pretty early deadlines for application.\n\nAsian American Journalist's Association\nhttp://www.aaja.org/benefits.html\nLearn about the AAJA's Job Hotline and find information on the Executive Leadership program, and fellowships to help Asian American journalists develop their journalism and leadership skills.\n\nAssociation for Education in Journalism and Mass Communication\nhttp://www.aejmc.sc.edu/online/home.html\nScroll down the home page of this association for journalism educators and locate the \"Placement Ads\" area. There are opportunities for graduate assistantships as well as postings for professor positions.\n\nDow Jones Newspaper Fund\nhttp://www.dowjones.com/newsfund/alumni.html\nFind out where you can get a monthly posting of editing and reporting positions available in the 70 newspapers that participate in the DJNF programs.\n\nJournalism and Women Symposium\nhttp://www.jaws.org/jobs.html\nJob postings on JAWS numbered more than 10 on a recent check. They have also put together links to other sites with Job Banks including:\n\nNational Diversity Journalism Job Bank\nhttp://www.newsjobs.com/\nSee the link to other job banks more than 15 listed!\n\nNational Association of Hispanic Journalists' Job Bank\nhttp://www.nahj.org/nahjjob.html\nNicely broken down into different types of positions, internships, print, broadcast, radio.\n\nGannett Newspaper Newsroom Opportunities\nhttp://www.gannett.com/job/jobs.htm\nCovers not only opportunities in the newsroom, but on the business side and production, too.\n\nThe Minorities in Broadcasting Training Program\nhttp://www.webcom.com/mibtp/OPENINGS.html\nFocused on jobs in television, broken down by administrative, technical, on-air and writing.\n\nEditor and Publisher Classifieds\nhttp://www.mediainfo.com/ephome/class/classhtm/class.htm\nThese listings are broken down into all the kinds of positions in a news organization from advertising to art and graphics to editorial and production. With dozens of listings, this may be the most comprehensive site for journalism job openings.\n\nListservs:\n\nNewslib\nhttp://sunsite.unc.edu/slanews/about/newslib.html\nInformation on how to join NewsLib, the listserv for news librarians and researchers. This listserv is often the posting place for internship, researcher and news library director positions available.\n\nNICAR-L\nhttp://www.nicar.org/list.html\nThis site tells you how to join the National Institute for Computer Assisted Reporting listserv. If you are looking for a CAR type position in a newsroom, read the postings, be an active (and intelligent) contributor and get your name out there!\n\nNora Paul can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-tech-tip","slug":"1997-march-tech-tip","url":"https://dwillis.github.io/uplink/article/1997-march-tech-tip.html","year":1997,"month":"March","monthNum":3,"headline":"Tech Tip","kicker":"What day is it?","author_name":"Richard Mullins","author_title":"Computer-Assisted Reporting Instructor","affiliation":"NICAR/Missouri School of Journalism","summary":"This tech tip explains how to use database functions in Access and FoxPro to determine the day of the week from date fields, allowing journalists to analyze patterns like which weekdays have the most arrests or traffic accidents without manually checking calendars.","keywords":["database","functions","dates"],"topics":["Technology","Data & Databases"],"technologies":["Access","FoxPro","SQL"],"full_text":"Tech Tip What day is it?\n\nBy Richard Mullins\nNICAR/Missouri School of Journalism\n\nIf you wanted to find out what day of the week you were born, you could find a calendar for that year and look it up.\n\nBut if you wanted to find out which weekday had the most arrests or traffic accidents, the calendar lookup strategy would be too time-consuming for 50 records, let alone 150,000.\n\nAccess and FoxPro have functions that look at a date and determine the day of the week for that date. I'll explain both and show examples, starting with the FoxPro functions. For illustration, I'll assume a table called Arrests, with date-type field called BookDate.\n\nYou can see how this works with a query like this:\n\nSELECT bookdate, dow (bookdate) ;\nFROM arrests\n\nThe dow() function returns the day-of-the-week as number, with Sunday being 1. If you want the name of the weekday, use cdow(). Here's how to remember the function names: DOW stands for DayOfWeek and CDOW stands for CharacterDayOfWeek.\n\nSELECT bookdate, dow (bookdate), cdow (bookdate) ;\nFROM arrests\n\nSince you can always retrieve this day-of-week information with the function, you don't need to store the weekday in a new field in order to use it for a query. To find the arrest counts by weekday, in descending order, you would write this group-by query:\n\nSELECT cdow (bookdate) count (bookdate)\nFROM arrests ;\nGROUP BY 1 ;\nORDER BY 2 DESC\n\nIn Access, you can get this same information with the format() function. The help file on this function is quite long, since the function is used for more than just dates.\n\nHere is a general tip for help files: If you think that the help file on a particular topic is too confusing or too long, then look through the examples for the stuff that seems to fit what you want to do. Paste the example into your query and try it out.\n\nHere is the general syntax for the Access format function as it applies to dates. You specify two parameters, or instructions. These go inside the parentheses after the function name and are separated by commas.\n\nThe first parameter is some date expression, like a literal date or a column containing a date. The second is format specifier, which is a quoted string of characters specifying how you want the day of the week to be expressed. Here is the syntax:\n\nFormat DateExpression, FormatTemplate\n\nYou have three choices for FormatTemplate:\n\nW day of week as a number\nddd as abbreviation (Sun-Sat)\ndddd as full name (Sunday-Saturday)\n\nApplying this to the example table, here are the queries in Access:\n\nTo get the day of the week as a number:\n\nSELECT bookdate, format (bookdate, \"w\")\nFROM arrests ;\n\nTo get the day of the week as an abbreviation:\n\nSELECT bookdate, format (bookdate, \"ddd\")\nFROM arrests ;\n\nTo get the day of the week as a full name:\n\nSELECT bookdate, format (bookdate, \"dddd\")\nFROM arrests ;\n\nThe Access query to show arrests by weekday, in descending order of the arrest count:\n\nSELECT format (bookdate, \"dddd\"), count (bookdate)\nFROM arrests\nGROUP BY format (bookdate, \"dddd\"),\nORDER BY count (bookdate) DESC ;\n\nCan't turn your text file into a table? This question and more were answered in Tech Tip columns in 1996. The bound edition of 1996 Uplinks is now available for $20 plus shipping. To order, call Wendy Charron at (573) 882-0684.\n\nRichard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-car-trips-first-ventures","slug":"1997-march-car-trips-first-ventures","url":"https://dwillis.github.io/uplink/article/1997-march-car-trips-first-ventures.html","year":1997,"month":"March","monthNum":3,"headline":"CAR Trips: First ventures","kicker":"Calculating cafeteria costs","author_name":"Derek Willis","author_title":"Reporter","affiliation":"Palm Beach Post","summary":"A Palm Beach Post investigation uncovered massive overspending in the school district's cafeteria division, revealing that the district paid exorbitant prices for items like business cards and tab dividers through a printer contract that exceeded its approval by nearly three times. The reporters created a database from over 500 invoices to expose the lack of spending controls and an unwritten policy permitting contract overages.","keywords":["overspending","cafeteria","investigation"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Paradox"],"full_text":"School board members were told that the Palm Beach County School District's cafeteria division would spend about $55,000 on printing costs in 1997. But it paid a printer nearly four times that amount in the last six months of 1996, paying 10 times more for a business card than the district did and nearly 23 times the norm for tab dividers to separate sections of cookbooks and manuals.\n\nPalm Beach Post reporters detailed how the school district's spending controls were virtually absent within the cafeteria division, which provides meals for approximately 135,000 school children, and that school board members were unaware of a district policy that permitted over-spending on contracts.\n\nSome of the findings:\n\nPink and teal business cards for employees of cafeteria department cost nearly $21,000 or 24 cents for each card. The typical school district business card costs 2.4 cents.\nThe cafeteria unit spent $85,000 on tab dividers - often spending $23.75 a set when the rest of the district paid $1 for each set.\n\nHow it Started\n\nSchools reporter Mary Warejcka investigated how the district spent money on vendors, businesses and individuals that supplied services and products. After examining paper records and getting a tip, she found that the cafeteria division was spending more than the school board had approved on its contract with a local printer, KC's Desktop Publishing.\n\nThe school system followed an unwritten rule that allowed it to exceed the limit of a school board-approved contract. In this case, the district spent nearly $400,000, almost three times the $135,000 specified in the contract.\n\nWarejcka requested copies of the printer's invoices from the district and was given a four-inch stack of paper records dating back to 1994, when the contract was awarded. The 500-plus invoices contained multiple orders and some were barely legible. We decided to create a database.\n\nWe spent more than six hours entering details from the invoices into a Paradox database, making note of the item purchased, the quantity and price, among other details. We also tried to match purchase orders from district records to the invoices.\n\nWhen we finished, we looked at the items the district paid for, including the business cards and tab dividers. Using queries, we were able to tell how much the cafeteria unit spent on most items and a per-item price.\n\nWe also discovered that the invoices contained incomplete information. Nearly a quarter of them did not have an order date. And 40 percent of the money was spent without any reference to how the items corresponded with the contract. The story was published the following day.\n\nReactions\n\nOne school board member had this reaction to the expensive tab sets: \"That's totally outrageous. Are they lined in gold?\" The district's internal auditor requested, and the superintendent ordered, an audit of the food service department. The director of the cafeteria was suspended with pay two weeks after the story ran. In addition, the district changed its policy to require that the school board be notified whenever a contract exceeds the amount it approved. Since then, additional tips - and the school district's own investigation of spending - have yielded more stories on the cafeteria unit director, who set up a private consulting firm with the same name as the cafeteria division and accepted $4,000 in speaking fees at conferences across the nation.\n\nAdvice\n\nSchool districts in Florida are county-wide and often are the county's largest employers and some of the biggest spenders. Tackling school spending piece-by-piece, or department-by-department, seemed to be the way to go in our case.\n\nAlthough certain school records were available on computer, the district did not enter all of the information from the printer's invoices into a database. Thus we had to make our own to see how much the cafeteria division spent on business cards and other items.\n\nDerek Willis can be reached at (561) 820-4463 or send e-mail to dwillis@gate.net.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-nicar-notes","slug":"1997-march-nicar-notes","url":"https://dwillis.github.io/uplink/article/1997-march-nicar-notes.html","year":1997,"month":"March","monthNum":3,"headline":"NICAR Notes","kicker":"NashCAR debriefing","author_name":"Brant Houston","author_title":"Managing Director","affiliation":"NICAR","summary":"The NashCAR computer-assisted reporting conference in Nashville showcased significant advances in journalism technology, featuring over 50 panels and 140 computers for hands-on training with more than 500 attendees. The conference covered expanded topics including mapping software, statistical analysis, database management, and practical applications for various beats and news formats.","keywords":["conference","journalism","technology"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet","Internet","World Wide Web","GIS","SPSS","Intranet","CD-ROM"],"full_text":"One of our flyers about NashCAR, our annual computer-assisted reporting conference, talked about racing to the future. Once we got to Nashville, however, it looked the future was racing toward us. Four years ago at a conference in Raleigh, our panels and training dealt mostly with spreadsheets and database managers. The use of the Internet was an afterthought and there was no mention of the World Wide Web. But in Nashville the ante was raised. Our speakers and instructors covered spreadsheets and database managers, two of the basic tools for journalists doing CAR. They also spoke about using mapping software, SPSS (statistical software), intranets, programs for cleaning up databases, and writing CD-ROMs. But they didn't stop there. They talked about the role of computers in the newsroom (whether broadcast or print), the role of database editors and the best ways to train colleagues. At the same time, they gave advice on how to use computers to cover particular beats and specialties, how to use CAR for breaking news and how to tailor CAR for broadcast. The range of offerings through more than 50 panels and dozens of hands-on classes left most first-time CAR conference attendees dazzled and left repeat attendees satisfied that new and advanced courses were plentiful. Furthermore, we had our highest number of attendees ever - more than 500 - and we strained our largest number of computers for hands-on training - about 140 - to meet demand. For those who could not get to this conference, we have tape recordings of panels. (They are still valuable even if speakers sometimes refer to computer screen projections that you can only imagine.) We also have a bag of handouts that is startling in its breadth and depth.\n\nHere's a sampling:\n\nPaul Overberg of USA Today, showed how to profile your community with Census data.\nElizabeth Marchak, recent winner of a Polk Award, gave 11 solid tips on covering aviation. while Byron Acohido of the Seattle Times described how to do an investigation such as the amazing ones he has done on Boeing.\nPenny Loeb of U.S. News and World Report and Carol Napolitano of the Omaha World Herald teamed up to do an actual newsletter on housing just for NashCAR. And Mike Himowitz checked in with another sharp tip sheet on housing.\nNeill Borowski of the Philadelphia Inquirer gave a quick lesson in adjusting for inflation and advice on avoiding math errors.\nIf it moves (transportation excluding planes), Heather Newman of the Detroit Free Press and Loeb of U.S. News put it together in tip sheets that tell where the databases are and how to use them.\nChris Schmitt of the San Jose Mercury News and Griff Palmer of the Daily Oklahoman analyzed the coverage of state government.\nRob Covey of U.S. News gave six steps to improve graphics.\nA CAR kit for local reporting was assembled by four journalists: Rose Ciotta of the Buffalo News, Margot Williams of the Washington Post, Napolitano of the World Herald and Tom Torok of the Philadelphia Inquirer.\nWhen it comes to crime, Bill Ruberry of the Richmond Times-Dispatch outlined a plan for coverage and Bob Warner of the Philadelphia Daily News reviewed a project that followed violence in schools.\nOn sports, Tom Witosky of the Des Moines Register, had nine story ideas and provided actual story clips.\nJeff South of the Austin American-Statesman created a handout of can't miss ideas for localizing federal databases.\nJim Specht of the Gannett News Service and April Lynch and Aurelio Rojas of the San Francisco Chronicle offered two primers on covering immigration issues.\nDiagnosing hospitals were Robert Benincasa of the Burlington Free Press, Rick Linsk of the Asbury Park Press, and Tom Gaumer of the Plain Dealer in Cleveland.\nLise Olsen and Pedro Enrique Armendares of IRE's Periodistas de Investigacion (our new program in Mexico City) collected and distributed a list of web sites on border issues while Robin Rowland of the Ryerson Institute for Computer-Assisted Reporting in Canada summarized the way international data can be shared.\nCAR pioneer Philip Meyer and Barbara Hansen of USA Today explained the careful use of significance tests.\nJanet Roberts of the Wilmington Morning Star in North Carolina gave out strategies for making CAR work in a small newsroom.\nAnd to help reporters on the run, Tom Boyer of the Seattle Times gave an overview of how to deliver relational databases through a Web browser and Mike Himowitz of the Baltimore Sun talked about friendly front ends.\nIn the coming weeks, the handouts will be collated and distributed in hardcopy form or at our Web site. (www.nicar.org)\nAt IRE and NICAR, no one believes that we did everything right at NashCAR. Panels can be adjusted, training improved and scheduling of events can go smoother. Although half of our speakers were first-time presenters, we are looking hard for even more new panelists and viewpoints at our next conference, which is planned for Indianapolis in March 1998. We will also try to get more broadcasters and new media people on panels and continue to push for more women and minority journalists to participate.\nWe do know that we should keep the program flexible enough so that impromptu meetings, special demonstrations and on-the-fly workshops can always find a home at the conference.\nThanks to those who came to Nashville and made it such a great conference and for those who didn't we hope to see you next year in Indiana.\nBrant Houston can be reached at (573) 882-0684, or send e-mail to brant@nicar.org\n\nNewcomer's tips\n\nThe sea of paper\n\nIf they give you too much information on paper, they want you to drown. Hire a keypunch firm.\n\nOur investigative reporter Carolyn Tuft asked the city (six months before the primary in the mayoral race) for a copy of cellular phone bills. The city replied that it would infringe on privacy of our city officials, that it would stall all work at City Hall and that they didn't keep the bills. They just pay them.\n\nTwo months later, the case was settled out of court and piles of paper started arriving every day. Soon the stacks were covering the floor in my office, and we were losing our sanity.\n\nI tried scanning the bills, but they were too dirty. We tried to do the data entry ourselves. We used Access, designed a form and assigned three clerks to punch the data.\n\nTwo days later, we realized it would take us months to finish. We hired a professional data entry firm.\n\nThe story documented widespread abuse: hundreds of calls to girlfriends, liquor stores, gambling boats and 2 a.m. calls to a city contractor.\n\nNever again will I attempt to enter thousands or records in-house. Measure the price in terms of your salary and time. Unless it's a developing database of unique things happening on a beat, it's silly to the waste newsroom's time on typing.\n\nA couple of quick points:\n\nWrite weekly (daily, if you can discipline yourself) memos summarizing your projects, data negotiations and technical progress. It helps to divide projects into more manageable chunks. On a slow week, it helps to see what I've been doing with my life - and keep my spirits up.\n\nDocument. I log all the data cleaning and restructuring and put it in a program right away. Often, start with a program. It helped me a great deal when I was updating databases or when I made mistakes and had to go back to the original data. Make a CD with your original data, the clean database and all the documentation and programs.\n\nAs I work with the data, I try to keep my notes in the same place-electronically. Lotus Notes, a groupware the Post-Dispatch uses for e-mail and Web publishing, works pretty well. I log my conversations with officials, write technical notes and keep my requests in a Lotus Notes database I call \"Fish Tank.\"\n\nNatalya Shulyakovskaya can be reached at (314) 340-8183 or send e-mail to natalya@pd.stinet.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-handout-of-the-month","slug":"1997-march-handout-of-the-month","url":"https://dwillis.github.io/uplink/article/1997-march-handout-of-the-month.html","year":1997,"month":"March","monthNum":3,"headline":"Handout of the month","kicker":"Sports and the Computer","author_name":"Tom Witosky","author_title":"Sports Reporter","affiliation":"Des Moines Register","summary":"This article provides sports journalists with five data-driven reporting ideas to improve coverage, including analyzing play-by-play statistics, tracking athletic spending and Title IX compliance, examining workers' compensation records, monitoring coaches' outside income contracts, and reviewing athletic department audits.","keywords":["database","analysis","sports"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["spreadsheet"],"full_text":"To improve the quality of news in your sports department, try these reporting ideas. There will be some resistance, but it will be worth it if you try.\n\nStories by the numbers\n\n1. Football play-by-play\n\nBuild a database of all offensive and defensive plays from all games played by State U or your local professional team. Analyze the most recent games or the team's seasons statistics in a weekly column. Best resource is the play-by-play provided by the home team's sports information office or public relations department.\n\nThe database should be structured so that it includes quarter, ball position, down, distance, team in possession, play, result of play, tackler(s), penalty and time of game.\n\nKeep individual game play-by-play in separate tables, then combine tables to determine team tendencies and individual performance.\n\nCollege and professional coaching staffs spend a lot of time each week analyzing their opponent's tendencies. We should do the same thing. Similar databases can be built for basketball, baseball or any other sport where a substantial amount of information is kept. Small papers could do high school sports though reporters will have to do own play-by-play.\n\n2. Spending on sports\n\nHigh schools, colleges and universities spend millions and millions of dollars annually on sports. Where does the money go, and does it make a difference in the outcome? These kinds of projects are exactly what the computer is to be used for. It allows for analysis of lots of information easily.\n\nAre the schools in compliance with federal gender equity laws? Last year, I found that the girls high school basketball team uses hall lockers to change for practice. Along that same hallway was a locked boy's basketball gym. Colleges and universities are a lot better about Title IX, but there are exceptions. Recently, I found five Division I-A schools still had a 30 percent gap between the percentage of women enrolled and the percentage of the women participating in sports.\n\n3. Workers compensation\n\nNo professional team owner or general manager ever has anything nice to say about the cost of their workers comp. Generally, workers comp records are public. Build a database with the last five years of injuries sustained by players for the local professional team. Is there a pattern? What does the team physician say?\n\n4. Outside income contracts\n\nThis will allow you to track over the years how much money your top coaches are making from endorsement contracts and other deals. Coaches are required to provide their presidents with a report of all outside income contracts. Get them and begin to build the database.\n\n5. College athletic department audits\n\nAlso keep drive charts.\n\nNCAA rules require that each athletic department have its books audited by an outside firm annually. Open records laws should be strong enough to get copies of the reports from the public institution. Private schools will resist, but ask anyway. Build a database from the reports to find trends in spending and revenue generation. Get copies from the other schools in your school's conference for comparison.\n\n6. Season Ticket Holders\n\nEver wondered who sits in the best seats at a college football game? Most athletic departments now have their season ticket holders in a database. Ask for it and have fun with the powerful people in your community. Use it against campaign disclosure lists and see how the similarities run.\n\nThese are just some of the ideas that I have used my computer to help me with. There are others many others to try. Sports is a game of numbers, and computers crunch them better than reporters. Get to know spreadsheets and relational software. It will give you some top notch Page 1 stories.\n\nTom Witosky can be reached at (515) 284-8522, or send e-mail to glrumper@msn.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-seminars-bootcamps-conferences","slug":"1997-march-seminars-bootcamps-conferences","url":"https://dwillis.github.io/uplink/article/1997-march-seminars-bootcamps-conferences.html","year":1997,"month":"March","monthNum":3,"headline":"Seminars, bootcamps, conferences","kicker":"Get your training","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR and The Associated Press offer specialized computer-assisted reporting training through on-the-road sessions, bootcamps at their Missouri headquarters, and international programs, with costs varying by location and program type.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","spreadsheet"],"full_text":"NICAR and, where indicated, The Associated Press provide specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases.\n\nCost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below.\n\nNote: In the following list, \"Open to all\" means any journalist may sign up. \"Closed\" means the session is open only to members of the host organization.\n\nLincoln, Neb., Alternative News Weeklies April 12. Closed.\nKansas City, Alternative News Weeklies April 12. Closed\nMilwaukee, Wis., Milwaukee Journal April 14-16. Closed.\nNew York, N.Y., CBS News April 16. Closed.\nLima, Ohio, Lima News - April 20-22. Closed.\nCharleston, W.Va., Marshall University- May 13-16. Closed.\nNew York, N.Y., Reuters America-May 29-30. Closed.\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969. IRE National Conference, Phoenix, Ariz., - June 12-15.\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo.\n\nAs with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size.\n\nFor information, call Wendy Charron at (573) 882-0684.\n\nMay 4-9. Advanced Bootcamp concentrating on stats and maps at the University of North Carolina-Chapel Hill with Philip Meyer, author of The New Precision Journalism. The seminar is held in Chapel Hill. For details, call (573) 882-0684.\nMay 18-23. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684.\nAug. 10-15. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684.\n\nSouth of the Border\n\nPeriodistas de Investigacion (IRE-Mexico) offers computer-assisted reporting training in Spanish to journalists in Mexico and Latin America. PI, based in Mexico City, has developed a two-day workshop that covers the basics of investigative reporting and computer-assisted reporting, and a four-to-five day bootcamp that covers CAR intensively.\n\nThe workshops can be held in almost any city with prior notice. The courses cover using internet, spreadsheets, database managers, and generally electronic records, and uses examples from the Mexican and Latin American press.\n\nThere are two bootcamps scheduled in April and May:\n\nMedellin, Universidad Pontificia Bolivariana April 7-12\nMexico City - May 14-18.\n\nA one or two-day CAR workshop will be held in April in Monterrey, Mexico. Dates and times to be announced.\n\nAnnual membership to IRE-Mexico is $25 (or $200 pesos mexicanos) and is open to journalists or students of journalism living in Mexico and Latin America.\n\nInstructors are Lise Olsen, managing director of IRE-Mexico, and Pedro Enrique Armendares, assistant director. Both have more than seven years of experience in investigative reporting, Olsen at The Virginian-Pilot (Norfolk, Virginia) and Armendares at La Jornada (in Mexico City). Olsen specializes in computer-assisted reporting, and Armendares in internet resources. Call Olsen at (52)-5-554-7613 for more information, or e-mail her at: lise@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-growing-collection-of-federal-databases","slug":"1997-march-growing-collection-of-federal-databases","url":"https://dwillis.github.io/uplink/article/1997-march-growing-collection-of-federal-databases.html","year":1997,"month":"March","monthNum":3,"headline":"Growing collection of federal databases","kicker":"From the NICAR library","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering aviation, transportation, crime, finance, health care, and safety records, useful for investigative journalism and data analysis projects.","keywords":["databases","federal","journalism"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW Federal Aviation Administration's accidents and incidents, including major plane crashes since 1971.\n\nNEW NASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\n\nA monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\n\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\n\nCoast Guard boating accidents, 1969-1994.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\n\nFederal procurement data, 1992-1995, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1995.\n\nNEW National Bridge Inventory System data, includes inspection grades. 1994-1995\n\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1995 data.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nThe National Inventory of Dams. 1991-1995.\n\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\n\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-march-bits-bytes-and-barks","slug":"1997-march-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-march-bits-bytes-and-barks.html","year":1997,"month":"March","monthNum":3,"headline":"Bits, Bytes and Barks","kicker":"Goodbye Nashville, Hello Phoenix","author_name":"","author_title":"","affiliation":"","summary":"The article announces an upcoming IRE national conference in Phoenix and lists job opportunities in computer-assisted reporting at the Associated Press and The Baltimore Sun.","keywords":["conference","journalism","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["SQL Server","Access","Excel","FoxPro","Internet","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nGoodbye Nashville, Hello Phoenix\n\nThere will be no trips to the Grand Ole Opry like at NashCAR, NICAR's national conference, and your chances of seeing Dolly Parton are slim, but the national IRE conference in Phoenix does promise a lot of surprises. And, it's just around the corner. The conference will be June 12-15 at The Arizona Biltmore Hotel. For more information or to register, call IRE at (573) 882-2042. Also, keep an eye on the IRE website (www.ire.org) for the latest details. For hotel reservations, call The Biltmore at (800) 950-0086.\n\nHelp Wanted\n\nThe Associated Press is looking for a director of computer-assisted reporting to lead its CAR programs and to train reporters and editors in AP bureaus worldwide. Applicants should have experience as an investigative reporter and/or editor and should have skills in the development and use of computer-assisted reporting tools including relational databases, database front ends, SQL Server, spreadsheets and the Internet.\n\nInterested journalists should e-mail Bob Port, special assignments editor, at bport@ap.org or Drew Sullivan, news data editor, at drew@ap.org.\n\nThe Baltimore Sun is offering a 90-day paid internship in computer-assisted reporting this summer. It is looking for a college student with strong journalistic skills and a familiarity with CAR tools and concepts. Experience with Microsoft Access, Excel and FoxPro is a plus.\n\nThe intern will work closely with the electronic news editor on a variety of CAR projects. The pay is $370 a week, and starting and ending dates can be flexible to accommodate school schedules.\n\nInterested college journalists should send resumes to Michael Himowitz, electronic news editor.\n\nHe can be reached via e-mail at mikeh@clark.net, or students can mail their resumes to Michael J. Himowitz, Electronic News Editor, The Baltimore Sun, 501 N. Calvert St., Baltimore, MD 21278.\n\nNICAR Net\n\nTo subscribe to IRE-L or NICAR-L, send e-mail to listproc@lists.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-lonely-hearts","slug":"1997-april-lonely-hearts","url":"https://dwillis.github.io/uplink/article/1997-april-lonely-hearts.html","year":1997,"month":"April","monthNum":4,"headline":"Lonely Hearts","kicker":"Statistics and Medicine","author_name":"Dave Davis and Joan Mazzolini","author_title":"journalists","affiliation":"The Plain Dealer","summary":"An investigation reveals that some hospital transplant programs secretly turned away donor organs matched to waiting patients without disclosing this to patients, who mistakenly believed the national organ shortage was responsible for their lengthy waits. The reporting used computer-assisted analysis to examine transplant program volumes and found that centers performing fewer transplants annually were more likely to reject organs for non-medical reasons.","keywords":["transplants","hospitals","investigation"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["FoxPro","SPSS","Medline"],"full_text":"Imagine you're so sick you can't walk from the bedroom to your kitchen and you gasp for every breath. Imagine you go to the doctor, and she tells you your heart is damaged, so damaged you'll need a transplant or you'll die within six months.\n\nThen imagine traveling to the hospital every week, even every day for tests blood tests, lung capacity tests, and every known heart test. Imagine you've spent thousands of dollars, and your insurer has spent tens of thousands of dollars, and that your family is praying you get on the waiting list and get a donor organ. Imagine you go through all of this, but there is really no chance you'll ever get a transplant. Your hospital has stopped performing them, but doctors didn't tell you or any of the other patients for fear you might go elsewhere.\n\nIt sounds horrible, but this is exactly what happened to patients awaiting heart transplants at University of Kansas Medical Center in Kansas City and William Beaumont Hospital in suburban Detroit.\n\nIn each case, doctors at the hospitals turned away for nonmedical reasons donor organs they were offered over the course of about a year. But they didn't tell patients.\n\nIn fact, they let patients believe it was the national organ shortage, and not their own stalled programs, that was causing the unending wait.\n\nIn a series published in February by The Plain Dealer, we used computer-assisted reporting techniques to examine in detail transplant programs that turned away organs that were matched to waiting patients. We also followed a 37-year-old Pennsylvania woman through the ordeal of a heart transplant, and entered waiting times into FoxPro to examine the great disparity in waiting times at different centers, and with the help of SPSS analyzed the relationship between the average annual number of transplants centers performed and their mortality rate.\n\nLooking for patterns\n\nVolume, or the average annual number of transplants a center performed, was one common thread which ran through each of the stories. For example, centers that performed a small number of transplants each year almost always had fewer resources in terms of medical staff experienced in transplantation. We found they tended to be the centers that would most often turn away donor organs for nonmedical reasons.\n\nWe were especially intrigued by this question of program volume because, for each type of major organ transplant, half the centers in the country had very small annual transplant numbers. For example, about half the centers in the country did fewer than nine heart transplants a year.\n\nResearching the question\n\nSince donor organs are scarce, we wondered if lives were being lost because a limited number of organs were being spread among too many transplant centers. It was this question, among others, that lured us into a statistical analysis.\n\nUsing the National Library of Medicine's Medline database we searched the medical literature online to see what had been published concerning transplant center volume and patient mortality. (You can get a Medline user ID online at http://igm.nlm.nih.gov or call 1-800-638-8480.)\n\nWe didn't find much, just one article from the Journal of the American Medical Association in which the authors found that heart transplant centers doing fewer than nine transplants a year had significantly higher one-year mortality rate.\n\nHoning in with stats\n\nWith this information in hand, we sought help from Phil Meyer and John Bare at the University of North Carolina-Chapel Hill. It was Bare who did the heavy statistical lifting, while Meyer watched over our progress with the big picture in mind: Credible results. We decided on a two-prong analysis. First, we wanted to determine whether low-volume transplant centers had a higher observed one-year mortality rate than high-volume centers. If so, we wanted to find out if the increased rate of death could be explained by differences in patient risk factors and donor characteristics, or whether it was partly attributable to transplant center volume.\n\nThe first part was straightforward data analysis. The second part, however, was trickier. It required using a statistical method known as logistic regression, which is commonly used in medical research to adjust data for certain patient risks.\n\nWe had patient-level data on transplant donors and recipients - one record for each of the 55,000 transplants that had been performed from 1987 through 1991. The records also contained a key field identifying the transplant center.\n\nFor each organ type, we computed the average annual number of transplants the centers performed. We then labeled each center either \"high volume\" or \"low volume\" depending upon whether they were above or below the average volume.\n\nWe found that the observed 12-month mortality rate was higher at all times for low-volume centers. In other words, patients who underwent a transplant at a low-volume center had a greater chance of dying in the first year following the transplant.\n\nThis held true for all six major types of organs. Next, using logistic regression, we adjusted for medical differences in donors and recipients and found that center volume was a significant predictor of mortality at one-year.\n\nThough she did not use CAR, Cheryl Thompson of the Kansas City Star uncovered similarly troubling trends at KUMC in a series of investigative stories published in May of 1995. That series, which revealed the hospital had stopped performing transplants and was turning away healthy hearts while continually adding patients to their waiting list, is available in the IRE Resource center.\n\nFor a copy of the volume analysis technical documentation or a reprint of the entire transplant series, call Davis at 1-800-688-4802 or send email to ddavis00@aol.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-getting-back-to-basics","slug":"1997-april-getting-back-to-basics","url":"https://dwillis.github.io/uplink/article/1997-april-getting-back-to-basics.html","year":1997,"month":"April","monthNum":4,"headline":"Getting Back to Basics","kicker":"For the Novice","author_name":"Neill A. Borowski","author_title":"","affiliation":"The Philadelphia Inquirer","summary":"This guide explains how to properly analyze population data in computer-assisted reporting, emphasizing the importance of visualizing data through plots and calculating basic descriptive statistics like mean, median, mode, and range rather than over-applying complex statistical tests.","keywords":["statistics","analysis","data"],"topics":["Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"At the outset, we have to agree on something: Much of the time we really aren't using \"statistics.\" We're mostly in the business of measuring populations. This distinction is important. Many statistical tools are designed to help the researcher use samples to make inferences about the population. In computer-assisted reporting and analysis, we usually have the population in front of us. There's no guessing what the full universe of data looks like. We have it all: Every teacher's salary in our state, every fatal traffic accident in the nation, or every sentence imposed by a judge. By over-testing our data, we may be complicating and confusing our analysis. Avoid using statistical tools \"because they're there.\" Always keep it simple. With that said, here are some fundamental steps to measure your data. Let's say our database is all of the teacher salaries in a state:\n\nAlways, always plot your data\n\nThis probably should be your first step after making sure the database is clean. Plot a histogram (for a description of a histogram, see the page margin to the right). It will show how your data are distributed. A normal distribution will be mound-shaped. Look for oddities in your data. Are there a large number of salaries far off to the right of your histogram? This could be a highly paid district.\n\nCalculate the 'basics'\n\nThese are the range, arithmetic mean, the median and the mode (remember \"ramm\"). Lets say we're looking at salaries. The mean — often called the average — is the sum of the salaries divided by their count. The median is the salary in the middle after the salaries are ranked (when there is an even count of salaries, there will be two salaries in the middle; the mean of these two salaries is the median). The mode is the salary that appears most often. The range is the highest salary minus the lowest salary.\n\nRank, rank and compare\n\nWho gets the most? Group the districts and calculate their averages and medians. Which districts pay the most? How do these compare with the state average?\n\nUse the total experience (in years) of each teacher to calculate the salary per experience year. Rank these. Do you have the salary from five years ago? Calculate the percentage change and rank the change. What district salaries changed the most? The least? For me, one of the handiest tools in software is the =rank function in MS Excel. It will tell you where things stand in a flash.\n\nLook at the spread\n\nTwo sets of data could have the same means, but would look far different when they are plotted. The average may be $45,000 for two sets of salaries. However, one may have a low salary of $30,000 and a high salary of $65,000. The other data set may have a low of $8,000 and a high of $90,000.\n\nThe range will help. But the standard deviation is a better tool to help with this. Keep in mind that there are two standard deviations — the sample and the population standard deviations. (In Excel, the sample standard deviation function is =stdev and the population standard deviation function is =stdevp).\n\nThe standard deviation offers a quick view of how the data are distributed around the mean. Rule of thumb: The mean plus and minus one standard deviation includes about 68% of the data and the mean plus and minus two standard deviation includes about 95% of the data. What are the extreme values or outliers? Check out how many salaries are beyond three standard deviations. How many go beyond two standard deviation? These may be newsworthy.\n\nStudy the scatterplot\n\nWe're not explaining regression analysis in this article (regression looks at the impact one variable, such as experience level, has on another variable, such as salaries). However, you can quickly look at the \"big picture\" through a scatterplot.\n\nThe \"x\" (horizontal) axis is the independent variable — in this case, total years of experience for each teacher. The \"y\" (vertical) axis is the dependent variable — the salary of each teacher. A scatterplot will show you the regression line and the data points scattered about the line. The points farthest from the line are outliers.\n\nA regression line in a scatterplot of teacher salaries would be upward sloping — it would go from the lower left (low experience/low salary) to the upper right (high experience/high salary). If there are points in the upper-left quadrant of your plot far from the line, it might indicate teachers who are paid a lot with very little experience. And if there are points in the lower-right quadrant, it might indicate teachers with a lot of experience but at low pay.\n\nNeill Borowski can be reached at (609) 779-3884, or send email to neill.borowski@phillynews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-statistics-resources","slug":"1997-april-statistics-resources","url":"https://dwillis.github.io/uplink/article/1997-april-statistics-resources.html","year":1997,"month":"April","monthNum":4,"headline":"Statistics Resources","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Nora Paul's column highlights internet resources for finding statistics, while journalists on the NICAR-L listserv recommend various statistics books and tools useful for reporters working with data and numbers.","keywords":["statistics","journalism","resources"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["SPSS","Data Desk"],"full_text":"Statistics Resources\n\nA trip to the bookstore\n\nNora Paul has dedicated this month's On the Internet (page 10) column to finding data sets and statistics guides on the Internet. Be sure to check it out to complete your statistics resource guide.\n\nMany of these resources can be ordered online through the Ernie Pyles Memorial Bookstore on the web at http://www.duff.net/ernie/. For a non-framed version go to http://www.duff.net/ernie/stats.html. The site also contains numerous other journalism-related books for order.\n\nRecently on the NICAR-L listserv, a series of messages arose pertaining to the subject of statistics resources for reporters. Many of the list members shared their knowledge and experiences with certain texts. Following are some of those books, along with members' comments and how to email them:\n\nThe SPSS Guide to Statistical Analysis by Maria Norusis may be the best primer around. My two additional references are: PDQ Statistics by Norman and Streiner, and Statistics: A Spectator Sport by Richard Jaeger. Both attempt to make the link between this subject and the layman, or at least the lay nerd. Ford Fessenden, Newsday fordfess@newsday.com\n\nThe Jessica Utts text, Seeing Through Statistics is the perfect book for CAR types. It teaches you why to use stats, without much math. This book is clearly written, including lots of examples (many taken from news media). It will work well for those who want to learn this on their own, outside of a classroom setting.\n\nI also recommend a program called Data Desk. It's built around the idea of \"exploratory data\" analysis, which uses some slick data visualization tools such as rotating 3-d plots, brushing, color ranges, et al. Stephen K. Doig steve.doig@asu.edu\n\nFor the lighter side of statistical reading, try these:\n\nHow to Lie With Statistics, the classic little book first published in 1954, by Darrell Huff. I've still seen it in college bookstores. The pictures alone are worth the price.\n\n200% of Nothing: From Percentage Pumping to Irrational Ratios, An Eye-Opening Tour through the Twists and Turns of Math Abuse and Innumeracy, by A.K. Dewdney, John Wiley and Sons. This one isn't really statistics per se, but it's good reading for reporters who are finding themselves working with numbers of any kind more often. Sarah Cohen, NICAR training director sarah@nicar.org\n\nI like Victor Cohn's News & Numbers. Cohn was a Washington Post science and medical reporter. Much of the book is devoted to deciphering statistical claims by scientists, doctors, environmentalists and others in the health arena, but he also addresses basic concepts too like variance, rates, significance, and so forth. Very readable. Rick Linsk, Asbury Park Press linsk@app.com\n\nPhilip Meyer's book The New Precision Journalism is an excellent resource. It has a statistics section, and deals with poll and survey design, data collection and analysis.\n\nBill Doskoch bdoskoch@compusmart.ab.ca\n\nIt never hurts to collect as many useful stats books as possible. Some specific suggestions:\n\nStatistical Concepts: A Basic Program by Brown, Amos and Mink. You'll probably have to go to a good used bookstore for this one. It does a terrific job of teaching basic stats. It starts with explaining terms such as variable, frequency, distribution, etc., in definitions that go a sentence or two.\n\nDictionary of Statistics and Methodology, by W. Paul Vogt., from Sage Publications.\nThe Practice of Social Research, by Earl Babbie is a good introductory methods textbook. It goes through theory/research, measurement, operationalization, etc.\nStatistical Reasoning for the Behavioral Sciences, by Richard Shavelson is another textbook used in graduate stats introductory courses.\nPolitical Science Research Methods, by Johnson and Joslyn, is a book someone passed along to me. The writing is generally clear, and the book is short on Greek letters. I always like to find stats books that explain things in sentences, not just equations, and this one does that.\nUnderstanding Political Variables, by Buchanan, is a book I don't use a lot, but it's generally readable.\n\nBasic Statistics: An Introductory Survey for Students of Mathematics, Social Sciences, and Business, by Parl. Again, unless there's a new edition, check the dusty shelves of a local used bookstore. John Bare jbare@interpath.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-power-to-the-people","slug":"1997-april-power-to-the-people","url":"https://dwillis.github.io/uplink/article/1997-april-power-to-the-people.html","year":1997,"month":"April","monthNum":4,"headline":"Power to the people","kicker":"CAR Trips: First ventures","author_name":"Janet Roberts","author_title":"Computer Resources Editor","affiliation":"The Morning Star","summary":"A North Carolina newspaper used computer databases and Excel to identify and interview voters in specific precincts to gain deeper insight into a contentious school board race involving disputes over redistricting and segregation, moving beyond traditional door-knocking journalism.","keywords":["journalism","database","voters"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Excel","Paradox"],"full_text":"When incumbent Republican school board member slugged the husband of another GOP incumbent in the parking lot after a candidates' forum, we knew last May's Republican primary election was like none we had seen before.\n\nOne faction of the New Hanover County Republican Party was calling for a redistricting plan based on neighborhood schools. So what if that would mean a return to segregation and the loss of federal money for our school system?\n\nThe county's more mainstream Republicans said they wanted neighborhood schools, too, but they weren't willing to throw out 20 years of civil rights gains to achieve them.\n\nThe Morning Star's news columns were full of daily stories about the debate. But none of those stories shed much light on the key question: How was the fight playing out with voters?\n\nOften, when newspapers want the answer to that question, they turn to old-fashioned shoe-leather journalism. You knock on doors.\n\nFor a small newspaper, the problem with that approach is the time it takes. You knock on lots of doors and you don't always find registered voters or people who are giving the election much thought. The crapshoot doesn't always result in an enlightening story.\n\nHere's where the computer came in. We thought we could use it to take some of the chance out of the process.\n\nWho we found\n\nHow did it work?\n\nIn four days' time, we found and interviewed a few dozen voters who had been following the school board race closely and had strong opinions about which side was right.\n\nWe interviewed and photographed one of two black Republicans registered in one of our target precincts. Imagine the likelihood of finding him had we just gone out randomly knocking on doors.\n\nThe story got beyond the candidates' rhetoric and painted the issues in real terms, telling of real parents with real kids in school. Their opinions of what course the school system should take likely helped other voters decide how they wanted to vote. We gave it showcase play on our Sunday cover, two days before the primary.\n\nHow we did it\n\nThis was by no means a scientific process. Rather, we used the computer to help sharpen our natural reporting instincts.\n\nExcel helped us zero in on precincts where we wanted to interview people. Paradox pinpointed voters who would make good interviews.\n\nWe decided to look for Republican precincts where voters tend to be independent-minded: They don't just flip the Republican lever; they sometimes vote for Democrats. Our theory was that these independent-minded voters probably give their decisions a lot of thought, were likely following the party dispute closely and would help turn the race for one faction or another. We also wanted precincts with a lot at stake, where kids would have to be bused to maintain racial balance in the schools.\n\nUsing simple sorts in Excel, we zeroed in on a handful of precincts that matched our criteria. We sorted first by demographics, finding the Republican-dominated precincts. Then we looked for voting trends, finding one precinct, for example, where Republicans had helped a black Democrat win the last general election.\n\nFrom the few interesting precincts we found, we decided to focus on three - a blue-collar neighborhood, a white upper-class neighborhood, and a suburb where a lot of kids are bused. This is where Paradox came in. We had a database of all registered voters in New Hanover County. It lists names, addresses, party affiliation, date of birth, sex, race. Even more helpful to our story: It tells if they voted in the past several elections and even lists some telephone numbers.\n\nFor each of the three precincts, we asked Paradox to give us a list of all the registered Republicans who had voted in the '94 primary. That eliminated the apathetic voters from the lists and gave us a better chance of landing decent interviews. I printed out these lists, and reporters Scott Whisnant, Tricia Vance and Phil Hervey took it from there, blitzing on the phones to conduct the interviews.\n\nThe follow-up\n\nWe used the same technique the day after the election for our post-primary analysis. We used Excel to find the swing precincts, and then culled lists of voters from our voter registration database in Paradox. This all resulted in a story where voters explained the outcome of the race rather than tired pundits.\n\nWhat I liked most about these stories is that they relied heavily on number crunching and data but were vastly devoid of numbers. Janet Roberts can be reached at (910) 343-2389, or send e-mail to starnews@wilmington.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-correction","slug":"1997-april-correction","url":"https://dwillis.github.io/uplink/article/1997-april-correction.html","year":1997,"month":"April","monthNum":4,"headline":"Correction","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A correction is provided for the Gini coefficient formula used in February's lead story, detailing the proper mathematical calculation with cumulative proportions of households and income.","keywords":["Gini","formula","correction"],"topics":["Data & Databases"],"technologies":[],"full_text":"There was an error in the formula for February's lead story about the Gini coefficient.\n\nThe correct formula is:\n\nGini coefficient=1 - SUM ( (Xi-Xj) * (Yi+Yj))\n\nwhere:\nX is the cumulative proportion of households;\nY is the cumulative proportion of income;\ni is a particular income category;\nj is i-1, or the preceding income category.\n*Xi, Xj, Yi, Yj must equal one or less","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-stats-from-the-road","slug":"1997-april-stats-from-the-road","url":"https://dwillis.github.io/uplink/article/1997-april-stats-from-the-road.html","year":1997,"month":"April","monthNum":4,"headline":"Stats from the road","kicker":"New monthly column","author_name":"Sarah Cohen","author_title":"Training Director","affiliation":"NICAR","summary":"Journalists increasingly use statistics and data analysis tools like crosstabs to report stories based on comprehensive records rather than surveys, requiring different analytical approaches than traditional statistics courses teach.","keywords":["statistics","crosstabs","journalism"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["SPSS","Excel"],"full_text":"statistics: numerical data assembled and classified so as to present significant information Webster's New World Dictionary\n\nWhen scores of otherwise street-wise reporters show up at conference panels with titles like \"significance testing\" and \"regression techniques\", it's clear that something is happening within the world of journalism.\n\nAnd when an Associated Press bureau chief actually asks for a lecture on the use of numbers in the newsroom, it's yet another sign that numbers - yes, statistics - are becoming an important element in the beat reporter's toolbox.\n\nMost statistics courses and books begin with a simple premise that is just not true for most reporters. They assume the bulk of our work comes from surveys, not from comprehensive reports that are collected from everyone.\n\nWe don't take a sample of hospital discharges; we look at all 40 million of them. Instead of surveying borrowers to find out if they feel discriminated against, reporters usually look to existing reports that compile information on all loan applications in their areas. In other words, instead of designing the perfect survey, reporters must decide whether to attempt a study at all, or if they should fight for records that may be of use.\n\nWe also need some simple measures that don't bog down our readers or our stories. Sure, experts will claim there's no analysis that will compensate for all the flaws in medical, education, or lending data. Fair enough. But our job is to use present information, not hide it, even when it's somewhat flawed. Our challenge is to decide when it's fatally flawed, or how to present it most fairly.\n\nThis month I'll focus on one of the tools that has become a staple in journalism - the humble crosstab. Crosstabs can reveal many relationships without clouding them through complicated formulas.\n\nHow? By giving you an apples-to-apples comparison and by providing you with information that conforms to the way reporters think - in neat categories like high and low, big and small, yes and no.\n\nA crosstab will do the job quickly and cleanly. We're going to use Philip Meyer's rule for crosstabs to get started: Put the \"independent variable\" in your crosstab as a column, and put the \"dependent variable\" as a row. Now generate the percents of the column, and you have a neat little table that you can describe easily to readers, editors and even your next-door neighbors.\n\nStep 1: Figure out your categories\n\nSome categories are natural: race, sex or vote. Others require a little work. One common category is high- versus low-income. You can make this sensitive to your area or your numbers by using the quartiles in your data as cutoffs for your categories. Another approach is to split detailed categories, like those found in Census data, into natural groups. An example is the highest level of school people in an area completed: No high school, high school, some college, college, graduate school. Now you have six groups where you began with 20 or so possible values for the number of years of school completed.\n\nMake sure every category is included in your analysis. This may mean doing a little arithmetic, or thinking of your data in a new way.\n\nHow can you get these groups? If you have SPSS, it's easy - just run Transform, Ranks, then choose quartiles instead of rank. It will give you a 1 for the lowest quartile, through 4 for the highest, as a new variable.\n\nIf you're using Excel, it takes a few more steps. The simplest way is simply to sort the values and assign them in order. You can also use the PERCENTRANK function, and round it to the next lowest quartile by using the FLOOR function. This won't be exact, but it will be close.\n\nStep 2: Decide which field, or variable, is independent, and which is dependent.\n\nOne clue is to look for which one came first in time. The choice of insurance (or the lack of it) probably came before the birth of your newborn. The ethnic group membership (black, white or Hispanic) obviously came before income. And the income probably came before the decision to apply for a mortgage. When you have this kind of obvious sequence, the later event could depend on the earlier event. It's dependent. Remember, statistics can't tell you this. So it involves some thinking before you get started.\n\nStep 3: Produce the crosstab (in SPSS) or a PivotTable (in Excel)\n\nThis will count up the number of people in each cell of a square. If you've set up your crosstab with the independent variable in columns, then you just need to ask for the column percents in a crosstab or PivotTable. In SPSS, the option is under the \"Cell Statistics\" button. In Excel, it's under \"Options\" when you double-click on the data field. Here is an example of a crosstab, using an Excel PivotTable, for 2,800 mortgage applications in 1995 from high-income home-buyers in Nashville, Tenn.\n\nIt doesn't take a lot of sophistication to see that the rate of denials for black applicants in this high-income group is about three times higher than the rate for white applicants. That's because you simply have to glance across the line you care about to see the difference, adjusted for the number of applications from each group.\n\nIs this legitimate? Did it happen by chance?\n\nWe'll tackle the use of significance testing in another column. But the short answer is: It happened. For the high-income applicants in Nashville in 1995, it may not matter whether it was a fluke. We haven't tried to make anything bigger of this analysis than it is. It's a description of the facts for that year and that place, not an indication of something bigger.\n\nStep 4 (or is it step 1?): Don't leave your brain behind\n\nYou can make all kinds of comparisons in tables like this, but please don't leave behind what you already know. For instance, think about the relationship between income and education. If you test it, you'll probably find big news: People with less education have higher incomes! The reason is the \"lurking variable\" that so many people forget about. In this case, it's age. People reach their prime earning power in their 40s and 50s. But people are also becoming more educated.\n\nSo just like we had to control this analysis for income, we also have to control other analyses for items like age, education, poverty or a whole rash of other factors. That's why this last step might easily be your first: If you don't know what you're looking for, then read the academic literature, talk to other reporters who have covered the issue and talk with local experts before you even start.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-foxpro-wrote-me-a-letter","slug":"1997-april-foxpro-wrote-me-a-letter","url":"https://dwillis.github.io/uplink/article/1997-april-foxpro-wrote-me-a-letter.html","year":1997,"month":"April","monthNum":4,"headline":"FoxPro wrote me a letter","kicker":"Lonely Days are Gone","author_name":"Jo Craven","author_title":"NICAR staff","affiliation":"NICAR","summary":"A journalist demonstrates how to use FoxPro database software to automate the creation of 133 personalized form letters with different addresses, saving hours of manual typing by using a scanloop program with the TEXT/ENDTEXT function.","keywords":["automation","database","programming"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["FoxPro","Word"],"full_text":"Why work if you can get a computer to do the job for you?\n\nRecently, when I needed to send a form letter to 133 different businesses, I thought of NICAR data guru Richard Mullins, who teaches this ethic. I could sit and type 133 different addresses on top of 133 different copies of the same form letter - or I could get a computer to do it for me. I decided to let the computer do the job.\n\nHere's what I had available: a FoxPro database that included, among other things, the names and addresses of the businesses that I wanted to contact.\n\nI knew I could write a scanloop in FoxPro to pull out the addresses. And I found out that I could use the TEXT/ENDTEXT function to insert a text block - in this case, my form letter - after each scanned address. Later, I found I could get FoxPro to use the same database to print out mailing labels. What would have taken me a couple of hours, FoxPro did in minutes. (Incidentally, you could accomplish this same task with Microsoft Word's Mail Merge without writing a FoxPro program).\n\nTo create the FoxPro program, which I called \"letters,\" in the command window, I typed:\n\nmodify command letters\n\nIn the .prg, or program, window that appeared, I wrote the following program (an explanation follows):\n\nCLEAR CLEAR ALL CLOSE ALL SET CENTURY ON SET TALK OFF SET STATUS BAR OFF USE C:\\business. files = VI \" SCAN files = alltrim (str (business.id)) SET ALTERNATE TO \"c:\\' + files SET ALTERNATE ON ?alltrim (cmonth (date + + alltrim (str (day (date + + alltrim (str (year (date )\n\n?name ?address ?alltrim \"+state +\" \"+zip ?\" \"\n\nTEXT\n\nTo Whom It May Concern:\n\nThis is where I typed my letter.\n\nENDTEXT CLOSE ALTERNATE ENDSCAN SET TALK ON SET STATUS BAR ON\n\nSincerely, Jo Craven\n\nMy database, business.dbf, had 133 records. This program uses an alias called \"files\" to create 133 different text files, each containing the form letter topped with the different business addresses.\n\nTo make this work, I needed each record to have a unique ID of no more than eight characters. My data did not have such a field, so I created one using the recno() function: I modified the database by adding a field called \"id\" and told FoxPro to:\n\nreplace all id with recno\n\nThis numbered each record, providing the necessary unique ID. Here is the program again, including explanations of its components:\n\nCLEAR CLEAR ALL CLOSE ALL\nThese three lines clear the slate, making sure that no other files are open when the program starts. They are standard in any FoxPro program.\nSET CENTURY ON\nThis sets the dateline that starts each letter.\nSET TALK OFF SET STATUS BAR OFF\nThese turn off FoxPro's bells and whistles in order to make the program run faster.\n\nUSE C:\\business.dbf\n\nThis tells the computer which database to use - the one with the business names and addresses\nfiles =\n\nThis reveals that an alias called \"files\" will be used. It is defined and plugged into the program later. This line can be omitted; including it is a matter of form.\nSCAN\nThis begins the scanloop. In FoxPro, a scanloop tells the computer to look at every record in a database and perform whatever functions are specified within the scanloop - in this case, setting up separate text files, typing in the various names and addresses from the database, and inserting the text block written within the scanloop.\nfiles = alltrim (business.id)\nThis defines the alias \"files.\" It says that, for this program, \"files\" equals \"id,\" which is the field in business.dbf containing the unique IDs. \"Alltrim\" makes sure blank spaces don't confuse the machine. And \"str\" tells the computer to consider the numeric, unique IDs as character data.\nSET ALTERNATE TO \"c:\\\" + files SET ALTERNATE ON\nThis is where the individual text files are created. \"Set Alternate To\" tells the computer that everything it does within this scanloop should be copied to another file on the c:\\ drive. \"Files\" has already been defined as \"id,\" so for every unique ID, a new text file will be created on the c:\\ drive. \"Set Alternate On\" tells the computer to act on the \"Set Alternate To\" command. \"Set Alternate To\" won't work without \"Set Alternate On.\"\n?alltrim (cmonth (date + + alltrim (str (day (date ( ) ) + + alltrim (str (year (date\nThis makes the date. If your computer's date and time are set correctly, this command will produce the current date. The question mark (?) tells FoxPro to type the information.\n?\" \"\nThis produces a blank line between the date and the address.\n?name ?address ?alltrim (city) \"+state \"+zip ?\" \"\nThis tells FoxPro to pull out the information in the fields \"name,\" \"address,\" \"city,\" \"state,\" and \"zip,\" inserting the comma where appropriate, and spacing between the state and ZIP code.\n\n(Remember, we already told the computer which database to use - business.dbf) The last line leaves a blank space between the address and the letter below.\n\nTEXT To Whom It May Concern:\n\nThis is where I typed my letter. Your letter can be as long, or short, as you like.\n\nSincerely, Jo Craven\n\nENDTEXT\n\nTEXT tells FoxPro a text block is coming and should be inserted in the scanloop after the address. Anything can be typed into the text block. A tip: Use carriage returns after each line to ensure proper line breaks. ENDTEXT must be typed at the end of the text block.\n\nCLOSE ALTERNATE\n\nRemember, FoxPro was typing everything from the scanloop into a text file named after the unique ID. This closes that file, so that when the scanloop begins again, FoxPro will name the next alternate file after the next unique ID and thus create the different text files. (When I ran this program, I ended up with 133 text files with names called 1.txt, 2.txt, on through 133.txt)\n\nENDSCAN\n\nThis tells FoxPro the scan is done, and should be started anew on the next record in the database, until there are no more records left.\nSET TALK ON SET STATUS BAR ON\nThese turn the bells and whistles back on. (Remember, I turned them off in the beginning to make the program run faster.)\n\nAfter typing this program into the .prg window, to save and exit, type ctrl W. To run the program, in the FoxPro command window, type:\n\ndo letters\n\nMaking mailing labels for envelopes is even easier because FoxPro offers a wizard that will draw the addresses from the same database. From the Run pulldown menu, choose Wizard and then \"labels\" and follow the steps. Tip: Standard mailing labels are size 5160.\n\nJo Craven can be reached at (573) 882-0684, or send e-mail to jo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-surfing-for-stats","slug":"1997-april-surfing-for-stats","url":"https://dwillis.github.io/uplink/article/1997-april-surfing-for-stats.html","year":1997,"month":"April","monthNum":4,"headline":"Surfing for stats","kicker":"On the Internet","author_name":"Nora Paul","author_title":"","affiliation":"The Poynter Institute","summary":"This article guides journalists on finding, interpreting, and using statistics responsibly to provide context and depth to their reporting, while warning against the misuse of data to distort truth.","keywords":["statistics","reporting","databases"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"\"He uses statistics as a drunken man uses lamp-posts - for support rather than illumination.\" -Andrew Lang\n\nNow, you don't want to be accused of that, right? Learning how to correctly find, interpret, and use statistics for illuminated reporting is one of the most challenging of all journalistic tasks.\n\nThe creative and proper use of databases and statistics can help us move, as godfather of precision journalism, Phil Meyer, says, \"from event reporting, to putting the event into context.\"\n\nEvent coverage says, \"There was a car accident at 3rd and Maple.\" Context journalism will go on to say, \"This is the thirteenth accident at that intersection in the past six months.\"\n\nThorough context journalism will go even further to say, \"This is the third most dangerous intersection in the city.\" And investigative journalism will find out why. Statistics and analysis of databases will be the fuel for this kind of journalism.\n\nBut there is a danger. As British Prime Minister Benjamin Disraeli, in a quote attributed to him in Mark Twain's memoirs, said, there are three kinds of lies, \"lies, damn lies and statistics.\" The use of statistics for context setting, \"truth-telling\" and reality checking in this age of easily accessed data, and misinformation, is a challenge and an opportunity for reporters. This column will give you some places to hunt for statistics on the Internet, some guides you can read on how to interpret statistics and some resources for locating experts who can help you understand the numbers you crunch yourself.\n\nLooking for statistics on the net\n\nI hate reinventing the wheel, and there has already been a great one created on this topic by information consultant Aggi Raeder (aggi@netcom.com) in the September 1996 (page 16 Searcher magazine at http://www.infotoday.com/searcher/srchrtop.htm).\n\nShe lists meta-sites, and then breaks down statistics sources into the subject categories of Agriculture, Banking, Business, Crime, Economics, Education, Energy, Government, Health, International, Local Government, Politics, and Population. The article itself isn't on the net, but the information about how to order reprints of the article is.\n\nOne of the sites she lists at the top as an example of a great meta-site compilation of the sites which are compilations of sites) for statistical information are Larry Schankman's pages. Schankman, a librarian at Mansfield University's Pennsylvania Library, has two pages, both of which should be given medals for organization, thoroughness and description of the resources he links to.\n\nBusiness and Economic Numeric Data http://www.clark.net/pub/lschank/web/ecostats.html\nFrom health care expenditures, to housing starts, to lottery funds by State, this page lists all manner of statistical resources from credible sources.\nBe especially sure to check out his \"Library Guides Series: Statistical Resources\" at http://www.clark.net/pub/lschank/web/govstats.html\nThe site is broken down by categories to help you find web sites and books and reports that are not on the web.\nElectronic Data Dissemination of Census Publications\n\nhttp://www.clark.net/pub/lschank/web/cendoc.html\n\nHe takes the mother of all statistics sources, the U.S. Census Bureau, and links to its resources, with clear explanations and descriptions of what you will find.\n\nThese pages are fascinating to read. Just looking at the types of statistics that are gathered can get story ideas flowing. Check these pages out, bookmark them, then send a note of thanks to Schankman (lschank@clark.net) for his great work.\n\nGuides for using statistics\n\nStatistics Every Writer Should Know at http://nilesonline.com/stats/ is the website every writer should bookmark. Robert Niles, Rocky Mountain News webmaster and former reporter, has put together the resource guide for understanding statistics using the language and framework of reporting. Read it all and you'll be much further along in understanding the basics.\n\nGot those numbers, but don't know what they really mean? Subject specialists spend years learning how to interpret those numbers. You need some help as you move from one subject to another in the course of your reporting. Check into the Reporting Tools area of The Foundation for American Communications' website for journalists at http://www.facsnet.org/report_tools/main.html\n\nFACS Guides and Primers, and News and Numbers are great backgrounders on understanding the numbers behind the news. Click on the \"Covering Business\" link and get an explanation of reading financial statements and doing financial analysis.\n\n*Research Methods Resources from the College of Education at Arizona State University at http://seamonkey.ed.asu.edu/~behrens/links to an incredible array of resources about statistical analysis.\n\nThe Statistical Instruction Internet Palette (SIIP) includes the Data Gallery with samples of data displayed in different ways and the Equation Gallery that gives you various formulas for statistics (i.e. - click on \"variance\" and you will get a formula, plus an in-depth explanation of what it means).\n\nExperts explain the numbers\n\nSometimes the numbers aren't enough. You have to understand their importance.\n\nSt. Petersburg Times researcher Kitty Bennett has pulled together an experts and sources directories, topped with a link to ProfNet's Expert Database at http://time.vyne.com/profnet/ped/eg.acgi\n\nShe has found university and think tank expert directories and listed them for you on the Special Libraries Association/NewsDiv website at http://sunsite.unc.edu/slanews/internet/experts.html Nora Paul can be reached at (813) 821-9494, or send email to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-ire-gets-joyce-grant","slug":"1997-april-ire-gets-joyce-grant","url":"https://dwillis.github.io/uplink/article/1997-april-ire-gets-joyce-grant.html","year":1997,"month":"April","monthNum":4,"headline":"IRE gets Joyce grant","kicker":"NICAR/IRE Notes","author_name":"Brant Houston","author_title":"Executive Director","affiliation":"IRE and NICAR","summary":"IRE received a $342,000 grant from the Joyce Foundation to create an online Campaign Finance Information Center that will serve as a repository of federal, state, and municipal campaign finance databases with training resources for journalists. The center will initially focus on the Great Lakes region and begin operations in early fall at the Missouri School of Journalism.","keywords":["campaign","journalism","database"],"topics":["Government & Politics","Data & Databases","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"IRE gets Joyce grant\n\nBy Brant Houston\nIRE Executive Director\n\nBy this time next year, there should be a new center on campaign financing open at the headquarters of Investigative Reporters and Editors and the National Institute for Computer-Assisted Reporting.\n\nIRE has just received the first half of a $342,000 two-year grant from the Joyce Foundation to create and operate the on-line Campaign Finance Information Center.\n\nThe center will serve as a repository of federal, state, and municipal databases on campaign contributions and spending. The center will also keep other related public records and offer training to journalists both on-line and at seminars.\n\nAlthough IRE Envisions a nationwide archive, the initial effort with the Joyce Foundation's funds will concentrate on the Great Lakes region, which includes the states of Illinois, Indiana, Michigan, Minnesota, Ohio, Iowa and Wisconsin.\n\nThe center will collect the databases of campaign finances and make them easy to use and search on the Internet. The center will not be in the business of doing data entry.\n\nIn addition to this electronic data repository the center will offer links to other pertinent Web sites, provide on-line tutorials to teach reporters how to access, analyze and work with campaign data, and publish a newsletter on the subject.\n\nAs usual, IRE and NICAR will coordinate its work with other campaign finance study groups to prevent duplication of efforts and to expand the number of resources available.\n\nIRE and NICAR expect the center to begin operations in the early fall once staff and equipment are in place in the offices at the Missouri School of Journalism.\n\nWhile the grant is a large one, the new center is an ambitious project and we're hoping that we may get additional funds.\n\nNICAR already has an extensive database library of government information, including campaign information that will become a part of the center's archives. The center hopes to collect databases of state and municipal campaign reports that have been created by news organizations and other groups.\n\nThe Joyce Foundation is a private foundation based in Chicago with assets of $650 million. Its Money and Politics Program supports efforts to reform the campaign finance system to reduce the influence of special interest and enhance the public's participation in elections.\n\nSome databases have already been created with the help of the Joyce Foundation.\n\nUp and coming\n\nWhile we gear up for the creation of the new center on campaign finance, we're working hard to keep our on-the-road training program up to date and useful.\n\nWe will continue to offer our traditional basic one to two-day regional seminars, but we plan to add more seminars on particular topics.\n\nLate this summer we hope to focus on education reporting and databases at a three-day seminar in Philadelphia.\n\nWe also plan to offer workshops in other areas such as medicine, business, and managing CAR.\n\nIn addition, you will see more seminars that combine workshops on better reporting techniques that IRE has traditionally offered with NICAR's hands-on training.\n\nIf you are interested in helping host one of these seminars or have ideas about workshops, we should do, please email brant@ire.org\n\nMore CAR stories in Resource Center\n\nIRE's recent contest attracted dozens of new CAR stories and each of those has been indexed and catalogued in IRE's Resource Center. You can search the indexes by going to www.ire.org and choosing the resource center pages. From there you can search the indexes for new and old CAR stories by key words.\n\nHandouts from our computer conferences are also available as abstracts and sometimes in pdf format, which can be easily downloaded and printed out.\n\nA complete set of the handouts, is available through the IRE resource center for $75, including shipping.\n\nAlso, to order a set of abstracts from any of NICAR's national computer conferences, call the resource center at (573) 882-3364 or email Bruce Moores at bruce@nicar.org\n\nBrant Houston can be reached at (573) 882-0684, or email him at brant@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-census-profiles-communities","slug":"1997-april-census-profiles-communities","url":"https://dwillis.github.io/uplink/article/1997-april-census-profiles-communities.html","year":1997,"month":"April","monthNum":4,"headline":"Census profiles communities","kicker":"Handout of the Month","author_name":"Paul Overberg","author_title":"","affiliation":"U.S.A. Today","summary":"The article highlights free government data sources from the 1990s that provide demographic information at the county level, including population estimates, migration patterns, and race/ethnicity breakdowns, which journalists can analyze to uncover meaningful demographic trends.","keywords":["census","migration","demographics"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":["FTP"],"full_text":"The 1990 Census remains a good reference, but more history than news. Cheer up! Here are several reliable government sources of 1990s demographic data on counties or even smaller areas. All of them are free or nominal, clean and electronic. On first glance, what they offer seems pretty obvious. The secret lies in analyzing and comparing the pieces to find patterns.\n\nYes, this data is estimated or incomplete. But remember: Only one person in seven got the long Census form, the source of most of what we write about the 1990 Census. In all cases below, read the documentation to understand the data's limits.\n\nAnnual county-level population estimates and components\nhttp://www.census.gov/population/www/estimates/countypop.html\n\nThe Census Bureau calculates these based on July 1 and releases the data the following March. Even better than getting the estimate is getting the underlying data used to make the estimate: births, deaths, elderly (65+), under 65, net domestic migration, net international immigration, federal citizen movement, group quarters population.\n\nOf these, the single most valuable is net domestic migration, especially when you line up several consecutive years. That's because it's the collective verdict of people voting with their feet. People usually don't move far enough to leave a county. When they do, often it's to go to college, retire or take a better job, meaning many of them won't come back.\n\nAnnual county-level estimates for race/Hispanic origin, sex, and age\n\nhttp://www.census.gov/population/www/estimates/co9094.html (Totals only)\n\nhttp://www.census.gov/population/estimates/county/casrh/ (Details by five-year age group)\n\nThe Census Bureau calculates these periodically on contract and releases them about a year after the estimate date. (Current data is 1990-94; 1990-96 data is due out in July).\n\nThese differ in methods and timing from the main estimate process above, so expect differences. You can use the race/Hispanic origin/sex totals for each state and county to watch for trends.\n\nThe same data, broken down by five-year age groups, lets you see changes in a county's age structure versus the state and nation.\n\nAnnual county-level migration files, with median household income\n\nFor county-to-county and most recent state-to-state files, call Emily Gross at the IRS Statistics of Income Division, (202) 874-0944. For previous years' state-to-state files, dial the division's BBS, (202) 874-9574. Or try the often-overloaded Fedworld address: ftp://ftp.fedworld.gov/pub/irs-soi/\n\nThe IRS uses the Social Security number on each tax return to check county of residence in consecutive years. For each county, it counts how many stayed put from one year to the next, and how many moved between counties. For each county-to-county flow, it reports returns (roughly, households), personal exemptions (roughly, persons), aggregate income and, since 1992-93, median household income.\n\nYou can match a county's inbound and outbound files to calculate net flows between nearby counties, but you can also examine the files separately for distinct patterns in where people come from and go to.\n\nAnnual County Business Patterns\nhttp://www.census.gov/epcd/cbp/view/cbpview.html\n\nThe Census Bureau bases these on a survey conducted in early March. They're released 15 months after the survey year. Data shows employment, payroll and number of establishments by industry. Some data on payroll and employment is masked to prevent disclosure in small categories. But the number of establishments isn't. Neither is their distribution by size.\n\nThis data is helpful in following regional trends by industry, especially in the hard-to-track services sector: clinics, churches, day-care centers, hair salons, etc. The data includes basic groups like retailing and construction and smaller groups like \"eating and drinking places.\"\n\nOther useful data\n\nAnnual estimates for towns and cities\nhttp://www.census.gov/population/www/estimates/citypop.html\n\nThese are \"places\" or \"minor civil subdivisions,\" in Census speak. Basically, they're any self-governing political entity smaller than a county. (\"Places\" covers all such things in 30 states. In 20 states in the Northeast and Midwest, a variety of local political units like townships, towns, boroughs and villages are self-governing, so the bureau lists them separately).\n\nEstimates are based on July 1 and released the following July. The most recent estimate was July 1, 1994. (The bureau skipped the 1995 release to focus on improving its methodology) These aren't as accurate as county estimates, but are helpful for watching patterns.\n\nAnnual county totals in various Social Security programs\n\nhttp://www.ssa.gov/statistics/oasdi94.html\n\nhttp://www.ssa.gov/statistics/ssi_st_cty_d95/\n\nhttp://www.ssa.gov/statistics/eedata93.html\n\n(For most current data on a few counties, call Social Security Administration at (410) 965-8904).\n\nThe Social Security Administration has placed several annual county-level statistical reports on the web. They're tabular, so download and import is simple. Currency varies.\n\nOther sources\n\nFinally, don't forget to check for customized analyses and research done by your state demographer and state or county planning department. Also, see if your marketing department pays for access to demographic data from a company like Claritas. It's often overkill for newsrooms and easy to misuse, but it can help portray customized or ZIP code areas, where between-Census data is rare.\n\nPaul Overberg can be reached at (703) 276 5427 or email him at 70534.3647@compuserve.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-get-your-training","slug":"1997-april-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-april-get-your-training.html","year":1997,"month":"April","monthNum":4,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR and affiliated organizations offer specialized computer-assisted reporting training through on-the-road sessions, bootcamps, and conferences for journalists to enhance their data analysis and investigative reporting skills.","keywords":["training","journalism","databases"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet","Internet"],"full_text":"NICAR and, where indicated, The Associated Press provide specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases.\n\nCost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below.\n\nNote: In the following list, \"Closed\" means the session is open only to members of the host organization.\n\nChicago, Ill., Chicago Tribune. April 30-May 2. Closed.\nCharleston, W.Va., Marshall University. May 13-16. Closed.\nDenmark, Sweden. May 26-June 6. Closed.\nReuters. May 29-30. Closed.\nReuters. June 26-27. Closed.\nWashington, D.C., Freedom Forum. July 14-15. Closed.\nTempe, Ariz. Reporting and Education seminar. July 25-26. Closed.\nMinneapolis, Minn. August 4. Closed.\nCharleston, W.Va. Aug. 19-22. Marshall University. Closed.\nMedill School of Journalism. Sept. 11-13. Open.\nNew York, N.Y. Sept. 26-27. Columbia University. Open.\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969. IRE National Conference, Phoenix, Ariz., - June 12-15.\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo.\n\nAs with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size.\nFor information, call Wendy Charron at (573) 882-0684.\nMay 4-9. Advanced Bootcamp concentrating on stats and maps at the University of North Carolina-Chapel Hill with Philip Meyer, author of The New Precision Journalism. The seminar is held in Chapel Hill. For details, call (573) 882-0684.\n-May 18-23. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684.\n-July 11-13. Mini Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684.\n.Aug. 10-15. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684.\n\nSouth of the Border\n\nPeriodistas de Investigacion (IRE-Mexico) offers computer-assisted reporting training in Spanish to journalists in Mexico and Latin America. PI, based in Mexico City, has developed a two-day workshop that covers the basics of investigative reporting and computer-assisted reporting, and a four-to-five day bootcamp that covers CAR intensively.\n\nThe workshops can be held in almost any city with prior notice. The courses cover using internet, spreadsheets, database managers, and generally electronic records, and uses examples from the Mexican and Latin American press.\n\nThere is one bootcamp scheduled in May:\nMexico City - May 14-18.\n\nAnnual membership to IRE-Mexico is $25 (or $200 pesos mexicanos) and is open to journalists or students of journalism living in Mexico and Latin America.\n\nInstructors are Lise Olsen, managing director of IRE-Mexico, and Pedro Enrique Armendares, assistant director. Both have more than seven years of experience in investigative reporting, Olsen at The Virginian-Pilot (Norfolk, Virginia) and Armendares at La Jornada (in Mexico City). Call Olsen at (52)-5-554-7613 for more information, send email to: lise@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-from-the-nicar-library","slug":"1997-april-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-april-from-the-nicar-library.html","year":1997,"month":"April","monthNum":4,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR maintains a comprehensive collection of federal government databases covering aviation, transportation, finance, health, crime, and safety data, available for investigative journalism and research purposes.","keywords":["databases","government","journalism"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["CD-ROM"],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW Federal Aviation Administration's accidents and incidents, including major plane crashes since 1971.\n\nNEW NASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\n\nA monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\n\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\n\nCoast Guard boating accidents, 1969-1994.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\n\nFederal procurement data, 1992-1995, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1995.\n\nNational Bridge Inventory System data, includes inspection grades. 1994-1995\n\nNEW FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1995 data.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nThe National Inventory of Dams. 1991-1995.\n\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\n\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information. For up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-help-wanted","slug":"1997-april-help-wanted","url":"https://dwillis.github.io/uplink/article/1997-april-help-wanted.html","year":1997,"month":"April","monthNum":4,"headline":"Help Wanted","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"","summary":"Two major newspapers are hiring technology-focused journalists: the Orange County Register seeks a technology reporter, while the Dayton Daily News is recruiting a computer-assisted reporter skilled in data analysis and FoxPro programming.","keywords":["journalism","technology","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["FoxPro","Internet"],"full_text":"The Orange County Register is in search of a technology reporter. Applicants for the position should submit a packet of current work, a resume and a one-page statement of philosophy to: John Doussard, 625 N. Grand Ave., Santa Ana, Calif. 92701. Telephone: (714) 953-2225. Fax: 714-565-3681.\nThe Dayton Daily News (160,000 daily; 220,000 Sunday) is looking for a computer-assisted reporter who knows FoxPro, can crunch data for daily and long-term computer stories, help other reporters on computer projects, pull information off the Internet and write a story when needed. This is an opportunity to work on a newspaper that is serious about computer-assisted journalism. The Daily News has won numerous national awards in the 1990s for computer projects, and was a Pulitzer finalist twice. If interested, send resume, cover letter, clips and references to John Erickson, News Manager, c/o Dayton Daily News, 45 S. Ludlow St., Dayton, Ohio, 45402. Email address: John_Erickson@CoxOhio.com Phone: 937-225-2266.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-nicar-net","slug":"1997-april-nicar-net","url":"https://dwillis.github.io/uplink/article/1997-april-nicar-net.html","year":1997,"month":"April","monthNum":4,"headline":"NICAR Net","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"Instructions for subscribing to NICAR and IRE listservs to connect with journalists discussing reporting techniques and professional development.","keywords":["listserve","journalists","community"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","Listserv"],"full_text":"Don't forget to keep up with NICAR on the Internet. Subscribe to our listserve and join in as reporters talk about how to do the job better. Email to LISTSERV@MIZZOU1.MISSOURI.EDU. In the message, on the first line, write: subscribe NICAR-L your name. To join IRE on the Internet, the instructions are the same except, on the first line, write: subscribe IRE-L your name.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-april-statistics-newsletter","slug":"1997-april-statistics-newsletter","url":"https://dwillis.github.io/uplink/article/1997-april-statistics-newsletter.html","year":1997,"month":"April","monthNum":4,"headline":"Statistics newsletter","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A newsletter from the CHANCE Course Project at Dartmouth examines how statistics are used and misused in journalism and publications, offering abstracts of recent articles and available through web and email subscription.","keywords":["statistics","newsletter","journalism"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","Email"],"full_text":"If you are interested in how we use (and sometimes misuse) statistics, there is a newsletter on the Internet that is good and fun. It's part of the CHANCE Course Project, which operates from Dartmouth and is supported by the National Science Foundation. The newsletter includes abstracts from recent newspaper and journal articles that have made use of statistics. It is published in electronic format and can on the web at http://www.geom.umn.edu/docs/education/chance/chance_news/news.html For an email subscription, drop an email message to jlsnell@dartmouth.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-disaster-on-deadline","slug":"1997-may-disaster-on-deadline","url":"https://dwillis.github.io/uplink/article/1997-may-disaster-on-deadline.html","year":1997,"month":"May","monthNum":5,"headline":"Disaster on Deadline","kicker":"Crash Test","author_name":"Ford Fessenden","author_title":"","affiliation":"Newsday","summary":"Newsday's newsroom mobilized quickly to cover the TWA Flight 800 crash. Reporters used computer-assisted reporting (CAR) to gather data from aviation databases, which helped give a fuller understanding of the plane's safety record. The coverage won a Pulitzer Prize for its depth and immediacy.","keywords":["TWA 800","aviation","Pulitzer"],"topics":["Public Safety","Investigative Methods","Journalism & Newsroom"],"technologies":["Internet"],"full_text":"The jumbo jet that crashed in a fireball and lay burning on the rolling surface of the Atlantic off Moriches on a July night last year quickly transformed Newsday's newsroom into a rolling thunder of organized pandemonium. Reporters were dispatched to various locations including beaches, docks, airports, and police stations. Newsday's coverage of TWA 800 owes a deep debt to the technology of computer-assisted reporting. Over the next few days and as the story stretched on into weeks and months, consulting the Internet, and querying safety databases became routine. The paper reported that the plane had no indications of trouble according to Federal Aviation Administration databases. TWA had an exemplary safety record. With a focus on the human tragedy, reporters compiled biographies of 100 victims for the paper shortly after the crash. Reporters used websites and databases to follow family stories and conspiracies surrounding the crash. Newsday's database team leveraged safety databases for leads. Computer-assisted reporting was a tool that transformed public understanding of aviation safety, showcasing hard, careful work by a staff of 55 reporters.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-may-probing-hud-woes","slug":"1997-may-probing-hud-woes","url":"https://dwillis.github.io/uplink/article/1997-may-probing-hud-woes.html","year":1997,"month":"May","monthNum":5,"headline":"Probing HUD woes","kicker":"Troubled Housing","author_name":"Deborah Nelson","author_title":"Reporter","affiliation":"The Seattle Times","summary":"The Seattle Times investigation into HUD's Indian housing program uncovered corruption and mismanagement, showing that funds for low-income housing were being misused. The series prompted reforms and won a Pulitzer Prize for Investigative Reporting.","keywords":["HUD","investigation","Pulitzer"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["spreadsheet","Email"],"full_text":"Despite the low-tech nature of The Seattle Times' investigation into HUD's Indian housing program, electronic journalism played a crucial role in the work that won the 1997 Pulitzer Prize for Investigative Reporting. The investigation, reported by Eric Nalder, Alex Tizon, and Deborah Nelson, highlighted pervasive waste and mismanagement in low-income housing funds amidst Native American communities. The federal aid was misdirected towards affluent tribal members' homes instead of aiding the genuinely needy. Weak regulations allowed incidences like the construction of a 5,300-square-foot house for a tribal executive right under HUD’s supervision. Examples across the U.S. demonstrated overlooked violations in the use of HUD funds. Correspondingly, e-mail served as a critical information source, revealing institutional failure and regulatory oversights. Furthermore, spreadsheets and a range of electronic resources aided in the manual tracking and analysis of funding data and granted quick access to crucial details for case studies. But the investigation entailed visiting communities where access to information through traditional reporting measures was restricted.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-may-hook-line-and-pulitzer","slug":"1997-may-hook-line-and-pulitzer","url":"https://dwillis.github.io/uplink/article/1997-may-hook-line-and-pulitzer.html","year":1997,"month":"May","monthNum":5,"headline":"Hook, line & Pulitzer","kicker":"","author_name":"John McQuaid","author_title":"Reporter","affiliation":"The Times-Picayune","summary":"An investigation into the global status of fisheries by The Times-Picayune exposed alarming trends and the effects of market pressures on fish populations. This led to the Pulitzer Prize for Public Service.","keywords":["fisheries","Pulitzer","investigation"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":[],"full_text":"When The Times-Picayune assigned reporters to examine global fisheries, they discovered that the issues were interconnected due to the global marketplace. This interconnectedness formed the basis for the 'Oceans of Trouble' series, which won the Pulitzer Prize for Public Service. The reporters gathered data concerning catch, economics, and fish stocks, which highlighted a drastic decline in fishing success and economic stability for fishermen. Despite extensive regulatory attempts, the fisheries were already depleted, showcasing the government's lag in addressing overfishing. CAR enabled the team to unravel these economic and social connections, despite information gaps. The study presented a stark picture of failure in fisheries management. Furthermore, digital communication platforms like 'Fishfolk' provided valuable discourse and source tracking, proving key in both framing the series and finding interview leads. This endeavor underlined not just the dire state of global fisheries but also the profound consequences of inadequate data management and policy adherence.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-may-investigating-crime-statistics","slug":"1997-may-investigating-crime-statistics","url":"https://dwillis.github.io/uplink/article/1997-may-investigating-crime-statistics.html","year":1997,"month":"May","monthNum":5,"headline":"Investigating crime statistics","kicker":"Crime watch","author_name":"Derek Willis","author_title":"","affiliation":"Palm Beach Post","summary":"The Palm Beach Post discovered that Florida's 1996 crime statistics were significantly flawed because the state included police departments that failed to submit reports or submitted inaccurate data, inflating the apparent crime decline.","keywords":["crime","data","investigation"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"Florida's analysis of crime in 1996 brought some good news: the state's overall crime rate declined again, allowing Arizona to take over the top spot among the most crime-ridden states. And for residents of Palm Beach County, the 6 percent drop in crime was much higher than the overall rate.\n\nOne big problem: both numbers were wrong. In its report, the Florida Department of Law Enforcement included in its calculations police agencies that had failed to submit a report or gave inaccurate figures.\n\nTwenty such departments were counted as having no crime at all by FDLE, which skewed the overall rate. The police department for Hialeah, a city of 200,000 people in Dade County, was the largest agency to report \"no\" crime.\n\nPalm Beach Post reporters also discovered that two local agencies turned in incomplete or inaccurate reports to the state.\n\nOne city said its crime rate declined 60 percent from 1995 to 1996; the reason was that officers were not turning in crime reports to a records clerk.\n\nHow the story was done\n\nIn advance of the state's report, police reporter Monika Gonzalez asked every police department in the county for a copy of the report they sent to FDLE. Our original intention was to compile a chart and run a story the day the report was to be released.\n\nBut when we compiled the numbers in Excel and did a few calculations, we noticed that several departments had huge swings in crime from 1995 to 1996.\n\nSo we called those departments. One city sent its six-month report to the state because the final version wasn't ready. Another sent a full report but knew it was incorrect. Two others didn't send a report. FDLE included all of them in its calculations.\n\nTo account for these agencies, we excluded them from both 1995 and 1996 totals and recalculated the crime rate. The county's rate actually increased during that time period.\n\nNext we asked FDLE for 1996 crime data from every police department in the state, which we received on a 3.5-inch floppy disk and imported into Excel.\n\nWe found 17 other departments that were counted as having no crime. Using our paper copies of 1995 data, we found seven more who didn't report any crime in that year.\n\nSo we again excluded all of those agencies from both years' totals and recalculated the change in the crime rate. Instead of a 1.7 percent drop, Florida's crime rate edged up 0.41 percent.\n\nLessons learned\n\nObviously, the lesson here is to check the data.\n\nFlorida has hundreds of police departments and the state's report presents summary data but not detailed information. FDLE made no mention of the fact that these 20 agencies reported no crime - we simply looked at the numbers in Excel.\n\nEyeballing (and multiple sorting) led us to most of the major discoveries for our stories, and a few minor ones that would raise questions. One county said its crime rate dropped nearly 84 percent during the two-year period. We also found that agencies in other counties seemed to have a habitual problem with reporting, filing no reports in both years.\n\nFDLE acknowledged that its report is only as good as the numbers it receives but said there were no plans to change the system.\n\nBut reporters, editors and, hopefully, readers now realize that the crime reporting system depends not just on the public, but also on police agencies who compile statistics.\n\nAnd that sometimes, these agencies make mistakes.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-continuing-with-crosstabs","slug":"1997-may-continuing-with-crosstabs","url":"https://dwillis.github.io/uplink/article/1997-may-continuing-with-crosstabs.html","year":1997,"month":"May","monthNum":5,"headline":"Continuing with crosstabs","kicker":"Stats from the road","author_name":"Sarah Cohen","author_title":"columnist","affiliation":"Uplink","summary":"This column explains three statistical methods—relative risk, odds ratios, and attributable risk—for extracting meaningful insights from crosstab data, using C-section rates among differently insured women as an example.","keywords":["crosstabs","statistics","analysis"],"topics":["Data & Databases","Investigative Methods","Health & Environment"],"technologies":[],"full_text":"Last month's Uplink column had all kinds of reasons to use crosstabs: They're easy to do, easy to understand and easy to explain. But a few key issues were missing. Among them were ways to paraphrase their results to get a key number for your story.\n\nIn this month's column, we'll look at three ways to get meaning from your crosstab: relative risk, odds ratios and attributable risk.\n\nHere's a crosstab of uncomplicated births derived from Florida hospital discharge data for one quarter last year. \"Insured\" means that the women had some kind of traditional fee-for-service insurance. \"Medicaid\" means the women were covered under the state's health insurance program for the poor. The question: Do women with expensive traditional insurance get C-sections more frequently than women with less generous government-supplied care?\n\nHere are the raw numbers, rounded off to make it easier:\n\nInsured\tMedicaid\tTotal\nC-Section\t3,000\t2,000\t5,000\nNo C-Section\t8,000\t10,000\t18,000\nTotal\t11,000\t12,000\t23,000\n\nHere's how it would look as a simple crosstab, like the ones described last month. The first row is the C-section rate, or the percent of all women within the insurance plans who got C-sections:\n\nInsured\tMedicaid\tTotal\nC-Section\t27%\t17%\t22%\nNo C-Section\t73%\t83%\t78%\nTotal\t100%\t100%\t100%\n\nOf course, this difference could be explained in other ways. For instance, women covered by traditional insurance are typically older than Medicaid recipients, and their pregnancies may be riskier.\n\nBut the same kind of analysis might be done within a single age group, like women in their 20s.\n\nLooking just at the row for C-sections, how can you describe the difference?\n\nYou could just use the percentage point difference. The C-section rate for women in Medicaid is 10 percentage points lower than those covered through traditional insurance.\n\nBut that's a rough sentence for your readers or viewers to swallow. If the difference were much smaller, there'd also be a problem with proportion: Is 1 percentage point a lot?\n\nThe simplest correction is to calculate the relative risk of getting a C-section by computing the simple ratio between the numbers. That's how the Wall Street Journal reported mortgage denial rates by race for the nation's top lenders a few years ago. In our case, the relative risk is 1.63.\n\nA typical sentence might read: Women in traditional insurance plans get C-sections 63 percent more often than those women covered by Medicaid.\n\nThe problem with this approach is that it doesn't work very well if there's no reason to choose one of the rows rather than the other in the crosstab.\n\nLet's say the data reflected pass and fail rates on an employment exam, and you wanted to compare these rates for signs of discrimination. Are you more worried about the difference in pass rates or fail rates? It could make a big difference.\n\nUsing the same ratios as above, but just reversing the math, women might have failed 63 percent more often than men, but men would have passed only 15 percent more often than women.\n\nOne solution to this problem is the odds ratio. It's pretty easy to compute: Divide the odds for one group by the odds for the other group.\n\nDon't confuse odds with percents. An odds calculation uses its counterpart as its base instead of the total of all groups. The easiest way to remember it is how you figure out betting odds: odds of 3-to-1 mean that you could get a $3 windfall or you could lose $1.\n\nSo the odds of getting a C-section with traditional insurance are 3,000-to-8,000 or 3-to-8. For every three women who get C-sections, there are eight who don't. The odds of getting one in under Medicaid are 2,000-to-10,000, or 1-to-5.\n\nThe odds ratio is (3/8)/(1/5) = 1.88. That means the odds of getting a C-section under traditional insurance are 88 percent greater than for women covered under Medicaid. (It always works like this: Odds ratios are always higher than relative risk for numbers over 1, and always lower for numbers under 1.)\n\nThe most compelling reason to use this ratio, instead of relative risk, is that it doesn't matter how you express your problem.\n\nIn the employment exam example, the odds ratio for men passing is the same as the ratio for women failing. Attributable risk is used most commonly in medical stories, but lawyers have used it for years in product liability and discrimination cases. When you see statements like, \"20 percent of all lung cancer could be avoided if everyone stopped smoking,\" the authors are talking about attributable risk. It's a little harder to calculate, especially if you have lots of age groups or other control factors. Here's a shortcut, which works for comparable numbers expressed as percents or any other kind of rate, such as a rate per 100,000 people: 1-(Rate in the control group, or the group not affected / Overall rate) So for the C-sections, you would calculate it using the top line of the crosstab: 1-(Medicaid rate/ Total rate), or 1-(17/22), or .23. This says that about one-quarter of C-sections could be avoided if doctors paid by traditional insurers acted like Medicaid providers, so long as that's the only important difference between the two groups. Put another way, you could multiply the result by the number of C-sections there were, and estimate the number that might have been avoided: 700. This is useful when there's a specific outcome you care about, and it works even when the proportions are very small, like in cancer clusters. So is did two-thirds more women get C-sections in traditional insurance, are the odds almost 90 percent higher, or could you have avoided one-quarter of the C-sections among insured mothers if they were in Medicaid? They're all true, assuming you've controlled for factors like age and health of the mother. Pick your weapon carefully.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-car-catches-big-fish","slug":"1997-may-car-catches-big-fish","url":"https://dwillis.github.io/uplink/article/1997-may-car-catches-big-fish.html","year":1997,"month":"May","monthNum":5,"headline":"CAR catches big fish","kicker":"Hook, line & Pulitzer","author_name":"John McQuaid","author_title":"Reporter","affiliation":"The Times-Picayune","summary":"A journalist describes using databases, mailing lists, and online communities to research fisheries management, discovering that regulatory efforts consistently lagged behind fish depletion while also meeting his future spouse through professional networking.","keywords":["fisheries","research","databases"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["Excel","Internet","Listserv"],"full_text":"scientist put it, this was a \"science of elegant post-mortems.\"\n\nOne other database trick helped demonstrate this point. Looking for a way to analyze whether the government's actions to protect fisheries were working, I got a list of all fishery management plans used by the Fisheries Service for the last 20 years, including hundreds of amendments and updates. Taking this as a raw indicator of regulatory activity, I used Excel to total the amendments by plan, by region, and by year. The results indicated that the government was consistently behind the curve, producing a mind-boggling complex pile of regulations to protect fisheries only after they were depleted.\n\nSchleifstein and I also subscribed to fisheries-related mailing lists that generated dozens of messages ranging from the useful to the ridiculously arcane each day. For example, the Fishfolk listserv administered by the Massachusetts Institute of Technology brings together social scientists, managers, biologists and fishermen in a freewheeling discussion. Because they offered a mosaic of expertise and opinion, the lists were useful for developing and refining some of the framing ideas of the series. They were also useful for more mundane tasks—tracking down sources, or answering technical questions. At one point, I was curious about the origin of the fishery science terms \"recruitment\" (what happens when fish reach a catchable age) and \"year class\" (all the fish born in a given year). So I posted the question to the lists. I got dozens of responses, including one from a curious scientist who went to a library and researched the question for me. The terms originated early in the century, apparently in the work of Russian scientists. I was a little disappointed that they didn't come from college football, as some had theorized. The Fishfolk list also provided me with one additional benefit. Trish Clay, a Fisheries Service anthropologist, was one of its founders. I was impressed with her posts to the list, and arranged via e-mail to interview her. We're now engaged to be married in October.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-reporter-lands-rudder-story","slug":"1997-may-reporter-lands-rudder-story","url":"https://dwillis.github.io/uplink/article/1997-may-reporter-lands-rudder-story.html","year":1997,"month":"May","monthNum":5,"headline":"Reporter lands rudder story","kicker":"Fear of Flying","author_name":"Kara Morrison","author_title":"Staff","affiliation":"NICAR","summary":"Seattle Times reporter Byron Acohido won a Pulitzer Prize for his investigative series on safety problems with Boeing 737 rudder-control systems, uncovering a pattern of crashes and near-crashes through extensive reporting and document analysis.","keywords":["aircraft","investigation","safety"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["FAA's Service Difficulty Reports database"],"full_text":"On a windy February morning in 1991, a Boeing 737 dove out of the sky and crashed into a Colorado Springs park, killing all 25 people aboard.\n\nThat crash, and the investigation that followed, alerted Seattle Times reporter Byron Acohido to a larger story that needed to be told. Crash investigators were finding what seemed to be a recurring problem with the rudder-control system on Boeing 737s.\n\nThree years later, when USAir Flight 427 crashed and exploded into flames near Pittsburgh, investigators again pointed to 737 rudder problems, and Acohido began a two-year reporting project that would earn him a Pulitzer Prize.\n\nAcohido, who has covered the aerospace industry for the Seattle Times since 1988, won the Pulitzer for beat reporting for his five-part series called \"Safety at Issue.\"\n\nThe series, published in October 1996, details the potentially fatal problems with rudder-control systems on the world's most popular aircraft.\n\nSpotting the story\n\nAcohido had kept his eye on the rudder problems since the Colorado Springs crash and was aware of more than a dozen foreign crashes similar to the 1991 incident. The Pittsburgh crash brought the problem closer to home.\n\nImmediately after that Sept. 8, 1994, crash, Acohido searched the FAA's Service Difficulty Reports database and found nearly 100 reports of serious problems with 737 rudders.\n\nAfter he wrote the story about those reports, a source suggested Acohido would find further information in a NASA aviation database. That database provided details of a Continental Airlines pilot's similar experience with rudder problems near Honduras in 1994.\n\nFrom there, Acohido says most of his reporting came from interviews and thousands of pages of documents, including National Transportation Safety Board records obtained through the Freedom of Information Act, FAA documents, Boeing service bulletins and court records.\n\nOther computer assistance came from the graphics department. Acohido said working with the graphics coordinator Karen Kerchelich and news artist James McFarlane to provide detailed graphics for the series helped him focus.\n\nThree perspectives gave him three new sets of questions to answer each time they met. And having to describe in detail each mechanical aircraft part to his editors in turn helped clarify Acohido's writing.\n\n\"That helped me forward,\" Acohido says, admitting the process was tedious at times. \"The more I found out, the more I found out I didn't know.\"\n\nPulling it together\n\nIn spring 1996, after having done 20 to 30 breaking stories over the two years, he began writing the series. It would take six drafts and more than 500 inches of material before he had finished.\n\nTwo more incidents gave Acohido the momentum to finish the project: the crash of Commerce Secretary Ron Brown's 737 in April 1996 and the near-crash of an Eastwind Airlines 737 in June 1996.\n\nIt wasn't just the Pulitzer that made the series a success. The FAA has since mandated changes to the 737 rudder-control systems.\n\n\"I know for sure we did meet our goal of providing readers with clarity about an issue that meant something to them,\" Acohido said. \"I know we helped raise the level of awareness to the pilot community. I know that for a fact.\"\n\nHe heightened his own awareness so much that Acohido avoids flying on 737s whenever possible.\n\n\"I don't like flying on them because I know too much about them. The same risk is out there. It's there now, and it will be there for two to three more years until they put the fix they are working on on the airplanes.\"\n\nHis best tip for a Pulitzer-winning project?\n\n\"Don't try to do it in one giant piece,\" Acohido advises. \"Get whatever you can in print. It will help your understanding. It will break out new insights and new sources.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-cheating-at-foxpro","slug":"1997-may-cheating-at-foxpro","url":"https://dwillis.github.io/uplink/article/1997-may-cheating-at-foxpro.html","year":1997,"month":"May","monthNum":5,"headline":"Cheating at FoxPro","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"computer-assisted reporting specialist","affiliation":"IRE and NICAR","summary":"This article explains how to use FoxPro batch files and SQL queries to automate repetitive data processing tasks, specifically demonstrating how to add and modify table fields without manual intervention using the SELECT INTO TABLE technique.","keywords":["automation","FoxPro","SQL"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["FoxPro","SQL"],"full_text":"Many tasks in importing and processing computer data are repetitive. This is good because it means you keep asking for data and someone keeps giving it to you.\n\nThe downside is you are too smart and your time is too valuable to sit at your keyboard and mechanically type a sequence of FoxPro commands. The answer - as many old-timers (\"six months or six databases, whichever comes first\") know is a simple batch of commands, saved in a FoxPro command or .prg file.\n\nHowever, one of your data processing steps could involve adding a field, changing the name of a field, or deleting a field after a new one is created and filled with the right data.\n\nIt would be nice if you could cheat your way around the manual two-step intervention this seems to involve: 1) Stopping at the appropriate point in the process to modify the table structure, and, 2) waiting.\n\nBefore I tell you how to cheat FoxPro out of tethering you to the keyboard, I should finish on the reasons why batch files for processing data are important:\n\nThey save your time, not the computer's.\nThey are repeatable. This is important if your data custodian has to send you a correction after you've invested four hours in your importing and preparation steps.\nThey explicitly document and define the processing steps taken with the data. The batch is developed by deliberating what steps need to be taken and then testing those steps.\n\nAfter process is refined, the commands are recorded in the batch. This ensures that the correct procedures are not left to chance or memory.\n\nThe batch should include comments (non-executing lines) about what you did and why. You should put in who wrote the batch and when it was changed.\n\nAll of these things are useful when you are asked about the integrity of the data or about decisions and changes made to the data.\n\nNow, here's how to cheat. The technique is based on two features of SQL and FoxPro: Using the into table keywords at the end of a SQL query, and putting a literal value in the SELECT clause of the query.\n\nAs an example, you have a table called contribs and you want to add a character-type column named NewOcc and 40 characters wide. Here is what your batch would look like:\n\nAdd a char (40) column to the table\n\nSELECT * space (40) as NewOcc ;\nFROM contribs ;\nINTO TABLE contribl\n* next step in batch\n\nThe query selects all the current columns, and creates one that is 40 spaces, using the built-in FoxPro function. The AS keyword is standard SQL and defines the column name you want. If you leave it out, FoxPro will name it EXPR_1 or something.\n\nHere is one more example from one of my batches. It does in one step what I used to have to do in two manual steps: creates a column called state, and then puts the third and fourth characters of the id column into that field. (See the second line.)\n\nAs the data comes from the FEC, the state column contains the state used in the candidate's mailing address, not the state where the office is sought. In importing that data, I call that column MailState.\n\nThe Candidate ID field contains the two-letter postal abbreviation of the state the candidate would represent. In this example, the columns are explicitly named in the query instead of using the asterisk wildcard; I wanted to dictate the order of the columns.\n\nSELECT cand.id, cand.name, ;\nsubstr (id, 3, 2) as State, ;\ncand.party1, cand.party3, ;\ncand.ico, i cand.status, cand.address1, ;\ncand.address2, cand.city, cand.mailstate, cand.zip, ;\ncand.prin_comm, cand.electyear, i cand.district ;\nFROM tmpcand CAND ;\nINTO table cand96\n\nRichard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-surfing-for-winners","slug":"1997-may-surfing-for-winners","url":"https://dwillis.github.io/uplink/article/1997-may-surfing-for-winners.html","year":1997,"month":"May","monthNum":5,"headline":"Surfing for winners","kicker":"On the Internet","author_name":"Nora Paul","author_title":"","affiliation":"The Poynter Institute","summary":"A guide to accessing Pulitzer Prize-winning journalism stories and information about award recipients from major news organizations including The Times-Picayune, Newsday, Seattle Times, Philadelphia Inquirer, and Wall Street Journal.","keywords":["Pulitzer","awards","journalism"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web","Internet"],"full_text":"The season of journalism awards and prizes are upon us.\n\nThe Pulitzers were just announced, so were the IRE award winners. In this column we'll look at some sites that can lead you to where to read the prize-winning stories, where you can find out more about journalism prizes and where you can learn about the stories behind the stories.\n\nThe Pulitzer Prize site: http://www.pulitzer.org/\n\nGet the listing of current winners and, for the journalism prizes in 1995-96, the winners, their bios, their award-winning stories and information on the judging panels and the runner-ups.\n\nThe listings for winners prior to 1995 are under construction. The stories from this year's winners will be available in June.\n\nPulitzer Winners\n\nThe Times-Picayune http://www.neworleans.net PUBLIC SERVICE\n\nSeries analyzing the conditions that threaten the world's supply of fish.\n\nRead all about the win: http://www.neworleans.net/prizepages/prizeindex.html\n\nThis article, covering the wins (Public Service and Editorial Cartooning) contain links to the prize-winning stories and samples of the winning editorial cartoons.\n\nNewsday http://www.newsday.com SPOT NEWS REPORTING\n\nStaff of Newsday, Long Island, N.Y., coverage of the crash of TWA Flight 800 and its aftermath.\n\nRead all about the win:\n\nhttp://www.newsday.com/about/pulitzer.htm\n\nThis page in the \"About Newsday\" area provides links to the 1996 Pulitzer winning coverage and lists of previous Newsday prize winners.\n\nSeattle Times http://www.seattletimes.com\n\nINVESTIGATIVE REPORTING and BEAT REPORTING\n\nInvestigative: Eric Nalder, Deborah Nelson and Alex Tizon of The Seattle Times investigation of widespread corruption and inequities in the federally-sponsored housing program for Native Americans, which inspired much-needed reforms.\n\nBeat: Byron Acohido of The Seattle Times coverage of the aerospace industry, notably an exhaustive investigation of rudder control problems on the Boeing 737, which contributed to new FAA requirements for major improvements.\n\nRead the stories:\n\nhttp://www.seattletimes.com/todaysnews/special.html\n\nThe texts of \"Safety at Issue: the 737,\" and \"Tribal Housing: From Deregulation to Disgrace\" are presented here.\n\nPhiladelphia Inquirer http://www.phillynews.com EXPLANATORY JOURNALISM\n\nMichael Vitez, Ron Cortes and April Saul of The Philadelphia Inquirer series on the choices that confronted critically-ill patients who sought to die with dignity.\n\nRead the stories:\n\nhttp://www.phillynews.com/packages/end_of_life/\n\nSeeking a Good Death series text is available here. The package of stories from the series, related stories from the archive and list of links to Internet resources is a great example of compiling a full array of coverage on a topic on the web.\n\nWall Street Journal http://www.wsj.com NATIONAL REPORTING\n\nStaff of The Wall Street Journal coverage of the struggle against AIDS in all of its aspects, the human, the scientific and the business, in light of promising treatments for the disease.\n\nNew York Times http://www.nyt.com INTERNATIONAL REPORTING\n\nJohn F. Burns of The New York Times coverage of the harrowing regime imposed on Afghanistan by the Taliban.\n\nI guess they are too used to winning prizes to make a big deal, so I couldn't find anything about this Pulitzer win. But do a search <http://www.nytimes.com/search/daily/> on \"Taliban\" and you'll get Burns' news stories (don't be thrown off by the dates on the list that is retrieved from the search - even though it says April 1997, the stories are from last October).\n\nBaltimore Sun\nhttp://www.sunspot.net FEATURE WRITING\n\nThe site includes Lisa Pollak's compelling portrait of a baseball umpire who endured the death of a son while knowing that another son suffers from the same deadly genetic disease.\n\nOther journalism awards:\n\nSure, the Pulitzer is journalism's Oscars, but there are a lot of other awards given by journalism associations. Here are links to the associations' pages with information about awards and winners:\n\nASNE - American Society of Newspaper Editors Writing Awards\nhttp://www.asne.org/kiosk/news/wabrel.htm\nIncludes 1997 winners with links to a page with their photo on it.\n\nAPME - Associated Press Managing Editors Awards\nhttp://www.apme.com/\nAwards for public service, FOI, Meritorious Service and the Associated Press awards are listed here with details about the award and listing of winners and finalists. (There is no link to the awards page from the home page, so click to any other page, go to the bottom and link to the APME Awards page.)\n\nCommittee to Protect Journalists http://www.cpj.org/attacks96/frontmatter/ipfa.html\n1996 Press Freedom Awards winners page with links to lengthy bios of the winners.\n\nEducation Writers Association http://www.ewa.org/\nClick on the \"Contest\" link in the left column to get to a news release about the 1996 winners, some with links to the stories that won the prize.\n\nInternational Women's Media Foundation http://www.iwmf.org/courage.htm \"Courage in Journalism\" Awards\n\nIRE - Investigative Reporters and Editors Awards\nhttp://www.ire.org/resources/contest/\nLinks to the 1996 award winners. Some of the winners' entries are linked to the text of their stories.\n\nNABJ - National Association of Black Journalists Awards http://www.nabj.org/CONVENTION/conv0.html Click on the Awards item in the left column to get a page about the categories of awards given by NABJ.\n\nNAHJ - National Association of Hispanic Journalists Contest http://www.nahj.org/noticias/contj97.htm Information about the award, to be presented at the national convention. For a listing of last year's winners: http://www.nahj.org/noticias/detalle.htm#jaward\n\nNGLJA - National Gay and Lesbian Journalists Association Awards\nhttp://www.journalism.sfsu.edu/www/nlgja/awards.html Description of the award and links to listing of last year's winners.\n\nRTNDA - Radio and Television News Directors Association http://www.rtnda.org/rtnda/ Scroll down the options to Awards and click to releases about past and current winners.\n\nSPJ - Society of Professional Journalists Awards\nhttp://www.spj.org/awdfell/awards.htm Although the page is still under construction, it does have a listing of the awards they give and a link to last year's winners. And now a plug for the too often unsung support group behind the great journalism.\n\nGet more tips from Nora Paul by going to the Poynter website at poynter.org. The site includes a list of upcoming seminars including \"Reporting with the Internet\" and also includes some CAR-related handouts.\n\nNews Research http://sunsite.unc.edu/journalism/awnwslib.html\n\nThis site links you to essays by some of the top researchers in the country and how they conducted the, often, award-winning research.\n\nCongratulations to all the winners and runner-ups. And good luck next year, everyone!","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-spreadsheet-story-ideas","slug":"1997-may-spreadsheet-story-ideas","url":"https://dwillis.github.io/uplink/article/1997-may-spreadsheet-story-ideas.html","year":1997,"month":"May","monthNum":5,"headline":"Spreadsheet story ideas","kicker":"Handout of the Month","author_name":"Stephanie Reitz","author_title":"","affiliation":"The Hartford Courant","summary":"This article outlines practical ways journalists can use spreadsheets to investigate and analyze public records, including employee payroll data, workplace injuries, budget allocations, and voting patterns to uncover newsworthy stories.","keywords":["spreadsheet","analysis","investigation"],"topics":["Investigative Methods","Data & Databases"],"technologies":["spreadsheet"],"full_text":"An engineer once told me that if he was stranded on a desert island, the one software he couldn't live without is the spreadsheet. It can be used to calculate, sort and graph with relative ease, he explained (although he neglected to mention why that would come in handy on a desert island).\n\nEngineers, accountants, budget analysts and the pencil-pushers that we cover in our reporting have known for years that spreadsheets are a great time-saving tool. For reporters, they also provide a new way to collect and analyze information - and to have fun while doing it.\n\nHere are some story ideas using spreadsheets, many of which you can start as soon as you return to your newsrooms:\n\nCity employees' earnings and overtime\nRequest these payroll records from your finance or personnel office, specifying that you want the detailed information turned over to the IRS at the end of the year - not just the gross salary for the positions.\n\nThen take a look at who makes what, and why. We analyzed this in one Connecticut city and found firefighters who had doubled their pay with overtime during a staffing shortage - but were exhausted on their life-or-death calls.\n\nWork-related injuries and illnesses\nThe U.S. Department of Labor requires these filings each year, broken down by department and detailing the number of work days lost. Build a spreadsheet that compares them over several years: why, for instance, did work-related illnesses jump over just a few years in the old library? Are police, firefighters and public works employees still those most likely to be hurt? These figures can prompt interesting stories about work conditions in certain buildings or departments, and the tax-funded worker compensation measures.\n\nCompare seemingly unrelated accounts\nAs the school department cuts textbook purchases because of budget problems, has the athletics account been touched? Analyze budgets over several years and see which are most vulnerable to reductions, and which are sacred cows. Also, end-of-year spending reports can help highlight which accounts keep showing surpluses and perhaps have been overbudgeted (and where does that surplus go?).\n\nKeep a running tab of a topic of interest\nFor example, does your town include a state unemployment report in its agenda packets every month? Or the number of marriage and birth licenses recorded each month in the town clerk's office? Update your spreadsheet every month or so, then go back to it in six months to a year - look for interesting spikes and dips, and you can whip up a quick daily or a deeper enterprise story.\n\nBuild a spreadsheet on voting turnout\nWith the basics from your town's election registrars, you can build a spreadsheet of registered voters and ballots cast in each election or referendum. Is one neighborhood particularly well-represented during budget referendums? How does your community stack up against others in voter turnout? This information is also good fodder for election advance stories.\n\nWhat's the cost of that union contract?\nWith the appropriate union contract and formula in front of you, figure out the true cost - payroll multiplied by the raises offered each year. Don't forget those hidden \"step increases!\" This allows a thorough and fair review of a contract's financial value. You can also use this on individual contracts, such as when a new administrator is hired with pay incentives built into his/her contract.\n\nA final hint\n\nThe more interested you are in the data, the more you'll enjoy learning to use the spreadsheet. Don't jump into any old project just to get one under your belt - pick a topic that makes you curious.\n\nAnd don't hesitate to call me if I can help with spreadsheet glitches, formulas or anything else- happy calculating!","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-so-many-car-options","slug":"1997-may-so-many-car-options","url":"https://dwillis.github.io/uplink/article/1997-may-so-many-car-options.html","year":1997,"month":"May","monthNum":5,"headline":"So many CAR options","kicker":"NICAR/IRE Notes","author_name":"Brant Houston","author_title":"Executive Director","affiliation":"IRE","summary":"IRE Executive Director Brant Houston announces an upcoming computer-assisted reporting (CAR) day at the National IRE Conference in Phoenix on June 12, featuring 20 panels and workshops, with details about the main conference program and the next CAR conference planned for Indianapolis in March 1998. The article also highlights IRE's growing international work and how global issues are becoming increasingly relevant to local reporting.","keywords":["conference","reporting","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"So many CAR options\n\nBy Brant Houston\nIRE Executive Director\n\nThe optional computer-assisted reporting day of the National IRE Conference is coming up fast. The so-called optional day will be on Thursday, June 12, at The Biltmore Hotel in Phoenix, Arizona and will offer about 20 panels and workshops. Some panels will be popular repeats from our NashCAR conference in March such as reporting on air, rail and car crashes. We will also have two-hour panels on special topics in which panelists will show attendees how to produce stories from education, health, and business databases. Other panels will be two to three hours long and broken into one-hour sessions. For example, Nora Paul of the Poynter Institute will lead a \"training the trainers\" program that will go from 9 a.m. to noon and be divided into three sections. Another two-part panel on using CAR at small to medium newspapers will be coordinated by reporters from smaller news organizations.\n\nAs usual at the national conference, NICAR will keep a demonstration room open from Thursday through Saturday for journalists to share techniques and talk about specific problems or stories. For the main program on Friday and Saturday, we have scheduled a series of exciting and ground breaking panels composed of award-winning journalists. Many of those panels will have CAR sprinkled throughout. And on Sunday the Poynter Institute will guide the program with a group of thoughtful sessions on writing and other topics. You can find information on the conference at www.ire.org both before and after it happens.\n\nIndianapolis, March 19-22\n\nMeanwhile, we are already planning for our next national CAR conference, which will be Indianapolis, March 19-22, 1998. This conference is a return to a city where IRE did two joint conferences with CAR pioneer and University of Indiana professor James Brown in the early 1990s.\n\nWe're happy that Jim has invited IRE and NICAR back to the university's campus in Indianapolis where there are great computer labs and lecture halls.\n\nPlease send us ideas and stories (as they emerge during the year) that you think should be subjects for panels and hands-on training in Indianapolis. You can e-mail them to brant@ire.org\n\nGlobal = Local\n\nAs IRE and NICAR do more international work it's becoming clear to us that global is becoming local and the local is becoming global.\n\nIn Nashville, reporters from Iowa talked to Lise Olsen (the director of our reporting institute in Mexico City) about the effects of NAFTA on agriculture in the Midwest. Our panels on immigration looked at immigration's effects on the United States, Latin America and China.\n\nSeveral IRE members have asked us to put together panels and handouts on global economics issues.\n\nColleagues in Mexico, Sweden and Denmark are sharing ideas and techniques as the Internet and Web browsers make international reporting easier.\n\nOne of our challenges in the coming year is to show how our programs for small to medium news outlets (print and broadcast) and for international journalists actually bolster each other.\n\nOne way we hope to meet that challenge is by holding a border seminar Nov. 14-16 this year along the Mexico and U.S. border for journalists from both countries. We expect that many of the journalists will be from smaller news organizations.\n\nCoordinating coordinators\n\nAnd speaking of our efforts for those organizations, IRE and NICAR are rejuvenating our regional coordinator system. We are looking for members to volunteer to be regional and state coordinators.\n\nWe are asking the volunteers to sign up more members, hold informal gatherings of local reporters, and eventually help us arrange workshops in every state over the next year or two. We believe this will keep IRE in touch with reporters no matter what size their organizations.\n\nCory de Vera, IRE's membership coordinator, is collecting the names of volunteers. You can e-mail her at cory@ire.org or call (573) 882-2042.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-get-your-training","slug":"1997-may-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-may-get-your-training.html","year":1997,"month":"May","monthNum":5,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR and The Associated Press offer specialized computer-assisted reporting training through workshops, bootcamps, and conferences for journalists at various locations throughout the year, with costs varying by program type and market size. Training covers data transfer, spreadsheet building, database management, and online research skills for investigative journalism.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","spreadsheet"],"full_text":"NICAR and, where indicated, The Associated Press provide specialized training in your newsroom.\n\nLearn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases.\n\nCost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below.\n\nNote: In the following list, \"Closed\" means the session is open only to members of the host organization.\n\nDenmark, Sweden. May 26-June 6. Closed.\nNew York, N.Y. Reuters. May 29-30. Closed.\nSan Francisco, Cal. June 21-22 (seminar for minority journalists) San Francisco State University.\nChicago, Ill. Reuters. June 26-27. Closed.\nWashington, D.C., Freedom Forum. July 14-15. Closed.\nHuntington, W.Va. Aug. 19-22. Marshall University. Closed.\nWashington, D.C. Sept. 11-13. Medill School of Journalism. Open.\nNew York, N.Y. Sept. 26-27. (Broadcast workshop) Columbia University. Open.\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969 IRE National Conference, Phoenix, Ariz., - June 12-15.\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo.\n\nAs with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size.\n\nFor information, call Wendy Charron at (573) 882-0684.\n\nMay 18-23. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684\n\nJuly 11-13. Mini Bootcamp on general computer-assisted reporting. New York University. New York, N.Y. For details, call (573) 882-0684.\n\nAug. 10-15. Regular Bootcamp on general computer-assisted reporting. For details, call (573) 882-0684\n\nSouth of the Border\n\nPeriodistas de Investigacion (IRE-Mexico) offers computer-assisted reporting training in Spanish to journalists in Mexico and Latin America.\n\nPI, based in Mexico City, has developed a two-day workshop that covers the basics of investigative reporting and computer-assisted reporting, and a four-to-five day bootcamp that covers CAR intensively.\n\nThe workshops can be held in almost any city with prior notice.\n\nThe courses cover using internet, spreadsheets, database managers, and general electronic records, and uses examples from the Mexican and Latin American press.\n\nThere is one bootcamp scheduled in May:\nMexico City - May 14-18.\n\nAnnual membership to IRE-Mexico is $25 (or $200 pesos mexicanos) and is open to journalists or students of journalism living in Mexico and Latin America.\n\nInstructors are Lise Olsen, managing director of IRE-Mexico, and Pedro Enrique Armendares, assistant director.\n\nBoth have more than seven years of experience in investigative reporting, Olsen at The Virginian-Pilot (Norfolk, Virginia) and Armendares at La Jornada (in Mexico City).\n\nCall Olsen at (52)-5-554-7613 for more information, send email to: lise@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-from-the-nicar-library","slug":"1997-may-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-may-from-the-nicar-library.html","year":1997,"month":"May","monthNum":5,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers a comprehensive collection of federal government databases covering aviation, crime, finance, safety, and infrastructure for investigative journalism research, with datasets spanning from 1937 to 1996.","keywords":["databases","federal","journalism"],"topics":["Data & Databases"],"technologies":["CD-ROM"],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW Federal Aviation Administration's accidents and incidents, including major plane crashes since 1971.\n\nNEW NASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\n\nA monthly CD subscription for all 1995-96 Federal Election Commission campaign contributions by individuals and political action committees, plus all presidential matching fund requests.\n\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings. 1991-1995.\n\nCoast Guard boating accidents, 1969-1994.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\n\nFederal procurement data, 1992-1995, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1995.\n\nNational Bridge Inventory System data, includes inspection grades. 1994-1995\n\nNEW FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the new 1995 data.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nThe National Inventory of Dams. 1991-1995.\n\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1995.\n\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1988 to 1995.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-may-nicar-net","slug":"1997-may-nicar-net","url":"https://dwillis.github.io/uplink/article/1997-may-nicar-net.html","year":1997,"month":"May","monthNum":5,"headline":"NICAR Net","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The Atlantic City Press and Orange County Register are hiring investigative reporters with technical and leadership skills, while IRE promotes membership and announces conference panels focusing on computer-assisted reporting techniques.","keywords":["hiring","journalism","reporting"],"topics":["Journalism & Newsroom"],"technologies":["Excel","Access","SQL"],"full_text":"Help Wanted\n\nThe Atlantic City Press is looking for an investigative and computer-assisted reporter with five years experience. Applicants should be proficient in Excel, be able to use Access to analyze complex databases and know SQL. Send resume and clips to Michael Diamond, 1000 W. Washington Ave., Pleasantville, N.J. 08232.\n\nThe Orange County Resister is looking for a leader for its investigative team. This is currently a reporter-editor job.\n\nSubmit a packet with resume, clips and a one-page introductory letter to: Larry Burrough, deputy editor for news, 625 N. Grand Ave., Santa Ana, Calif. 92701. Fax: (714) 565-3681.\n\nMembership has its privileges\n\nYou can join IRE via our web server, or you can download a copy of our membership form in Adobe Acrobat format from the web site and submit it by fax to (573) 884-5544.\n\nFor more information, check out http://www.ire.org/membership.html\n\nMore Phoenix Information\n\nThe National IRE Conference fun begins June 12 with CAR optional day. Here is a tentative list of the panels that will be offered that day:\n\n50 Of The Newest Ideas In CAR\nUsing the Internet For Daily And Beat Reporting\nBroadcasters' Workshop: Good Answers To Special Problems\nCrime Reporting: More Than The Usual Numbers\nTraining The Trainers\nEthics Of Data And Legalities On The Net\nCovering Education With CAR\nLocal Reporting For Larger Markets-Newspapers & Broadcast\nCovering the Environment With CAR\nSmall To Medium Newspapers\nCovering Social Issues With CAR\nBusiness Reporting With CAR\nUsing CAR to Analyze Campaign Finances\nHow To Find Data For Both Deadline And Long Term Stories\nTransportation: Crash Data\nHow To Keep Your Data Organized\nCovering Health Care With CAR\nThe Latest On Intranets\nMapping, Graphics On The Web","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-uplink-update","slug":"1997-june-uplink-update","url":"https://dwillis.github.io/uplink/article/1997-june-uplink-update.html","year":1997,"month":"June","monthNum":6,"headline":"Uplink Update","kicker":"Summer Session","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"This journalism newsletter features computer-assisted reporting (CAR) techniques used by various news organizations to investigate topics including public housing, FDA procedures, day-care centers, school rankings, and FCC bid rigging.","keywords":["investigation","reporting","data"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["Internet"],"full_text":"This month's Uplink offers a smorgasbord of CAR offerings to get your summer started off right. Michael J. Himowitz tells how The Baltimore Sun discovered \"dreams, debts and demolitions\" in the city's public housing. Tom Gaumer from The Cleveland Plain Dealer tells how he investigated the FDA's experiment procedures and found alarming results. Ames Alexander of The Charlotte Observer teaches the ABCs of day-care investigations. Sarah Cohen uses her statistics column to share how the Arkansas Democrat-Gazette tackled the tricky, but important job of ranking area schools. Bob Sullivan of MSNBC also tells about his first computer-assisted reporting project, which uncovered bid rigging at the FCC. And Nora Paul gives the must haves of Internet sites in her monthly Internet column.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-dreams-deferred","slug":"1997-june-dreams-deferred","url":"https://dwillis.github.io/uplink/article/1997-june-dreams-deferred.html","year":1997,"month":"June","monthNum":6,"headline":"Dreams deferred","kicker":"Housing Investigation","author_name":"Michael J. Himowitz","author_title":"Reporter","affiliation":"The Baltimore Sun","summary":"Investigates the flawed efforts in Baltimore's public housing that resulted in financial burdens on low-income homeowners due to liens and a mismanaged demolition program.","keywords":["public housing","liens","demolition"],"topics":["Investigative Methods","Government & Politics","Business & Economy"],"technologies":[],"full_text":"Martha Fields' West Baltimore rowhouse was a dump with a leaky roof that brought down her bedroom ceiling and forced her into the street. Barely subsisting on Social Security, the 65-year-old woman had no money to fix up the eyesore, and when neighbors complained, the city responded with a work crew. Fields thought that was wonderful - she was getting help at last. What she wound up with was a boarded-up derelict saddled with $37,000 in liens that she can never hope to repay. In fact, with interest accruing at 20 percent a year, Fields now owes three times what the house is worth. But no one is ever likely to buy it. Her tale of woe and others sparked a year-long investigation of Baltimore's decaying housing stock by Sun reporters John O'Donnell and Jim Haner. The outcome was a three-part series titled 'Dreams, Debts, Demolitions' that ran April 6-8. With the help of a sophisticated computer-assisted reporting effort, Haner and 'Donnell discovered that Fields' case was far from unique. They found a well-intentioned government effort gone awry, a campaign to save neighborhoods that actually encouraged their abandonment while lining the pockets of politically-connected contractors. The result: a city where 10 percent of the residential addresses are owned or controlled by the government. A city with as many as 40,000 vacant houses and empty lots. A city where millions have been spent repairing and renovating homes that were torn down within a few years. O'Donnell and Haner outlined the decline of a politically popular program set up in the 1980s to battle slumlords. It gave the city the power to repair run-down houses and bill the owners for the cost. To guarantee a return if the owners didn't pay up, the city put liens on the houses that would be satisfied when the properties were sold. But the system wasn't designed to deal with a 20-year exodus of 200,000 people that sent inner-city property values plummeting. Instead, the liens encouraged thousands of landlords to abandon their property and turned low-income homeowners into debtors. Thousands of homes became albatrosses burdened with liens so large that no one could afford to buy and renovate them. With interest accruing every year at punitive rates, some properties worth $10,000 have been saddled with liens worth $100,000. 'We found many cases of vacant houses that were definitely worth rehabbing if it was affordable,' O'Donnell said, 'but it isn't affordable with a $30,000 lien on the property.' The Sun's reporters also found a housing department so backlogged and disorganized that it has no idea how many properties it actually owns. They uncovered an aggressive but willy-nilly program of demolition that has left some crumbling blocks untouched and others gap-toothed. And they also found a bureaucracy churning along with an outdated system that routinely mails out bills for repair work to long-dead homeowners and long-departed landlords. But within days after the housing series ran, Mayor Kurt L. Schmoke announced a drastic slowdown in the city's demolition program and promised a new look at the liens and interest payments that have tied up so many properties.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-june-experimenting-with-drugs","slug":"1997-june-experimenting-with-drugs","url":"https://dwillis.github.io/uplink/article/1997-june-experimenting-with-drugs.html","year":1997,"month":"June","monthNum":6,"headline":"Experimenting with Drugs","kicker":"CAR & FDA","author_name":"Tom Gaumer","author_title":"","affiliation":"Plain Dealer","summary":"Plain Dealer reporters investigated FDA inspection records spanning two decades and found that more than half of drug experiments lacked adequate informed consent from human subjects, with some cases showing no consent at all. The reporters obtained and analyzed a computerized FDA database of over 8,000 inspection records to uncover patterns of inadequate disclosure of risks and benefits to research participants.","keywords":["consent","FDA","investigation"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["FoxPro","floppy disk"],"full_text":"When the Food and Drug Administration inspects drug experiments, it finds more than half the time that the human subjects have not been sufficiently informed about their possible risks and benefits. That's the conclusion in articles by Plain Dealer reporters Bill Sloat and Keith Epstein. The series ran over four days beginning Dec. 15. The project began more than a year ago after Sloat and Epstein finished stories about Norplant and how some women did not give written consent when Norplant was tested on them. Managing Editor Gary Clark suggested that the two try to find out if there were other instances where the FDA found inadequate consent.\n\nThe two reporters obtained more than 5,000 pages of federal documents along with a computer database containing the FDA inspections from 1974 through 1995 of clinical research involving experimental drugs being tested on humans.\n\nThe reporters found the database almost by accident. They had filed many FOIA requests for FDA inspection reports, but didn't get any on the grounds there were none. They started calling employees from the FDA and someone told them to ask for the 483s, which was the actual inspection report. Armed with that information, they got a printout of some 483s.\n\nBut then someone mailed them a huge printout of the actual database. With that, they knew for sure the database existed on a computer.\n\nIt took another three months, but Epstein, who works in The Plain Dealer's Washington bureau, finally got the records on floppy disks.\n\nThat database provided numbers that were the basis for much of the series as well as most of the information for the graphics that ran with the project.\n\nThe database contained 8,039 records. There was no single field for each inspection so we used FoxPro 2.6 for Windows to calculate that the FDA conducted 4,154 inspections of medical researchers.\n\nThe database had a column for type of deficiency, and we found that in more than half the inspections - 2,201 - the consent was inadequate.\n\nThe database even revealed 47 instances where there was no consent.\n\nOne of the major facts Sloat and Epstein wanted to know was how federal research locations compared with those at universities, hospitals or private facilities such as clinics.\n\nBut the database had no classification indicating what type of facility was being inspected.\n\nIn addition, it had the usual pattern of using different spellings or words to describe the same institution.\n\nSo Sloat and I spent a week in early 1996 going through the entire database (we were able to throw out some records that did not have a location).\n\nWe added a new column and gave each location a code letter such as G for government or U for university. Private facilities or those we could not identify went into the same category, which we called other. But we were able to identify most of them from the name.\n\nOnce that new database was completed, we were able to go far beyond the information available at first.\n\nWe found out very quickly that the government locations, such as Veteran's Administration hospitals, had the worst track record for providing information to the people used in experiments.\n\nBecause we now had a unique number for each location, we were able to pinpoint the government locations and universities that had the most violations over the years. All these facts were used in graphics that ran every day.\n\nTo make the story interesting, Sloat and Epstein had to find the names of people who were part of studies. The task was painstaking because identities of patients were omitted or deleted from many records. In other cases, the government used code names.\n\nSloat and Epstein traveled extensively throughout the country and even to Europe tracking experiments inspected by the FDA.\n\nSloat and Epstein even found people who died during a drug experiment, including a Pennsylvania woman who received a substance also used as a gasoline additive to try to dissolve gallstones.\n\nThat woman was only known in records as TE3. Sloat and Epstein were able to match public death records with FDA documents to find the clues that enabled them to identify the woman as Laura Michalski.\n\nThey talked to Michalski's widower and children who were unaware that Michalski had been part of a drug experiment.\n\nTom Gaumer can be reached at (216) 999-4500, or send e-mail to TGAUMER@plaind.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-beyond-the-nightmares","slug":"1997-june-beyond-the-nightmares","url":"https://dwillis.github.io/uplink/article/1997-june-beyond-the-nightmares.html","year":1997,"month":"June","monthNum":6,"headline":"Beyond the nightmares","kicker":"Day-care discovery","author_name":"Ames Alexander","author_title":"Reporter","affiliation":"Charlotte Observer","summary":"The Charlotte Observer conducted a data-driven investigation using state databases to reveal widespread child abuse and neglect in North Carolina day cares, finding that over 100 children are injured monthly and that illegal facilities have higher abuse rates than registered ones.","keywords":["daycare","abuse","databases"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["FoxPro"],"full_text":"We'd all read stories about day-care horrors, the tales of abuse and neglect that leave readers with a sense of disgust and fear but little understanding.\n\nSo when other Charlotte Observer staffers and I decided to assess the state of child care in North Carolina, we knew we needed to do more than simply lay out the parental nightmares.\n\nWe needed an authoritative way to find out how often day-care kids are mistreated, where it's most likely to happen and why. The challenge was daunting, but computer databases made it less so. Using Foxpro, we analyzed tens of thousands of records in databases kept by state day-care regulators. That - along with scores of interviews, visits to day cares and trips to state filing cabinets-helped us illustrate the consequences of the state's lax day-care rules. Here's some of what we found: Each month, more than 100 North Carolina children are hurt so badly in day care that they require medical treatment. That's partly because day-care staffing requirements in North Carolina are among the nation's most lenient. Children are more likely to be neglected or abused at centers that meet only minimal state requirements. Complaint rates are particularly high at the state's smallest day cares, which are based in homes and subject to the least state scrutiny of all. The rate of substantiated complaints at the state's two largest chains - KinderCare and La Petite Academy- is more than twice that of the average center in North Carolina. Over the past five years, state officials have discovered nearly 1,700 North Carolina day cares operating illegally. More children were neglected and abused in those homes than in all the legally registered child-care homes combined. Inspectors don't punish or close many problem day cares. Though state inspectors substantiated more than 1,400 complaints against day cares during the past two years, they revoked just 36 licenses and issued only 138 fines. The databases proved useful in other ways as well. They steered us to the child-care centers with the worst track records. And they helped us produce comprehensive summaries of all recent state actions against day cares in the Piedmont. Lots of work - and a little resourcefulness - went into those summaries.\n\nWe used the databases to generate a list of all recent state actions against day cares in the 10-county region. Trouble was, the computer files didn't give us a full account of what happened and how the day care responded to the state's action. So we took our list Raleigh, and pulled the files on each day care cited.\n\nArmed with laptops, we plowed through the files and created our own databases, entering information about what happened and how the day cares responded.\n\nTo be fair and accurate, we later tried to contact the owners of each day care. Then we printed the summaries in their entirety.\n\nIf you're looking for databases with information on child-care centers, start by asking the state agency that regulates them. In North Carolina, it's the state Division of Child Development, an arm of the state Department of Human Services.\n\nWithout too much coaxing, we got databases with a great deal of valuable information: complaints against day cares; injuries among children; names of day care owners and their phone numbers; whether each day care was licensed or registered; how many children each center was licensed to care for, etc.\n\nOne thing to keep in mind: It's not fair to compare the raw number of complaints at each center.\n\nYou need to calculate complaint rates, based on the number of children each center is licensed to care for. And when figuring those rates, it's best to use substantiated complaints, rather than total complaints. Not every parent's gripe is well-founded, experts note.\n\nIn the end, computers could do only a small part of the work. We spent much of our time visiting day cares, poring through state records and interviewing teachers, parents and experts.\n\nBut used well, databases can make your child-care stories more powerful, accurate and downright interesting.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-ire-resource-center-notice","slug":"1997-june-ire-resource-center-notice","url":"https://dwillis.github.io/uplink/article/1997-june-ire-resource-center-notice.html","year":1997,"month":"June","monthNum":6,"headline":"IRE Resource Center notice","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"The Uplink newsletter highlights investigative journalism stories from outlets like the Cleveland Plain Dealer and Arkansas Democrat-Gazette, which can be accessed through the IRE Resource Center with reporter tips available by phone.","keywords":["investigation","journalism","resources"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Several of the stories featured in this issue of Uplink including the Cleveland Plain Dealer's FDA story and the Arkansas Democrat-Gazette's school ranking story are available through the IRE Resource Center. To order copies of the stories and tips from the reporters who did them, call (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-head-of-the-class","slug":"1997-june-head-of-the-class","url":"https://dwillis.github.io/uplink/article/1997-june-head-of-the-class.html","year":1997,"month":"June","monthNum":6,"headline":"Head of the Class","kicker":"Stats from the road","author_name":"Sarah Cohen","author_title":"On The Road trainer","affiliation":"Arkansas Democrat-Gazette","summary":"The Arkansas Democrat-Gazette developed a straightforward methodology for ranking schools by combining multiple quality indicators like test scores, teacher training, and diversity without complex statistical techniques, finding that top-performing schools weren't necessarily in wealthy areas.","keywords":["schools","ranking","methodology"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"When the Arkansas Democrat-Gazette set out to measure the quality of its schools, it didn't look to fancy statistical techniques to do its work.\n\nInstead, reporters and editors thought through the project, read about school reports, evaluated the information they could get and culled the reporting of three experienced education reporters.\n\nThen they came up with a simple set of criteria that they would use to define the word \"best\" for their readers.\n\nYes, test scores are good measures of a school's success.\n\nBut when parents look at schools, they often look also at the environment, diversity and other measures of quality.\n\nDemocrat-Gazette reporters took that into account when devising their index of excellence in schools.\n\nThey included measures of teacher training and coverage, racial disparity, performance, parent activity, and motivation. And they combined them without the regression techniques that have been used elsewhere to find over- and under-performing schools based on poverty.\n\n\"We may do that in the future,\" said Jeff Porter, the computer-assisted editor who crunched the numbers for the series. \"But what was interesting was that the elementary and junior high schools that were at the head of the rankings weren't in the wealthy parts of town. The elementary school at the top is almost a rural school, on some road that kind of meanders around west of town.\"\n\nBeginning at the beginning\n\nThe effort shows why thinking through a project as perilous as the annual school reports is just as important as learning advanced statistical techniques.\n\nEducators tell us there's no way we'll compare schools fairly. But if our job is to provide our readers and viewers with information, not hide it, then our other job is to do it as responsibly as possible.\n\nThat's where Democrat-Gazette's indexing technique comes in. In Excel, Porter essentially ranked each indicator, then added up the ranks. The school with the biggest total came out on top.\n\nThe only numbers trick he played was to turn the rankings upside down: The school with the \"best\" performance got the lowest rank, giving it the most points toward the total score.\n\nThe school with the \"worst\" performance got the highest rank - the No. 1 we always think of in rankings - contributing the least to the total score.\n\nCombining upside-down rankings of many indicators is one way reporters can cut the school reports down to size.\n\nThe approach seems embarrassingly simple for those who have struggled with the numbers morass of school reports.\n\nLots of indexes are done this way.\n\nMost business \"Top 100\" lists are driven by these kinds of rankings. So are the \"best places to live\" indexes.\n\nEven U.S. News & World Report, which ranks institutions from hospitals to colleges, uses a system something like this.\n\nIt might be more sophisticated, but the basic idea is the same - figure out what is important, assign a score for each element, and come up with a total that puts everyone on the same footing.\n\nIs an index made up of rankings scientific?\n\nNot entirely. Does it have drawbacks? Certainly.\n\nYou've lost a lot of information that you'd retain if you used z-scores instead.\n\nAll of these indexes, though, use some combination of judgment, reporting and sometimes even polling involved in deciding what's important.\n\nSome words of caution:\n\nFigure out your indicators carefully.\n\nIf you decide that racial disparity is a bad thing for schools, then you have to find some indicator of this that will treat all schools equally. Democrat-Gazette reporters had criteria laid out by the court system to decide how to measure it.\n\nThe lesson is clear - find some reputable and impartial source for your standards whenever possible.\n\nAdjust most of your numbers for the number of students. This is the \"per capita\" calculation that NICAR classes harp on so often. Arkansas used many measures \"per 100 students\" instead. It's the same thing.\n\nMake adjustments that educators say matter, but only when they do.\n\nMany school report stories use some kind of adjustment for poverty, often using the percent of the students who qualify for free or reduced-price school lunches.\n\nYou might want to give schools with high poverty rates (or other difficulties like English language skills or high migration rates) a leg up in your ratings.\n\nBy focusing on more than just test scores as a measure of quality, Democrat-Gazette reporters and editors didn't need to make these kinds of statistical adjustments.\n\nThrow out redundant indicators or others that don't tell you much.\n\nSchool systems often throw out reams of information, much of which is useless in judging the quality of schools.\n\nFor instance, New York's Web site scores of indicators, some of which are nothing more than the percentage of kids who passed competency tests in each subject for each grade. This doesn't help when they're always 99 or 100 percent.\n\nDecide what you'll do if there's a lot of missing data.\n\nIn Arkansas, like many other places, reporters simply couldn't get reliable information for some grades.\n\nYou may decide to focus only on the very early grades, then junior high schools.\n\nYou may have to skip a story on middle schools if there aren't enough to make a fair ranking.\n\nOr you may decide to include an average of the schools' other indicators as an estimate. Just don't let the missing values sit there, though. It will pull down the scores of otherwise fine schools.\n\nCollect raw numbers, not rates. Democrat-Gazette reporters wanted to make sure each indicator was calculated the same way.\n\nSo instead of collecting the usual drop-out rates, reporters had to ask school officials for the raw number of dropouts.\n\nThis is different, and raises the ire of some school officials, Porter said. But it's sometimes the only way to make sure your numbers are all calculated the same way.\n\nOnce you've gotten your indicators, though, making an index of rankings is straightforward. Get your final indicators into Excel, in any order.\n\nNow use the RANK() function to assign the scores for each indicator. Then add up all the scores. You have to figure out which direction is \"good\" and which direction is \"bad\" for each indicator.\n\nFor instance, a lower teacher-student ratio is usually considered better than a higher one. So a low number is \"good.\"\n\nOr a higher test score is better than a lower one, so a high number is \"good.\"\n\nHere's how a formula might look for a \"high is good\" indicator.\n\nThe RANK() function requires three pieces of information: the number you want ranked, or this test score; the group you want it ranked within, or the entire range of test scores; and the direction you want it ranked.\n\nA \"1\" means that a higher number will end up higher.\n\nA \"0\" means that it will rank normally, with a lower number assigned to a high score.\n\n=RANK(B3,B$3:B$77,1)\n\nNote the dollar signs.\n\nThey mean that, as you copy down, the formula will remain correct. But they also mean you can copy it across, only changing the \"1\" at the end when you come to an indicator in which \"low is good.\"\n\nTo see the Arkansas Democrat-Gazette's full story about the quality of Arkansas schools, go to www.ardemgaz.com\n\nOf course, using more sophisticated techniques can help you find the stories in your school system.\n\nIn the July issue of Uplink, we'll use the statistics column to profile a set of stories that used statistical techniques to find schools that overcome difficulties well, or sit on their advantages.\n\nSarah Cohen can be reached at (301) 942-2199 or e-mail her at sarah@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-more-databases-volunteers","slug":"1997-june-more-databases-volunteers","url":"https://dwillis.github.io/uplink/article/1997-june-more-databases-volunteers.html","year":1997,"month":"June","monthNum":6,"headline":"More databases; Volunteers","kicker":"IRE/NICAR Notes","author_name":"Andy Lehren and Cory de Vera","author_title":"NICAR staff","affiliation":"NICAR/IRE","summary":"The National Institute for Computer-Assisted Reporting is expanding its databases and data analysis services for journalists, offering access to government records on health, safety, and spending, while also seeking volunteers to serve as local coordinators to strengthen IRE's grassroots presence.","keywords":["databases","journalists","volunteers"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["GIS"],"full_text":"The National Institute for Computer-Assisted Reporting continues adding and updating databases for journalists, and is enlarging its data analysis services.\n\nAmong the new databases are: the Centers for Disease Control's reports on AIDS patients across the country, the Food and Drug Administration's collection of complaints about drugs and medical devices, and the U.S. Census Bureau's Consolidated Federal Funds Reports, which allow you to track federal spending in your community.\n\nThe updated databases include: the U.S. Department of Transportation's reports of accidents involving hazardous chemicals, the U.S. Small Business Administration's 7A loan guarantees for companies, FBI crime reports, railroad accidents, federal contracts, and files on legal immigrants to the United States.\n\nTo continue enlarging its offerings, NICAR has numerous FOIA letters and other pending requests for data. We will announce those offerings in this newsletter and on the NICAR-L listserv as they are completed.\n\nNICAR has been expanding its analysis services for broadcasters and newspaper reporters. This goes far beyond helping reporters process tricky tapes - from restaurant inspections to property records. NICAR is doing more and more data analysis for reporters. This year, we've begun analyzing records using mapping programs. For more information about how NICAR can help you, call NICAR at (573) 882-0684 or e-mail Andy Lehren at andy@nicar.org.\n\nWe Want You\n\nIRE is reviving and restructuring our system of state and regional coordinators and volunteers. If you'd like to volunteer to be a local coordinator or just work on specific projects, now is the time to let us know.\n\nAs a local coordinator, you might organize an evening reception for journalists in your area, or you might report back to IRE about the great investigations and CAR projects being conducted by local news organizations. We especially need people who will help us recruit new IRE members. Becoming a local coordinator or volunteer is a great way to get in touch with other journalists and keep yourself current with the best work in the business.\n\nStrengthening our presence locally is an effective, grassroots way we can reach out to journalists who can't always make it to a regional or national conference. We know that IRE has a good thing going, and we need you to help get the word out.\n\nIf you believe in the IRE mission and have some time to lend, call Cory de Vera at (573) 882-2042 for a volunteer application form. We'll let you know who else has volunteered in your area, and send you a volunteer's information pack.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-rigging-the-results","slug":"1997-june-rigging-the-results","url":"https://dwillis.github.io/uplink/article/1997-june-rigging-the-results.html","year":1997,"month":"June","monthNum":6,"headline":"Rigging the Results","kicker":"CAR Trips: First ventures","author_name":"Bob Sullivan","author_title":"Reporter","affiliation":"MSNBC","summary":"MSNBC investigated potential bid-rigging in FCC spectrum auctions by analyzing massive datasets to uncover whether telecommunications companies used coded bid amounts to signal competitors and suppress competition. The investigation combined long-term computer-assisted reporting with spot news coverage after a public complaint between two companies sparked a Department of Justice inquiry.","keywords":["auctions","bidding","spectrum"],"topics":["Investigative Methods","Business & Economy","Technology"],"technologies":["Access","DBF"],"full_text":"MSNBC's investigation into bid-rigging in FCC spectrum auctions was an interesting mix of a long-term CAR project and spot CAR reporting.\n\nMy editor, Mark Pawlosky, had charged me with finding databases regarding the FCC's auctioning of spectrum bands to telecommunications companies.\n\nThere is so much money changing hands in these auctions - $23 billion has been raised since the auctions started three years ago that the process was worth taking a look at.\n\nWith our telecommunications reporter, David Bowermaster, I had been in touch with Andy Lehren at NICAR regarding what data could be found or FOIA'ed.\n\nWe had asked for some pretty thorough data, not knowing what we were looking for, and NICAR was assembling it in Missouri.\n\nA public fight\n\nThen on May 1, Reuters reported that two small companies involved in the January auction for Personal Communications Services licenses (PCS devices) were in a public squabble. High Plains Wireless filed a complaint with the FCC accusing Mercury PCS of \"bid-signaling\" in January's DEF block auctions. The complaint spurred an investigation by the Justice Department, and reports said the last three digits of bid amounts were somehow involved. That was the burst of sunlight which we needed. Now we had something to look for. Companies issued bids that ended in unexpected numbers, like $1,100,136, as a signal to competitors. The signal might tell others to stop bidding on that market, or it might contain a message about bidding in another of the 1,479 licenses available in that auction. In any case, the hypothesis was that companies used this signaling to limit competition for licenses and by mutual unspoken agreements, keep prices down.\n\nNow, NICAR was able to point me to the enormous and unruly data the FCC had posted on its Web site regarding the PCS license auctions. In the DEF blocks, there were 276 rounds of bidding. Sure, all the data was on the Web site- in over 1,000 different .dbf files. There were six files for round one with all the round's bids, one with the high bid before the round, one with the high bid after the round, one with that round's withdrawn bids, etc.\n\nA sample test\n\nAt first, we agreed it might be futile to download all of it. But we figured there might be a way to test a sample of the data to see if it might bear fruit.\n\nFirst, we looked at the winning bids for the 1,479 licenses, to see if anything about the last three digits of the winning bids stuck out. Not really.\n\nSo then we cherry-picked every 20th round, and ran some tests on a mini-version of the data. We had some notion that bidders were allowed to lower their bids as the process progressed if they weren't challenged in a market.\n\nThat seemed odd, so we went hunting for markets where the winning bid was less than a bid made during the rounds. We found a few, but that was also basically a dead end.\n\nIt did point to a few markets, though, and when examined, the bidding in those markets exhibited what seemed to be odd bidding behavior - and about half of them involved a company named Western Wireless.\n\nThere seemed to be multiple instances of the same dollar amount being bid by the same company for the same license. That didn't make any sense why would they have to make redundant bids?\n\nExpanding the search\n\nWe now figured it was worth our while to get a picture of what happened in all 276 rounds of bidding.\n\nIt took most of a weekend, but we grabbed all the bids made in all 276 rounds. We then imported the tables into Microsoft Access.\n\nHad I more experience, I would have written some kind of script to bundle them all into one table.\n\nBut not knowing any other way, I used one append query at a time to compile into one table all the bids made in all the rounds on the DEF block.\n\nFortunately, each round had a field with the round number, so we could assemble them into one table without losing data or being forced to add markers.\n\nThe results were now quite good - I built a quick front end which allowed me to query one market at a time and showed the bidders and bids as they marched from first round to winning bid.\n\nThe results and more work\n\nWhat emerged? The most noticeable was Western, which seemed to have a habit of making the same bid over and over - in fact, it won 10 licenses in the 136th round with that kind of redundant bid.\n\nThere were plenty of other signals, too like Comcast's bid for a Philadelphia license that ended in 346, when that was also the city code for Philadelphia - or Western's three round 83 winning bids, which ended in 083.\n\nBut we only had half the picture, and Bowermaster's reporting here pointed us to the other half.\n\nConsultants told him that a frequent method for signaling was to make a bid, withdraw it quickly (sometimes in the same round), then make the same bid again.\n\nSo, it was back to the Web site for the 276 .dbf files of withdrawals.\n\nI then had to merge the withdrawals and the bids into one table - because my goal was to follow all relevant activity in a market as bids progressed.\n\nTo do this I had to add a field, because the data didn't indicate whether a bid was a withdrawn.\n\nI just added a withdrawal field to my combined table, then pasted in the withdrawal tables, with \"default for new records\" set at \"yes.\"\n\nThe 552 tables assembled into one, I now had a complete picture of the competitive bidding for a market.\n\nSure enough, those 10 licenses Western had won in Round 136? In round 135, they had withdrawn a bid for the same amount in every case.\n\nWhen we called Western to comment on our findings, they refused, but did say they had been contacted by the Justice Department.\n\nBut there was no reason to pick on Western dozens of other companies had done similar things.\n\nOmnipoint Corp., bidding as OPCSE-Galloway Consortium, made and then withdrew bids on 171 occasions - 65 times Omnipoint entered bids and then withdrew them in the same round.\n\nOn our Web site, we were able to post round-by-round bids for about 20 communities which showed this kind of bid-signaling.\n\nBowermaster's reporting, combined with work from Brock Meeks, revealed that this behavior actually had its roots in \"game theory,\" which was applied in the design of the auctions.\n\nGame theory is elaborate, but suggests temporary alliances (as you might do in the board game \"Risk\") and poker tactics to succeed in battle.\n\nSurprise - some of the consultants who designed the auctions actually consulted with companies doing the bidding.\n\nSeveral say the bid-signaling was fairly well-known and accepted as part of the process but not by all.\n\nThe Justice Department is currently trying to determine if bid-signaling was in fact bid rigging if it was fair game or collusion.\n\nMercury PCS has replied to the FCC saying they did use these tactics but argued that they were not illegal, and used our story extensively in their request for dismissal of High Plains' complaint.\n\nThe debate on legality continues-as does the debate on whether \"everybody was doing it.\"\n\nAfter all, even if it wasn't illegal, and most companies knew about it, those who didn't were disadvantaged.\n\nAnd the FCC did issue a warning after round 135 complaining that withdrawals were being misused. The Justice Department investigation is ongoing.\n\nThe good news is, this is a CAR story that broke off a beat, and because we were already doing some homework in the area, we pulled together a pretty exhaustive story five days after the initial story broke.\n\nBob Sullivan can be reached at (202) 936-7751, or send e-mail to Bob.Sullivan@MSNBC.COM","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-whats-your-function","slug":"1997-june-whats-your-function","url":"https://dwillis.github.io/uplink/article/1997-june-whats-your-function.html","year":1997,"month":"June","monthNum":6,"headline":"What's Your Function?","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"","affiliation":"NICAR/Missouri School of Journalism","summary":"This article explains how to create User Defined Functions (UDFs) in FoxPro programming language, demonstrating the process with simple examples like noDot() and rSplit() functions that simplify common operations.","keywords":["FoxPro","functions","programming"],"topics":["Technology"],"technologies":["FoxPro"],"full_text":"The FoxPro programming language is composed of command and function words. You use these for simple interactive commands and for programs, simple and complex. It's also possible to combine the existing functions and command words, package and label them as a new function, then use that new function just like the built-in functions.\n\nThis make-your-own-tool capability is a fundamental feature of most programming languages. I'll come back to the uses and advantages of making functions. First, let's make a really simple one.\n\nThe first function we'll write doesn't really add anything new to FoxPro, though that's the usual reason for writing one. The function noDot() returns a string with all the periods stripped out. It's equivalent to, but shorter than the second command.\n\n*The function we'll write will be used like this:\n? noDot (address)\n\nIt is equivalent to:\n? strtran (address, )\nExample code\nFUNCTION noDot Parameters cString RETURN strtran ( cString, )\n\nThat's it. Type three lines, save it, and you have a new function for FoxPro. The manual calls them UDFs, or User Defined Functions. The next function we'll write doesn't add anything new to FoxPro either, but a handy substitute for its hard-to-memorize-and-type equivalent.\n\nThe function rSplit() returns the part of a string that's on the right side of a separating character.\n\nThe function we'll write will be used like this:\n? rsplit (name)\nIt is equivalent to:\n? substr (name, at name) )\n\nThis one is a bit more complicated. Also, we'll write it in several passes, adding and explaining features, improvements along the way.\n\nFirst version FUNCTION rSplit\n\nparameters cString nSplitPos = at (',' , cString) RETURN alltrim (substr (cString, nSplitPos + 1\n\nMost of the time, the comma is the separating character in the string we want to split. Our function, as written, assumes that. However, if we want to use this function to split apart a string based on another character, we'd have to edit the code to change the quoted character in the second line, where the at() function finds its location.\n\nWhen we were done with that odd, special case, we'd have to remember to change the code back to the original version, which assumed a comma as the separator.\n\nOur second revision makes a better way of doing this: When calling the function, we specify the string to split and the separating character.\n\nSecond version: add a parameter for separator FUNCTION rSplit\n*usage: rSplit (name, ) parameters cString, cSeparator jnsplitPos = at (cSeparator, cString) RETURN alltrim (substr (cString, nSplitPos + 1\n\nBut this still isn't as smart as it should be. We know that most of the time, the comma is the separating character; our function should reflect that. So we'll make the comma the default choice, unless another separating character is supplied to the function.\n\nWe can handle this default-or-optional feature if we can tell the function to find out how many parameters were passed to it. That's what the parameters() function does: counts how many parameters were supplied to the current function. If rSplit() has two parameters supplied to it, then a separating character has been explicitly defined. If it has one parameter, that means the function has to explicitly fill in what we let the user imply: the separator is a comma.\n\nThird version: make separator DEFAULT to a comma FUNCTION rSplit usage: rSplit (name) rSplit (offenses, parameters cString, cSeparator if parameters ( ) = 1 cSeparator = 'i' endIF\n\nnSplitPos = at ( cSeparator, cString) RETURN alltrim (substr ( cString, nSplitPos + 1 * end of rSplit ( )\n\nThe function seems perfect now: It's as smart and simple as it can be. Still, we're going to make one more improvement.\n\nIt may not even seem like a real improvement or a necessary one. It probably seems a bit arcane an attempt to guard against the improbable.\n\nI'm assuming three things about programs:\n1) Nothing is improbable.\n2) If the improbable happens, make sure that either there is no damage or there is a graceful way out.\n3) The last improvement involves a significant programming concept.\n\nInside this function we are creating a new variable, nSplitPos, by assigning a value to it. We want to be able to use this new function in any new or old FoxPro script we create and we have to ensure that there are no surprises.\n\nWe may think that it is not very likely that there is a variable with the name nSplitPos already alive and working somewhere else in the currently running FoxPro script.\n\nBut if there were already such a variable, the assignment statement in our function would overwrite the current value held by nSplitPos.\n\nAnytime your code does something you didn't intend or, worse, weren't even aware of, that's bad. While it may not be immoral, it can cause you real grief trying to find and fix the damage.\n\nTo make sure our function couldn't possibly interfere with any other variables, we'll add the statement:\n\nprivate inSplitPos\n\nThis hides any other variables outside the function so it can't interfere with them.\n\nFourth version: make sure variables created by the function do not clash with any others.\n\nFUNCTION rSplit parameters cString, cSeparator private nSplitPos if parameters ( ) = cSeparator = ',' endIF nSplitPos = at (cSeparator, cString) RETURN alltrim (substr (cString, nSplitPos + 1 * end of rSplit ( ) Richard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-must-have-sites","slug":"1997-june-must-have-sites","url":"https://dwillis.github.io/uplink/article/1997-june-must-have-sites.html","year":1997,"month":"June","monthNum":6,"headline":"Must-have sites","kicker":"On the Internet","author_name":"Nora Paul","author_title":"","affiliation":"The Poynter Institute","summary":"This article highlights useful internet resources for journalists, including sites for locating statistics from federal agencies and accessing news archives from newspapers across the country.","keywords":["statistics","research","journalists"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Summertime and the living is easy. The Internet can help make some of your reporting tasks easy, too.\n\nSince this issue of Uplink, according to our esteemed editor, is a hodgepodge, I thought I would honor the theme with a hodgepodge of Internet sites that you really should know about.\n\nLocating statistics\n\nFinding the right statistics to illustrate your story or to verify what a source might tell you can be hard. Here are two great sources to make locating statistics easy:\n\nFedStat: http://www.fedstats.gov/\n\n\"More than 70 agencies in the United States Federal Government produce statistics of interest to the public.\"\n\nThe Federal Interagency Council on Statistical Policy maintains this site to provide easy access to the full range of statistics and information produced by these agencies for public use.\"\n\nIn a beautifully organized site, this compilation of statistic pages gives you tables and charts on everything from breastfeeding to union membership from such sources as the CDC, the Bureau of Labor Statistics, the Social Security Administration. The search function lets you target the topic you need or browse through the alphabetical listing.\n\nhttp://www.sau.edu/cwis/internet/wild/Refdesk/Stats/staindex.htm#useful\n\n\"Where the Wild Things Are: Librarian's Guide to the Best Information on the Net How Much, How Many? Statistical Sources and Calculation Tools on the Net.\"\n\nThis directory, compiled by the St. Ambrose University library, links you to great statistic sources, evaluated for their usefulness and currency with the librarian's skills. Organized by economic statistics, social and demographic statistics, useful data and formulas.\n\nLocating news stories\n\nGetting ready to cover a story that has been covered in other papers? Check out what the local coverage has been through the Net. Here are a couple of great sites that can make locating the local media easier:\n\nhttp://www.mediainfo.com/ephome/inpaper/nphtm/online.htm\n\nEditor & Publisher's Online Newspapers - sure, there are several listings of newspaper Web sites but Steve Outing's continues to be one of the best, most comprehensive and well maintained. Find a local site and, hopefully, they will have a couple days of news stories available.\n\nhttp://sunsite.unc.edu/slanews/internet/archives.html\n\nNewspaper Archives - I call them angels of the Internet, those people who compile useful listings of Internet resources and eliminate so much of the hunting and pecking.\n\nOne of the angels is Margot Williams, Washington Post researcher, who had her class at George Washington University find all the newspapers on the Internet that have archives of past stories available.\n\nThe handy chart lists by state the newspapers with archives, the date range available and the pricing (an amazing number still are free, get to them quickly, though!)\n\nWhile you are at the Special Libraries Association, News Division site, check out another angel's listing - Kitty Bennett's, researcher at the St. Petersburg Times, compilation of expert directories and sources on the Internet: http://sunsite.unc.edu/slanews/internet/experts.html\n\nDoing calculations\n\nQuick, how do you convert meters to feet? Ha! I don't know either. Check out this site to make hard calculations easier:\n\nhttp://www-sci.lib.uci.edu/HSG/RefCalculators.html\n\nCalculators On-Line Center - \"Links to over 3,750 calculators created by over 1,090 very 'CREATIVE' Individuals, Businesses & Tax Supported Entities World Wide.\" Find the calculator for meters to feet in Part II, Mathematics - Weights and Measures. The site it links you to will even convert to \"football field\" equivalents. Calculators On-Line Center is one of those sites that is just interesting to read through the possibilities available.\n\nDirectories & dictionaries\n\nHaving a virtual reference desk of basic books with facts, figures, dates and definitions is easy with the help of these great listings of ready reference sources on the Internet:\n\nhttp://lib-www.ucr.edu/infomine/reference/\nInfomine: Scholarly Internet Resource Collections - click on one of the categories (Acronyms, Biographies, Calendars, etc.) and get a small selection of evaluated reference sources to help answer quick questions. Put together by the University of California, Riverside.\n\nipl.si.umich.edu/ref/RR/\nInternet Public Library - organized by type of reference work (almanac, dictionary).\n\nk12.oit.umass.edu/rref.html\nReady Ref. On the 'Net - arranged alphabetical by subject.\n\nGoing traveling\n\nIt's easy to find information about the places you will be traveling to this summer with the following, one of the best compilations of city information:\n\nhttp://local.yahoo.com/local\n\nYahoo: Get Local - this beautifully designed package of local information, links and news is browsed by putting in the ZIP code of the place you are going.\n\nWhat you'll get back is information about the local weather, major league sports results, the searchable Yellow Pages, links to local news sites, and the regular of Yahoo categories like: business, city guides, education, entertainment and arts, government, health, lodging, etc.\n\nThere is even a list of local UPI stories that can help you pitch your trip as a reporting excursion. Have an easy summer, everyone.\n\nNora Paul can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-a-question-of-diversity","slug":"1997-june-a-question-of-diversity","url":"https://dwillis.github.io/uplink/article/1997-june-a-question-of-diversity.html","year":1997,"month":"June","monthNum":6,"headline":"A Question of Diversity","kicker":"Census Tracking","author_name":"Jennifer LaFleur","author_title":"Reporter","affiliation":"San Jose Mercury News","summary":"The San Jose Mercury News applied the USA Today Index of Ethnic Diversity to analyze population changes in their region, using a probability-based formula to measure how diverse different census tracts are rather than just reporting raw demographic percentages. The index, which ranges from 0 to 1, provides a more nuanced understanding of diversity by showing clustering patterns and can be applied to other demographic measures beyond race and ethnicity.","keywords":["diversity","methodology","demographics"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":["spreadsheet"],"full_text":"After the 1990 Census, the San Jose Mercury News had charted the changes and breakdowns of our diverse area. A few years later, Miranda Ewell wanted to look at what has happened since then in the workplace and in our neighborhoods. After testing a few different measurement tools, I decided to try applying a diversity index to our populations.\n\nI chose the USA Today Index of Ethnic Diversity developed by Philip Meyer of the University of North Carolina at Chapel Hill and Shawn McIntosh of the Dallas Morning News. The USA Today diversity index eliminates the need to analyze the distribution of several factors with one probability-based index.\n\n\"The index has a range from 0 to 1, and its value represents the probability that two people chosen at random from the study population will differ along at least one ethnic dimension.\" (International Journal of Public Opinion Research, Spring 1992).\n\nWe chose this index because it seemed to both show trends of diversity and be easy enough to explain to readers in a few paragraphs.\n\nHere's the formula to calculate the Index of Ethnic Diversity which we applied to each Census tract:\n\nStep 1: Probability that two persons chosen from a population at random will be members of the same racial group: P. R = (A2+B2+C2+D2) where A, B, C & D are the proportions in the population of whites, blacks, native Americans, and Asians or Pacific islanders.\n\nStep 2: Probability that two persons chosen from a population at random will either be both Hispanic or both no Hispanic. This is a separate value because Census questionnaires ask Hispanic origin in a different question from race. Therefore it is possible to be both white and Hispanic or black and Hispanic.\n\nP=(H2+N2)\n\nStep 3: Calculate the probability that two randomly chosen persons are the same in both race and Hispanic/non-Hispanic status: PR * PH Subtract the result from 1: 1 - (PR*PH)\n\nWe created a spreadsheet with the populations and put the formulas into the spreadsheet. We applied the index to our local county totals, but also took our results by Census tract and mapped them to show the clusters of diverse areas.\n\nWe also applied another measure to the counties of a whole over time: the Dissimilarity Index. This is another index from 0 to 1 - the closer to 1 the higher level of segregation in an area. For both Asians and Hispanics, the index has increased. The data also showed decreases for blacks; however, our county has too low a black population for most statistical measures to be accurate.\n\nSo why would you want to do any of this instead of general ethnic breakdowns for your area or even mapped for small areas like Census tracts. Those measures don't really show clustering over the mix of different populations. This would show low diversity for areas that are all Asian in our area or mostly white. This index provided a different look at our area. In addition this Index can be applied to other demographic measures, not just race and ethnicity.\n\nTo view the on-line version of the Mercury News piece on diversity, go to www.sjmercury.com/news/local/census/index.htm\n\nTechnology also changed the way we brought this story to the reader, in addition to showing maps in the newspaper, readers could go on-line, click on their neighborhood and see what the demographic breakdown is. This is something we just couldn't do on paper.\n\nJennifer LaFleur can be reached at (408) 920-5000 or send e-mail to jenster@sjmercury.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-get-your-training","slug":"1997-june-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-june-get-your-training.html","year":1997,"month":"June","monthNum":6,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers computer-assisted reporting training for journalists through various workshops, bootcamps, and on-the-road sessions throughout 1997-1998, teaching skills like data transfer, spreadsheet building, and database management.","keywords":["training","journalism","databases"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","spreadsheet","PC"],"full_text":"NICAR provides specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases. Cost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below. July 9, 1997 - New York, N.Y. On-the-road training for CBS News July 10, 1997 - New York, N.Y. On-the-road training for Newsday July 11-13, 1997 - New York, N.Y. Computer-Assisted Reporting Workshop at New York University Sponsored by NICAR and New York University. There are 16 spots available for this three-day hands-on seminar. Register online or call NICAR offices at (573) 882-0684 for a flier and registration form. July 14-15, 1997 - Washington, D.C. On-the-road training for the Paul Miller Program Sponsored by the Freedom Forum. July 17-18, 1997 - Chicago, Ill. Join NICAR for hands-on training at the National Association of Black Journalists' convention in Chicago. Limited to conference participants. July 25-26, 1997 - Philadelphia, Pa. Reporting with Education Statistics Workshop at Temple University The first of NICAR's specialized workshops on beat reporting, hosted by Temple University and The Philadelphia Inquirer. The workshop includes a day of panels that pair reporters experienced in education statistics with experts in the fields, and a day of hands-on training using the techniques they've described. Register online or call NICAR offices at (573) 882-0684 for a flier and registration form. August 13-17, 1997 - Boston, Mass. Join NICAR for hands-on training at the Asian American Journalists Association in Boston. Limited to conference participants. August 19-22, 1997 - Huntington, W.Va. On-the-road training for Marshall University September 11-13, 1997 - Washington, D.C. Computer-Assisted Reporting Workshop Sponsored by NICAR and the Medill School of Journalism October 20-24, 1997 - Ft. Lauderdale, Fla. On-the-road training for the Ft. Lauderdale Sun-Sentinel February 1-3, 1998 - Minneapolis, Minn. On-the-road training for the Minneapolis Press Foundation\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969 March 5-8, 1998 - Indianapolis, Ind. NICAR Conference June 4-7, 1998 - New Orleans, La. IRE National Conference\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo. As with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size. For information, call Wendy Charron at (573) 882-0684. August 10-15, 1997 - Columbia, Mo. NICAR Regular Bootcamp September 28 - October 3, 1997 - Columbia, Mo. CAR Workshop for Minority Journalists Sponsored by NICAR and the Multicultural Management Program January 4-9, 1998 - Columbia, Mo. NICAR Regular Bootcamp","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-from-the-nicar-library","slug":"1997-june-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-june-from-the-nicar-library.html","year":1997,"month":"June","monthNum":6,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering aviation, transportation, finance, health care, and crime data from the 1970s through 1990s. The resources are available for investigative journalism purposes with contact information for pricing and details.","keywords":["databases","government","journalism"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\nFederal Aviation Administration's accidents and incidents, including major plane crashes since 1974.\nNASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\nFederal Election Commission campaign contributions by individuals and political action committees, 1990-1997.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings, 1991-1996.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration, 1974-1997.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\nFederal procurement data, 1992-1996, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1996.\nNational Bridge Inventory System data, includes inspection grades, 1994-1995.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes 1995 data.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1989-1995. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nThe National Inventory of Dams, 1991-1995.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1996.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1975 to 1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-june-bits-bytes-and-barks","slug":"1997-june-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-june-bits-bytes-and-barks.html","year":1997,"month":"June","monthNum":6,"headline":"Bits, Bytes and Barks","kicker":"Membership Message","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR announce membership information, highlight recent listserv discussions, thank volunteers from the Phoenix conference, and preview upcoming conferences in Indianapolis and New Orleans.","keywords":["membership","conference","journalists"],"topics":["Journalism & Newsroom"],"technologies":["Adobe Acrobat","Access","World Wide Web","FTP"],"full_text":"Bits, Bytes and Barks\n\nMembership Message\n\nYou can join IRE via our Web server, or you can download a copy of our membership form in Adobe Acrobat format from the Web site and submit it by fax to (573) 884-5544. For more information, check out http://www.ire.org/membership.htm\n\nNICAR Net\n\nRecent topics on the IRE and NICAR listserv have included campaign finance data, troubles with traffic accident data, ethics, medical investigation, medical investigations and Access tips. To subscribe to IRE-L or NICAR-1, send e-mail to listproc@lists.missouri.edu In the body of the message, type: subscribe NICAR-L<your name> subscribe IRE-L<your name> Also, check out the IRE-L and NICAR-L mailing list archives on our website at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.\n\nThanks for the Memories\n\nPhoenix turned out to be a great conference thanks largely to the tireless efforts of volunteers and speakers. NICAR and IRE would like to send special thanks to speakers and demo room lecturers.\n\nThe conference would have been impossible to put on without the help of reporters, editors, researchers and professors across the country. Plans are already being made for next year's conferences. NICAR will hold its national conference March 5-8 in Indianapolis, IN. The IRE National Conference will be held in The Big Easy - New Orleans, LA - June 4-7. For information or to register, call Lisa Barnes at (573) 882-8969. We're also looking for any suggestions for improvements for next year's conferences and welcome any volunteers or speakers who want to participate. Submit your ideas to lisa@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-july-getting-by-on-a-shoestring","slug":"1997-july-getting-by-on-a-shoestring","url":"https://dwillis.github.io/uplink/article/1997-july-getting-by-on-a-shoestring.html","year":1997,"month":"July","monthNum":7,"headline":"Getting by on a shoestring","kicker":"","author_name":"Loraine Anderson","author_title":"Regional Editor","affiliation":"Traverse City Record-Eagle","summary":"Loraine Anderson discusses how the Traverse City Record-Eagle manages computer-assisted reporting with limited resources, sharing tips for small newspapers.","keywords":["small","resources","reporting"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Excel","Access","Word","PC","Internet","FEC Info"],"full_text":"We don't have a CAR post at the Traverse City Record-Eagle. It would be a luxury we couldn't afford. We have 11 reporters to cover our 13 mostly rural counties in northwestern Lower Michigan. We have a daily circulation of about 26,000 and a Sunday circulation of 40,000. I am the regional editor and oversee our project reporting as well as our theme beats: business, environment/health, family/education. Our newspaper recomputerized a year ago, so we are set up beautifully for CAR. We are now completely PC-based in our newsroom with e-mail on all computers. We have Internet access on five of the PC-120 terminals in the newsroom. None of these computers are loaded with spreadsheet or database management software, except for the elementary spreadsheet functions available in Microsoft Word. We do have Excel and Access loaded on computers elsewhere in the building (business department and in the office of our weekly newspaper), so we can take disks to those machines and work with them. The challenge now is to learn how to maximize the use of our computer system. I've picked it up because CAR has so many applications for a small newspaper with a small staff and large coverage area. I'm trying to do three things: get myself up to speed so that I can work efficiently with databases; determine for management what our newsroom needs so that we can gather and analyze data more conveniently; build a front-end system for our databases so that our reporters can easily access the information without having to know the ins and outs of Access and Excel. Most of our CAR work so far has been quite elementary, designed to teach me how to efficiently use Access and Excel so that I can know it well enough to share it with the reporters. I have both Access and Excel on my computer at home, where I've done a lot of the work. It's meant an incredible amount of my own time, but I don't mind too much because I'm teaching myself. Here are some of the things we have done: Congressional campaign finance. Our biggest CAR project this year has been the campaign finance records for our three congressional districts and six state legislative districts for the 1994-96 election cycle. Last fall, I used the American University site to download congressional data and spent many a weekend cleaning the data and checking business directories and websites so that I could correctly categorize them using codes listed in Larry Makinson's book, 'Follow the Money,' published by the Center for Responsive Politics, which has an excellent web site. This enabled me later to do sorts to find out how various special interest groups - health, oil and gas, etc. - gave to our congressional leaders. I did the number crunching and gave them, as well as the assignment, to the reporter who had two days to report and write his story. Our stories were quite basic because we didn't have enough time to analyze the data and question some of the donations. State campaign finance. Getting the data for the state representatives was more involved because Michigan's election data are not online. It all had to be typed and categorized. We used information found at the FEC Info page (www.tray.com), looking up all of the over $200 contributors by ZIP code and sifting through the information for anyone who gave more than $5,000. This provided not only an interesting story, but the beginning of a better understanding of the politics and business of our coverage area. We also have downloaded state health statistics, school proficiency scores and EPA Toxic Release Inventory figures for in-depth reports. I also have all of the 13-county demographic information on disk in Excel files, plus area tax base statistics obtained from county equalization offices and area construction permits obtained from county building departments. My long-term goal for all this information is to build a front-end system and get it networked into our newsroom computers so that our reporters can easily use it without having to understand all the ins and outs of Access and Excel. I am in the process of learning how to do that now, using Baltimore Sun's electronic news editor Mike Himowitz's front-end tutorial, which I downloaded from his site (http://www.clark.net/pub/ mikeh/mike/car.htm). In order to do this, I've needed to learn more of the intricacies of Access, so I am currently going through the Que Access 95 Tutor book and CD.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-july-joining-hands","slug":"1997-july-joining-hands","url":"https://dwillis.github.io/uplink/article/1997-july-joining-hands.html","year":1997,"month":"July","monthNum":7,"headline":"Joining Hands","kicker":"Papers help start CAR","author_name":"Joanna Kakissis","author_title":"","affiliation":"NICAR","summary":"Journalists at small newspapers are forming computer-assisted reporting networks to share training, data, and resources, with examples including CARwash in Washington state and similar consortiums in Wisconsin and across Knight Ridder publications.","keywords":["journalism","data","collaboration"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["World Wide Web"],"full_text":"When David Cuillier, an editor at a small daily in Washington state, decided to form a computer-assisted reporting network for journalists in his state two years ago, he was listening - in exasperation - to members of the governor's task force debate limiting access to records. 'I was the only reporter there,' says Cuillier, assistant city editor of the Tri-City Herald in Kennewick, Wash. 'The state had already closed records for gun permits a while ago. I realized then that if more people were asking for data, the state wouldn't be so quick to close things down.' Shortly after the meeting, Cuillier called every one of the 23 newspapers in the state, speaking with reporters who were interested in obtaining and using computerized data in their publications. About 10 of the newspapers Cuillier contacted have become regular contributors to the CAR network, which Cuillier has named 'CARwash.' By mid-August, Cuillier plans to have a Web site with information about the network as well as a library of CAR stories done in Washington state. 'It's helped a couple of papers get rolling on using CAR in smaller stories and building up to using it in bigger projects,' he said. Smaller newspapers face high costs and limited resources when trying to incorporate computer-assisted reporting and often don't have the money or time to fight extended public information battles. As a result, several journalists from small newspapers - and even journalists from newspaper corporations like Knight Ridder - have formed networks to coordinate CAR training and data acquisition. In Wisconsin, journalists from the Associated Press and the Wisconsin State Journal have joined forces with other state reporters and editors to acquire state data in electronic form, have a third party clean up the data and then distribute it to members of the informal CAR network in the state at significantly reduced prices. 'So far, this (consortium) hasn't had a regular meeting schedule,' said Andy Hall, an investigative reporter with the Wisconsin State Journal and a founding member of the state's CAR network. 'It's just been an erratic series of phone calls and faxes But we've made progress. We're in the process right now of acquiring a huge database from the state department of transportation listing traffic accidents in the past seven years Information like this could be used for anything from in-depth projects to quick hits.' That CAR methods could help enhance both spot and investigative coverage in part motivated executives at Knight Ridder news service to sponsor a CAR network for all their newspapers. The newspaper chain currently sponsors KRICAR, the Knight Ridder, Inc. Computer Assisted Reporting Task Force. KRICAR's coordinator, the Philadelphia Inquirer's Neill Borowski, said the task force's goal is to nurture a 'big brother, big sister' program where bigger newspapers whose journalists are already trained in CAR techniques help journalists from smaller Knight-Ridder papers. 'We want to create a network where small newspapers don't feel intimidated by calling the bigger newspapers and asking questions about computer-assisted reporting and analysis,' said Borowski, who heads CAR work at the Inquirer. 'And so far, it seems to be working. About once a month, we get calls from smaller papers and journalists who want to know if we have any suggestions on subjects ranging from data analysis or to how to do certain queries.'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-july-abatement-analysis","slug":"1997-july-abatement-analysis","url":"https://dwillis.github.io/uplink/article/1997-july-abatement-analysis.html","year":1997,"month":"July","monthNum":7,"headline":"Abatement analysis","kicker":"Tax abatements","author_name":"Geoff Dutton","author_title":"","affiliation":"The News-Herald","summary":"A journalist's computer-assisted reporting project on tax abatements revealed flawed data, but the setback led to a comprehensive five-part series exposing how local municipalities were caught in bidding wars and how most relocated companies simply moved between nearby communities rather than bringing genuine economic growth.","keywords":["reporting","taxes","development"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["spreadsheet"],"full_text":"Computer-assisted reporting magnifies everything, for good and for ill. A previously indistinguishable mess of details will suddenly zoom into focus, often revealing surprising trends. At the very least, CAR can generate solid numbers to accurately gauhge and explain already perceived trends. That's why I got into it. Veterans of CAR also had repeatedly warned me about a downside - the tedium involved in database reporting. Seeing the phenomenal results of their work, however, I was willing to endure. Still, my biggest CAR disappointment one I was not prepared for - came recently, after months of compiling a database for my first larger scale CAR project. Industrial and commercial tax abatements were the subject. They had become commonplace over the years within The News-Herald's territory of eastern suburban Cleveland. Even The News-Herald had received a big one. The property tax cuts are given by local municipalities to encourage development, retention of existing businesses and job growth. Never had the program been independently and comprehensively analyzed locally. But the individual tax cuts had been monitored on the local, county and state level since 1994. I wanted to compile the various individual reports, plug them into a spreadsheet and determine the impact of the tax cuts. How many tax dollars have been forgone? How much have businesses invested? How many jobs have been created? How much do those jobs pay? Which companies weren't living up to their promises? Unfortunately, my conclusion - after months of hard work - was that flimsy data precluded a solid evaluation. As it turned out, it wasn't a dead end. The predicament just forced a new approach to the story. I made a list of roughly a dozen interesting story angles and disparate conclusions. With the help of an insightful editor, I grouped the pieces into what would become a five-part series. The series focused on my biggest stumbling block the lack of credible data. It became a topic for the final day of the series. The series broke down this way: Ironically, one of the strongest parts of the series focused on companies that were given tax abatements to relocate. There had been 70 tax abatements granted in Lake County since the program began in 1990, forgiving more than $56 million in taxes. Most of the 70 tax breaks were to help companies expand. But of the 23 given to bring in new companies, an obvious pattern existed. Most of the relocating companies 18, to be exact moved from one local community to another. Nine of those 18 actually moved within the county. As it turned out, some communities felt they were being caught in a tax abatement 'bidding war' with their neighbors. Another emerging issue in Ohio is urban sprawl. Urban folks complain about the government investing in building new highways, sewers and other infrastructure to allow people to move out of the city. Instead, critics argue, the money should be spent to improve older urban communities. The local industrial relocations identified in the first day's story clearly followed the trend of moving outward into greener pastures literally and figuratively in terms of tax cuts. A small spreadsheet of basic Census data also demonstrated the corresponding shift in population and wealth during the past 30 years. For the final day, I tackled the issue of the slippery numbers. In monitoring the tax abatements - which are given in exchange for promises of job creation - communities often rely on annual estimates provided by the companies. The numbers were suspect. The company receiving the county's largest tax cut of $10 million, for example, was getting credit for creating 143 jobs while openly downsizing. And some companies obviously never could live up to their promises. When the raw payroll and job numbers - those that were the basis of granting the abatements - were calculated, they didn't add up. Some deals, if fulfilled, would equate to creating jobs paying an average of more than $100,000 annually, while others promised full-time jobs paying $12,000 a year.\n\nstate wouldn't be so quick to close things down.\"\n\nThe Washington connection\n\nShortly after the meeting, Cuillier called every one of the 23 newspapers in the state, speaking with reporters who were interested in obtaining and using computerized data in their publications. About 10 of the newspapers Cuillier contacted have become regular contributors to the CAR network, which Cuillier has named \"CARwash.\" By mid-August, Cuillier plans to have a Web site with information about the network as well as a library of CAR stories done in Washington state.\n\n\"It's helped a couple of papers get rolling on using CAR in smaller stories and building up to using it in bigger projects,\" he said.\n\nJoining forces in Wisconsin\n\nSmaller newspapers face high costs and limited resources when trying to incorporate computer-assisted reporting and often don't have the money or time to fight extended public information battles. As a result, several journalists from small newspapers - and even journalists from newspaper corporations like Knight Ridder - have formed networks to coordinate CAR training and data acquisition. In Wisconsin, journalists from the Associated Press and the Wisconsin State Journal have joined forces with other state reporters and editors to acquire state data in electronic form, have a third party clean up the data and then distribute it to members of the informal CAR network in the state at significantly reduced prices.\n\n\"So far, this (consortium) hasn't had a regular meeting schedule,\" said Andy Hall, an investigative reporter with the Wisconsin State Journal and a founding member of the state's CAR network. \"It's just been an erratic series of phone calls and faxes. But we've made progress.\n\n\"We're in the process right now of acquiring a huge database from the state department of transportation listing traffic accidents in the past seven years. Information like this could be used for anything from in-depth projects to quick hits.\"\n\nCross-country connection\n\nThat CAR methods could help enhance both spot and investigative coverage in part motivated executives at Knight Ridder news service to sponsor a CAR network for all their newspapers.\n\nThe newspaper chain currently sponsors KRICAR, the Knight Ridder, Inc. Computer Assisted Reporting Task Force.\n\nKRICAR's coordinator, the Philadelphia Inquirer's Neill Borowski, said the task force's goal is to nurture a \"big brother, big sister\" program where bigger newspapers whose journalists are already trained in CAR techniques help journalists from smaller Knight-Ridder papers.\n\n\"We want to create a network where small newspapers don't feel intimidated by calling the bigger newspapers and asking questions about computer-assisted reporting and analysis,\" said Borowski, who heads CAR work at the Inquirer. \"And so far, it seems to be working.\n\n\"About once a month, we get calls from smaller papers and journalists who want to know if we have any suggestions on subjects ranging from data analysis or to how to do certain queries.\"\n\nKRICAR also operates a listserv that is accessible to all Knight-Ridder papers.\n\nAny questions about data acquisition or analysis can be posted to this list as well, said Borowski.\n\nAn Inquirer news researcher recently used the list to distribute Uniform Crime Report data - which she had converted into spreadsheet form - to Knight Ridder newspapers. Two newspapers - one large and one small - scooped their competition because of access to that information, said Borowski.\n\nFor more information on the KRICAR, see the Web site at http://CARA.phillynews.com/KRICAR/KRICAR.html\n\nFor more information on CARwash, check out the Web site (after August 15) at http://www.wnpa.com\n\nAccess and Excel on my computer at home, where I've done a lot of the work. It's meant an incredible amount of my own time, but I don't mind too much because I'm teaching myself. Here are some of the things we have done:\n\nCongressional campaign finance\n\nOur biggest CAR project this year has been the campaign finance records for our three congressional districts and six state legislative districts for the 1994-96 election cycle.\n\nLast fall, I used the American University site to download congressional data and spent many a weekend cleaning the data and checking business directories and Web sites so that I could correctly categorize them using codes listed in Larry Makinson's book, \"Follow the Money,\" published by the Center for Responsive Politics, which has an excellent Web site.\n\nThis enabled me later to do sorts to find out how various special interest groups - health, oil and gas, etc. - gave to our congressional leaders. I did the number crunching and gave them, as well as the assignment, to the reporter who had two days to report and write his story. Our stories were quite basic because we didn't have enough time to analyze the data and question some of the donations.\n\nState campaign finance\n\nGetting the data for the state representatives was more involved because Michigan's election data are not online. It all had to be typed and categorized.\n\nWe used information found at the FECInfo page (www.tray.com), looking up all of the over $200 contributors by ZIP code and sifting through the information for anyone who gave more than $5,000. This provided not only an interesting story, but the beginning of a better understanding of the politics and business of our coverage area.\n\nWe also have downloaded state health statistics, school proficiency scores and EPA Toxic Release Inventory figures for in-depth reports. I also have all of the 13-county demographic information on disk in Excel files, plus area tax base statistics obtained from county equalization offices and area construction permits obtained from county building departments.\n\nMy long-term goal for all this information is to build a front-end system and get it networked into our newsroom computers so that our reporters can easily use it without having to understand all the ins and outs of Access and Excel. I am in the process of learning how to do that now, using Baltimore Sun's electronic news editor Mike Himowitz's front-end tutorial, which I downloaded from his site (http://www.clark.net/pub/mikeh/mike/car.htm). In order to do this, I've needed to learn more of the intricacies of Access, so I am currently going through the Que Access 95 Tutor book and CD.\n\nTips\n\nIf you are from a small newspaper and want to learn CAR, here are some things that will help:\n\nA real desire to learn CAR and a willingness to spend lots of your own time and possibly even some of your own money. I attended the week-long NICAR boot camp at the University of Missouri in May 1996. Though our editor and publisher are supportive of CAR, I had to pay for it myself because there is little money for training.\nHave patience, but be diligent. You can't learn everything overnight, but you have to keep at it. Learn what is in front of you and the next part will make itself clear to you. When I started out, I surfed a lot, finding and bookmarking sites that would be good research sources for data, reporting projects and editorials.\nIf you don't have it, get good spreadsheet and database manager software on a computer with Internet access. I got Access and Excel because our office word processing is Microsoft Word and our systems people were most familiar with that.\nGet on the NICAR listserv and monitor it daily.\nFind other CAR sites that explain some of the basics. In addition to the handouts page at NICAR's site (http://www.nicar.org), I've learned a lot at these sites: Mike Himowitz's Home Page (http://www.clark.net/pub/mikeh/mike/mikehome.htm), the Detroit Free Press training tips page (http://www.freep.com/jobspage/academy/intel.htm), CAR/CARR (http://www.ryerson.ca/~dtudor/carcarr.htm).\n\nLoraine Anderson can be reached at (616) 933-1468, or by e-mail at loraine@traverse.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-july-students-rank-schools","slug":"1997-july-students-rank-schools","url":"https://dwillis.github.io/uplink/article/1997-july-students-rank-schools.html","year":1997,"month":"July","monthNum":7,"headline":"Students rank schools","kicker":"Class Project","author_name":"Daniel J. Foley","author_title":"Professor","affiliation":"University of Tennessee, Knoxville","summary":"A University of Tennessee journalism class partnered with the LaFollette Press newspaper to produce an eight-page special report on Campbell County schools using state education database analysis, resulting in a comprehensive 'report card' of the school district's performance across multiple metrics.","keywords":["journalism","schools","database"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"A public affairs reporting class got some 'real world' publishing experience and a weekly newspaper got a special report it normally wouldn't be able to produce when the two combined on a database study of a Tennessee school system. The joint effort by the School of Journalism at the University of Tennessee, Knoxville, and the 9,000-circulation LaFollette Press resulted in the May 29 publication of an eight-page special tabloid section on schools in Campbell County. The headlines on stories written by the 10 students in my senior-level journalism class convey the flavor of their 'report card' on the school district: 'Campbell schools, students rank among lowest in state' 'Absenteeism is highest in state' 'Test scores: Student trail statewide averages' 'Value added: Schools aren't making the grade' 'Schools get extra $$$ from state' ' But test scores, salaries low' 'Low scores may be linked to poverty' 'None of county's elementary schools accredited' 'Success in college? Scores say no' 'Success at work? It's same story' The students spent five weeks of a 15-week semester learning to use a spreadsheet, analyzing a database compiled by the Tennessee Department of Education, interviewing school officials and then writing, critiquing and rewriting their stories. The state's 21st Century Report Card database includes more than 300 columns of information on each of the state's 139 school districts on such topics as achievement test scores, local funding resources, state aid, per-pupil spending, average teacher salaries, demographics, attendance and accreditation. Similar data are available in many states, so the project methodology undoubtedly could be used elsewhere. The journalism students used Excel 5.0 on Macs to examine the database, but any good spreadsheet will work. The primary database skill used on this particular project was sorting and ranking school districts in various categories. One of the challenges was to identify which category would give the best overall assessment of the district. It is well-known that student achievement on standardized tests in any school district usually reflects the socioeconomic makeup of that district. Students who come from wealthy families with well-educated parents generally score higher than students who come from poor families with lesser-educated parents, regardless of the quality of the schools. Tennessee, like many other states, has implemented a program to measure school impact on student learning, apart from such socioeconomic factors. The 'value-added' assessment system, which was designed by a statistics professor at the University of Tennessee, measures each student's progress from year to year. Results of the value-added assessment have been interesting. In some school districts, students do well on achievement tests, but nevertheless don't make the expected progress from year to year. In other districts, test scores are comparatively low, but students nevertheless make better-than-average progress. One of the things UTK students tried to determine was which school districts do the best on both student achievement and school performance. Here is how they used the data to do the rankings: Student achievement: Tennessee Comprehensive Assessment Program test scores are now available for seven grades (second through eighth) in five subjects - reading, language, mathematics, science and social science. Each districts' percentile scores for those 35 grade-subject areas were added and districts were then ranked from highest to lowest. Campbell County students ranked 107th of 134 districts. (Five districts were not included because they do not have all grades from second to eighth.) School performance: Value-added scores, based on student progress as a percentage of national norms, are now available in five subject areas. Each districts' five scores were added and districts were then ranked from highest to lowest. The Campbell County system ranked 119th. Overall ranking: Rankings for student achievement and school performance were then combined and districts were ranked based on the combined score. Campbell County ranked 124th. The district's poor showing on both student achievement and school performance and its even worse overall ranking provided a theme for the special section. Each student was assigned to evaluate data from a specific portion of the database and to determine how Campbell County did in each category. One student, for example, examined data on dropouts, absenteeism, suspensions, expulsions and promotions. Her key finding may help explain the district's low overall ranking: students in the upper grades in Campbell County had the highest rate of absenteeism in the state. The students used several benchmarks for purposes of comparison: The district's ranking among the 134 districts in the study. The district's score, percentage or number compared to the statewide average. The district's score, percentage or number compared to the national norm. The year-to-year trends within the district (three years of data were available for most categories). In order to avoid stories that were a mass of mind-numbing statistics, each student was instructed to pick and write about only two or three or four of the most revealing numbers. The class interviewed the school superintendent, the county executive and the president of the teacher's union to get their perspectives. Journalists with even modest skills in using spreadsheets could duplicate this project in any community where the state compiles similar information in database format. The project does, of course, involve a lengthy commitment of reporter time. Smaller papers with fewer resources may want to explore a cooperative effort with a college journalism class.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-july-keeping-car-local","slug":"1997-july-keeping-car-local","url":"https://dwillis.github.io/uplink/article/1997-july-keeping-car-local.html","year":1997,"month":"July","monthNum":7,"headline":"Keeping CAR local","kicker":"CAR Trips: First ventures","author_name":"Jim Toler","author_title":"Reporter","affiliation":"The Free Lance-Star","summary":"A newsroom implements computer-assisted reporting (CAR) at The Free Lance-Star to investigate local issues through data analysis, resulting in investigations into police chases, law enforcement spending disparities, and crime statistics that prompt policy discussions and provide important context to the community.","keywords":["data","investigation","newsroom"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["Lotus Approach","Excel"],"full_text":"Tapping a world's worth of information via the Internet. Adding depth and perspective to stories by analyzing data. Probing how governments spend tax dollars and how laws are enforced. Those were the objectives my editor Virginia Demaree and I set for starting CAR at The Free Lance-Star in Fredericksburg, Va. The idea was to get someone in the newsroom comfortable enough with spreadsheets, databases and Internet searches so that other reporters would be able to apply technology to their beats. Considering we used calculators to figure election results last fall, this was a worthy goal. Here are some of the stories that have resulted:\n\nChasing Data\n\nPolice reporters seemed to be writing an increasing number of high-speed chase stories last year. The pursuits often ended in injury-causing crashes and several suspects died. Based on those observations, I set out looking for data on police pursuits. Working with cop reporters, we quickly found out that only some local police and sheriff's departments had electronic records and even then the information was insufficient for comparison.\n\nSo I built a database of 65 chases from stories in our clip files. Using Lotus Approach (an easy database for first-timers), I constructed 25 information fields, and from that I was able to determine trends. Our findings: Police initiate chases for minor infractions a majority of the time, most drivers who flee are under 25 and chases typically end in crashes.\n\nBy examining state police and local figures, we also found that National Transportation Safety Board fatal accident statistics undercounted the number of deaths in Virginia in one of the years we examined.\n\nPolice in the Fredericksburg area haven't rewritten pursuit policies, but the stories and a follow up have prompted debate.\n\nCop and dollars\n\nLocalities in our area between Washington, D.C., and Richmond face similar problems with crime. Yet there's wide disparity in the amounts of money spent on law enforcement among the city of Fredericksburg and neighboring Stafford and Spotsylvania counties. Using an Excel spreadsheet, we compared law enforcement budgets and the number of cops on the street in each locality. In a story published in April, we showed that Spotsylvania County spends the least on law enforcement, though it's one of the fastest growing counties in Virginia. Spotsylvania officials are now in the process of hiring additional deputies.\n\nConcern about crime has been on the rise. Spotsylvania was the scene of an abduction and murder of a teen-age girl last year. In May, two young sisters were abducted and murdered. Like the earlier case, their bodies were found in an adjacent county. The cases have drawn national attention and concern there's a serial killer at work.\n\nAfter the May slayings, we used spreadsheets to analyze violent crime statistics and homicide rates in the region. The stories provided context for the unusual cases.\n\nEducational effort\n\nWhere does all money for area schools go? How much gets to the classroom?\n\nThose were the basic issues examined in an analysis of instructional expenditures over a seven-year period.\n\nEducation Kiran Krishnamurthy and I put the data into spreadsheets. We found one county, which had a reputation for its school system, had fallen behind a neighboring county in per pupil expenditures. The reason: the system known for its schools had to put more into building schools to cope with growth while the other had increased classroom spending.\n\nThe story challenged long-held perceptions about which systems were getting dollars into the classroom.\n\nOverall, our focus is local.\n\nWe want to examine issues that make a difference in the lives of people who live in the community. We try never to forget that behind the Census numbers, government expenditures or the crime rates, there are real people with stories to tell.\n\nJim Toler can be reached at (540) 374-5416, or by e-mail at jimtoler@flstarweb.com/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-july-using-regression-analysis","slug":"1997-july-using-regression-analysis","url":"https://dwillis.github.io/uplink/article/1997-july-using-regression-analysis.html","year":1997,"month":"July","monthNum":7,"headline":"Using regression analysis","kicker":"Stats from the road","author_name":"Sarah Cohen","author_title":"training director","affiliation":"NICAR","summary":"Reporters at the Omaha World-Herald used regression analysis to evaluate school effectiveness by comparing actual test scores to predicted scores based on student demographics, then verified findings through on-site observation. The approach demonstrates that statistical analysis requires both sophisticated methodology and ground-truth reporting to accurately assess school performance.","keywords":["regression","analysis","schools"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["SPSS","SAS"],"full_text":"Using regression analysis\n\nWhen Carol Napolitano and Paul Goodsell wanted to study the effectiveness of Omaha schools, they turned to a tool that reporters increasingly use: regression analysis.\n\nThe method is growing in popularity among education reporters, as NICAR found in our first-ever seminar on advanced computer-assisted reporting techniques in Philadelphia last month, which focused on reporting on education statistics.\n\nBut the Omaha World-Herald's experience shows why using multiple regression analysis a technique that's become a buzzword in newsrooms-require more than some sophisticated software and a desire to show it off.\n\nNapolitano and Goodsell spent weeks inside the schools they identified long after their statistical procedures were completed. What they saw not only put faces on their results, it explained them. In underperforming schools, Napolitano saw chaos. In exceptional schools, she saw teachers who returned year after year, discipline that kept disruptions at bay and strong principals.\n\nDefining Reality\n\nFirst, a few definitions. What is regression analysis? It's just a tool that helps you level the playing field by predicting one number based on values of other numbers. In Omaha and other places, that meant predicting test scores based on demographics of the student population, such as poverty, English language skills and stability (called \"mobility\" in school reports).\n\nTypically, reporters subtract the predicted scores from the actual scores. What's left over is what U.S. News & World Report has started calling \"value added\" in its annual evaluation of American colleges and universities. It's the improvement that's attributed to the school's effort rather than the advantage or disadvantage that students bring to the schools from home.\n\nThe results can be striking. In Omaha, almost half of the schools that performed worse than the analysis would have predicted on California Achievement Tests scored better than the median county-wide. Almost half that outperformed the prediction scored below the district median. Hanging out at the schools confirmed their statistical results. At the most effective schools, teachers kept order, gained the respect of the kids and challenged them to work hard. At the least effective, kids slept at their desks, flashed gang signs and wandered around aimlessly with a helpless teacher standing by.\n\nLessons Learned\n\nOmaha's statistical technique can become a lesson for many reporters hoping to make sense of the proliferating school reports. The first battle was getting good data. What Omaha learned was that more isn't necessarily better-just more flexible.\n\nAfter years of fighting, Omaha won access to a database containing 70,000 individual student records on the California Achievement Test, the home ZIP code, the school and a couple of other identifiers including participation in school lunch programs. What it meant was that Omaha, unlike others who have tackled this project, had actual scores rather than the percentage of kids who scored in certain ranges. It also meant they could match the ZIP codes with demographic information about the home neighborhood, not the school neighborhood, in a district with busing.\n\nWhat they ended up using in their analysis, though, was familiar: All but one was a traditional measure of poverty or advantage, ranging from the percentage of parents in the home ZIP code with college degrees to the percent of students on free or reduced lunch. The last was the percentage of minorities taking the tests. Together, these factors accounted for up to 84 percent of the difference in median test scores among Omaha schools. The other 16 percent was, well, news.\n\nIf Omaha wanted to predict school performance, why did it exclude information on, say, dollars spent or PTA hours logged? Its goals, unlike those of the Arkansas Democrat-Gazette featured last month in this space, were to isolate the characteristics of a school that teachers, principals, or officials, couldn't control. That way, it could evaluate the extra value each school brought to education rather than predict more exactly the school's performance on a series of test measures.\n\nThe method wasn't without controversy. School officials told Omaha reporters and editors that they'd ruin the lives of each 2nd grader in the district. They condemned the method. A school official's daughter, who was a graduate student nearby, e-mailed daily to question another detail of the method after the series ran. But the analysis had one strength the school officials couldn't deny: It confirmed internal, unpublished studies conducted by the school system itself in its attempts identify the best and worst of Omaha's schools.\n\nAnd it did so with compelling anecdotes from inside those schools. Here are some tips for reporters who want to tackle something like Omaha's project:\n\nFight for information on students rather than settling for the school reports offered up on the Web. This information was crucial to Omaha's ability to look at a range of student demographics using Census data.\n\nIdentify your goals early on. In Omaha, reporters wanted to evaluate the performance of the education system in each school. In Arkansas, reporters decided that it didn't matter why schools were good - they wanted to develop a model of quality that ignored socioeconomic factors. Both are legitimate goals. Just decide which is yours.\n\nWatch for bad data, but don't get paralyzed. In Omaha, for example, certain schools were designated \"universal free lunch\" schools. So many of the kids qualified that the schools just gave the program to everyone and stopped counting how many would have gotten the reduced price lunches elsewhere. They didn't show up as poor in the database.\n\nLook at more than one year's worth of information. Omaha looked at schools that under- or over-performed five years in a row.\n\nUse the scientific method, and understand all of your results.\n\nIf you need help, hire it. Omaha, in fact, used two consultants, John Bare and Brenda Sugrue, to lead them through the maze of numbers and tests.\n\nDon't let the school officials intimidate you. They'll always tell you that you've made big mistakes, and you're irresponsible. If you have been responsible, don't let it dissuade you.\n\nSarah Cohen can be reached at (301) 942-2199 or e-mail her at sarah@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-july-backgrounding-individuals","slug":"1997-july-backgrounding-individuals","url":"https://dwillis.github.io/uplink/article/1997-july-backgrounding-individuals.html","year":1997,"month":"July","monthNum":7,"headline":"Backgrounding individuals","kicker":"On the Internet","author_name":"Alan Schlein","author_title":"","affiliation":"Schlein News Bureau","summary":"A guide to online search engines and databases useful for journalists to background and locate individuals, including people finders, email locators, and archived newsgroup searches.","keywords":["search","journalists","backgrounding"],"topics":["Investigative Methods","Technology"],"technologies":["CompuServe","America Online","CD-ROM","Netscape","Switchboard","Whowhere","Yahoo","Lycos","Excite","Alta Vista","Usenet","Infospace","Four11","Internet Address Finder","Saavysearch","Dejanews","PR Newswire","World Wide Web"],"full_text":"Dozens of search engines exist to give you first clues on backgrounding individuals. This is not a comprehensive list. Just ones I've found useful in finding people online. The best test for most search engines is to put your own name in and see what turns up. Compuserve offers Phone File, America Online offers \"White Pages\" and CD-ROM's are available with the nation's phone numbers. Netscape also offers a \"People\" icon that provides various searches for addresses, phone numbers or e-mail addresses of people and companies.\n\nSwitchboard: www.switchboard.com Good tool for finding people and e-mail addresses.\nWhowhere: www.whowhere.com Also good as phone book, business resource and e-mail address finder.\nYahoo's People Page: www.yahoo.com/search.people\nLyco's People Find: www.lycos.com/pplfndr.html\nExcite's People Finder: www.excite.com/reference/people.html\nBroad search coverage by concept. See later reference to Excite for best journalism usage.\nAlta Vista: www.altavista.digital.com My favorite (so far) of all the search engines especially powerful if you know exactly what you are looking for. If you do put it in quotes and check it out, use the advanced search capability to designate word proximity in a phrase. Also a full-text index of Usenet newsgroup archives.\nInfospace: www.infospaceinc.com This one is good for finding people, e-mail addresses and directions to the location. It's one of several good mapping programs and gives directions in simple terms (like go right, instead of go west).\nFour11: www.four11.com 411 is great for searching e-mail addresses, white and yellow pages phone books and also has a reverse directory capability, when you have a number and want to know who lives there.\n\nInternet Address Finder: www.iaf.net This seems to be the most comprehensive for finding e-mail addresses, including obsolete and old ones.\nSaavysearch: www.cs.colostate.edu/~dreiling/smartform.html\nIt looks through multiple search programs and gives you a single list of hits back. I find it saves time.\n\nBackgrounding Tools\n\nDejanews: www.dejanews.com\n\nYou can search for a topic, returning all the archived newsgroup messages that mentioned that topic, or search for an e-mail address to get all the messages that someone has posted (great for finding subject's interests). BUT BE AWARE: while you can read others postings on the usenet or news group, they can also read yours. A recent New York Times story noted that a well-known privacy advocate had posted 527 messages to usenet groups in the past 18 months, with postings to 32 different newsgroups, everything from a cyberpunk mailing list, a beer newsgroup and a Debbie Reynolds fan club.\n\nFraternity and Sorrority Connection: www.greek.com\nYou can link to this site for some alumni links.\nFinding Faculty: www.petersons.com\n\nThey publish the textbook Register of Higher Education, a great resource for finding administration folks at universities. Not on-line yet. They do, however, have the best on-line service available to search for a university employee.\n\nFinding Professionals\n\nProfnet: www.prnewswire.com Originally run by Dan Forbush at the State University of New York - Stony Brook, it has now been merged with PR Newswire, a public relations wire service. They still take your request and broadcast it to thousands of university press flacks nationwide. Include your affiliation, a very specific description of the type of expert you are seeking and the type of questions you need answered and information on how to contact you (e-mail and phone). They usually get an answer.\n\nAmerican Medical Association Physician Select: www.ama-Assn.org\n\nTo find physician addresses and credentials, click from the home page to \"AMA Health Insight\" and then \"Physician Select,\" AMA's reference source of professional information on individual physicians in the U.S.\n\nMartindale-Hubbel: www.martindale.com/maps/./locator/home.html\nExcellent resource for finding lawyers and backgrounding them.\nFinding Officers: www.officer.com/officers.htm\n\nThis site is an unofficial collection of links to personal home pages of law enforcement officers. If a cop has a home page on the Web, it's here.\n\nBiography Dictionary: www.tiac.net/users/parallax\nIncludes over 18,000 prominent persons.\n\nAlan Schlein can be reached at (202) 544-5893, or send e-mail to Schlein@access.digex.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-july-the-data-they-are-a-changin","slug":"1997-july-the-data-they-are-a-changin","url":"https://dwillis.github.io/uplink/article/1997-july-the-data-they-are-a-changin.html","year":1997,"month":"July","monthNum":7,"headline":"The data, they are a changin'","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"","affiliation":"NICAR","summary":"This article explains how to perform basic data manipulation tasks like updating and deleting records in both Access and FoxPro databases, comparing the syntax and safety features of each system.","keywords":["database","SQL","FoxPro"],"topics":["Data & Databases","Technology"],"technologies":["FoxPro","Access","SQL"],"full_text":"Sometimes you need to make changes in your data - cleanup, standardization, etc. A question often heard in NICAR bootcamps is something like \"I know how to do this in FoxPro, but I don't know how to do it in Access.\" Since the reverse of this question also comes up, I'll cover the basics of both.\n\nUpdating data\n\nHere is the query in Access:\n\nUPDATE Contribs SET NewOccupation = \"Homemaker\" WHERE Occupation = \"Housewife\"\n\nThis is how to do the same thing in FoxPro: there are two commands typed in the command window.\n\nUSE Contribs REPLACE ALL ; NewOccupation WITH \"Homemaker\" ; FOR Occupation = \"Housewife\"\n\nIn FoxPro, the table you are working on is defined first, in a separate command (\"use,\" which is the same as going through the File/Open dialog box.) In Access, the table to work on is specified after the word UPDATE. The words are different, but they have in common a way to define the column to work on, the text to put in it and which rows to perform the change.\n\nDeleting data\n\nAccess:\nMaybe you won't use this as often, and never on the only copy of original data. Uplink readers would never commit a blunder like that.\n\nAnyway, in creating a subset of a larger data set, sometimes you make the first cut by creating a new table based on rows meeting a certain criteria.\n\nYou could do subsequent delete commands to further whittle down that table. This SQL command in Access will delete all rows where the occupation field contains \"ATTORNEY\" DELETE FROM Contribs WHERE Occupation = \"ATTORNEY\"\n\nAccess tells you how many records will be deleted before it actually deletes the records. If you click cancel on the message box, nothing happens and the records remain. If you click OK, Access deletes the records and you can't retrieve them.\n\nFoxPro\n\nUSE Contribs DELETE FOR Occupation = \"ATTORNEY\"\n\nAgain, FoxPro takes two commands to do what is specified in one command in SQL and Access. After FoxPro command, the records fitting the specified condition are only marked for deletion; they still remain in the table. You can still RECALL the records. This next command does the deletion and is not reversible.\n\nPACK\n\nFoxPro will not give you a prompt before this command is carried out. If you say \"oops,\" the next thing you say as the smart Uplink TechTip reader that you are: \"I'm glad I have a backup.\"\n\nRichard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-july-quick-hits","slug":"1997-july-quick-hits","url":"https://dwillis.github.io/uplink/article/1997-july-quick-hits.html","year":1997,"month":"July","monthNum":7,"headline":"Quick Hits","kicker":"Handout of the month","author_name":"Andy Hall","author_title":"","affiliation":"Wisconsin State Journal","summary":"The article provides practical methods for journalists to conduct meaningful investigative stories in limited time on topics like education achievement gaps, population trends, and crime patterns by leveraging available data sources and databases.","keywords":["investigation","data","community"],"topics":["Investigative Methods","Data & Databases","Demographics & Census"],"technologies":["Internet","World Wide Web"],"full_text":"Any of these issues may be worth of an in-depth project. But if you've got only a day or a week, you can produce meaningful stories examining portions of these issues.\n\nEducation\n\nThe racial achievement gap: Is it widening in your community's schools? Even if you're handed a news release and told to write up the latest test scores, you can quickly dig up additional data from the schools or your state's education agency. (Some state agencies are listing this data on the Internet.)\n\nMethod: Compare current high school cumulative GPAs, listed by race, to figures five years ago and, if available, 10 years ago. Use the same procedure for standardized test scores administered at various grade levels. Don't let the school district simply release the \"average\" score for the entire student populations. Also, examine the scores, listed by race, for each school. You'll see some interesting differences.\n\nQuestions: Is the gap between whites and minorities expanding? Why or why not? Can the differences be explained by other \"risk factors\" such as poverty and high mobility? National education experts say they know of no major school district in the nation that has substantially reduced the racial achievement gap. If your schools have, you've got a huge national story. If they haven't, you've still got a huge story.\n\nTip: Look for students or schools whose experiences defy the stereotypes. They'll provide inspiring stories of success - and poignant stories of failure.\n\nPopulation trends\n\nHow has the population of your community changed since the 1990 Census? Don't wait for the 2000 Census to find out.\n\nMethod: Contact your State Data Center (list of centers available through 301-457-1305). Also check the Census Bureau's homepage, www.census.gov, and particularly the annual county-level estimates for race/Hispanic origin, sex and age at www.census.gov/population/www/estimates/co9094.html (See handout from USA Today's Paul Overberg for details; call IRE at 573-882-0684.) Be sure you understand the limits of the data. Tip: New data became available in July.\n\nCrime\n\nWhere is violence concentrated in your community? Which police officers handle most of the violence?\n\nMethod: Obtain the police department's database for calls for service, or for reported crimes. Or persuade the department to provide you with such information as the address having the greatest number of violence-related calls, the block with the greatest number of calls, and the officer responding to the greatest number of violence calls. Long after you've published your major package of stories, many quick-hit-stories remain hidden in the data.\n\nTips: There's great value in stories demonstrating that crime can be traced to specific buildings and to a relatively small number of people. Your stories give the community a place to focus efforts such as rooting out bad landlords and bad tenants, or obtaining more police patrols.\n\nQuick-hit angles include: which cops handle the greatest number of violence-related calls, and what are their perspectives on trends in the community? What types of calls do officers in your community handle? Which date, day of the week or time are most frequently associated with reported violence? How long does it take the police to arrive to incidents? What patterns do you find when you analyze the sex, age and race of violence victims?\n\nRedlining\n\nAre insurance companies excluding minorities from homeowners coverage?\n\nMethod: Examine state insurance department records. Start by focusing on a particular company. Determine how much of the market that company holds across the metropolitan area. Determine what share of the market the company holds within each ZIP code. Obtain Census data listing minority population within each ZIP code. Is there an interesting pattern? In 1995, Madison-based American Family Insurance settled a lawsuit over its Milwaukee policy-writing practices for $14.5 million.\n\nTip: You can do this within one week. Medical waste, voting, state aid, affordable housing, wardens. Got time for more?\n\nDo any hospitals in your area burn medical waste to generate electricity? The pollution may be making patients and employees sick.\n\nAs politicians spend millions trying to win votes, check to see how many pols and other local leaders bother to vote.\n\nWhen your state doles out economic development aid, where does it go? If any ends up in funny places, such as a shaky deal to prop up a factory in the governor's hometown, you've got a hot issue.\n\nWhen a developer or politician brags that they're creating affordable housing, check it out: How many people can really afford these places?\n\nWho are the wardens of the prisons in your area? These people wield extraordinary power over the lives of inmates and employees, but they're seldom subjected to scrutiny.\n\nAndy Hall can be reached at (608) 252-6100.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-july-campaign-finance-assistance","slug":"1997-july-campaign-finance-assistance","url":"https://dwillis.github.io/uplink/article/1997-july-campaign-finance-assistance.html","year":1997,"month":"July","monthNum":7,"headline":"Campaign finance assistance","kicker":"IRE/NICAR Notes","author_name":"Brant Houston and Jo Craven","author_title":"","affiliation":"IRE and NICAR","summary":"Electronic filing of campaign finance reports is gaining momentum across states, potentially eliminating the need for time-consuming data entry in journalism investigations. However, concerns exist about restricted access to certain information, such as contributor addresses, which could hinder identification of multiple contributions.","keywords":["filing","campaigns","disclosure"],"topics":["Government & Politics","Investigative Methods","Journalism & Newsroom"],"technologies":["World Wide Web","floppy disk"],"full_text":"One of the worst tasks in computer-assisted reporting is data entry. Yet doing data entry is usually the only way to do one of the best kind of stories: analysis of state and local campaign finance reports.\n\nBut, in the coming years, there is a chance that campaign finance stories may not have to rely on massive and time-consuming data entry projects.\n\nElectronic filing of campaign finance reports has become a hot topic among state election officials and campaign finance reformers.\n\nA recent report (soon to be updated) by the Center for Governmental Studies in Los Angeles reveals that all but four states are considering ways to encourage or require electronic filing.\n\nAnd in July, 40 state election officials from more than a dozen states met to share questions and ideas about electronic filing and disclosure.\n\nAs outlined in the center's report, electronic filing and disclosure of campaign contributions and expenses can streamline administrative costs, permit faster reporting on the information and aid the public in making better decisions during elections.\n\nFor journalists, electronic filing and disclosure through diskette or through World Wide Web sites would allow dozens of timely and meaningful stories during an election, rather than afterward.\n\nThere already is progress in some states. While the systems are not perfect, here is a list of some of their efforts:\n\nHawaii has implemented electronic filing of campaign finance reports and is in the process of upgrading its system. The Hawaii system visually mimics the state's paper forms. It provides diagnostics, including spell checking. And it supplies housekeeping functions, such as generating letters notifying filers of missed deadlines.\nIowa has developed an electronic filing system and expects voluntary filing to begin in August. The first reports expected to be available are PAC filings in October and candidate committee filings in January.\nMichigan permits electronic filing, but has not developed filing software.\nTwenty-two states have pending statutory legislation for electronic filing. Among those already considering how to start their own systems are Michigan, Missouri, Oklahoma, New Jersey, Illinois and Ohio.\n\nWhat's missing\n\nAlthough steps have been made, we are concerned about moves not to disclose street addresses or to restrict information electronically that can be obtained on hard copy.\n\nFor example, California has voted to omit contributors' street addresses from data that it puts on the Web and is debating whether to omit cities as well — decisions that will make it impossible to clearly identify multiple contributions by the same individual.\n\nIRE and NICAR are pushing for electronic filing and have the ear of election officials who are trying to come up with standardized reports and coding for electronic filing.\n\nWith help of a two-year grant from the Joyce Foundation, located in Chicago, IRE and NICAR have begun the Campaign Finance Information Center.\n\nThe center, which is just getting started, plans to be a repository for federal, state and local campaign finance data and to help journalists analyze that data.\n\nWe already are reviewing the electronic filing issues with state officials and campaign finance groups, representing the interests and needs of journalists.\n\nIn September, we plan to attend a larger conference of election officials where we will push hard for meaningful changes.\n\nLet us know by e-mailing jo@nicar.org and brant@ire.org what we need to do and any news of what your state or local government is doing.\n\nTo get a copy of the Center for Governmental Studies report, write to Craig Holman, Project manager, National Resource Center, 10951 W. Pico Blvd., First Floor, Angeles, Calif. 90064.\n\nTo get a copy of the COGEL study surveying 39 states regarding the conversion from paper filing to electronic filing, write to Charles G. Lamb, executive director, Missouri Ethics Commission, 221 Metro, Jefferson City, Mo. 65109\n\nAs part of the campaign finance project, IRE and NICAR will develop a Web page with tips from reporters, examples of stories and data for state and federal races. If you have additional suggestions of items that should be on the page, send e-mail to jo@nicar.org or jody@nicar.org\n\nBrant Houston can be reached at (573) 882-0684, or send e-mail to brant@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-july-get-your-training","slug":"1997-july-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-july-get-your-training.html","year":1997,"month":"July","monthNum":7,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers specialized computer-assisted reporting training through on-the-road sessions at various newsrooms and conferences, as well as intensive week-long bootcamps at their Missouri headquarters, with upcoming sessions scheduled throughout 1997 and early 1998.","keywords":["training","journalism","database"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"Check out the NICAR Web site nicar.org to find out times and dates for upcoming on-the-road training and bootcamps.\n\nOn-the-road training\n\nNICAR provides specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases. Cost varies. For information, call Lisa Barnes at (573) 882-8969, or to register, call the numbers below. August 13-17, 1997 - Boston, Mass. Join NICAR for hands-on training at the Asian American Journalists Association in Boston. Limited to conference participants. August 19-22, 1997 - Huntington, W.Va. On-the-road training for Marshall University September 11-13, 1997 - Washington, D.C. Computer-Assisted Reporting Workshop Sponsored by NICAR and the Medill School of Journalism September 20, 1997 - Coeur D'Alene, Idaho Join NICAR for hands-on training at the Idaho Press Club's annual conference in Coeur D'Alene. Limited to conference participants. September 24-26, 1997 - Arlington, Va. On-the-road training for USA Today October 20-24, 1997 - Ft. Lauderdale, Fla. On-the-road training for the Ft. Lauderdale Sun-Sentinel\n\nConferences\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969 March 5-8, 1998 - Indianapolis, Ind. NICAR Conference June 4-7, 1998 - New Orleans, La. IRE National Conference\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo. As with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size. For information, call Wendy Charron at (573) 882-0684. August 10-15, 1997 - Columbia, Mo. NICAR Regular Bootcamp September 28 - October 3, 1997 - Columbia, Mo. NICAR Bootcamp with eight fellowships available for minority journalists Sponsored by NICAR and the Multicultural Management Program January 4-9, 1998 - Columbia, Mo. NICAR Regular Bootcamp","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-july-from-the-nicar-library","slug":"1997-july-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-july-from-the-nicar-library.html","year":1997,"month":"July","monthNum":7,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to numerous federal government databases covering topics ranging from medical devices and aviation safety to campaign finance and small business loans, enabling journalists to investigate government programs, safety records, and lending patterns.","keywords":["databases","government","investigation"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW U.S. Food and Drug Administration's Medical Devices Reports, detailing complaints about drugs, pacemakers and other medical products. 1974-1996.\n\nNEW U.S. Centers For Disease Control's AIDS database, providing case-by-case demographic information about those with the HIV virus. 1995.\n\nNEW U.S. Census Bureau's Consolidated Federal Funds Reports, showing which communities get how much under various federal programs. 1983-1995.\n\nFederal Aviation Administration's accidents and incidents, including major plane crashes since 1974.\n\nNASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\n\nFederal Election Commission campaign contributions by individuals and political action committees, 1990-1997.\n\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings, 1991-1996.\n\nCoast Guard boating accidents, 1969-1994.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration, 1974-1997.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\n\nFederal procurement data, 1992-1996, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1996.\n\nNational Bridge Inventory System data, includes inspection grades, 1994-1995.\n\nNEW. FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the 1995 final report.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration loan guarantees, 1980-1997. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nThe National Inventory of Dams, 1991-1995.\n\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1996.\n\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1975 to 1995.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-july-bits-bytes-and-barks","slug":"1997-july-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-july-bits-bytes-and-barks.html","year":1997,"month":"July","monthNum":7,"headline":"Bits, Bytes and Barks","kicker":"Reader Response","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"A journalism newsletter seeks reader feedback on content improvements and announces multiple job openings at IRE and NICAR, including positions for campaign finance specialists, database administrators, and a deputy director.","keywords":["journalism","employment","database"],"topics":["Journalism & Newsroom"],"technologies":["Access","Internet","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nReader Response\n\nIt's time we heard from you. We are looking to improve Uplink and want to know your ideas. Are there certain columns that you would like to add? Are there certain topics you would like covered? Have you done a story that you think should be featured in an upcoming issue? This is your chance to play armchair editor. Send comments, critiques and suggestions to Uplink's editor, Jody Sowell, at jody@nicar.org We look forward to hearing from you.\n\nNICAR Net\n\nRecent topics on the IRE and NICAR listserv have included campaign finance data, troubles with traffic accident data, ethics, medical investigation, medical investigations and Access tips.\n\nTo subscribe to IRE-L or NICAR-1, send e-mail to listproc@lists.missouri.edu\n\nIn the body of the message, type: subscribe NICAR-L<your name> subscribe IRE-L<your name>\n\nAlso, check out the IRE-L and NICAR-L mailing list archives on our website at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.\n\nIRE and NICAR Want You\n\nIRE and NICAR have several positions open:\n\nDirector/Trainer for Campaign Finance Information Center and Database Library Administrator for Campaign Finance Information Center\n\nWe are looking for excellent journalism skills and/or excellent computer skills. The center will archive state and local campaign finance data, offer on-site and on-line training and will publish a newsletter. Salaries for the jobs range from $35,000 to $40,000.\n\nIRE Deputy Director\n\nIRE is also looking for a deputy director who will report to the executive director and help oversee IRE's six divisions: training, publications, the Mexico program, NICAR database library, operations and the new campaign finance information center. Salary range is $40,000 to $46,000. The job may include teaching at the school of journalism as an adjunct professor.\n\nDatabase Library Administrator for NICAR\n\nThis position involves supplying data and data analysis to more than 120 print and broadcast organizations and overseeing a team of graduate research assistants. Salary range is $35,000 to $45,000.\n\nInquiries and resumes should be sent to Brant Houston, Executive Director, IRE, 138 Neff Annex, Columbia, Mo. 65211 or brant@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-back-to-school","slug":"1997-august-back-to-school","url":"https://dwillis.github.io/uplink/article/1997-august-back-to-school.html","year":1997,"month":"August","monthNum":8,"headline":"Back to School","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Journalists across multiple newsrooms have uncovered significant issues in public school systems, including middle-income student exodus, uncertified substitute teachers, unequal teacher assignments, and disparities in minority hiring practices.","keywords":["schools","reporting","districts"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"The binders have been bought. The pencils have been sharpened. Now it's time to see what's really going on behind the school house door.\n\nMark Skertic and The Cincinnati Enquirer found out that middle-income parents have yanked their kids from the city's public schools. Heather Newman and a team of reporters at the Detroit Free Press discovered that uncertified substitutes were often being called on to teach classes and that principals were being paid for outside activities that other principals did for free. NICAR's Kara Morrison reports on how the Baltimore Sun showed that the Baltimore school district's greenest teachers were assigned to the area's most disadvantaged schools.\n\nAnd Griffin Palmer of The Daily Oklahoman tells how reporters there used computer-assisted reporting to analyze minority hiring in the school district.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-separate-and-unequal","slug":"1997-august-separate-and-unequal","url":"https://dwillis.github.io/uplink/article/1997-august-separate-and-unequal.html","year":1997,"month":"August","monthNum":8,"headline":"Separate & Unequal","kicker":"Uplink Update","author_name":"Mark Skertic","author_title":"","affiliation":"The Cincinnati Enquirer","summary":"An analysis revealed that Cincinnati's public schools are struggling to retain middle-income families, leading to a disproportionate number of low-income students in public schools compared to state and national averages.","keywords":["Cincinnati","public schools","demographics"],"topics":["Education","Data & Databases","Demographics & Census"],"technologies":["CD-ROM"],"full_text":"Some school administrators refused to believe it when they read that parents in Cincinnati were less likely to choose public schools than their counterparts in almost every other large city in the nation. After all, the school chiefs argued, walk into any of the district's magnet schools and you see happy children from all over the city working together. But anecdotes don't always give a clear picture. Census Bureau reports make clear that the district's hold on middle-class families is tenuous. Instead of a public school system that mixes together children from different social and economic classes, Cincinnati schools are in danger of becoming a system that exists only to teach the community's poorest children. The analysis that appeared in The Cincinnati Enquirer showed that high-income families have given up on public schools and many middle-income earners have opted for private schools or other options. Today, about 78 percent of Cincinnati households with school-age children choose the public schools. That compares to 86 percent on average in Ohio and 87 percent nationally. Parents are more likely to choose public schools in most cities in the nation, including Detroit, Los Angeles and Miami. The news was troubling for public school advocates who see local elementaries and high schools as great melting pots, where different races and social classes come together to learn. Instead, their school system's population is an imperfect reflection of the community, with a disproportionally poor population. More than half the children enrolled in Cincinnati schools come from households where the annual income is less than $20,000 annually.\n\nCensus tracks school kids\n\nSchool administrators had said for years that they didn't collect the information necessary to really know how well the district's enrollment represented the overall population, nor did they know what percentage of parents were using the public schools.\n\nBut it was all there in the 1990 Census files, waiting to be sifted out. For reporters in cities where there have been dramatic population shifts or school enrollment has jumped or plummeted wildly that makes this a difficult story to replicate. Most of it was collected in 1989 and it's just too outdated if your area has changed substantially in that time.\n\nBut if population and enrollment have been stable, the information will prove useful. In this case, I talked with demographers and education researchers at the University of Cincinnati about the validity of the data. They checked elements of the data listed below against some current population estimates. In their judgment, the school-enrollment trends I was preparing to cite were still valid.\n\nThe sources I used:\n\nThe School District Data Book, a series of CD-Roms from the National Center for Education Statistics. That software uses information from the 1990 Census to provide detailed information broken down by school district. Most states are available on a single disk. For details and current prices, call the center at (800) 424-1616. The same information is also available online at http://govinfo.kerr.orst.edu/sddb-stateis.htm.\n\nThe CD-ROM version is user friendly and fast. It operates in DOS, but the menus are easy to use. The reference manual that comes with the disks is expansive, but also easy to use.\n\nThe National Center for Education Statistics also has detailed information on enrollment and poverty trends. Much of this is available on the center's web site: http://www.ed.gov/NCES/.\n\nA series of tables the Census Bureau ran for a Center for Education Reform study on where teachers send their children to school. When I saw that study a few months before doing this project, I realized that before the bureau could break out details on teachers, an analysis had to be done on where the general population sent their children.\n\nThe bureau's analysis for the Center for Education Reform resulted in several tables: A state-by-state listing of the percentage of parents who chose public school and the percentage who chose private, a listing of the 100 largest cities with the same information and similar tables. In all cases, information was provided for the overall population broken down by income bracket. That analysis was paid for by the Bradley Foundation of Milwaukee, Wisc. But once the Census Bureau created the \"document\" for that study, I got a copy by calling the population division and asking for it. Special Tabulation Package 181 cost $60 for three disks. The text files were easy to call up in a spreadsheet program. Contact the bureau's population division at (301) 457-2429 for a copy.\n\nPutting it together\n\nThe School District Data Book contains more than 100 pieces of information for every school district in the state. The menu system makes it easy to break data down into subsets for analysis.\n\nFor example, I was able to choose between looking at economic information for all households in the school district or households with children. I could examine demographic profiles of children or for all school-age children.\n\nThat allowed me to do detailed breakdowns, looking at where middle income earners sent their children to school.\n\nThe Special Tabulation purchased from the Census Bureau had all the information needed to look at the nation's largest cities, giving readers a larger perspective on the situation.\n\nFinally, the National Center for Education Statistics reports had details about population shifts and trends.\n\nAfter the story appeared, the school district's treasurer got copies of the School District Data Book CDs, hoping to double-check my work and prove how inaccurate I had been. He didn't have much luck.\n\nIn fact, Cincinnati Public Schools recently began doing a series of demographic studies, hoping to take the same sort of research and project it over five years. The analysis will be used to develop a recommendation for some school closing and repair prioritization. I'm already looking forward to FOling their raw data for some analysis of my own.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-record-fight","slug":"1997-august-record-fight","url":"https://dwillis.github.io/uplink/article/1997-august-record-fight.html","year":1997,"month":"August","monthNum":8,"headline":"Record Fight","kicker":"Going Private","author_name":"Heather Newman","author_title":"","affiliation":"Detroit Free Press","summary":"Detroit Free Press reporters obtained salary data for over 22,000 Detroit Public School employees to investigate principals earning extra compensation for after-school activities ahead of upcoming union negotiations.","keywords":["salaries","investigation","principals"],"topics":["Data & Databases","Education","Investigative Methods"],"technologies":[],"full_text":"It started with a tip last November: Detroit Public School principals were raking in extra cash by doing after-school activities other principals did for free. Detroit Free Press education writer Jennifer Juarez Robles realized that with teachers' union salary negotiations coming up in 1997, how much principals - and regular staff members - were making above their base salaries could quickly become a hot debate topic. Salaries and benefits are a perennial issue in the district, making up 83 percent of all discretionary dollars spent. So Robles and Ariana Cha, an intern now at the San Jose MercuryNew filed an open records request for the district's salary database. They asked for race, sex and age information, as well as gross salaries for the past two years, base salaries, names, position titles and where the people worked. Detroit Public Schools has more than 22,000 employees, making it one of the largest public payrolls in the state. ","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-august-inexperience-reigns","slug":"1997-august-inexperience-reigns","url":"https://dwillis.github.io/uplink/article/1997-august-inexperience-reigns.html","year":1997,"month":"August","monthNum":8,"headline":"Inexperience Reigns","kicker":"Poor schools get rookies","author_name":"Kara Morrison","author_title":"NICAR representative","affiliation":"NICAR","summary":"Baltimore Sun reporters discovered that disadvantaged schools with the lowest test scores and highest poverty rates are staffed disproportionately with inexperienced teachers, limiting equal access to quality education. The finding was substantiated through teacher turnover data showing a concentration of less experienced educators in schools serving the largest minority populations.","keywords":["teachers","inequality","schools"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Mike Himowitz had often heard Baltimore County's neediest schools were getting stuck with the greenest teachers. But it wasn't until Baltimore Sun reporter Marego Athans ran across some teacher turn-over data that the two were able to substantiate the problem. Athans discovered one school administrator disturbed by the trend had been keeping track of teacher transfers and turnover and years of teaching experience. Himowitz, the Sun's electronic news editor, said the numbers jumped off the page. They supported complaints that Himowitz had heard for years as a former education reporter and Baltimore County bureau chief. The result was a story in the June 4 Baltimore Sun detailing how teachers without much experience are concentrated in disadvantaged schools with the lowest test scores, highest poverty rates and usually the largest minority populations. In other words, Athans said, students do not have equal access to experienced teachers. She reported that according to several national reports, teacher experience is one of the most important factors in student achievement. ","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-august-adding-numbers","slug":"1997-august-adding-numbers","url":"https://dwillis.github.io/uplink/article/1997-august-adding-numbers.html","year":1997,"month":"August","monthNum":8,"headline":"Adding Numbers","kicker":"Minority teachers missing","author_name":"Griffin Palmer","author_title":"","affiliation":"The Daily Oklahoman","summary":"A reporter used data analysis to uncover that Hispanic and Native American students had the largest representation gap compared to educators, rather than Black students as initially assumed, by carefully comparing student enrollment and certified educator databases while accounting for data inconsistencies.","keywords":["data","educators","minorities"],"topics":["Education","Data & Databases","Demographics & Census"],"technologies":["FoxPro"],"full_text":"Data analysis doesn't have to drive an education story. Sometimes it can be a nice augment. When reporters Susan Parrott and Melissa Nelson-Varela began preparing a feature about the State Education Department's efforts to recruit minority educators, I reached for two data sets I already had. One was a racial breakdown of Oklahoma's student body. The other was a racial breakdown of certified educators. The analysis turned up some surprising numbers that helped shape the story: The greatest discrepancy in numbers between student body and educators was not among blacks, but among Hispanics and Native Americans. Before the analysis, Susan and Melissa had instinctively been focusing mostly on black students and efforts to recruit more black teachers. Our analysis showed them they needed to focus more on other racial minorities. The analysis wasn't terribly complex, but we did have to be careful to select comparable groups from both databases. The State Education Department's student racial breakdown includes students enrolled at private schools, while the educators' database does not include those working at private schools. The enrollment data documentation warned that students enrolled in bilingual programs were counted twice, a fact that we also had to account for before we could do our analysis correctly. Unlike the U.S. Census, the data counted Hispanics as a distinct ethnic group and not as a subset of other racial groups. To make sure we were comparing like groups, we needed to select only those cases from the student enrollment file that contained data for districts included in the educators' file. The tables' design made this challenging. In the student file, each record indicated a grade in a particular school. Each record contained columns listing number of class members by race and by gender. The educators' file contained a single record for each teacher or administrator, with a field denoting the individual's race. Each of the files listed district numbers many times. In the enrollment file, district number occurred in each grade. In the educators' file, district number occurred in each individual's record. Doing a straight relational join in this situation would result in thousands of duplicate matches. A simple nested query, in FoxPro, allowed us to find only those districts appearing in the educators' file and to filter out the duplicate numbers from bilingual enrollments. The tables were called enroll96.dbf and edcert96.dbf The join field was called district. Any time the grade field in the enrollment table contained the string 'BI', it was duplicate information that had to be eliminated. Here's how the query looked: select * from enroll96 where grade <> 'BI' ; and district in (select district from edcert96) ; into table public96 We now had a table giving us enrollments only for public school districts, and could make meaningful comparisons between the racial distribution among certified educators and the racial makeup of the student body. We knew that Oklahoma City and Tulsa, the state's two largest, most urbanized districts, would have sharply different racial makeups than the rest of the state, so we analyzed them separately. In fact, we broke the data into four separate groups, based on size of enrollment. Our analysis showed, however, that only Oklahoma City's and Tulsa's racial makeup differed significantly from the makeup of other districts. Our analysis showed that blacks made up 36.6 percent of students in Oklahoma City and Tulsa, while blacks comprised 21 percent of the two cities' educators. In the rest of the state, blacks made up 6.6 percent of students; less than 2 percent of teachers. American Indians comprised 6.4 percent of the urban student bodies and 2.3 percent of the urban districts' educators. In the balance of the state, the discrepancy was more stark: 16.3 percent of students and 3 percent of educators were American Indian. Similar discrepancies showed up for Hispanics. The ratio of Hispanic students to educators was 9.2-to-1 in the urban districts. The data analysis contributed only a few paragraphs to a 85-column-inch story. It lent authority to the story, though, and helped shape its lead. Griffin Palmer can be reached at (405) 475-3311 or send e-mail to palmer@qns.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-august-handout-of-the-month","slug":"1997-august-handout-of-the-month","url":"https://dwillis.github.io/uplink/article/1997-august-handout-of-the-month.html","year":1997,"month":"August","monthNum":8,"headline":"Handout of the Month","kicker":"CAR coverage syllabus","author_name":"Rose Ciotta","author_title":"Reporter","affiliation":"Buffalo News","summary":"This IRE conference handout guide provides education reporters with computer-assisted reporting (CAR) techniques to investigate and compare school district data, including demographic information, teacher salaries, and long-term trends, enabling readers to see how their districts compare locally and statewide.","keywords":["education","analysis","districts"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"This is an excerpt of a handout given at the National IRE Conference in Phoenix. To order the complete handout or to get a list of other available handouts, call the IRE Resource Center at (573) 882-3364. Using CAR to investigate school trends is no longer of interest only to the education reporter. There are scores of local stories based on the analysis of data gathered at the local or state level. In many cases, the data match national trend data - is collected by the U.S. Department of Education. These stories can be done by reporters covering education, suburbs or just one school district. Readers want to see how their district compares to other districts. By getting the same data for all of the districts in your state you can easily report how your district compares on any number of variables. It's an excellent way to size up your district. Here is a list of ideas to get you started. CAR Education Topics 1. Demographic Data Schools have to report their racial breakdown. In a growing community, those numbers alone can help you compare and contrast districts as a reflection of housing patterns. Or, in established areas, the demographic data can help you identify districts and/or schools to feature for other stories. Also look for income information which is usually reported as percent eligible for a free or reduced lunch or the Census poverty index. At-risk data includes suspension and dropout rates. There's also percent of students in special education, number of students with limited English proficiency, percent of graduates who went to college, percent who graduated with a state diploma. 2. Salaries Taxpayers everywhere want to know how the teacher salaries in their district compare to teachers in other districts. The best way to do this story is to get all of the salaries and figure your own medians and averages to determine which number best reflect the teachers' salaries. If salaries increase with seniority, a district or school with a high number of veteran teachers with advanced degrees will have high salaries compared to districts hiring younger teachers with bachelor's degrees. If actual salaries are not readily available, the state usually can give you median salaries. The state also collects other details related to seniority, education, race, certification and pupil-teacher ration. Don't stop with teachers. Some of the most interesting figures are the salaries paid to top administrators. Once you get that database, you can also figure salaries on a per pupil basis to compare salaries between large and small districts and number of administrators per pupil. 3. Taking the Long View Pick an issue and look at it over a long period of time, i.e. a decade. Whether it's teacher salaries or amount raised in local revenues, any numbers would have to be adjusted for inflation. To account for changes in enrollment, the value needs to be figured on a per pupil basis. The result is a per pupil number adjusted for inflation that allows anyone to compare data such as total expenditures, amount spent on special education, amount raised in local taxes and amount raised in state or federal aid. 4. School Taxes School taxes probably account for the largest portion of the local property tax bill if your district relies on local property taxes for its funding. This is another area that can benefit from a long view analysis. When analyzing revenues, make sure you know what's included an what isn't. Local revenues may include sales tax. Is that a major or minor portion of the total? To be sure you have the whole picture of a district, ask for all of the revenue pieces including how much the district receives in state aid and federal grants. To compare districts over time, calculate these numbers on a per pupil basis adjusted for inflation. 5. Enrollment It's reasonable that spending increases when enrollment increases. But, you can often get interesting explanations when the opposite occurs. Looking at data simultaneously over a 10-year period allows you to see these trends. 6. Spending This is the trickiest area in school finance and another very good place to calculate per pupil spending over time with adjustments for inflation. The pitfalls are in the definition of spending. Remember, readers get the total expenditure number on their tax bill. They may be confused if you use anything else to calculate spending per pupil. Rose Ciotta can be reached at (716) 849-5548, or send e-mail to ciotta@buffnews.com There's approved operating spending with only includes day to day operating expenses. There's total expenditures which includes everything including busing, and there's total general fund expenditures which is spending for the program itself.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-august-stats-from-the-road","slug":"1997-august-stats-from-the-road","url":"https://dwillis.github.io/uplink/article/1997-august-stats-from-the-road.html","year":1997,"month":"August","monthNum":8,"headline":"Stats from the road","kicker":"To adjust or not to adjust","author_name":"Sarah Cohen","author_title":"training director","affiliation":"NICAR","summary":"A journalist questions the methodology of adjusting home prices for inflation, arguing that housing prices shouldn't be adjusted using traditional inflation measures since homes are treated as financial transactions rather than consumption in government economic data.","keywords":["inflation","housing","economics"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Stats from the road\n\nTo adjust or not to adjust\n\nThe subject came up again recently in a private e-mail. A writer wanted to know how to adjust the change in prices of homes for inflation.\n\nThe underlying question makes some sense. Have home prices outpaced more general inflation over the past 10 years? But the question got me onto my soapbox.\n\nIf inflation is defined as the increase in prices, how can you adjust them for inflation? It's like expressing a murderer's prison time as the difference from the average felon's. OK. Housing prices, like stock prices, are different.\n\nBut I doubt many reporters know why. The reason is that these investments aren't counted in traditional inflation measures, like the Consumer Price Index or the Gross Domestic Product's implicit price deflator.\n\nHome purchases are counted as purely financial transactions, not consumption or useful production, so they're excluded from virtually all of the government's inflation and production figures.\n\nWrong question\n\nOne answer to the correspondent's question is, \"Do something else.\" Another writer asked a seemingly more relevant question: How would you measure how affordable homes are today, compared with a decade ago? That's easier, and easier to explain. A simple approach is to is find the median home price for your area for a year, and divide it by the median income for that year. You'll something that tells you how many years' income the typical family would have to pay to buy a typical home that year. More accurately, you'd factor in the cost of buying a home by adjusting for interest rates, insurance requirements, and local tax changes over time. You may not even have to do the work. The National Association of Home Builders has an index called the Housing Opportunity Index. It compares affordability in 190 metro areas, and includes property taxes, interest rates and closing costs in its calculation.\n\nInflation consideration\n\nIf you decide to go ahead with the inflation-adjustment, though, here are some things to consider:\n\nFirst, decide which year you want to express your dollars in. I usually choose the most recent year available, because our readers and viewers are intimately aware of how much a dollar buys today.\n\nThey get fuzzy when you ask them to go back in history. They attribute halcyon days of low prices and prosperity to every year before about 1990.\n\n(Economists and statisticians will tell you that you have to use the base period of the deflator as your reference period. That's getting a bit picky for my taste.)\n\nThe next question is whether to use local inflation rates or national ones. Most places you won't have a choice. There are only about 15 metro areas that have published inflation figures.\n\nNext, just to be picky, find out the month that your home prices were collected. There are usually two types - an end-of-year value and an annual average. Many people gravitate to the annual average, while I gravitate toward the end of the year.\n\nWhy? A year is a long time. You'll mush together a lot of information if you use an annual average.\n\nAnnual averages will also lag consistent trends. If home prices rise a steady 0.3 percent a month, then your annual average will be somewhere near the June level, not the December level.\n\nThe big issue, though, is to stay consistent. If you use averages for one year, use them for all years. That way you won't confuse seasonal patterns with more meaningful changes.\n\nNow you have your reference period: December 1996, if that's possible with your home prices. And you want to compare it to, say, December 1986.\n\nLook up the CPI or the GDP deflator for the years you want to use. If you want annual numbers for a graph, just pick them all up. You can get the CPI at the Bureau of Labor Statistics' Web site for free.\n\nIf you subscribe to STAT-USA (about $100 a year from the Commerce Department) you can also get the GDP deflator for consumer expenditures, which some people think is a more accurate but quarterly version of the Consumer Price Index.\n\nMedian home prices are a little harder. The federal government only publishes median prices for new homes, not the vast majority of existing homes that are sold each month. Those are from the National Association of Realtors and some other real estate research companies. Locally, you can probably get your local real estate sources to give you historical data.\n\nExisting home prices rose from about $80,300 to $118,000 for the decade ending last year, according to the national consultant used for the NAHB Housing Opportunities Index. The CPI for All Items rose from 109.6 to 156.9 over the decade. Here's how to calculate the change in home prices, adjusted for inflation.\n\nInflation-Adjusted: = CPI then) Value then Putting it into numbers, then: CPI annual average for 1996 = 156.9 =1.432 (or so) CPI annual average for 1986 = 109.6\n\nSo home prices for 1986 expressed in 1996 dollars = 1.432 * 80,300 = about $115,000 So real home prices, as the economists call them, have risen only $3,000, or 2.6 percent, over the decade. Not such a good investment, when looked at under those conditions. That's less than 0.3 percent a year. Having done this calculation, though, what does it mean? It doesn't take into account changes in the nation's housing stock, like the trend toward bigger homes or longer commutes. It doesn't take into account the payments that people have to make, including lower interest rates. Nor does it take into account tax caps that have made new homeowners pay more tax at the expense of more stable families. Finally, when you compare it to other products gasoline, furniture, clothing, or even computers, you usually don't think of those prices after inflation. So use it with care. Sarah Cohen can be reached at (301) 942-2199 or e-mail her at sarah@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-car-tech-tip","slug":"1997-august-car-tech-tip","url":"https://dwillis.github.io/uplink/article/1997-august-car-tech-tip.html","year":1997,"month":"August","monthNum":8,"headline":"CAR Tech Tip","kicker":"Splitting up isn't hard to do","author_name":"Richard Mullins","author_title":"","affiliation":"NICAR","summary":"This column demonstrates how to use string functions in Access to split fields, building on similar techniques previously covered for FoxPro, a practical skill for anyone working with databases.","keywords":["Access","string","databases"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Access","FoxPro","SQL"],"full_text":"CAR Tech Tip\n\nSplitting up isn't hard to do\n\nBy Richard Mullins NICAR\n\nThis column will go over the same ground as an earlier column - how to use string functions to split a field - but with the functions and commands used in Access instead of FoxPro.\n\nIf you work with databases for very long, you will face the question, \"How do I split apart a field?\"\n\nIf your search for help leaves you with a half-dozen offers of \"I don't know Access. I like FoxPro better. I can show you how to do it FoxPro,\" then there's this month's column. It's not person-to-person, but you can tape it to your monitor.\n\nLittle difference\n\nThe differences between FoxPro and Access are not all that great, especially if you think about it right. You probably don't need anyone else telling you how to think, but that's OK because I might not do a very good job and so no harm will be done.\n\nHere is one way to state what's needed to solve the problem: You need to understand the three FoxPro functions, left (), at (), and substr ()\n\nHere is a better way to state what you need to know. You need to understand that all database programs have a way to look at strings and:\n\n1) Pluck out a certain number of characters starting from the left, or from anywhere in the string.\n\n2) Locate the position of a certain character.\n\nTo illustrate the solutions, I'll assume a table called Contribs with the original field called name, which contains a last name, then a comma, then the rest of the name: the first name, and perhaps middle name or initial. There are two new columns called last and rest.\n\nThe examples will be in SQL, which you can type into the SQL view of an Access query.\n\nOne of the benefits of SQL being the lingua franca of databases is you can pass along plain text of a query and not have to describe the specific screen of a specific database program. Last month's column presented the FoxPro/Access-SQL versions of updating columns and deleting rows.\n\nFollowing the usual typographic convention, the SQL words are all caps. The string function words are in italic, a departure from the usual convention.\n\nTo split off the last name into the field called last.\n\nUPDATE Contribs SET last = Left (name, InStr (name, \",\")-1) WHERE (Len (name) >0 AND name IS NOT NULL AND InStr (name, \",\") > 0 )\n\nNow for the explanations: The function, instr (), which you can remember as being short for in-string, is the Access equivalent of the FoxPro function at (). These functions give the position of the comma within the field called name.\n\nTo avoid taking the comma trailing the name, you subtract 1 from the number that indicates the comma position.\n\nWhere conditions\n\nIn SQL update queries, all rows are changed unless you include a WHERE condition. In this case, the WHERE statement tells Access to split all the name fields apart, but only if three conditions are true: 1) the field is not empty, 2) the field is not NULL, and 3) the comma can be found in the field.\n\nThis is necessary in Access to avoid error messages that FoxPro would never trouble you with.\n\nI know that conditions one and two seem to be saying the same thing, but in Access, which is more precise than FoxPro in the way it deals with nothing, a field can be NULL or it can contain a string whose length is zero.\n\nTo split off the rest of the name into the field called rest.\n\nUPDATE Contribs SET rest = Mid (name, InStr (name, \",\") +1) WHERE (Len (name) > 0 AND name IS NOT NULL AND InStr (name, \",\") > 0 )\n\nThe Access function for going to any position within a string is mid. The FoxPro equivalent is substr().\n\nAgain, we have to find the comma, but not put it in the destination field. So for the right split, we add 1 to the position of the comma.\n\nRichard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-on-the-internet","slug":"1997-august-on-the-internet","url":"https://dwillis.github.io/uplink/article/1997-august-on-the-internet.html","year":1997,"month":"August","monthNum":8,"headline":"On the Internet","kicker":"Education on the Web","author_name":"Pete Weitzel","author_title":"","affiliation":"Poynter Institute","summary":"A guide to useful web resources for education reporting and research, compiled by Pete Weitzel, featuring sites like Education Week that provide statistics, articles, and background information on education issues.","keywords":["education","resources","reporting"],"topics":["Education","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","gopher"],"full_text":"To add depth to dailies or to make your series as hard-hitting as possible, it is worth a trip to the Web. Here are dozens of useful sites collected by Pete Weitzel. These sites offer everything from education statistics to education-related magazines. These locations can help you earn an A on your next education assignment.\n\nSites we've found most useful\n\nEducation Week-http://www.edweek.org/ Articles from the current Education Week; background information on education issues; links to a range of education sites, listed by subject, and to education organizations. Education Week's \"The Daily News\"-http://www.edweek.org/context/clips/clips.htm Education stories from select newspapers, magazines and other reports on education. U.S. Department of Education - http://www.ed.gov/ A good place to get started on research. Links to resource directories, research reports, education organizations and associations, including: Associations: http://www.ed.gov/EdRes/EdAssoc.html Organizations: http://oeri.ed.gov:8888/STATES/direct/SF National Center for Education Statistics http://www.ed.gov/NCES/ or gopher.ed.gov:10000 Numbers on just about anything you want to know about. The Condition of Education, 1997-http://www.ed.gov/NCES/pubs/ce/index.html The NCES analysis of education in America, with statistical comparisons, and including these chapters: The Social Context of Education - http://www.ed.gov/NCES/pubs/ce/c97004.html Public and Private Schools, How Do They Differ? - http://www.ed.gov/NCES/pubs/ce/c97006.html Climate and Diversity-http://www.ed.gov/NCES/pubs/ce/c97sece.html Educational Research Information Center (ERIC) - http://www.aspensys.com/eric/barak.html\n\nERIC is a national information system providing access to education-related literature. It is supported by the Department of Education and the National Library of Education.\n\nERIC Clearinghouses-http://www.aspensys.com/eric/barak.html#1 The clearinghouses collect, abstract and index education materials, and respond to requests for information in specific subject areas.\n\nEric Search Wizard - http://ericae2.educ.cua.edu/scripts/ewiz/amain2.asp Designed to help you search for information on specific subjects.\n\nCurriculum Standards on the World Wide Web http://cdp.mde.state.mi.us/info/standards.html Links to national and state standards, by subject. A good starting place if you are interested in doing a local report card. Edlinks - http://webpages.marshall.edu/~jmullens/edlinks.html A strong list of education links, with short explanations of each site. And a 'Daily Report Card' of media coverage of education issues. Maintained by Marshall University.\n\nOrganizations and interest groups\n\nAmerican Association of School Administrators - http://www.aasa.org/ \"Front Burner Issues\" plus a comprehensive lists of links to education organizations & associations, materials & resources, regional laboratories, and to state superintendent organizations. Council for Great City Schools - http://www.cgcs.org/ Includes \"Urban Educator,\" and descriptions of \"programs that work.\" Center for Research on Evaluation, Standards, and Student Testing - http://cresst96.cse.ucla.edu/index.htm Education Commission of the States - http://www.ecs.org Hot topics backgrounders Education Writers Association - http://www.ewa.org/ Links to prize-winning education stories and various education sites American Federation of Teachers - http://www.aft.org//index.htm National Education Association - http://www.nea.org/ Council for Basic Education-http://www.c-b-e.org/index.htm Washington-based interest group focused on curriculum standards. National Association of Secondary School Principals - http://nassp.org/ National PTA - http://www.pta.org/ U.S. Charter Schools Association - http://www.uscharterschools.org/\n\nReports and research studies\n\nThe Condition of Education, 1996 http://www.ed.gov/NCES/pubs/ce/home.html National Education Summit - http://www.summit96.ibm.com/ Americans' Views on Standards - http://www.summit96.ibm.com/brief/papers/public.html#executive A Public Agenda Foundation study for the National Education Summit. Goals 2000 Update - http://www.ed.gov/G2K/ProgRpt96/ A 1996 update on the federal initiative. Third International Mathematics and Science Study - http://www.ed.gov/NCES/timss/brochure.html or http://www.ed.gov/NCES/timss/ The first is the official site; the latter seems to get to the point quicker. Teachers of English to Speakers of Other Languages - http://www.cal.org/cal/html/eslindex.htm Proposed standards for ESL teaching. US News, Special Report on Standards - http://www.usnews.com/usnews/news/stanhigh.htm March 1996 cover story, with poll results, and links to related resources\n\nReport cards\n\n(Note: Many states, school districts and schools post \"report cards\" on Internet home pages. Here are links to two urban school district pages, as examples, and to one newspaper's guide to providing comparative information on local schools.) Boston Public Schools - http://www.boston.k12.ma.us/ The system's website includes statistical reports on its schools (enrollment by race/grade and test scores) and school home pages. San Francisco Public Schools \"Accountability Report Cards\"-http://www.sfusd.k12.ca.us/schlhome.htm Seattle Times High School Guide - http://www.seattletimes.com/schoolguide/altindex.html The newspapers' consumer report on high schools, with report cards and statistics on each.\n\nMiscellaneous\n\nHarvard Graduate School of Education - http://gseweb.harvard.edu/ Links to the school's education newsletter, abstracts of articles in the Harvard Education Review. Columbia Teachers College - http://www.tc.columbia.edu/rschact.stm Centers on Urban Education and Technology and School Change. Edweb - http://edweb.cnidr.org:90/resource.cntnts.html Discussions on education and the use of computers and the internet. Phi Delta Kappan - http://www.pdkintl.org:80/karticle.htm Home page for education fraternity's journal Lists of schools The American Schools Directory - http://www.asd.com/ Provides links to web sites at 106,000 K-12 schools. HotList of K-12 Internet School Sites - http://rrnet.com/~gleason/k12.html Links to state departments of education, state school districts. Web66: International WWW School Registry -http://web66.coled.umn.edu/schools.html Links to school websites around the world. Pete Weitzel can be reached at (919) 489-4151, or send e-mail to pweitzel@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-irenicar-notes","slug":"1997-august-irenicar-notes","url":"https://dwillis.github.io/uplink/article/1997-august-irenicar-notes.html","year":1997,"month":"August","monthNum":8,"headline":"IRE/NICAR Notes","kicker":"New strategies in CAR","author_name":"Brant Houston","author_title":"Executive Director","affiliation":"IRE and NICAR","summary":"News organizations must restructure their traditional hierarchical management to break down barriers between newsrooms, libraries, and training departments to effectively manage the explosion of electronic information. A new collaborative approach is emerging as the solution for news organizations to survive and thrive in the 21st century.","keywords":["information","structure","collaboration"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"For journalists in the 21st century the road of excess information can lead to the palace of wisdom, but only if news organizations rethink the way they gather, archive and retrieve that information.\n\nOver the past decade, news organizations in the United States have struggled to use traditional management structures to deal with terrifyingly new technologies and an overabundance of electronic news tips, faxes, on-line services and electronic government databases.\n\nThe old management structure rests on hierarchies of publisher-editor-reporter and on fiefdoms of \"desks\" such as \"national\", \"state\" and \"metro\" and on departments separated into the categories of newsroom, library, information technology, and lately the World Wide Web site. In addition to sometimes inhibiting communication and cooperation, the structure also makes few arrangements for training for reporters and editors.\n\nIn the newest \"age\" of electronic information (these ages seem to emerge every two years), the old management structure can lead to poor planning, duplication of efforts, and news stories that make little use of information that could provide depth and context.\n\nNew strategies\n\nBecause of the shortcomings of the traditional approaches, a new structure has begun forming as news organizations realize they must tear down the virtual walls between the newsroom, the library, and the trainers. While this new structure is only a preliminary one and only beginning to be implemented, it is a first step in re-thinking how a news organization can survive and thrive in the 21st century.\n\nThe new structure calls for requiring cooperation between reporters and editors, research librarians and trainers. Although the new structure may initially seem only appropriate for large organizations, smaller news organizations will need to look toward incorporating the new approaches for research and training.\n\nHow information comes in\n\nOne way to understand the need for the new approach is to look at what is happening at many U.S. newspapers now. Traditionally, reporters have been responsible for storing and organizing their information in their own private files. Often this storage is haphazard and inconsistent because of the daily requirements and the innate volatility and unpredictable nature of news.\n\nIf the reporters share their information, it is generally on an informal basis in conversation or a slightly formal basis in memos or in \"mentor\" programs. Indeed, some reporters see little sense in sharing too much information because of the internal competitiveness in a newsroom, a competitiveness often condoned by management in hopes of producing better stories.\n\nMeanwhile, secondary sources such as news stories, atlases, directories and reports come to the news library, which then reminds the news staff of the documents and resources it has gathered.\n\nIn recent years, however, more information has come in electronically. Because of its electronic form, the information can be more easily stored and distributed if a news organization is up to date in its technology.\n\nTo see how a new structure can work, here are some of the current best techniques of electronic information gathering.\n\nHow new databases are dealt with\n\nA reporter, as part of working on a story, acquires a database that contains thousands of relevant records on the subject of a story or a database that is so complete or intriguing that a story springs from it.\n\nThe reporter then seeks the help of an experienced \"database editor.\" The database editor is usually a veteran journalist who has been largely self-taught in technology and software. The database editor is well-trained in reviewing a database, finding its flaws and determining its accurate uses. The database editor also has the abilities, sometimes with help of more expert data processor, to carefully correct some of the database's imperfections, perhaps by standardizing names and coding categories.\n\nThe reporter, who may have received in-house or outside training in data analysis for journalistic purposes, reviews and analyzes the database, conducts interviews, and goes into the field to gather more information for the story.\n\nDuring the interview stage, the reporter also consults with the originating editor and again with the database editor, who may be called on to do more \"data cleaning\" and more advanced data analysis. When the database has been used for its first story or series of stories, it then goes to the news library for archiving and distribution. At that point, the database editor writes a document detailing the kinds of information it contains and the caveats involved in its use. The document is reviewed by the librarian and possibly receives further editing. This database and its documentation is then archived on CD-ROM and/or made available through an Intranet or some other kind of an internal network for journalists. During the initial reporting, of course, the library is involved, pointing the reporter to other databases and sources of information. In a third path of information gathering, the reporter does not obtain a database, but goes to the Intranet or to the library to search its archives, seeking advice from the news researchers on how to better search and on what other archives might be available. The reporter also searches public sources and private on-line databases with guidances from researches. In another path, the reporter keeps his or her own notes in a software program such as Lotus Notes and retrieves various pieces of information through key word searches. At the same time, the network technology and library keeps track of the reporter's interests. When a fax or other electronic information comes into the newsroom or library or is entered into the electronic archives, the reporter could be alerted electronically to the new information.\n\nWhat's needed\n\nLooking at just these examples of how information gathering has developed helps identify the kind of coordinating that is needed. This new reporting requires a trained editorial staff, trained library researchers, a training curriculum, a World Wide Web and network staff, and a trainer staff. At smaller news organizations, one staff member might still perform two or three of these jobs, but these are some of the titles and duties that have evolved:\n\nDatabase editor. Coordinates acquisition and analysis of electronic databases and information by the news staff.\n\nWorks with news librarian, trainer, and Intranet coordinator to determine strategies, prevent duplication of efforts, and to develop curriculum for training.\n\nNews Library Researcher. Coordinates acquisition by library staff, archival, retrieval and distribution of electronic databases and information.\n\nTrainer. Coordinates training of news staff use and understanding of electronic databases and the software for the use and understanding. Works with database editor and news library researcher to develop curriculum, schedule training and choose appropriate trainers. Works with Intranet coordinator to make training and materials available on Web.\n\nIntranet Coordinator. Works with database editor, news library research and trainer to ensure search tools and material on the Web are useful and relevant.\n\nThe editorial staff needs a basic knowledge of how electronic information is stored and how it can be used. The library researchers need to make sure that information is being archived and distributed and that the news staff knows what the library has or can get. The Web site and network staff needs to know how the news staff produces stories and what kind of distribution works bests. The trainers need to ensure the editorial and library has the capability to use all the electronic tools and information and to realize the potential of those tools. This involves creating a curriculum with those other groups.\n\nThe push into the electronic information age has been primarily spurred by reporters and news librarians. Editors and managers have often been overwhelmed by daily crises and the need to put out news. They frequently have regarded the acquisition of electronic information as mysterious, threatening or frivolous.\n\nHowever, the advent of Web technology, through its visual representations, has defogged the importance of electronic information and editors are beginning to realize the treasure trove for stories and services it provides.\n\nEditors and managers can now step forward to ensure that the acquisition is organized and efforts are not duplicated.\n\nWith this beginning structure, news organizations can ensure they continue to do journalism and are not simply information providers.\n\nBrant Houston can be reached at (573) 882-0684, or send e-mail to brant@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-seminars-bootcamps-conferences","slug":"1997-august-seminars-bootcamps-conferences","url":"https://dwillis.github.io/uplink/article/1997-august-seminars-bootcamps-conferences.html","year":1997,"month":"August","monthNum":8,"headline":"Seminars, bootcamps, conferences","kicker":"Get your training","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers specialized computer-assisted reporting training workshops, bootcamps, and seminars at various locations throughout 1997-1998, teaching journalists how to extract data from government files, create spreadsheets, and navigate databases for investigative reporting.","keywords":["training","journalism","databases"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","spreadsheet","PC"],"full_text":"NICAR provides specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases. Cost varies. For information, call Lisa Barnes at (573) 882-8969. September 11-13, 1997 - Washington, D.C. Computer-Assisted Reporting Workshop Sponsored by NICAR and the Medill School of Journalism September 15-17, 1997 - Atlanta, Ga. On-the-road training for CNN September 20, 1997 - Coeur D'Alene, Idaho Join NICAR for hands-on training at the Idaho Press Club's annual conference in Coeur D'Alene. Limited to conference participants. September 24-26, 1997 - Arlington, Va. On-the-road training for USA Today October 4, 1997 - Tucson, Ariz Join NICAR for hands-on training at at the Society of Environmental Journalists' annual conference. Limited to conference participants. October 16, 1997 - New York, N.Y. Join NICAR for hands-on training at the Media and Democracy Conference. Limited to conference participants. October 16-17, 1997 - Washington, D.C. On-the-road training for Reuters October 20-24, 1997 - Ft. Lauderdale, Fla. On-the-road training for the Sun-Sentinel December 11-13, 1997 - Washington, D.C. Computer-Assisted Reporting Workshop Sponsored by NICAR and the Medill School of Journalism\n\nConferences & Events\n\nNICAR will offer training and seminars at the following professional conferences, including the IRE and NICAR national conferences. Costs vary. For information or to register, call Lisa Barnes at (573) 882-8969. November 15-16, 1997 - Ciudad Juarez, Mexico Gathering at the Border March 5-8, 1998 - Indianapolis, Ind. NICAR Conference June 4-7, 1998 - New Orleans, La. IRE National Conference June 3-6, 1999 - Kansas City, Mo. IRE National Conference\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo. As with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size. For information, call Wendy Charron at (573) 882-0684. January 4-9, 1998 - Columbia, Mo. NICAR Regular Bootcamp May 10-15, 1998 - Chapel Hill, N.C. NICAR Advanced Bootcamp May 17-22, 1998 - Columbia, Mo. NICAR Regular Bootcamp August 9-14, 1998 - Columbia, Mo. NICAR Regular Bootcamp","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-from-the-nicar-library","slug":"1997-august-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-august-from-the-nicar-library.html","year":1997,"month":"August","monthNum":8,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering topics from aviation and crime to healthcare and finance, enabling journalists to investigate government spending, safety records, and regulatory compliance across multiple agencies.","keywords":["databases","government","investigation"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW U.S. Food and Drug Administration's Medical Devices Reports, detailing complaints about drugs, pacemakers and other medical products. 1974-1996.\n\nNEW U.S. Centers For Disease Control's AIDS database, providing case-by-case demographic information about those with the HIV virus. 1995.\n\nU.S. Census Bureau's Consolidated Federal Funds Reports, showing which communities get how much under various federal programs. 1983-1995.\n\nFederal Aviation Administration's accidents and incidents, including major plane crashes since 1974.\n\nNASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\n\nFederal Election Commission campaign contributions by individuals and political action committees, 1990-1997.\n\nThe Health Care Financing Administration's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings, 1991-1996.\n\nCoast Guard boating accidents, 1969-1994.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration, 1974-1997.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\n\nFederal procurement data, 1992-1996, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1996.\n\nNational Bridge Inventory System data, includes inspection grades, 1994-1995.\n\nNEW. FBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the 1995 final report.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration loan guarantees, 1980-1997. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nThe National Inventory of Dams, 1991-1995.\n\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1996.\n\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1975 to 1995.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-bits-bytes-and-barks","slug":"1997-august-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-august-bits-bytes-and-barks.html","year":1997,"month":"August","monthNum":8,"headline":"Bits, Bytes and Barks","kicker":"Help Wanted","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are hiring for multiple positions including a Director/Trainer for the Campaign Finance Information Center, Database Library Administrator, IRE Deputy Director, and NICAR Database Library Administrator, with salaries ranging from $35,000 to $46,000.","keywords":["hiring","journalism","database"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Bits, Bytes and Barks\n\nHelp Wanted\n\nIRE and NICAR have several positions open:\n\nDirector/Trainer for Campaign Finance Information Center and Database Library Administrator for Campaign Finance Information Center\n\nWe are looking for excellent journalism skills and/or excellent computer skills. The center will archive state and local campaign finance data, offer on-site and on-line training and will publish a newsletter. Salaries for the jobs range from $35,000 to $40,000.\n\nIRE Deputy Director\n\nIRE is also looking for a deputy director who will report to the executive director and help oversee IRE's six divisions: training, publications, the Mexico program, NICAR database library, operations and the new campaign finance information center. Salary range is $40,000 to $46,000. The job may include teaching at the school of journalism as an adjunct professor.\n\nDatabase Library Administrator for NICAR\n\nThis position involves supplying data and data analysis to more than 120 print and broadcast organizations and overseeing a team of graduate research assistants. Salary range is $35,000 to $45,000.\n\nInquiries and resumes should be sent to Brant Houston, Executive Director, IRE, 138 Neff Annex, Columbia, Mo. 65211 or brant@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-reader-response","slug":"1997-august-reader-response","url":"https://dwillis.github.io/uplink/article/1997-august-reader-response.html","year":1997,"month":"August","monthNum":8,"headline":"Reader Response","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"Uplink newsletter is soliciting reader feedback and suggestions for improving content, columns, and topic coverage, with readers invited to submit their ideas directly to the editor.","keywords":["feedback","improvement","engagement"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Reader Response\n\nIt's time we heard from you. We are looking to improve Uplink and want to know your ideas.\n\nAre there certain columns that you would like to add? Are there certain topics you would like covered? Have you done a story that you think should be featured in an upcoming issue? This is your chance to play armchair editor.\n\nSend comments, critiques and suggestions to Uplink's editor, Jody Sowell, at jody@nicar.org We look forward to hearing from you.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-august-nicar-net","slug":"1997-august-nicar-net","url":"https://dwillis.github.io/uplink/article/1997-august-nicar-net.html","year":1997,"month":"August","monthNum":8,"headline":"NICAR Net","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The article provides information about subscribing to IRE-L and NICAR-L mailing lists for journalists, which discuss topics like campaign finance, traffic accidents, ethics, and medical investigations, with archives available online.","keywords":["listserv","journalism","archives"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web","Email"],"full_text":"Recent topics on the IRE and NICAR listserv have included campaign finance data, troubles with traffic accident data, ethics, medical investigation, medical investigations and Access tips.\n\nTo subscribe to IRE-L or NICAR-1, send e-mail to listproc@lists.missouri.edu\n\nIn the body of the message, type:\n\nsubscribe NICAR-L<your name>\n\nsubscribe IRE-L<your name>\n\nAlso, check out the IRE-L and NICAR-L mailing list archives on our website at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-september-fun-with-car","slug":"1997-september-fun-with-car","url":"https://dwillis.github.io/uplink/article/1997-september-fun-with-car.html","year":1997,"month":"September","monthNum":9,"headline":"Fun with CAR","kicker":"Uplink Update","author_name":"","author_title":"","affiliation":"","summary":"This article showcases the diverse applications of Computer-Assisted Reporting (CAR) beyond traditional investigative journalism, featuring examples from multiple newsrooms including the Wilmington Star, Detroit Free Press, and Colorado Springs Gazette that used CAR for feature stories, entertainment analysis, and reader engagement projects.","keywords":["reporting","journalism","analysis"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Access"],"full_text":"For any of those who doubted that CAR could do it all, this issue is for you. CAR is not just for byte heads and government reporters any- more. In this issue, Janet Roberts shares how the Wilmington Star has used CAR for features ranging from Afri- can-American women's difficulty in finding love to an analysis of cities in other states that shared Wilmington's name. Heather Newman and the Detroit Free Press turned their lenses on movie critics and reviewed their re- views using computers. The Colorado Springs Gazette asked readers to rate their favorite comics and then used Access to ana- lyze favorites according to age. NICAR's Kara Morrison also looks at a report on aging trends of parents popular baby names done by the Minneapolis Star-Tribune. Also in this issue, North Carolina reporter Hart Matthews shares his first CAR experience.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-september-number-assisted-features","slug":"1997-september-number-assisted-features","url":"https://dwillis.github.io/uplink/article/1997-september-number-assisted-features.html","year":1997,"month":"September","monthNum":9,"headline":"Number-assisted features","kicker":"CAR for all","author_name":"Janet Roberts","author_title":"","affiliation":"Wilmington Morning Star","summary":"Janet Roberts discusses how the Wilmington Morning Star utilizes computer-assisted reporting (CAR) to enhance feature stories with data analysis, such as the challenges African-American women face in finding love and comparisons of cities sharing Wilmington's name.","keywords":["CAR","features","Wilmington"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Excel","Netscape","SelectPhone CD","Internet"],"full_text":"Gone are the days when computer- assisted reporting was the playground of big-splash, hard-news project reporters. Today, any reporter worth his or her salt is in on the game, using computers just like they use the telephone, the notepad and the morgue. And includes the features depart- ment. At the Wilmington \"Morning Star,\" a 55,000 daily on the North Carolina coast, we have made it our goal to get all reporters - in all corners of the news- room using computers as a daily reporting tool. The effort is paying off. If you look at the newsroom PCs with Internet access, you won't just find City Hall reporters sitting in front of the screens. You'll see features and sports reporters, too. You'll find the same re- porters plugging numbers into Excel for quick crunching. They load Netscape search forms to query the newsroom's voter registration records, property tax records and other government databases. They pop in the SelectPhone CD to find sources for their stories. For readers, the result is stories that often have more depth, more varied sources and more reach than they would otherwise. Consider these examples: Coming to their Census Feature writer Deirdre McGruder set out to do a story about educated black women who have found successful ca- reers but can't find love. She had plenty of anecdotes from women frustrated because they can't find educated black men to date. From there, she sought figures to lend the story weight. Fortunately for her, the Census Bu- reau posted a report on its home page in July with 1996 estimates on the educa- tion and income levels of black Ameri- cans. She found there are about 50 per- cent more educated black women in this country than educated black men. The number of black female professionals in the workplace continues to rise as do their salaries. And a trip to the Bureau of Justice Statistics Web page gave her another eye-opening figure: In 1995, black men were seven times more likely to be in prison than their white counterparts. McGruder was able to weave the statistics throughout the story, resulting in a far more thoughtful piece than it would have been other- wise. This was her first experience with the Census Bureau's Internet site, and she recommends it to other feature reporters. \"The Census is your friend,\" she said. \"Visit often.\" Naming names Feature writer Lee Roberts did just that when asked to do a piece about the other Wilmingtons in the United States. He headed to the Census Bureau's home page and searched for the name Wilmington. That found him 12 other communities that share our name, plus a statistical profile of each. Roberts downloaded statistics on each com- munity and imported them into an Excel spread- sheet for comparison. The computer work paid off. Before he even picked up the phone to start calling people in the different Wilmingtons, Roberts knew the racial makeups of the towns, whether they are rural or urban, how many people commute to work, the median income, etc. That allowed him to frame his questions and focus his interviewing time on the anecdotes and descriptions that gave the story life. Relocating Feature writer Clifton Daniel was doing a piece on families who leave higher-paying jobs in other cities to move to Wilmington for its slower pace and beautiful beaches. The cost of living doesn't hurt, either, a point Daniel drove home with a visit to the Internet's Relocation Calculator. He was able to plug in the salaries people were earning in the towns they left and show they needed to earn substantially less to lead a similar lifestyle in Wilmington. What's in a name? Lee Roberts used the SelectPhone CD, a cross-reference directory for the entire United States, to do a story on just plain folks from our readership area who share famous names with celebrities and historical figures. The CD led him to Michael Jordan not the basketball superstar who hails from Wilmington, but the self-proclaimed \"short, white and poor\" one, a roofer who lives just up the road in a neighboring county. The short Jordan said kids often telephone thinking they've found the bas- ketball star. \"The younger kids who call, I feel sorry for them,\" Jordan said. \"They have high hopes, and then they get me.\" Roberts also found Joan \"Can we talk?\" Rivers living in Wrightsville Beach, John Kennedy living in rural Whiteville, Robert E. Lee living near a Civil War fort in Kure Beach and John Wayne, a huge fan of the movie great. Janet Roberts can be reached at (919) 343-2389, or send e-mail to starnews@wilmington.net","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-september-paper-rates-critics","slug":"1997-september-paper-rates-critics","url":"https://dwillis.github.io/uplink/article/1997-september-paper-rates-critics.html","year":1997,"month":"September","monthNum":9,"headline":"Paper rates critics","kicker":"Hooray for CAR","author_name":"Heather Newman","author_title":"Reporter","affiliation":"Detroit Free Press","summary":"The Detroit Free Press conducted a computer-assisted analysis of movie reviews from seven major newspapers, tracking ratings for over 200 films from January to March 1996 to compare critical opinions with box office performance and Academy Awards. The study found that critics rarely agreed with each other or with the public, with only eight movies receiving unanimous ratings across all reviewers.","keywords":["analysis","critics","movies"],"topics":["Investigative Methods","Business & Economy"],"technologies":["Paradox","Nexis"],"full_text":"Readers love movie critics, and movie reviews are one of the most- read parts of the paper. That's what led us to try a com- puter-assisted analysis of film re- views for the Detroit Free Press. Knight-Ridder/Tribune Graph- ics puts together a weekly summary of starred reviews of current mov- ies. We typed in their record of the number of stars given to each film by reviewers from nine newspapers into a Paradox database, starting with the ratings given in January 1996 and ending the last week of March. We ended up with ratings for more than 200 movies. Eventually we cut the number of papers we used to seven, because two of the newspapers only appeared in the graphic for about half a year each. Appearing in the final story were reviews from the Dallas Morning News, the Miami Herald, the Phila- delphia Inquirer, the New York Daily News, the Seattle Times and, of course, the Free Press. We typed in the Motion Picture Associa- tion of America rating (PG, R, etc.) for each movie, the number of critics who had re- viewed a particular film - useful for doing averages - and the month, day and year when reviews first appeared. We also included an arbitrary \"category\" for each film. Free Press movie critic Terry Lawson and I sat down to go over each film, deciding whether it was a comedy, a drama, horror, suspense, a children's film and so on. This let us do averages by category, to see which types of films were getting the best ratings that year. Critical analysis Once the grunt work was done, we moved on to the analysis. First up: a quick check to see what the most popular rating was (two stars) and averages by category. Dramas averaged 2.51 stars each, followed by children's films at 2.44. Horror movies, to no one's surprise, were dead last, with an average of 1.8. Next was a check to see what the best-rated films were for 1996, we wanted to use the year to compare critics' opinions to the public's vote (as measured by box office take) and the Academy Awards. The movie with the most stars? \"The Hunchback of Notre Dame,\" whose average of 3.86 put it firmly at the top of the pack. Unfortunately, the \"Hunchback\" had not done so well at the box office, where it didn't even make the top 10, or at the Oscars, where it didn't get a major award nomination. \"People at Disney didn't even think that was a great film,\" we were told by Bruce Davis, executive director of the Academy of Motion Picture Arts and Sciences (the folks who give out Academy Awards). Thumbs up or thumbs down? Now we had the basis for our story. We knew, after further checking, that critics didn't often agree with the public or the Academy. The next numbers we crunched told us the final link in the story: They didn't agree with each other, either. We found just eight movies in the study where critics unanimously gave the same rating. Four of those were bombs. (\"Remember Eye for an Eye,\" \"Kazaam?\"). Just one made three stars (\"Fly Away Home\"). None made four. In fact, critics didn't even agree if some mov- ies were good or awful. We had four movies that had gotten both one and four stars by different critics. Sixty-five films got both one star and three stars from those who reviewed them. We used Lexis-Nexis to look up some of those reviews, then called the critics who wrote them. In what turned out to be our lead anec- dote, Rene Rodriguez of the Miami Herald talked about reviewing \"Austin Powers,\" Mike Myers' spoof on secret agents. He loved the film, laughing out loud in the near-empty the- ater, and called Myers \"a comic genius\" in his review. We followed it up immediately in the story with a quote from the \"Austin Powers\" review by Tom Maurstad of the Dallas Morning News, who opened that the movie \"fails to provide a single reason why we'd want to sit through it.\" We included comments from a half-dozen reviewers, including one-half of the most-dis- agreeing critic duo in America: Siskel and Ebert. (We got Gene Siskel. Rogert Ebert was at Cannes.) To round out the piece, our critic Lawson wrote a first-person story about being a movie critic, which led with his confession about how he deals with people looking over his shoulder on airplanes and wanting to know why he's writing about movie stars: \"With no shortage of professional pride, I reply: 'I do internal system analysis for a Korean portable computer manufacturer.\" Heather Newman can be reached at (313) 223-3336, or send e-mail to newman@det-freepress.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-september-car-goes-to-the-dogs","slug":"1997-september-car-goes-to-the-dogs","url":"https://dwillis.github.io/uplink/article/1997-september-car-goes-to-the-dogs.html","year":1997,"month":"September","monthNum":9,"headline":"CAR goes to the dogs","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Pet license data analysis is a popular and accessible first computer-assisted reporting project for journalists, offering quick stories that generate significant reader interest and demonstrate that CAR doesn't require lengthy investigations.","keywords":["pets","data","journalism"],"topics":["Investigative Methods","Data & Databases"],"technologies":[],"full_text":"One of the most popular features and one that is often used as a first project for beginning CAR reporters is an analysis of pet licenses. Several newspapers and television stations have tried their hand at this story, analyzing the most popular breed, name, sex and color dogs by city or county.\n\nMany cities already have their pet license information stored electronically, but some do not, requiring reporters to do their own data entry.\n\nThe results, however, are often worth it. It's a quick story that often gets the biggest of results reader interest. The Lawrence, Mass., Eagle-Tribune wrote a dog story in 1994 that was featured in IRE's \"100 Computer-Assisted Stories.\" \"It was a fun feature story that delighted our readers,\" Eagle-Tribune reporters wrote in the guide book. \"It also showed other reporters and editors that not all computer-assisted reporting has to be lengthy investigative pieces.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-september-computers-get-comical","slug":"1997-september-computers-get-comical","url":"https://dwillis.github.io/uplink/article/1997-september-computers-get-comical.html","year":1997,"month":"September","monthNum":9,"headline":"Computers get comical","kicker":"Funny CAR","author_name":"Victor Greto","author_title":"","affiliation":"The Gazette","summary":"The Gazette in Colorado Springs conducted a reader survey of over 4,000 responses to identify the most and least popular comics, using Microsoft Access and Excel to analyze preferences by age and gender demographics.","keywords":["comics","survey","database"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Access","Excel"],"full_text":"They're inane, funny, silly, idiosyncratic - and humbling because they're one of the most popular features of the daily newspaper.\n\nThey're the comics.\n\nRecently, The Gazette in Colorado Springs conducted a survey of its readers to discover what its most-read and most-hated comics were. Readers could cut out a form and send it back to us within two weeks. Sure enough, we received over 4,000 replies, better than any other cut-out-a-form-and-send-it-in-on-your-own-nickel survey we've done.\n\nReaders chose what they thought were our best and worst comics, and then rated each comic with one of the three choices: ALWAYS READ, SOMETIMES READ or HATE IT.\n\nThe Gazette's Research Center volunteered to tabulate the results into Microsoft Access and run queries that dealt with ages and gender of respondents, and how a mix of each of these categories judged the comics.\n\nTo facilitate inputting, I converted genders into the numerals 1 and 2. I converted age variables into the numerals 1 (17-younger), 2 (18-34), (35-54), and 4 (55-older). The three ratings from which they chose were also converted into numerals.\n\nWe hired temps to input the data. It took about three days.\n\nFor best and worst\n\nFirst of all, after all the data was inputted, we wanted to find out the overall tally: what was considered the BEST COMIC?\n\nWhen you choose peculiar queries, you don't want to run the Query Wizard Access 95 provides. So pick QUERY, then DESIGN VIEW. Choose the table (which in my case I called, simply, COMICS). This query entailed two steps: I chose BEST COMIC within the FIELD slot; in the box beneath entitled TOTAL, I chose GROUP BY. In the next column over, I chose BEST COMIC as the Field name, COUNT in the TOTAL field, then DESCENDING in the Sort field.\n\nI got my answer: the winning comic was \"For Better or Worse.\" Running a similar query, and using the same terms - except using the WORST COMIC field instead of the BEST COMIC - I calculated what people had voted as the most-hated comic: \"Where's Waldo\" and \"Judge Parker\" came out on top- bottom, actually.\n\nTo ascertain the top 5 ALWAYS READs and HATE ITs by age and sex was a little more time-consuming. I had to use Access in tandem with Excel to do it as expeditiously as I could. If you have a better way, let me know.\n\nI discovered I could not run a single query (like I had hoped) by grouping age and sex, then asking Access to count the individual comics' 1-2-3 variables. Access evidently needs to group, then count what it has grouped, one-by-one. That would mean I would have had to have done 42 queries on one category of age and sex. Because I wanted to do 8 categories, that would be a total of 336 queries. Excel came to my rescue.\n\nI exported my original table into Excel. Selecting all the responses to the ALWAYS READ, SOMETIMES READ or HATE IT (i.e., 1,2,3), I eliminated the 2s and 3s, saved the table under COMICS1 (i.e., a different name), then imported it back to Access.\n\nWithin Access and with the new table, I ran this query:\n\nChoose QUERY, then DESIGN VIEW; in the first column, I picked my COMICS1 table, the AGE field, the GROUP BY \"1\" (17-younger); in the next column, I chose the SEX field, GROUP BY \"1\" (male); in the next column, I chose our first comic, B.C., then COUNT; in the next column, I chose the next comic with the same COUNT, and continued until all the comics were done.\n\nIt worked fine. But the results were not sorted in descending order.\n\nTo sort the query, I hit the \"Analyze it in Excel\" button. From there, I selected the results, went to DATA, chose SORT, asked it to sort by ROW 2, which contained the numeric results of the Query, clicked on OPTIONS, asked it to sort from left to right, and hit OK. Finally! The top comics in descending order.\n\nThis first query counted all the \"1\" numerals for each comic as grouped by males 17-younger. I did the same for all the other combinations of male/female and the four different age-groups, analyzing or sorting each in Excel. After the initial set-up query, it didn't take long.\n\nTo get the 5 worst for each group, I exported the same original table into Excel, this time eliminating the 1s and 2s, saved under COMICS2, imported it into Access, and ran the same queries as above.\n\nVictor Greto can be reached at (719) 636-0182, or by e-mail at dorazio@gazette.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-september-testing-for-significance","slug":"1997-september-testing-for-significance","url":"https://dwillis.github.io/uplink/article/1997-september-testing-for-significance.html","year":1997,"month":"September","monthNum":9,"headline":"Testing for significance","kicker":"Stats from the road","author_name":"John Sullivan and Sarah Cohen","author_title":"","affiliation":"NICAR","summary":"Journalists can use statistical significance tests, particularly chi-square tests, to determine whether patterns in data like employment discipline differences represent real trends or random chance when investigating potential discrimination.","keywords":["statistics","significance","discrimination"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["SPSS","Excel"],"full_text":"Many journalists spend hours staring at data and wondering if they are looking at something that happened by accident. When a sheriff disciplines relatively more female deputies one year than male, is it possible that the pattern is just a fluke? The sheriff will argue it was a random blip in the numbers. A reporter has been hearing of other signs of sex discrimination. A significance test can help guide you toward the truth.\n\nPick your test\n\nMany journalists who look for significance use a chi-square test. Pollsters often use a T-test. These may sound confusing, but just think of them as tools that help us look at differences. The decision for which test you can run has to do with the kind of data you start with, how it's collected and whether it represents a universe or a sample.\n\nCategories or numbers?\n\nLet's talk about the different types of data. Statisticians classify variables into four measurement scales: nominal, ordinal, ratio and interval. But we often deal with just three. As you move up the data ladder, you get more power to describe an event that happened in the real world. Nominal data are used to classify things. Race or gender for example. They are categories that do not run on a scale. Ordinal data are in categories that run on a scale. Think of them as ranks. Pure ranks, like No. 1 and No. 2, are ordinal. So are words like poor, middle-income and rich. They're easy to think about. You get more information than just categories, but you don't know anything about the distance between the values. Rich may not be twice as rich as poor.\n\nContinuous variables\n\nThese are numbers: Test scores, incomes, or years in prison. Most commonly, we use numbers that can be added, divided and multiplied with ease: Five is half as big as 10. You can always convert a number into a word. You can't convert a word to a number.\n\nChi-Square\n\nChi-squares, the most common significance test used in journalism, tells us whether data arranged in categories reflect differences that happened by dumb luck. But keep in mind that we rarely deal with samples. Some would argue there's no point to a chi-square test. If the sheriff disciplined more women than men, it happened. It's true. On to the interview.\n\nBut we still have no way to evaluate the sheriff's claim of a fluke. The nice aspect of chi-square tests is that, unlike many other statistical tests, it makes no assumptions about the sample you drew. So choosing one year because it was the most recent can still create a reasonable chi-square test.\n\nYou normally get chi-square tests by setting up a crosstab, most frequently using SPSS. Although it's possible to generate them in Excel, it's much harder, because you have to hand-calculate much of its input.\n\nBefore we go on, we should probably talk a little about what makes something significant. Social scientists often say you have a \"significant\" gap if it's unlikely - about 5 percent of the time - you would have seen answers like yours purely by coincidence. It has nothing to do with whether or not the difference is meaningful, or even true. It's just by chance. The other thing to keep in mind is that significance tests rule out coincidence as a possible explanation. They don't prove anything else.\n\nAlso keep in mind that a chi-square is sensitive to two things: The number of cases you're examining, and the difference between the numbers you get and the numbers that you'd expect based on their proportions. As soon as you start looking at thousands or even hundreds of cases, chi-squares begin to report \"significant\" results, even if the difference is teeny. So a denial rate for mortgages of 7 percent white homeowners and 8 percent for black homeowners can be significant if you have enough applications. It might not stand up to the \"who cares?\" test we use for news.\n\nT-tests\n\nThe T-test is used to examine the difference between two means. They're used when you're working with one variable, or field, with numbers in it and another that comes in two categories.\n\nSarah Cohen is always looking for ideas for future statistics columns. Has a statistic got you stumped? Have you finally mastered a statistical test? Got an idea for a future story? E-mail Sarah at sarah@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-september-writing-the-story","slug":"1997-september-writing-the-story","url":"https://dwillis.github.io/uplink/article/1997-september-writing-the-story.html","year":1997,"month":"September","monthNum":9,"headline":"Writing the story","kicker":"Handout of the Month","author_name":"Richard Galant and Don Walker","author_title":"","affiliation":"Newsday/Milwaukee Journal Sentinel","summary":"This journalism handout from the IRE National Conference provides guidance on writing compelling stories using computer-assisted reporting and data analysis, emphasizing the importance of identifying a central message, keeping the narrative simple, and telling engaging stories rather than getting bogged down in technical details.","keywords":["reporting","writing","data"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["FoxPro","Pentium"],"full_text":"This is a combination of two handouts given at the National IRE Conference in Phoenix. To order the complete handouts or to get a list of other available handouts, call the IRE Resource Center at (573) 882-3364. It's easy to remember to add color and life to your story when writing features about movie critics of baby names. It is harder, but no less important, to do so when writing about county government or aviation. The following is a combination of two writing handouts given in June at the IRE National Conference in Phoenix. Not all of these tips are new, but all are worth repeating. Got data. Now what? One of the toughest things about computer- assisted reporting is that it's not enough to be a dogged reporter and a skilled user of data- base software. The best CAR work stands out in large part because the reporters have mas- tered the skill of writing compelling enterprise stories. There are no simple recipes for effective writing in newspaper projects. But there are some tips that can serve as a guide. Hone the message Sometimes the process of reporting - ob- taining data from reluctant officials, getting it into a usable format, crunching the numbers, drawing and verifying the conclusions can be so mind-numbing that it's hard to stand back and see the significant theme that runs through your findings. But it's this part of the process that's most crucial - you have to figure out the single important message that needs to be communicated. The best stories have a single, central theme which the data, the interviews, the documents, the examples support. Keep it simple You spent three weeks on the story, inter- viewed 50 people, examined 212 documents and put 6,000 record into a Foxpro database. You used a Pentium 133 with 32 megs of ram to crunch the data. That's impressive and im- portant. But don't let it get in the way of telling the story. How it was done How important is it to explain to the reader how we did this wonderful computer research? I've found that it's far more helpful to separate the \"How We Did This Story\" from the actual reporting and writing. Ithink most readers couldn't give a flip what kind of database we used. There's nothing wrong with reporting quickly how we reached our conclusions, but save the heavy lift- ing for a sidebar inside Part A. Nerd boxes and graphics If there's important details about how you used the data and about definitions of terms used in the story, consider breaking it out into a sepa- rate box. It will be there for those who need to know the details, but it won't deter the general reader. When using graphics, keep it simple. Highlight boxes on what you found out are ex- tremely helpful. Readers tend to look toward those first, then read the story. Tell a Story The same people who balk at reading an eight inch story on county government will devour a 500-page John Grisham novel. When stories are \"too long,\" it's often because writers have failed to make them compelling enough. How do you do that? Again, there's no right formula, but often the key is to use a narrative format. Narratives keep people reading by introducing believable characters and setting colorful scenes. The best stories of this type follow the form of a novel - with a beginning, middle and end. It's not an easy thing to do in the reporting you need to nail down details of the characters, the color, the dialogue - that you might not gather in reporting a more traditional survey story. There's no excuse for embellishing the details or creating composite characters. But if you do the reporting painstak- ingly and craft the story carefully, there can be a big dividend in more readership. The computer did most of your work, but do you have enough human faces in your report? Even the most computer literate will long for a human element to your story. The real question will be whether to include that in your main story, or break out a sidebar. There's no hard and fast rule. But having someone tell your story without the benefit of Paradox or Excel makes the story. Don't forget the basics We talk often about the nut graf at the Journal Sentinel, as most papers do. But you would be surprised to see how often it gets buried in the story. If you're going to report proudly that bad guys are living in your neighborhood because of some bureaucratic screwup, then explain why that's important to life as we know it. Many reporters seem to get too caught up with the \"gee whiz\" part. Vary the tone Avoid large, uniform sections - chunks of data or extended interview segments - in the story, if possible. To keep readers interested, mix anecdotes, key findings, data and quotes. Caution Be conservative in interpreting the data. Try to imagine all the potential counter-arguments to your central finding. Bounce it off experts. You may have to go back to the data to see if your findings and methodology are really bul- letproof. Make sure your story states promi- nently the point of view of those who can make reasonable arguments against the central find- ing. Do you have an adequate response to your blockbuster? Are they buried in the story, or did they get their shot higher in the story? You have to find the right place in the story, and not bury it, even if they think your story is full of holes. Killer Comparisons Strive for ways to derive statistical findings that can fairly be stated in the most concrete terms. In Newsday's series last year on racial discrimination in mortgage lending, the central finding from an analysis of nearly 100,000 mortgage loans was that African-American ap- plicants are rejected nearly three times as often as whites. That was stated prominently. But the story also brought that point home to readers: \"The gap is so wide that a black applicant making more than $150,000 a year stands a greater chance of being rejected than does a white person making less than $35,000 a year.\" Suspense If you use the narrative form, readers will stay with the story because it keeps them in suspense about the way things will turn out. The narrative can build momentum, propelling read- ers through the story. The End Bring it back home at the end of the story. Explain what happened to the people you wrote about at the top - or go back to the community you began writing about- or both. The last few grafs of a story can be the most powerful and can leave readers with a lasting impression. Dynamic Sometimes the data is strong and the finding so urgent, that the story works best with a hard lead, rather than an anecdotal top and narrative approach. Go for it. Richard Galant can be reached at (516) 843- 3274, or by e-mail at rgalant@newsday.com. Don Walker can be reached at (414) 224-2051.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-september-sites-on-the-borderline","slug":"1997-september-sites-on-the-borderline","url":"https://dwillis.github.io/uplink/article/1997-september-sites-on-the-borderline.html","year":1997,"month":"September","monthNum":9,"headline":"Sites on the borderline","kicker":"On the Internet","author_name":"Lise Olsen","author_title":"","affiliation":"Periodistas de Investigacion","summary":"Periodistas de Investigacion has compiled a curated list of over 250 border resource links organized by theme, including environmental archives, government agencies, and health initiatives for journalists covering Mexico and U.S./Mexico border issues.","keywords":["resources","border","environment"],"topics":["Health & Environment","Business & Economy","Technology"],"technologies":["Internet","World Wide Web","Listserv","Email"],"full_text":"This special list of border links, organized by theme, is part of a larger and ongoing effort by Periodistas de Investigacion to compile lists of resources for journalists covering Mexico and covering U.S./Mexico border issues. The complete list, which now includes more than 250 links, is available in Spanish directly from our office. Health and Environment Borderlands Environmental Archives http://www.txinfinet.com/mader/ecotravel/ border/borderlands.html Archives of environmental information on the border compiled by Ron Mader, a freelance journalist who specializes in environmental themes. Along with its sister page, Border- lands Links, it includes announcements, press releases, activities of various institutions and independent groups as well as an extensive biography and list of contracts in Mexico and the United States. Borderland Links http://www.txinfinet.com/mader/ecotravel/ border/borderlands1.html A long list of links related to the border and to environmental issues, which includes a bib- liography, and an index of related sites. The author of the site, Ron Mader, offers a rating of each link from the most extensive to the hardly adequate. It includes links to publications, NGOs, government links in the U.S. and in Mexico, academic sites and others. Environmental Protection Agency http://www.epa.gov:8080/epahome/ search.html EPA's homepage includes lots of informa- tion on environmental themes, including di- rectories of databases. EPA U.S./Mexico Border Activities http://www.epa.gov/compendium/ index.html This page specifically offers information on the EPA projects in the border region. Programa Frontera XXI http://semarnap.conabio.gob.mx:80 OPINIONAprograma.htm This program, administered jointly by the U.S. and Mexican governments, is meant to fos- ter environmental protection and sustainable de- velopment. Regional Environmental Information System for the U.S./Mexico Border http://begss1.beg.utexas.edu:8888/ This organization, also known by its initials REIS, is being developed by the University of Texas in Austin with funds from the EPA. It includes information from the EPA as well as a directory of resources of other organizations in- terested in environmental activities in the border. The Sierra Blanca Legal Defense Fund http://www.compassionate.org/sbldf/ This is a non-profit group dedicated to oppos- ing the installation of a radioactive waste dump in Sierra Blanca, Texas, 25 kilometers from the border. The page, obviously with a point of view, offers information related to the case, as well as general information about radioactive waste health effects and the position of the Mexican-Ameri- can community on the issues, etc. U.S./Mexico Border Health Home Page http://www.ncfh.org:80/border/index.html Information about public health projects in the border that are funded by the Department of Health and Human Services of the U.S. Govern- ment. U.S._MEXBORDER A listserv established by the EPA to foster a dialogue between people in both countries who do research on border environmental issues and other types of related work. To subscribe, send a list by e-mail to listserver@unixmail.rtpnc. epa.gov. Economics and Commerce Border Trade Institute http://www.tamiu.edu/coba/bti Official page of the institute, based at Texas A&M University. It dispenses information on trade between Mexico and the United States, specifically following 25 products and by request offers information about the trade between both countries from 1986-1996. NAFTALAB http://naftalab.bus.utexas.edu Experimental page on border themes pro- duced by graduate students at the University of Texas. It includes the NAFTA agreement, infor- mation on economic development, telecommu- nications, health and other issues. NATAnet - Electronic Commerce Port-of-Trade for Small Business http://www.nafta.net A commercial service for businesses that includes links to various special interest publi- cations such as \"Nafta Monitor,\" U.S. govern- ment information and intonation trade sites. General Information Border and Latin American Information http://lib.nmsu.edu/subject/bord/lat.html List of resources in the Internet, along with complete text of many articles written on border issues from 1986 to the present, compiled by Molly Molloy. Border Research Institute http://www.nmsu.edu:80/~bri/index.html The border research institute site, based at the University of New Mexico, is dedicated to sup- porting better understanding and cooperation be- tween Mexican and U.S. citizens. Its information includes statistics and links to demographic infor- mation, businesses, government, environment, energy, commerce, technology, etc. Borderbase http://borderbase.utep.edu/bbplus.html Statistics about border states in Mexico and the United States. BorderLines http://lib.NMSU.EDU/subject/bord/ bordline/ (In English) http://lib.NMSU.EDU/subject/bord/ bordline/esp/ (En espanol) A monthly electronic magazine produced by the Interhemispheric Resource Center, includes news about border themes. Tijuana/San Diego Border Research http://gort.ucsd.edu/mw/tj/tj.html Information about this border area, which includes maps, graphics and data about various border projects. Frontera Norte Star http://www.nmsu.edu/Research/frontera/ public_html/frontera.htm) Anelectronicnewsletteraboutborder themes published by the Center of Latin American Studies of New Mexico State University. International Boundaries Research Unit http://www-ibru.dur.ac.uk Information about border areas worldwide produced by the British University of Durham. Latin American Network Information Cen- ter (LANIC) http://lanic.utexas.edu The Institute for Latin American Studies of the University of Texas-Austin offers access to library card catalogs, specialized databases, internet tools, socioeconomic data and links to many other sites. North American Institute http://www.santafe.edu/~naminet This is the Internet site of a non-profit dedi- cated to exploring the development of a North American community and identity, with an emphasis on commercial, environmental and sociocultural themes. It has offices in the U.S., Canada and Mexico. Rio Grande free-net http://www.rgfn3.epcc.edu In addition to offering Internet access for border communities, the service offers a page with information about health, education, sci- ence, technology, politics and culture in the border. U.S.-Mexico Border Information http://lib.nmsu.edu/subject/bord/bord.html Information on the border with numerous links. Federal Government Information Mexican Government: Diario Oficial de la Federacion http://www.infosel.com.mx/infosel/serv/dof This is a service that you must subscribe to but it keeps you up-to-date on Mexican laws and legislative activities. (In Spanish) Mexican Government: Gobierno de Mexico http://info.juridicas.unam.mx/cnsinfo/ indice.htm Archives based at UNAM in Mexico city which include the Mexican constitutions, and state and local laws. (In Spanish) Nora Paul will soon return to writing her On the Internet column. If you have any suggestions for possible Internet topics, write Paul at npaul@poynter.org or send e-mail to Jody Sowell, Uplink's managing editor, at jody@nicar.org Lise Olsen can be reached at 011-525-554- 7613 or by e-mail at lise@ire.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-september-tracking-tourism","slug":"1997-september-tracking-tourism","url":"https://dwillis.github.io/uplink/article/1997-september-tracking-tourism.html","year":1997,"month":"September","monthNum":9,"headline":"Tracking tourism","kicker":"First Ventures","author_name":"Hart Matthews","author_title":"Reporter","affiliation":"The Outer Banks Sentinel","summary":"A journalist describes their first experience using spreadsheet software to analyze six years of Dare County tourism tax data on an extremely tight deadline, discovering that fall tourism revenues had grown 43 percent contrary to expectations.","keywords":["spreadsheet","tourism","deadline"],"topics":["Investigative Methods","Business & Economy","Data & Databases"],"technologies":["Excel"],"full_text":"My first spreadsheet project started with a classic error in reportorial judgment: Late on a Monday afternoon, the day before a weekly deadline, with six years of monthly tax rev- enues just in on a barely legible fax, I told my editor about a project I'd conceived that day. Since Dare County collects occupancy and meals taxes as a percentage of sales at hotels, rental agencies and restaurants, the numbers are often used to assess the health or growth of the tourism industry, the only real economy here on the Outer Banks of North Carolina. The county tourist bureau, charged with increasing visitation in the spring and fall seasons and decreasing the area's reliance on the hurricane-prone summer season, operates on 1.5 percent of those tourist receipts. The more growth the bureau stimulates in the off seasons, the theory goes, the more revenue earns. Our numbers, however, would question that seemingly obvious premise. CAR on the run Ididn't know how the figures might add up. From what the tourism and marketing director told me, I expected to show the shoulder sea- sons growing faster than the summer. My editor \"Can you get it in this week?\" Deadline in 24 hours, two beat stories to finish, and I had never used spreadsheet soft- ware in my life. \"Sure,\" I said. It was probably 10:00 that night, after I'd entered all the monthly totals using a magnify- ing glass and back-checking with a calculator, before I realized what I had gotten into. When I began doing a simple sum to com- pare the first two years of receipts, I couldn't even remember what command to type in the third column or how to extend that command down the column for the rest of the rows. Luckily, Ihad Brant Houston's \"Computer- Assisted Reporting: A Practical Guide\" on the bookshelf behind me. (I was working at home because the office had no spreadsheet soft- ware. I had Excel.) Though mainly a matter of juggling col- umns, it took some time to do the first set of numbers. Since each yearly comparison re- quired at least three columns - one for a new year, one for the difference, and one for the percentage change - I quickly found myself using eight sheets and 20 columns. With numerous errors and restarts, I com- pleted my first table, totalling only spring and fall numbers, at about midnight. I would have been better off had I done the entire table first, but I was anxious to know the angle of my story and excited to learn that fall revenues had grown 43 percent in five years. The second table, with totals for all months of the year, Ifinished sometime around 7:30 the next morning after five hours of sleep. Yearly growth came out to 35 percent. So far, so good. I printed out two reports, knowing that I'd have to find time to come back to the house later that morning for the final grouping of summer and winter revenues. At work by 8:30. Phone calls; one beat story completed. Back at the house by 10 a.m. The last table I finished before noon. Back at the office with three printouts, I still had to make calls. I finished the second beat story in between phone interviews with tourism officials. An hour and a half before deadline, I got a wakeup call from the Chamber of Commerce president, John Bone. I had already written the story in my head when Bone told me that, accord- ing to his retail figures and despite high fall growth, the summer still accounted for the same percentage of year-round business as it had six years ago. According to Bone, the county was just as dependant on good summer numbers in 1996 as it had been in 1991. Refusing to let go of my angle, I thought my tax numbers would show something different. I was wrong. Stranded at the office away from my PC, a hand calculator told me that despite the amazing growth rate, the fall had only gained 1 percent on the summer in yearly revenue share. In fact, with the spring lagging seven points behind the yearly average, the summer was now worth 58.5 percent of the year's total, up one-half of a percent from 1991. What I had failed to understand was the pro- portional difference between 1 percent of fall receipts ($4,936) and 1 percent of summer re- ceipts ($13,894), almost a three-fold difference. The phenomenal 5-percent growth the county had seen last October could be neutralized by a mere 8 percent drop in July. In fact, Hurricane Bertha had caused a 6 percent drop in July 1996. Another point I had missed was the differ- ence between summer and off-season rental rates. The weekly cost of a vacation cottage rises roughly 90 percent when the season begins in mid-June and yields much higher occupancy- tax returns than it does in a week in the fall. The results showed that, in terms of its own budget, the tourist bureau would be better off concentrating on the over-full summer season, since even slight increases would yield greater growth in the bureau's revenues than larger percentage changes during the spring or fall. I led my story with the huge fall increases, but hedged near the end of the piece with that last point. The tourist bureau took cautious credit for the fall increases, as well as for a stunning rise in early-June receipts. I turned in the story a little after 6 p.m, and my editor was profoundly pleased. I could almost see his chest swell with pride when he heard a local radio brief two days later that began: \"According to a computer analysis of county tax receipts conducted by the Outer Banks Sentinel. \" Hart Matthews can be reached at (919) 480- 1785 or by e-mail at hart@interpath.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-september-setting-the-data-free","slug":"1997-september-setting-the-data-free","url":"https://dwillis.github.io/uplink/article/1997-september-setting-the-data-free.html","year":1997,"month":"September","monthNum":9,"headline":"Setting the data free","kicker":"Tech Tip","author_name":"Andrew Lehren","author_title":"NICAR","affiliation":"NICAR","summary":"This article explains how to use Windows NT 4.0's AT command scheduler to automate data processing tasks like running FoxPro during off-hours, allowing more time for analysis and reporting instead of routine processing work.","keywords":["automation","scheduling","FoxPro"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Windows","FoxPro"],"full_text":"You're walking on a moonlit beach with a loved one. No, you're processing data. Wait a second - you can do both at the same time. Using Windows NT 4.0, there is a clunky little command to schedule jobs while you're away from the office. With some planning, you can spend less time at your desk processing data, leaving more time for analyzing and reporting. Setting up The command to run NT's schedule service is called AT. For the most part, Microsoft's documentation shows how you can use this to back up your computer. But you can use it to run, say, FoxPro, and then use a program to do your legwork. First, you need to get it to work properly with a Windows program. Go to Start Settings Control Panels Click on the Services icon (it's the one with a couple gears grinding away). You get a menu box. Scroll down to Schedule. Highlight it. Hit the Startup Button. Now make sure your settings say the Startup type is Auto- matic, it will log on as System Account, and check the box to Allow Service to Interact with Desktop. Hit OK. Now, to make sure NT knows quickly about your changes: with Schedule still highlighted, hit the Stop button, and then hit the Start button. Now NT will let you run a Windows program at will. Where you're AT Now we're ready to actually work with the command scheduler using AT. Go to Start Run and then type Command. You get a command line prompt, but don't get scared. The command line you' need is easy to type. Here is an example to just launch FoxPro: At 11:00/next:s \"c:\\fpw26\\foxprow\" This means at 11 a.m. on the upcoming Saturday, start up FoxPro. The program will startup FoxPro every Saturday if you change next:s to /every:s. If you get lost in the syntax, there are two ways to on-line help. Atthe command prompt, you can type at ?, or you can go to Start Help and search for AT. Doing the work Just starting a program when you're not in the office is pretty silly, and your co-workers will only wonder more about you. The trick is to get the program to actually do something. Here's how I got FoxPro to do something. It's a two-step process. In the following case, let's assume I've got raw data sitting in a folder called FBI. And let's assume I've written a FoxPro program to import the data into empty tables. Let's say the program is called fbi.prg. (You could add on to your programs to include cleanup tasks like date conversions and elimi- nating leading spaces in text columns). Go to a text editor and type: KEYCOMP = WINDOWS CATMAN=OFF COMMAND = do c:\\FBl\\fbi.prg Now save that in a file called bi.fpw. You've just created a FoxPro configuration file that, when called on, will automatically do your cleanup routine. Now go to a text editor again and type: c:\\fpw26\\foxprow-Cc:\\fbi\\fbi.fpw And save that in a file called fbi.bat. You've just created a batch file that will automatically launch FoxPro 2.6 and, by using -C, you've redirected FoxPro to look to your alternate config file. Don't worry about affecting future uses of FoxPro. The program will only go to that alter- nate config file when you tell it. The Last Part Now let's go back to the AT command. Go back to your prompt, like you did before. Type: at 11:00 /every:s \"c:\\fbi\\fbi.bat\" Now, every Saturday, FoxPro will automati- cally process your latest update of that database. If you want to check on any jobs, go to the prompt and type AT, then hit return. It will list all your jobs. Trouble Shooting What we'v done is tell our computer to start a program at a certain time, then look to that program's configuration file, which then tells it to process a certain database. Here are two problems I encountered along the way. First, make sure you're using the latest ver- sion of NT. Go to www.microsoft.com, click on support and go to Windows NT. Somewhere in that patch was the stuff that gets the Command Scheduler to actually work as well as promised. To make matters worse, the patch requires you to update your NT registry. To do this, look at the readme.txt file that comes with the patch. That, of course, refers to another part of the web site. For service patch 3, in section 3.6 of the readme file, it recommends going to www.microsoft.com/kb/. Then go to article Q166730 for Windows NT. Follow the six reso- lution steps. In addition, for those who got NT Resource Kit, you'll get a gui version of the Command Scheduler. That may make the AT command a little clearer. Andy Lehren can be reached at 1-800-NBC- NEWS x 7739, or by e-mail at andy@nicar.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-september-whos-bringing-up-baby","slug":"1997-september-whos-bringing-up-baby","url":"https://dwillis.github.io/uplink/article/1997-september-whos-bringing-up-baby.html","year":1997,"month":"September","monthNum":9,"headline":"Who's bringing up baby?","kicker":"CAR delivers","author_name":"Kara Morrison","author_title":"","affiliation":"NICAR","summary":"A Minneapolis Star-Tribune reporter used birth records database and Microsoft Access to analyze trends in parental ages in Minnesota, discovering that parents are having children later in life, with particular cultural patterns among Hmong families.","keywords":["database","analysis","trends"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Access","Excel"],"full_text":"Bill Loving was looking for new feature story ideas from a commonly used database: birth records. When a 63-year-old California woman gave birth in May, Loving found what he was look- ing for. He decided to look at the ages of Minnesota mothers and fathers to see whether they were waiting longer to have children. Loving, computer-assisted reporting editor at the Minneapolis Star-Tribune, worked with reporter Tatsha Robertson on the story called \"Birth of a Trend.\" The story ran June 8. Aging trends To do the analysis, Loving used Microsoft Access. Rather than ranking ages by birth dates, he created new age fields using Access's date arithmetic function. He entered the func- tion syntax into an update query, adding a new field called MomsAge. Here's how it is ex- pressed in Access: DateDiff MomsDOB] [ BabysDOB] ) where MomsDOB is the mother's birthdate field and BabysDOB is the baby's birthdate field. The \"y\" means express the difference in full years. After doing this, he was able to group and sort the ages in descending order to find the oldest mothers. \"We did include fathers as well and saw a surprising number in their 70s and even early 80s,\" Loving said. By flipping the query into an Excel table, the trend became obvious. He graphed the age query using birth records from 1980, 1985, 1990 and 1995, and found a marked shift toward parents having children later in life. Between 1985 and 1995, he found the aver- ageage of Minnesota women giving birth jumped from 26.6 to 28.5. In 1995, the oldest woman to give birth was 53; the oldest father of a newborn was 83. And the number of Minnesota teenagers having children dropped, while more women became mothers in their 30s and 40s. From the database, they also saw a distinct cultural difference in the Minnesota population. \"In the oldest parents, the majority were Southeast Asian,\" Loving said. Knowing this, Robertson researched why especially Hmong men and women were more likely to have children later in life. She found that in Laos, few couples use birth control and the mortality rate is high, so they typi- cally keep having children. The Hmong families who live in Minnesota continue the cultural practice. Because the database contained addresses, Robertson was able to interview the perfect Min- nesota woman to embody the trend toward late motherhood. Carole Kowitz, a new mother of twins in her 50s, was the focus of the story. \"Guys have been doing this for years and years,\" Kowitz said in the story. Now women are doing it and everyone's aghast.\" Naming names Loving said Robertson, a recent NICAR bootcamp graduate, also wanted to use the database for a story on baby names. She and Loving decided rather than focusing on the most popular baby names, they would focus on the most unusual. The hardest part here was narrowing the list. Among the names they cited: Praise T.L. (for \"Praise the Lord\"), Pepsi, California Seven, Lexxus, Queen Sarauniya, Moon Man and Gentle. Loving also noticed through his queries that the number of names given out had increased dramatically. In addition to an increase in ethnic names, Loving said there appeared to be less of a copycat effect than in earlier years, when several parents gave children common names. In the story that ran July 13, Robertson inter- viewed several parents who had searched for unique names. She also interviewed experts about popular ethnic names and historic trends in choos- ing names. Robertson and Loving also contrasted popular names given in 1995 with those given a decade earlier. In all, Loving said the computer analyses were straightforward and each took only an afternoon to complete. And, he maintains there are other stories hiding in the database he hopes to pursue. \"The database is kind of a standard every CAR program should have,\" Loving said. Loving had no problems obtaining the data- base from his state health department, which charges $50 for the annual database. The one glitch he encountered is that the database does not contain names of babies born out of wedlock. Kara Morrison can be reached at (573) 882- 0684, or by e-mail at kara@nicar.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-september-get-your-training","slug":"1997-september-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-september-get-your-training.html","year":1997,"month":"September","monthNum":9,"headline":"Get your training","kicker":"Seminars, bootcamps, conferences","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR offers specialized training programs in computer-assisted reporting for journalists, including on-the-road workshops, bootcamps, and conference sessions scheduled throughout 1997-1998. The training covers data transfer, spreadsheet building, database management, and online research skills for investigative reporting.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"NICAR provides specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and on-line databases. Cost varies. For information, call Lisa Barnes at (573) 882-8969. October 4, 1997 - Tucson, Ariz. Join NICAR for hands-on training at the Society of Environmental Journalists' annual conference. Limited to conference participants. October 10-12, 1997 - Boulder, Co. On-the-road training for University of Colorado October 16, 1997 - New York, N.Y. Join NICAR for hands-on training at the Media and Democracy Conference. Limited to conference participants. October 16-17, 1997 - Washington, D.C. On-the-road training for Reuters October 20-24, 1997 - Ft. Lauderdale, Fla. On-the-road training for the Sun-Sentinel October 30, 1997 - El Paso, Texas On-the-road training for Texas Association of Broadcasters November 6-7, 1997 - Yonkers, N.Y. On-the-road training for Consumer Reports\n\nWorkshops\n\nIRE and NICAR can also bring their road-tested workshops to sites around the country. We match regional public records experts and experienced reporters with our own trainers for unique seminars ranging from one-day demonstrations and panels on regional public records laws to three-day hands-on sessions on computer-assisted reporting or advanced techniques for data-intensive beats. Don't wait for someone else to organize a computer-assisted reporting workshop in your area. IRE and NICAR depend on members, news organizations, journalism schools and other journalism organizations to help us bring our seminars to you. Contact Sarah Cohen at (301) 942-2199 or e-mail her at sarah@nicar.org November 21-23, 1997 - Columbia, Mo. Intermediate Computer-Assisted Reporting Workshop\n\nDecember 11-13, 1997 - Washington, D.C. Computer-Assisted Reporting Workshop Sponsored by NICAR and the Medill School of Journalism\n\nBootcamps\n\nBootcamps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo. As with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and on-line databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size. For information, call Katy Fanning at (573) 882-0684. January 4-9, 1998 - Columbia, Mo. NICAR Regular Bootcamp May 10-15, 1998 - Chapel Hill, N.C. NICAR Advanced Bootcamp May 17-22, 1998 - Columbia, Mo. NICAR Regular Bootcamp August 9-14, 1998 - Columbia, Mo. NICAR Regular Bootcamp\n\nConferences\n\nNICAR will offer training at the following professional conferences, including the IRE and NICAR national conferences. These conferences feature practical tips and story ideas shared by prize-winning journalists in quick-hit sessions. Keep up with our conference schedule through our Web site, at www.ire.org. Costs vary. For more information or to register, call Lisa Barnes at (573) 882-8969. November 15-16, 1997 - Ciudad Juárez, Mexico Border Conference March 5-8, 1998 - Indianapolis, Ind. NICAR National Conference June 4-7, 1998 - New Orleans, La. IRE National Conference June 3-6, 1999 - Kansas City, Mo. IRE National Conference","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-september-from-the-nicar-library","slug":"1997-september-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-september-from-the-nicar-library.html","year":1997,"month":"September","monthNum":9,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering topics from transportation safety and healthcare to campaign finance and crime statistics, with data ranging from the 1930s through the 1990s.","keywords":["databases","government","federal"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"NICAR offers a number of federal government databases. Here is a list of our growing collection:\nNEW National Endowment for the Arts, 1987-1996, includes information on individual or organization receiving money.\nNEW Immigration and Naturalization Service legal residency, 1980-1995, includes information on the characteristics of aliens who were admitted as immigrants.\nNEW National Highway Traffic Safety Administration vehicle recalls and complaints, 1966-1997, includes information about vehicles that have been recalled by certain manufacturers.\nU.S. Food and Drug Administration's Medical Devices Reports, detailing complaints about drugs, pacemakers and other medical products. 1974-1996.\nU.S. Centers For Disease Control's AIDS database, providing case-by-case demographic information about those with the HIV virus. 1995.\nU.S. Census Bureau's Consolidated Federal Funds Reports, showing which communities get how much under various federal programs. 1983-1995.\nFederal Aviation Administration's accidents and incidents, including major plane crashes since 1974.\nNASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\nFederal Election Commission campaign contributions by individuals and political action committees, 1990-1997.\nThe Health Care Financing Administion's 1995 database of all Medicare-funded inpatient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings, 1991-1996.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration, 1974-1997.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\nFederal procurement data, 1992-1996, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records. 1993-1996.\nNational Bridge Inventory System data, includes inspection grades, 1994-1995.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the 1995 final report.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1980-1997. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nThe National Inventory of Dams, 1991-1995.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1996.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1975 to 1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org\n\nThe IRE Resource Center has copies of hundreds of stories that have used the databases listed here. If you are interested in seeing how reporters have used these databases, call the center at (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-september-bits-bytes-and-barks","slug":"1997-september-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-september-bits-bytes-and-barks.html","year":1997,"month":"September","monthNum":9,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR is expanding its conference offerings with new seminars and specialized beat-focused sessions, while also recruiting for several open positions at IRE and NICAR. The newsletter invites reader feedback on content and highlights recent listserv discussions on investigative journalism topics.","keywords":["conference","training","journalism"],"topics":["Journalism & Newsroom"],"technologies":["SQL","Access","GIS"],"full_text":"Bits, Bytes and Barks\n\nConferences and Seminars\n\nNICAR is forging ahead with new seminars and new approaches to conferences and workshops. Our annual conference, March 5-8 in Indianapolis, will not only include the usual panels and hands-on classes for beginners to the advanced, but also a set of special half-day sessions that will take in depth looks at how to use electronic information for particular beats.\n\nWe also are expanding our seminars for the intermediate range. Earlier this year we did a seminar on education and statistics. In the coming months, we plan to hold seminars on database design and data cleaning, use of SQL servers for the newsroom, mapping software, and business reporting.\n\nWatch our website at www.nicar.org for dates and times.\n\nFor more information on seminars and bootcamps, call (573) 882-0684.\n\nHelp Wanted\n\nIRE and NICAR still have several positions open: Director/Trainer for Campaign Finance Information Center and Database Library Administrator for Campaign Finance Information Center, IRE Deputy Director and Database Library Administrator for NICAR.\n\nInquiries and resumes should be sent to Brant Houston, Executive Director, IRE, 138 Neff Annex, Columbia, Mo. 65211 or brant@ire.org\n\nReader Response\n\nIt's time we heard from you. We are looking to improve Uplink and want to know your ideas.\n\nAre there certain columns that you would like to add? Are there certain topics you would like covered? Have you done a story that you think should be featured in an upcoming issue? This is your chance to play armchair editor.\n\nSend comments, critiques and suggestions to Uplink's editor, Jody Sowell, at jody@nicar.org We look forward to hearing from you.\n\nNICAR Net\n\nRecent topics on the IRE and NICAR listserv have included campaign finance data, troubles with traffic accident data, ethics, medical investigation, medical investigations and Access tips.\n\nTo subscribe to IRE-L or NICAR-L, send e-mail to listproc@lists.missouri.edu\n\nIn the body of the message, type:\n\nsubscribe NICAR-L<your name>\n\nsubscribe IRE-L<your name>\n\nAlso, check out the IRE-L and NICAR-L mailing list archives on our website at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-car-in-latin-america","slug":"1997-october-car-in-latin-america","url":"https://dwillis.github.io/uplink/article/1997-october-car-in-latin-america.html","year":1997,"month":"October","monthNum":10,"headline":"CAR in Latin America","kicker":"","author_name":"Kara Morrison","author_title":"","affiliation":"NICAR","summary":"Carlos Subero, a Venezuelan journalist, introduced computer-assisted reporting (CAR) in his country with unique political analyses using SPSS software, inspiring other Latin American journalists.","keywords":["Venezuela","journalism","CAR"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["SPSS","Excel","Internet","World Wide Web"],"full_text":"When Carlos Subero decided to attend last year's advanced computer-assisted reporting seminar in Chapel Hill, he was already an experienced political reporter in Caracas, Venezuela. But the knowledge he came home with led to projects no one else in his country had tried. The projects included an analysis of all 257 members of Venezuela's Congress using SPSS software, and a global study of the increasing number of women in high political offices. Subero did both analyses for his newspaper, El Universal, using databases he built from his own research. \"My first important project in CAR was very profitable,\" said Subero, who says he is still the only professional journalist practicing CAR in Venezuela. But he says the idea is spreading. Lise Olsen, managing director of Periodistas de Investigación, IRE's counterpart in Mexico City, said Subero is one of several Latin American journalists now embracing the computer as a powerful reporting tool. PI trains 40-60 journalists each month in computer-assisted reporting workshops. Olsen says two main factors are fueling the trend, the first of which is an increase in online information. \"It has become fashionable for government offices to have Web sites,\" she explained. This has made retrieving government records easier than having to request databases from government public relations offices that often do not know such information exists. The second main factor is that other journalists have been encouraged by watching some of their colleagues complete successful computer-assisted reporting projects. \"I think more and more people are trying [CAR], showing people who are doubtful that it can be done,\" Olsen said. Jose Roberto de Toledo is a Brazilian journalist who has done just that, completing numerous CAR projects for his newspaper, Folha, in Sao Paulo. After finding numbers compiled by a Brazilian congressman, Toledo used Excel to analyze public works projects in the country by state, department and type of expense. Hether calculated the amounts spent on each and compared the amounts to expenditures from previous years. Toledo also used data from Sao Paulos Pro-Aim institute to analyze the causes of death in the city. He was able to link many homicides to alcohol abuse. Finally, Toledo used INS data to track Brazilian emigrants. He found the five American states that were their most popular destinations - Massachusetts, New Jersey, Florida, New York and California - and profiled several Brazilians there. Toledo said each project took about a week. \"The most difficult part,\" Toledo said, \"is always to find the source of good and reliable data.' Starting simple Olsen said most computer-assisted projects in Latin America are still done using spreadsheets and Internet sources, rather than more complex databases and statistical analyses. Francisco Vidal, a reporter with Milenio magazine in Mexico City and formerly with the newspaper Reforma's investigative team, has done several such projects. Vidal analyzed the economics of Mexican soccer teams using data he compiled. He was able to estimate how profitable each team was. Another project Vidal did was a spreadsheet analysis of cancer rates among Mexican women. He found the rates were higher in some Northern Mexican states, and that women in other states were dying less frequently largely because they had detected cancer earlier. Olsen said another reporter, Claudia Fernandez of El Universal in Mexico City, recently downloaded data from the National Security Archives Web site. With this, Fernandez was able to decode some information about Cold War KGB activities with a code sheet that she also downloaded from the Web site. Good with the bad She was then able to identify a journalist, a former politician and other former KGB agents living in Mexico, one of whom was still alive. She then interviewed him and the family mem- bers of others for a riveting story. While Olsen admits Latin American journalists have more problems gaining access to as much computerized information as American reporters do, she says they do have one advantage. Olsen said journalists there do not have to deal as often with translating old 9-track government tapes, since most government offices in Latin America use PCs. And so far, she says, laws in Mexico at least have not made any distinctions between electronic and print records. Olsen also said sharing information helps reporters get started with CAR projects, since some ideas, like exploring campaign finance, analyzing budgets and comparing crime rates, can be done in almost any country. \"Information sharing is just as important internationally as nationally,\" she said. Subero for one doesn't plan to stop learning about and using computer-assisted reporting. He is currently working with the Venezuela electoral institution Consejo Supremo Electoral to ensure campaign finance data is compiled there. I have no experience with Access, Paradox or Mapinfo, but it is my plan to do it soon.\" Subero said Phil Meyer has agreed to travel to Caracas next November to teach CAR courses. \"This will give you an idea about the importance that El Universal sees in CAR.\" Toledo also said his experience has taught him much and has given him an edge in a market still new to computer-assisted reporting. \"Besides the conclusions of the articles them- selves, I learned a lot of things doing this job from operating the software to finding good sources of data,\" Toledo said. \"More than that, I learned that the computer is the best friend of the reporter. Unfortunately, I believe that CAR is still almost unknown in Brazil, I believe this is going to change in the next [few] years. Until then, I and a few others will continue to use CAR techniques to be ahead of the competitors.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-october-busted-tracking-bankruptcy","slug":"1997-october-busted-tracking-bankruptcy","url":"https://dwillis.github.io/uplink/article/1997-october-busted-tracking-bankruptcy.html","year":1997,"month":"October","monthNum":10,"headline":"Busted Tracking bankruptcy","kicker":"","author_name":"Stefan Lisinski","author_title":"Reporter","affiliation":"Dagens Nyheter","summary":"An investigation into bankruptcy abuse in Sweden reveals it as a tool for economic criminality, highlighting challenges in proving cases and detailing involved business networks.","keywords":["bankruptcy","Sweden","economic criminality"],"topics":["Business & Economy","Investigative Methods","Crime & Justice"],"technologies":["Access"],"full_text":"How do you tell a story about bankruptcy as a tool for economic criminality? A lot of people, at least in Sweden, know that bankruptcy is a business itself with a lot of criminality, but the trick is to prove it. A lot of people involved in bankrupt companies are suspected, some are prosecuted but only a few are convicted for economical criminality. I tried to do the story by starting at the other end. I made a top list and investigated the people who are involved in the biggest number of bankrupt companies. I could show that a lot of them had never actually ran the companies. I could describe the methods they were using. Criminals or not, that was for the courts to decide, but the reader got a picture of companies where the basic idea was not to sell anything but to fool the authorities and other companies. The story was published Oct. 1996 in Dagens Nyheter, Sweden's biggest morning newspaper. Story behind the story For Swedish reporters the story about how I did the story was just as interesting as the story itself. This was one of the first times a reporter could use data from a Swedish authority and run it in his own computer. Reporters in my country are used to getting information from authorities, but only on paper. Every citizen has a right to read most of the documents that you can find at the authorities, both local and national. But when you get to computers we have rules that look like a joke. When I asked the Swedish company register for information on discs about all Swedish joint- stock companies that filed bankruptcy during one year, they told me I first needed permission. After four months, the Swedish Computer Inspection Authority said Okay. They had trouble before when a reporter asked for permission to write a book on a computer and they found that this was against the law! At that time the Swedish government decided that this was a question of freedom of the press and gave permission. This decision helped me and I was one of the first reporters to get this kind of information on discs. This also explains why one of the first CAR stories in Sweden was about economic criminality. The Swedish company register knows how to use computers and they are used to selling information to other authorities and market companies. Other authorities don't understand what you are talking about, and they don't know that they are allowed to provide electronic information. I guess in a few years this will change. More Swedish reporters will ask for more electronic information and the authorities have to be taught to give us the information. The war has just started. Crunching the data For one year there were more than 18,000 bankrupt companies with about 50,000 members of the boards. The information was 43 megabytes. The newspaper had to pay about $2,000 for the information. The company registers argued that they had to create a program to sort out the bankrupt companies. I used Access, simply because my newspaper already had it, and I didn't think much about other alternatives. The computer work was quite easy, even if I had to learn the program from start. The problem was that I got a lot of different ideas from the data, and I had to realize that I couldn't make a story of all of the ideas. I got a list of people involved in 10 or more bankruptcies. Total there were 13 people. Then I started the old-fashioned research work, looking into documents from courts, trustees reports interviews, etc. This took a long time and had little to do with CAR work. A big part of the story was to make a description of all these 13 people. Most of them were anonymous in the story. Of the 13 people there were seven who were what we call goal keepers. These were drug addicts, thieves and people with no money at all. They had been put in the companies when it was time for bankruptcy. Eleven of 13 were involved in police investigations. Only two were entrepreneurs who had failed in business. I published an interview with one of them. Most of the people were in their 40s, 12 were men and 12 also were of Swedish origin. This was important since there is a lot of discussions about immigrants and criminality. I also found some new interesting examples of economic criminality. We found a lot of accountants who were involved in hundreds of bankruptcies. Some of them were about to lose their permission to work as accountants. The conclusion was that there is a lot of economic criminality in bankruptcies and that this is a profitable criminality with a good chance for the criminal to go free. One disappointment was that I couldn't find the big guys behind the goal keepers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-october-analyzing-property-tax","slug":"1997-october-analyzing-property-tax","url":"https://dwillis.github.io/uplink/article/1997-october-analyzing-property-tax.html","year":1997,"month":"October","monthNum":10,"headline":"Analyzing property tax","kicker":"Tax Investigation","author_name":"Neill A. Borowski","author_title":"Reporter","affiliation":"Philadelphia Inquirer","summary":"A New Jersey investigative project analyzed property tax changes from 1993 to 1996 using computer-assisted reporting on over 2 million records, finding a median 12% increase in residential property taxes while examining whether the governor's income tax cuts offset these increases. The article demonstrates how reporters can conduct sophisticated economic analysis using CAR tools and publicly available data sources like the Bureau of Economic Analysis.","keywords":["taxes","analysis","data"],"topics":["Business & Economy","Investigative Methods","Government & Politics"],"technologies":["Access","SPSS"],"full_text":"New Jersey has the highest per-capita property-tax burden in the nation. With this in mind and a governor's election in November we embarked on a mission to determine what happened to property taxes in New Jersey.\n\nThe target period was 1993, the year before Gov. Christie Whitman entered office, and 1996, which was the latest year available.\n\nThe result was a project in the Aug. 24 Inquirer that I reported and wrote with James M. O'Neill from our Trenton bureau. We looked at every residential property in New Jersey and compared each property's tax bill between 1993 and 1996. The analysis of more than 2 million records in each year taxed MS Access, but SPSS picked up the slack.\n\nHowever, the important computer-assisted reporting revelation wasn't how to do a tax story. The important step was coming up with a unique economic analysis using CAR tools.\n\nEvery reporter with CAR skills and some knowledge of statistics can put together a regional economic analysis that may rival similar analyses done by local planning boards, academics or others.\n\nThere may be reluctance to venture into economics because the subject seems so technical and full of ambiguities. Don't worry. You can make a contribution. In our tax piece, we found the median increase in a residential property tax bill was about 12 percent between 1993 and 1996. Yet, Gov. Whitman said state income tax cuts made up for the increases. That was hard to pin down, but we used 1989 (1990 census) median household income statistics to estimate incomes in municipalities in 1993 and 1996. We calculated income tax bills each year and found the change. In the higher-income communities, the income-tax cut often made up for the increase in property taxes. How were those income estimates made? We turned to the Bureau of Economic Analysis (U.S. Commerce Department), which makes wage, salary and personal income estimates for counties. We applied the county-level income increases from the BEA to the 1989 median household incomes by municipality. We tested the estimates using the CPI and found the BEA method didn't yield crazy results. (One key assumption, though, was that the typical household in a community saw increased income. We noted in the story that if the household's income were flat between 1993 and 1996, the tax cuts would have been even greater.)\n\nData sources\n\nHere are some of the sources of regional economic data we used for this story and others involving regional economic analysis:\n\nIncome data (BEA). A wonderful source is at http://www.stat-usa.gov. Estimates of personal income, earnings, wages and salaries are available for states, metros and counties. Unfortunately, there is a subscription fee (no press privileges). The Bureau of Economic Analysis' site is http://www.bea.doc.gov. This is free and you can at least get the news releases and maybe some data. The BEA also makes income projections by metro area to the year 2045.\n\nIncome data (BLS and Census). Hourly earnings and other data from the Current Population Survey and other sources available at http://www.bls.gov, the BLS (U.S. Department of Labor) site. The Census Bureau's income data tends to be more dated than the BEA's. The site is http://www.census.gov/hhes/www/income.html. The BLS' Consumer Price Index also is available at the BLS site.\n\nEmployment data (BLS). Many reporters say their state labor departments have a lot more data than BLS. However, in most cases the data released by state departments originate with the BLS. The two leading employment series: civilian labor force (residence-based) and payroll employment (employer-based). The civilian labor force number comes from the Current Population Survey and is used to calculate the unemployment rate. The civilian labor force number is national, with estimates for the states. For example, total civilian employment in Pennsylvania is the number of Pennsylvania residents who are employed. By contrast, payroll employment data (also called \"establishment employment\" or \"series 790 data\") are based on employer location and not job holder residence. For example, payroll employment in Pennsylvania is the number of jobs within the state borders - even if residents of New Jersey fill some of them. The 790 data typically are available only to the metro area level although the central cities of Philadelphia, Baltimore, Washington and New York are released. The payroll employment data also breaks industries down to sub-industry groups. Local residence-level employment data are available through the BLS Local Area Unemployment Statistics (LAUS) program. Some economists call these numbers the laus-y statistics because they are gross estimates and probably subject to a great deal of error (which isn't measured). However, as questionable as these data are they are available each month on the county or municipal levels. Finally, the Series 202 data now available from state labor departments and available in the future from the BLS is based on a quarterly survey of every employer and includes wage and employment information. It is available down to at least the county level (and is viewed as relatively solid; the BLS uses the 202 numbers to benchmark the 790 numbers). Some data are available at http://www.bls.gov.\n\nBusinesses (County Business Patterns). The Census Bureau each year publishes County Business Patterns, tracking employment, payroll and establishment totals down to four-digit SIC codes. For example, in 1994 there were 92 video-rental establishments employing 894 people and paying them a total of $10,482,000 a year in Philadelphia. So? You may be studying the size of one industry versus another. The CBP series isn't based on surveys, so there is no sampling error. The Census Bureau uses administrative records for CBP. However, there is a lag. The latest available, 1994, was released last fall.\n\nA problem is that the bureau withholds some data information about one employer might be disclosed. This becomes common when you move to three- or four-digit SIC codes or you concentrate on smaller counties in your analysis. Find out more at http://www.census.gov/epcd/cbp/view/cbpview.html. Complete reports are available for download by state using Acrobat Reader (pdf files).\n\nEconomic Censuses. While we know a bit about local businesses from County Business Patterns, the Census Bureau's economic censuses can fill in some of the \"holes.\" The censuses of Retail Trade, Wholesale Trade, Services, Services/Tax-Exempt, FIRE (Finance, Insurance and Real Estate), Manufacturing, Mining and Construction show the number of establishments, annual receipts and number of employees. These censuses come out for years ending in \"2\" and \"7.\" For information: http://www.census.gov/epcd/www/92results.html\n\nGross output. We all know that Gross Domestic Output (GDP) measures the output of goods and services for the national economy. However, there is GDP available at a state level, called appropriately Gross State Product. The database is simple: the total output of goods and services down to two-digit SIC code for each state. \"Output\" in this case is the value added to the good or service by an industry within a state. Because it is a value-added measure, there is no double counting (for example, the value of raw materials brought into Pennsylvania to make a product would not be counted in Pennsylvania's output).\n\nState/Local government finances/taxes. The Census Bureau's government page at http://www.census.gov/govs/www/index.html is overflowing with useful statistics. Included are 1992 through 1996 state tax collection data, state and local government finances from 1991 through 1994, financial information for larger counties and 1992 through 1995 state government payrolls.\n\nFederal tax data. The IRS' Statistics of Income (SOI) program has a useful site packed with tax statistics at http://www.irs.gov/tax_stats/index.html. Besides the statistics, you can download the IRS tax-exempt organizations file by state. Links to state tax departments are available at http://www.irs.gov/prod/forms_pubs/ftaframes.html. Other tax-related sites are available at http://www.irs.gov/prod/forms_pubs/ftaframes.html.\n\nOther local economic data. Each state has a data center affiliated with the Census Bureau. The links to these centers is at http://www.census.gov/sdc/www/. If you rummage through your data center's site, you can come up with local economic statistics, building permits and other useful data.\n\nNeill A. Borowski can be reached, by e-mail at neill.borowski@phillynews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-covering-housing","slug":"1997-october-covering-housing","url":"https://dwillis.github.io/uplink/article/1997-october-covering-housing.html","year":1997,"month":"October","monthNum":10,"headline":"Covering Housing","kicker":"Handout of the Month","author_name":"George Rodrigue","author_title":"","affiliation":"IRE","summary":"George Rodrigue provides investigative journalism tips for covering public housing, using his own investigation of racial segregation in East Texas public housing as an example. He recommends observing patterns firsthand, examining government files through FOIA requests, and using data analysis to expose systemic discrimination.","keywords":["housing","segregation","investigation"],"topics":["Investigative Methods","Government & Politics","Demographics & Census"],"technologies":[],"full_text":"In this handout, George Rodrigue gives a variety of tips for covering public housing. The IRE Resource Center has several handouts and stories dealing with public housing. To order any of these files, call (573) 882-3364.\n\n1. Look under your own nose.\n\nPublic Housing segregation in East Texas Everybody knew that public housing in rural East Texas was segregated.\n\nAnyone could drive into pretty much any small town and see two projects: One white and well-tended, and the other black and decrepit.\n\nWhat Craig Flournoy and I did, twelve years ago now, is stop and ask why.\n\nPublic housing authority directors said things like, \"If you let black kids and white kids grow up together, they might like each other. And then you know what you'd have? Race mixing!\"\n\nThe same pattern, in different guises, was equally obvious in other rural areas. Not to mention the big cities. Urban renewal had razed hundreds of inner-city neighborhoods, usually occupied by minorities but almost all of them poor.\n\nThe occupants were herded into high-rises and, quite often, denied chances to move to neighborhoods with lower poverty rates and more jobs.\n\nWhy?\n\nSome public officials confessed that racism played a role, as did the convenience of politically connected developers.\n\nIn other words, \"ghettos\" didn't just happen. They were subsidized by the federal government.\n\n2. Examine the files.\n\nHUD had known about problems in East Texas, and elsewhere, for decades.\n\nNext question: Discrimination had been illegal for 20 years. Why did HUD fail to intervene? We FOIA'd pretty much every HUD inspection file of East Texas public housing authorities.\n\nThe agency had known about the authorities' bigoted directors for decades. It had them on the record opposing \"race mixing.\" Yet it had done nothing.\n\nPrivately owned but HUD-subsidized projects showed the same pattern.\n\nOdd, because federal law required HUD to ensure that projects in non-minority residents. Time after time, we found HUD had approved marketing rules that specified a goal of \"zero\" recent minorities.\n\nWhen we called the former HUD regional administrator for comment he said, \"It looks like we didn't give a damn about those regulations, and we didn't.\"\n\nLast piece of the record chase: When Housing and Urban Development delayed providing us with certain reports, The Dallas Morning News asked the agency to send us copies of the docket cards listing the name and phone number of every attorney who had filed suit against the agency for civil-rights violations.\n\nThat led to warehouses full of HUD documents those lawyers had obtained, including depositions with city officials around the country who'd taken federal funds designed to \"free\" poor people, and used the money to ghettoize them.\n\n3. Reverse-engineer the bureaucracy.\n\nComputer-aided inspection of Dallas Housing Authority waiting list drew quick confession.\n\nLike most big cities, Dallas' public housing has long been segregated.\n\nThe Dallas Housing Authority's leaders, however, denied deliberately \"steering\" of applicants to particular projects.\n\nBy law, the authority must maintain a waiting list that tracks each applicant, noting that applicant's race and the projects that each applicant is offered.\n\nWe cross-tabulated the race of the applicant with the racial composition of the projects the tenants were offered.\n\nWhites almost always got offered \"white\" projects; blacks almost always were offered \"black\" projects.\n\nI called back the staff person who was responsible for making these offers and told her there was less than a one-in-ten-thousand chance of this being mere coincidence.\n\nShe confessed then and there that the agency was \"trying to make people offers for places it knew they'd like.\" This was illegal, she knew, but she said her bosses required her to follow this policy. Other things to look for in waiting lists: People who jump ahead of others on the list, or never get called for apartments. That can be a sign of favoritism, or prejudice. Every agency has to keep certain records, whether by law, regulation, or common-sense necessity. Find those records and use them to reverse-engineer the agency's real operating patterns. Also with HUD, we created a national database of public housing occupancy by race. We applied HUD's own criteria for segregation within a public housing authority, and found that the overwhelming majority of housing authorities were segregated.\n\n4. Huge, immensely promising projects bear especially close scrutiny.\n\nEmpowerment and Enterprise Zones\n\nYears ago Dallas' Crow Companies got $4 million in Housing and Urban Development grants to create 2,000 jobs for Hispanics in the impoverished West Dallas area. Time passed, the project was built, the federal funds were disbursed.\n\nNet result: 16 jobs for West Dallas Hispanics. Each with a federal subsidy of $250,000.\n\nThe history of even grander schemes, such as \"empowerment zones\" and \"enterprise communities,\" contains many tales of more grandiose flops. HUD and the Congressional Republicans share an ideological bias toward these kinds of programs, however, so they never really die. Keep an eye on them, and compare participants' contractual promises to what they actually deliver.\n\nBear in mind, also, that previous studies have found these zones mostly shift development around a bit within a city, without fundamentally altering development dynamics in the region.\n\n5. But then, so do routine business transactions.\n\nLandlords with HUD Section 8 subsidized projects often take the money and run.\n\nHere's a scandal so blatant HUD recently called a press conference to denounce it.\n\nToo many landlords whose projects carry HUD Section 8 subsidies have been pocketing the federal checks but then declining to pay their mortgage holders, or to spend money on maintenance.\n\nBecause of the weak way Congress wrote the laws, HUD must continue sending those checks even after the landlord declares bankruptcy leaving HUD to pay mortgage insurance. As a result of such \"skimming,\" taxpayers are stuck with bad debts on decrepit projects, and tenants live in hell-holes.\n\nHUD local or regional offices may be willing to provide information under FOIA, about projects where maintenance is particularly bad and where landlords may be in default on their loans.\n\nAnother routine problem: Housing and Urban Development guarantees loans up to $25,000 to lower-income people needing home renovations.\n\nBut it also lets contractors handle the loan paperwork. This is an invitation to fraud, and HUD has found plenty.\n\nBuilders get folks in debt they can't manage and then flee, often leaving shoddy or half-finished work behind.\n\n6. Connections matter.\n\nWelfare reform will affect the lives of everyone in public housing, driving up rents and, perhaps, leaving more kids unattended.\n\nPublic housing doesn't work in isolation. Changes in food stamp and welfare laws will cause huge changes.\n\nSome families will become better off (and face higher rents, and may leave.)\n\nOthers will become even poorer. Some face benefit cuts.\n\nOthers must cope with sporadic employment or face new costs for transportation, health and child — especially after they lose the single year of \"transitional\" benefits guaranteed them by most states.\n\nAnd as millions of mothers are pushed into the workforce, who'll be watching the children?\n\nGeorge Rodrigue can be reached at (202) 331-8426 or send e-mail to grodrigue@belo-dc.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-international-wrap-up","slug":"1997-october-international-wrap-up","url":"https://dwillis.github.io/uplink/article/1997-october-international-wrap-up.html","year":1997,"month":"October","monthNum":10,"headline":"International Wrap-up","kicker":"Around the Globe","author_name":"Brant Houston","author_title":"Executive Director","affiliation":"IRE","summary":"IRE and NICAR are expanding computer-assisted reporting (CAR) training globally, conducting workshops across multiple countries including Mexico, Sweden, Finland, Denmark, Germany, Peru, and England, while also receiving visits from international journalists interested in learning these investigative techniques.","keywords":["training","journalism","international"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","Email"],"full_text":"Spurred by the wealth of information on the Internet and the sharing of techniques by their U.S. colleagues, journalists in other countries are quickly catching up on CAR.\n\nThis month we will conduct a border workshop for U.S. and Mexican journalists in Juarez. The workshop, set up by IRE's Mexico division, will be a bilingual excursion into CAR and traditional investigative techniques.\n\nIn the next six months, IRE and NICAR trainers will travel to Sweden, Finland, Denmark, Germany, Peru and England.\n\nIn Sweden and Finland, IRE and NICAR this month will train journalists not only in the basics, but also in the wise use of statistical methods. In Denmark, IRE and NICAR will hold another weeklong boot camp in March before joining Swedish journalists at their annual investigative conferences. In Germany this January, we will provide training for the staff of Stars and Stripes while also reaching out to German journalists.\n\nIn Peru, NICAR and its fellow division, IRE Mexico (Periodistas de Investigacion) will do an \"exploratory\" visit next month to Lima to make plans for seminars for Peruvian journalists that will take place next year both in Missouri and Lima.\n\nAnd in England, IRE and NICAR will help conduct an all-day training session at a new media conference in London in July. Last year trainers from the U.S. spoke and taught at the conference that combined panels and training. For the 1998 conference, organizers plan to have one day of panels and a separate day of training. We are customizing lessons and data especially for foreign journalists in order to make it as relevant as possible.\n\nMeanwhile, we continue to receive numerous inquiries and visits from other foreign journalists. A Fulbright scholar from Spain is currently doing a study on CAR here at Missouri. Earlier this year another Fulbright scholar from Venezuela attended both our basic and advanced boot camps while working with CAR pioneer James Brown at the University of Indiana in Indianapolis.\n\nAt another boot camp, we had a working journalist from Argentina who wants to set up CAR workshops there. A New Zealand journalists is currently using E-mail to arrange a tour later this year of campuses and news organizations doing CAR.\n\nAt the same time, IRE and NICAR hope to help the Center for Public Integrity in Washington D.C. in its new project to put together an international consortium of international journalists.\n\nSo, as you can see, International CAR is really moving ahead.\n\nBrant Houston can be reached at (573) 882-2042 or send e-mail to brant@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-finland-update","slug":"1997-october-finland-update","url":"https://dwillis.github.io/uplink/article/1997-october-finland-update.html","year":1997,"month":"October","monthNum":10,"headline":"Finland update","kicker":"","author_name":"Jennifer LaFleur","author_title":"","affiliation":"NICAR","summary":"European journalists, particularly in Finland, are pioneering computer-assisted reporting by requesting raw data from government agencies despite weak open records laws, with training programs helping reporters develop these skills. Finland's high internet adoption and readily available government databases provide advantages for data journalism, though reporters still face challenges similar to those in the United States.","keywords":["Finland","reporting","databases"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","Adobe Acrobat","spreadsheet"],"full_text":"In spite of weak (or a complete lack) of open record laws, some European journalists are pioneering computer-assisted reporting in their own countries.\n\nMuch like the pioneers in the United States (particularly those at small to medium size news organizations) these journalists are breaking ground by being the first to ever request raw data from government agencies - and in some cases, they're getting them.\n\nYLE, public broadcasting of Finland, has also begun training television, radio and teletext reporters in using computers at its Radio and Television Institute in Helsinki. In January and October of this year, NICAR did hands-on sessions at the Institute. Heikki Hiilamo, a YLE investigative reporter and NICAR boot camp attendee has been leading the charge there. Hiilamo, currently on leave from the company, helped develop the October training session.\n\nThe difference in the January and October sessions was notable. Fewer participants expressed concern with not being able to get information from the government and more of the participants had used the Internet regularly as a reporting tool. Two of the reporters even came to the October class with economic data they had acquired and wanted to analyze.\n\nIn some cases, databases - such as tax information on individuals - is more readily available in Finland than in the United States. And the same techniques we use for building databases from paper records (especially in smaller communities) can be used in countries where an electronic database might not be available.\n\nFinland has one CAR advantage: It is a country that embraces technology with a higher rate of Internet connections than most U.S. cities. (With the exception of the text being in Finnish, many Helsinki billboards bear a resemblance to Silicon Valley.)\n\nMost government agencies in Finland put information on the Internet. As is happening in the U.S., the government is providing information in Adobe Acrobat files that are not easily transferred to a database or spreadsheet and as in the U.S., reporters are faced with retyping the information into their computers. Among the government agencies providing databases in a useful format are:\n\nThe bank of Finland (www.bof.fi) provides downloadable economic databases. (And also provides information in English for all of us non-Finnish speakers.\nStatistics Finland (www.statfin.fi) is their Census Bureau - plus more.\nEuropean Forest Institute (www.efi.joensuu.fi/) provides logging and forestry information.\n\nIn addition many European universities provide useful research data on their Web sites.\n\nReporters in Finland are also realizing what other journalists outside the United States have found that data from U.S. agencies also can be valuable. Stat-USA (www.stat-usa.com) for example, provides information about imports and exports by country.\n\nWe'll have another report since NICAR is visiting Finland in November.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-denmark-update","slug":"1997-october-denmark-update","url":"https://dwillis.github.io/uplink/article/1997-october-denmark-update.html","year":1997,"month":"October","monthNum":10,"headline":"Denmark Update","kicker":"","author_name":"Gwen Carleton","author_title":"","affiliation":"The Capital Times","summary":"Danish journalists are using Computer-Assisted Reporting (CAR) to investigate government effectiveness and access closed databases, with Jyllands Posten leading investigations into supreme court justices' workloads and legislators' side jobs. A consortium of Danish journalists continues pushing for standardized public access to government information, showing early signs of success.","keywords":["journalism","Denmark","databases"],"topics":["Investigative Methods","Government & Politics","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"In Denmark, a land of powerful government that keeps its databases closed, journalists are using CAR to shed light on their leaders' effectiveness and learn more about its people.\n\nIn recent months, the country's largest national daily newspaper, Jyllands Posten, has focused on the effectiveness of the country's supreme court.\n\nThe 18 appointed justices have many side jobs, the paper found, so journalists used CAR to analyze each justice's workload, number of cases and decisions, as well as the amount of time spent on cases.\n\nThe paper also has written about moonlighting legislators in the national parliament, and has published a list of legislators' side jobs on its Web site.\n\n\"CAR has been deeply involved in a number of lead stories,\" said Nils Mulvad, a Jyllands Posten business reporter and graduate of NICAR's boot camp. \"There have been a number of consequences politically, and on the court system's efforts to improve effectiveness - a number of votes concerning these issues are now being discussed.\"\n\nA consortium of Danish journalists and journalism educators also continue their efforts to unlock the nation's detailed and sophisticated governmental databases.\n\nThe country's tradition of zero access and the public's own reluctance to open such information has hindered the effort thus far. But journalists say encouraging signs are beginning to appear.\n\nIn response to pressure from the consortium, for example, the government has agreed to standardize rules for access to information in its various branches, something members say constitutes a small but important step forward.\n\n\"It's the character of the information that should decide access, not where the information is hidden,\" Mulvad, who also is a consortium member, said. \"The government has now accepted that.\"\n\nIn the seven months since NICAR held its first boot camp in Denmark, Jyllands Posten also has used CAR to analyze city demographics in advance of various city elections, and has expanded its Internet version of the paper. As other journalists nationwide jump on the bandwagon, Mulvad said, Jyllands Posten's CAR reporters are receiving increasingly frequent requests for lectures and training.\n\n\"Where there was uncertainty over the new methods when NICAR was here, now that attitude is gone,\" he said. \"The CAR machine is working.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-making-data-entry-easy","slug":"1997-october-making-data-entry-easy","url":"https://dwillis.github.io/uplink/article/1997-october-making-data-entry-easy.html","year":1997,"month":"October","monthNum":10,"headline":"Making data entry easy","kicker":"Tech Tip","author_name":"Richard Mullins","author_title":"","affiliation":"NICAR","summary":"The article provides practical guidance on managing paper-to-database data entry using Microsoft Access, emphasizing the importance of tracking data entry through automated sequence and timestamp fields to ensure data quality and auditability.","keywords":["database","tracking","entry"],"topics":["Data & Databases","Investigative Methods"],"technologies":["Access"],"full_text":"Sometimes the data you want is only on paper, but you really want to analyze it in a database. So, even though data entry is the proverbial pain, you decide to do it. If you're lucky, you can get some help on the data entry. But even if you do, make sure you do some of the entry yourself - a couple dozen records is probably enough to give you an idea of the judgments that have to be made when reading forms and typing some value in a particular field.\n\nFurthermore, whether you do it all or have some help, you should create some scheme to make it possible to quickly and unmistakably match an individual record on the paper to the corresponding row that was entered in the table.\n\nAccess assistance\n\nMicrosoft Access has some features to make this easy. When you design the table and specify the field names and types, add two more fields to track data entry: one to show the sequence of the data entry, the other to show the time the row was entered. After you create these fields, they require no extra effort on your part, and they can come in very handy. I'll talk about how to do it first, then the ways you might use the fields. For the sequence indicator, make a field called seq or sequence. For the data type, choose AutoNumber (in Access version 2, it's called Counter.) The first row will have the number 1 in this field, 2 for the second row and so on. The field increments the next value automatically. Once the number is there for a row, it doesn't change. Deletions from the table do not renumber the field and you can't modify it. If your data doesn't have a unique identifier for each case or row, this field makes a good choice. And if your data does have a unique identifier, the sequence field requires no effort on your part and it explicitly shows the order of data entry.\n\nFor the data entry time, make a field called EntryTime or RowTimeStamp. For data type, choose Date/Time. The bottom part of the Table Design view is where you specify properties for a field. In the property called default value, type the word Now. You might notice that when you click somewhere else on the screen, Access adds parentheses to the end of the word that you typed. This is a sort of automatic syntax-checking. Access guesses (correctly) that you want to call the built-in function Now(), which returns the time and date according to your computer. After you save the table and start entering data, you have two fields that Access is filling in for you: the sequence and the date/time.\n\nWhen planning and assigning the data entry, break it up into small manageable batches, especially if there are more than a few hundred records to work with. When a batch is done (say 50 records), log who did it and when, and file the paper originals by batch. With this filing and entry system, coupled with the entry-tracking fields Access is handling for you, you'll have an unambiguous way to match and audit your work. Here is an example use for the sequence field. Say you're typing in questionnaire responses. Instruct all the people doing the data entry to write the sequence number generated by Access on the piece of paper containing the data. If there is a question about the integrity or accuracy of a particular row in the table, you can find the paper original. The timestamp field will help if you need to look at all the records that were entered, say, on Tuesday afternoon, or all the records entered from Friday onward.\n\nExtra entry tips\n\nRichard Mullins can be reached at (573) 882-2127, or send e-mail to richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-around-the-world-on-the-web","slug":"1997-october-around-the-world-on-the-web","url":"https://dwillis.github.io/uplink/article/1997-october-around-the-world-on-the-web.html","year":1997,"month":"October","monthNum":10,"headline":"Around the world on the Web","kicker":"On the Internet","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute","summary":"Computer-assisted reporting can connect local news to global perspectives through Internet resources such as international news databases, statistics compilations, and government directories that journalists can access on deadline.","keywords":["reporting","international","resources"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","FTP"],"full_text":"One goal of computer-assisted reporting is to help make the local global and the global local,\" according to Brant Houston, our fearless IRE/NICAR leader. How can Internet resources help do this on a timely, comprehensive, credible basis, ie: on deadline?!\n\nHere is a listing of some resources with great compilations of international resources that can help bring a broader perspective to your reporting. Because, as we all know, we are no longer just citizens of our respective countries, we are members of a global village.\n\nOften the best source is to find a good \"scout\" page, a page of links that is well organized and contains the key resources. Here are some good listings of International Links:\n\nEuropean Journalism Page http://www.demon.co.uk/eurojournalism/ They have a set of links relevant to breaking stories as well as links organized by government and politics, the media, and organizations.\n\nThe World-Wide Web Virtual Library: International Affairs Resources http://www.pitt.edu/~ian/ianres.html Well organized by types of resources, by source of the information, and by topics.\n\nInternational statistics\n\nCIA World Factbook http://www.odci.gov/cia/publications/nsolo/wfb-all.htm A great compilation of statistics and country profiles.\n\nInternational Data Base http://www.census.gov/ipc/www/idbnew.html \"Statistical tables of demographic, and socio-economic data for all countries of the world.\" Check out the \"population pyramid\" feature for countries.\n\nInternational Federation of Data Organizations: http://ssda.anu.edu.au/other/other_archive.html A page of links, by country, to national data archive collections.\n\nInternational news\n\nWhat are the big stories? Check out local media coverage of breaking news after finding them in one of these online news directories:\n\nOneWorld www.oneworld.org \"A community of over 150 leading global justice organizations\" compiles news items from around the world, nicely searchable and focused on human rights issues.\n\nReliefWeb http://wwwnotes.reliefweb.int:81 Keep up with coverage of international natural disasters and the current status of \"complex emergencies\" with this Web site from the United Nations Department of Humanitarian Affairs.\n\nInternational directories\n\nChiefs of State and Cabinet Members of Foreign Governments http://www.odci.gov/cia/publications/chiefs/chiefs-toc-view.html If you need the name of the Minister of Education in Niger, head for this page, brought to you by those friendly folks at the CIA. They provide links to listings of all the government officials, with the date it was updated clearly marked. Unfortunately, doesn't give any contact or e-mail information.\n\nThe Embassy Page http://www.embpage.org/index.html Fighting breaks out in Cyprus and you want the American Embassy's take? Go to their Web site (and find all countries' embassy sites with this listing).\n\nInternational organizations\n\nNATO: The North Atlantic Treaty Organization http://www.nato.int It only has 16 members, but for those members, there is a handy chart linking you to the country's senate, head of state, military chief and other government Web sites that are available (http://www.nato.int/family/countries.htm). Check out, too, the Web archive with press releases, speeches, treaties and other documents, some dating back to 1949.\n\nSo you've got all this information and now all you have to do is pack. But you have no idea what the weather is going to be. Point your browser to http://www.worldclimate.com/ and find out if you should pack the raincoat.\n\nNora Paul can be reached at (813) 821-9494 or send e-mail to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-get-your-training","slug":"1997-october-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-october-get-your-training.html","year":1997,"month":"October","monthNum":10,"headline":"Get your training","kicker":"Seminars, boot camps, conferences","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR offers specialized computer-assisted reporting training through workshops, on-the-road seminars, and intensive boot camps at various locations, teaching journalists to analyze government data and build stories using spreadsheets and databases.","keywords":["training","journalism","data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"NICAR provides specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and online databases. Cost varies. For information, call Lisa Barnes at (573) 882-8969. December 3, 5 and 8, 1997 - Columbia, Mo. On-the-road training for the Bureau of National Affairs December 15-17, 1997 - Chicago, Ill. On-the-road training for The Chicago Tribune\n\nIRE and NICAR can also bring their road-tested workshops to sites around the country. We match regional public records experts and experienced reporters with our own trainers for unique seminars ranging from one-day demonstrations and panels on regional public records laws to three-day hands-on sessions on computer-assisted reporting or advanced techniques for data-intensive beats. Don't wait for someone else to organize a computer-assisted reporting workshop in your area. IRE and NICAR depend on members, news organizations, journalism schools and other journalism organizations to help us bring our seminars to you. Contact Sarah Cohen at (301) 942-2199 or e-mail her at sarah@nicar.org November 21-23, 1997 - Columbia, Mo. Intermediate Computer-Assisted Reporting Workshop December 11-13, 1997 - Washington, D.C. Computer-Assisted Reporting Workshop Sponsored by NICAR and the Medill School of Journalism January 23-25, 1998 - St. Louis, Mo. NICAR Health Care Workshop\n\nBoot Camps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo. As with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and online databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size. For information, call Katy Fanning at (573) 882-0684. January 4-9, 1998 - Columbia, Mo. NICAR Regular Boot Camp January 9-11, 1998 - Columbia, Mo. NICAR Advanced Boot Camp May 10-15, 1998 - Chapel Hill, N.C. NICAR Advanced Boot Camp May 17-22, 1998 - Columbia, Mo. NICAR Regular Boot Camp July 12-17, 1998 - Columbia, Mo. NICAR Regular Boot Camp August 9-14, 1998 - Columbia, Mo. NICAR Regular Boot Camp\n\nNICAR will offer training at the following professional conferences, including the IRE and NICAR national conferences. These conferences feature practical tips and story ideas shared by prize-winning journalists in quick-hit sessions. Keep up with our conference schedule through our Web site, at www.ire.org. Costs vary. For more information or to register, call Lisa Barnes at (573) 882-8969. November 15-16, 1997 - Ciudad Juárez, Mexico Border Conference March 5-8, 1998 - Indianapolis, Ind. NICAR National Conference June 4-7, 1998 - New Orleans, La. IRE National Conference June 3-6, 1999 - Kansas City, Mo. IRE National Conference","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-from-the-nicar-library","slug":"1997-october-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-october-from-the-nicar-library.html","year":1997,"month":"October","monthNum":10,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering topics ranging from transportation safety and healthcare to campaign finance and crime statistics, with data spanning from the 1930s through the 1990s.","keywords":["databases","government","records"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\nNEW National Endowment for the Arts, 1987-1996, includes information on individual or organization receiving money.\nNEW Immigration and Naturalization Service legal residency, 1980-1995, includes information on the characteristics of aliens who were admitted as immigrants.\nNEW National Highway Traffic Safety Administration vehicle recalls and complaints, 1966-1997, includes information about vehicles that have been recalled by certain manufacturers.\nU.S. Food and Drug Administration's Medical Devices Reports, detailing complaints about drugs, pacemakers and other medical products, 1974-1996.\nU.S. Centers For Disease Control's AIDS database, providing case-by-case demographic information about those with the HIV virus, 1995.\nU.S. Census Bureau's Consolidated Federal Funds Reports, showing which communities get how much under various federal programs, 1983-1995.\nFederal Aviation Administration's accidents and incidents, including major plane crashes since 1974.\nNASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\nFederal Election Commission campaign contributions by individuals and political action committees, 1990-1997.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded in-patient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings, 1991-1996.\nCoast Guard boating accidents, 1969-1994.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration, 1974-1997.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\nFederal procurement data, 1992-1996, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records, 1993-1996.\nNational Bridge Inventory System data, includes inspection grades, 1994-1995.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the 1995 final report.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1980-1997. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nThe National Inventory of Dams, 1991-1995.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971-1996.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1975-1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nThe IRE Resource Center has copies of hundreds of stories that have used the databases listed here. If you are interested in seeing how reporters have used these databases, call the center at (573) 882-3364.\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-october-bits-bytes-and-barks","slug":"1997-october-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-october-bits-bytes-and-barks.html","year":1997,"month":"October","monthNum":10,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR announces expanded campaign finance resources, conferences, and seminars for journalists, including an annual March conference in Indianapolis and upcoming specialized workshops on database design, SQL servers, and data visualization.","keywords":["campaign","seminars","journalists"],"topics":["Journalism & Newsroom","Government & Politics","Technology"],"technologies":["SQL","Access","World Wide Web"],"full_text":"The Campaign Finance Information Center is in full swing. Data, tips, links and stories are up at the center's Web site, www.campaignfinance.org\n\nAlso, coming soon to a mailbox near you will be the Campaign Finance Information Center's newsletter. Stories from the Fall '97 newsletter include reports about states with electronic filing systems, a look at how states are going to inform the public of the connection between money and politics and an explanation of in-kind contributions.\n\nConferences and Seminars\n\nNICAR is forging ahead with new seminars and new approaches to conferences and workshops. Our annual conference, March 5-8 in Indianapolis, will not only include the usual panels and hands-on classes for beginners to the advanced, but also a set of special half-day sessions that will take in depth looks at how to use electronic information for particular beats.\n\nWe also are expanding our seminars for the intermediate range. Earlier this year we did a seminar on education and statistics. In the coming months, we plan to hold seminars on database design and data cleaning, use of SQL servers for the newsroom, mapping software, and business reporting. Watch our website at www.nicar.org for dates and times. For more information on seminars and boot camps, call (573) 882-0684.\n\nNICAR Net\n\nRecent topics on the IRE and NICAR listserv have included campaign finance data, troubles with traffic accident data, ethics, medical investigations and Access tips.\n\nTo subscribe to IRE-L or NICAR-L, send e-mail to listproc@lists.missouri.edu\n\nIn the body of the message, type: subscribe NICAR-L<your name> subscribe IRE-L<your name>\n\nAlso, check out the IRE-L and NICAR-L mailing list archives on our website at www.ire.org and www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.\n\nReader Response\n\nWe are still looking for readers' ideas of how to improve Uplink.\n\nAre there certain columns that you would like to add? Are there certain topics you would like covered? Have you done a story that you think should be featured in an upcoming issue? Do you have something you would like to contribute? A Tech Tip? A column about a CAR story you reported?\n\nHere's your chance. Send comments, critiques and suggestions to Uplink's editor, Jody Sowell, at jody@nicar.org We look forward to hearing from you.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-slam-dunk","slug":"1997-november-slam-dunk","url":"https://dwillis.github.io/uplink/article/1997-november-slam-dunk.html","year":1997,"month":"November","monthNum":11,"headline":"Slam Dunk","kicker":"Uplink Update","author_name":"","author_title":"","affiliation":"","summary":"News organizations are using computer-assisted reporting to move beyond homers and cover sports as a business, including salary comparisons, NCAA investigations, and statistical analysis. The article features examples from the AP, Kansas City Star, and The Daily Oklahoman showcasing how CAR is transforming sports journalism.","keywords":["sports","journalism","investigation"],"topics":["Investigative Methods","Business & Economy","Journalism & Newsroom"],"technologies":[],"full_text":"For years, the media have been dropping the ball when it comes to covering sports. But in many places homerism has been replaced by hard-hitting investigations and stories that cover sports as a business thanks to CAR. This issue of Uplink examines how some news operations are tackling sports' issues. NICAR's Kara Morrison analyzes how the Associated Press created its own database to compare the salaries of Texas high school football coaches with classroom teachers. Greg Reeves of the Kansas City Star explains the computer-assisted reporting work that helped produce the Star's latest series on the NCAA. Griffin Palmer diagrams how The Daily Oklahoman uses CAR on a weekly basis to rank local teams. NICAR's Sarah Cohen explains how statistics can help sports' staffers, and columnist Nora Paul shares what is available on the Internet for sports lovers and writers.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-coaching-for-dollars","slug":"1997-november-coaching-for-dollars","url":"https://dwillis.github.io/uplink/article/1997-november-coaching-for-dollars.html","year":1997,"month":"November","monthNum":11,"headline":"Coaching for dollars","kicker":"Salary Comparison","author_name":"Kara Morrison","author_title":"IRE Resource Center","affiliation":"IRE","summary":"This issue highlights investigative reporting in sports, focusing on coaching salaries in Texas high schools and NCAA's role in college sports finances. It discusses the AP's local data-driven analysis comparing coaches' and teachers' salaries, as well as Kansas City Star's use of databases in examining NCAA money issues.","keywords":["sports","salaries","NCAA"],"topics":["Investigative Methods","Education","Data & Databases"],"technologies":["Excel"],"full_text":"When Linda Leavell heard her AP story being discussed on a three-hour Dallas radio program, she knew her investigative series was a winner. The story, Leavell said, was one nearly everyone in Texas could relate to: hometown high-school football coaches, revered in many communities, were making about 75 percent more than the state's teachers. Leavell, a day supervisor at AP's Dallas bureau, along with reporters Jaime Aron, Peggy Fikac, Ed Montes, Mark Babineck and Mike Cochran, put together seven-story package called 'Pigskin Paychecks' that was run by nearly every media outlet in Texas in October 1996. The initial legwork was extensive, because there was no ready-made database compiling the information the AP needed. The story idea stemmed from false rumors that coaches often earned more than their superintendents. To gather the data, AP sent questionnaires - all hand-tailored and addressed by name to each superintendent and complete with the name of each football coach - to 238 school districts. Leavell even anticipated compliance problems and sent a state attorney general's open records ruling with each letter. 'Schools are often notorious for refusing first and complying later, and we were trying to head that off,' Leavell explained. Eventually, all 238 districts representing 428 coaches in the largest Texas high schools, responded with the information. Leavell said most schools responded quickly, but about 75 or 80 of them had to be recontacted. Eventually, she ended up calling all the districts again to double check the information. Finally, all the data was entered into an Excel spreadsheet, which contained each school name, its district's average teacher salary, the superintendents' and coaches' names and salaries and the school size. The numbers spoke for themselves. Leavell learned head football coaches at the largest Texas high schools earned about $54,000 on average - 75 percent more than the average teacher's $31,000 paycheck. The highest paid coach made more than $82,000. Texas coaches receive state-set, base salaries determined by teaching experience, just like any other teacher. But in addition, coaches are paid a district-set stipend. Leavell said she received some help and suggestions from AP computer-assisted reporting experts Bill Dedman and Drew Sullivan on analyzing and explaining the data. Dedman's suggestion that Leavell explain the median salary became a key graph in the overall piece, making the findings easier to understand: 'The statewide averages are not inflated by just a few high-earning coaches,' the story read. 'The typical coach makes $54,239. Half make more, half make less.' The series also used U.S. Census data to show coaches' salaries were about $25,000 to $30,000 higher than average Texas household incomes and those of federal, state and local government employees. Along with an overall story on coaches' salaries, AP's series included profiles of the highest- and lowest-paid coaches, reactions from non-coaching teachers, education experts and state legislators, and even a personal essay by Mike Cochran, who explained what his own hometown football coach had meant to the community. The series, which took six months to complete, raised questions about the large salary disparity AP found. The AP team had also considered comparing each school's academic ranking with its economic focus on sports, but the reporters soon learned they couldn't assume a direct link between the salaries and academic performance. One graph in the series, however, mentioned: 'Only two schools among the top 50 in coaches' salaries rank as 'exemplary' in state academic ratings. Meanwhile, six are considered 'low performing.' Leavell said the series brought mixed reactions from the coaches. 'Some were mad, some were accusing us of singling them out or trying to humiliate them or stirring up trouble,' Leavell said. 'Even after it came out, I had a coach call who said, 'It's not anybody's business what I make.' Other coaches were willing to talk frankly about their pay, which they said often compensated for lack of job security. From compiling the data, Leavell learned coaching turnover was in fact high. 'It didn't really occur to me that coaches could be fired in the middle of the season or would leave,' said Leavell, who learned to specify she needed only the salary of the coach who had started the 1995-96 season, rather than the current coach. Leavell said if done again, she would have made this clear in the questionnaire and would also have asked for the principal's salary. Because of the series, AP's Dallas bureau became a model for successfully using the Texas open records law. Leavell also won the Headliners Award for the best sports story of 1996 and received honorable mention in a national AP contest. 'I was wholly satisfied,' Leavell said of the final project. 'It was really successful. It brought a lot of satisfaction to me personally and brought a lot of praise to AP in Texas nationally.'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-november-bending-the-rules","slug":"1997-november-bending-the-rules","url":"https://dwillis.github.io/uplink/article/1997-november-bending-the-rules.html","year":1997,"month":"November","monthNum":11,"headline":"Bending the Rules","kicker":"Money & Sports","author_name":"Greg Reeves","author_title":"","affiliation":"The Kansas City Star","summary":"The Kansas City Star used eight databases including NCAA-provided data on graduation rates and legislative votes to investigate the role of money in college sports, learning valuable lessons about data entry quality control and computer-assisted reporting techniques.","keywords":["NCAA","databases","journalism"],"topics":["Investigative Methods","Data & Databases","Education"],"technologies":["Paradox","FoxPro","XPL","floppy disk"],"full_text":"Eight databases contributed to The Kansas City Star's six-day series Oct. 5-11 on the role of the National Collegiate Athletics Association and money in college sports. The NCAA provided two of the databases on diskettes, including a history of student-athlete graduation rates by school, sport, race and sex, and 100,000-plus votes by member schools on NCAA legislation in the past four years. The NCAA also gave us early access to its Web site that will contain text files of all major NCAA infraction cases since 1987. We built several other databases by scanning the NCAA membership directory and 1,200 pages of NCAA convention books containing 694 legislative proposals since 1993 with accompanying discussion. And, like several other news organizations, we used data entry to create tables from the 1996 Gender Equity Disclosure Act forms we requested from 305 NCAA Division 1 colleges and universities. None of the databases in this project were particularly complicated. The lessons for computer-assisted reporting had more to do with scanning, data entry and text analysis. The generated files gave us the ability to search for statements on NCAA legislation by school officials and others. It also enabled us to build a searchable database of every NCAA legislative proposal since 1993. The data entry taught me a valuable lesson. We hired temps, some of them very good, to input gender equity forms and NCAA infraction summaries. Then we hired other temps to proofread the work. At the end we still had errors embedded in the data - fewer than a dozen, but dangerously hidden until uncovered by a query or report. It would have been cheaper to have two persons enter the data, then compare the results. That's how I plan to approach future projects requiring data entry. The NCAA graduation-rates database comes from a complex report submitted annually by Division 1 schools detailing enrollment and degrees received in 42 categories by student-athletes and other students. Most of the same data is published in the NCAA's yearly Graduation Rates Report. The book lists a number range for athletes graduating in each sport; the database lists the actual number. We asked the NCAA for this database and they provided us with four Paradox tables zipped onto a single diskette. The database makes it possible to rank schools by graduation rates of student-athletes or other students. We created flags for whether schools were public or private, and which had the best won-loss records over the past decade. These flags, along with fields on gender, sport, race and athlete type (aid or transfer) made it possible to analyze graduation rates in literally hundreds of ways. The infractions files, which we downloaded from the NCAA's yet-to-be-released Web site, totaled more than four megabytes of text. There were 124 files, some as long as 10,000 words. As I read each file I typed in brief markers, or flags, for the source of the infraction (the NCAA, the school, the press, etc.) and for every penalty listed (coach fired, sport suspended, etc.). Then I wrote a short program in XPL, the Xywrite scripting language, that flew through the files and threw the file name and flagged text (usually just a few words) into a file. I appended this 1,700-line text file to a FoxPro table, then linked that with the NCAA Enforcement Summary, which lists probation, post-season bans and television bans for major infractions since 1952. We now had a powerful way to analyze NCAA enforcement trends and show, that tough measures like bans on post-season play and television were being replaced by often meaningless probation. We collected gender equity forms from 305 Division I NCAA schools and analyzed them about the same time as several other newspapers. The reports showed that women are 52 percent of students at Division I schools, but only 37 percent of athletes. It also showed schools overall lost $200 million on sports programs in 1996. The federal law requires schools to fill out eight tables, on enrollment, athletic participation, number of coaches, average coaches' salaries, operating expenses, recruiting expenses, athletic aid and overall revenue and expenses. The reports are due Oct. 15 each year. The NCAA also provided us with diskettes containing FoxPro tables of more than 100,000 votes by the 933 member schools at the organization's past four annual legislative conventions. This database showed how hard it is to categorize political legislation. We examined each of nearly 700 proposals and tried to flag each one with some kind of meaningful evaluation. In the end, however, the database told us mainly what the NCAA did and did not address.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-open-records-tips","slug":"1997-november-open-records-tips","url":"https://dwillis.github.io/uplink/article/1997-november-open-records-tips.html","year":1997,"month":"November","monthNum":11,"headline":"Open records tips","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This article provides practical tips for journalists navigating open records act requests, including being specific about data needs, personalizing communications to school administrators, and maintaining detailed documentation of all follow-up efforts.","keywords":["records","compliance","transparency"],"topics":["Investigative Methods"],"technologies":[],"full_text":"Leavell's tips for successful open records act compliance:\n\nKnow exactly what data you need.\nTailor specific questionnaires for schools to fill out.\nSend the letters directly to administrators by name rather than title.\nEnclose a self-addressed, stamped envelope and a fax number.\nKeep a detailed log of each follow-up call you've made to the school, including names of people you talked to, when you called and what they said.\nEnclose a state attorney general's decision about a similar open records case with each letter.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-hunting-for-data","slug":"1997-november-hunting-for-data","url":"https://dwillis.github.io/uplink/article/1997-november-hunting-for-data.html","year":1997,"month":"November","monthNum":11,"headline":"Hunting for data","kicker":"Lax Enforcement","author_name":"Kara Morrison","author_title":"","affiliation":"NICAR","summary":"A Minneapolis Star-Tribune computer-assisted reporting project analyzed Minnesota DNR enforcement data from 1987-1994, revealing a 28.2% drop in citations despite increased outdoor activity, with resources shifted from hunting/fishing enforcement to motorized vehicle enforcement.","keywords":["enforcement","database","violations"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["Access","Excel","FTP"],"full_text":"Bill Loving was looking for a quick-hit, computer-assisted, sports story this spring when he recalled reading stories analyzing hunting accidents.\n\nLoving and reporter Dennis Anderson then came up with an even better twist: analyzing hunting and fishing violations. Anderson had heard the Minnesota Department of Natural Resources had been lax in its enforcement of violations in recent years.\n\nLoving, computer-assisted reporting editor at the Minneapolis Star-Tribune, set out to see what databases were available. To his delight, the Department of Natural Resources was helpful.\n\n\"Not only did they have good data going back to 1987, but they let us have it for free,\" Loving said. \"They also agreed to let me download it via anonymous FTP, instead of dealing with tapes, etc.\"\n\nDissecting the data\n\nThe database contained a wealth of information, including:\n\nType of violation\nAmount of fine\nDate, time and county in which the violation occurred\nBadge number of the conservation officer\nThe outdoor activity involved\n\nFrom it, Loving was able to confirm Anderson's hunch.\n\nDNR enforcement had in fact dropped, while ironically, outdoor activity was at an all-time high.\n\nThe story, which ran on the front page of the March 27 Star-Tribune, detailed a 28.2 percent drop in total DNR citations from 1991 to 1994.\n\nIn that period, hunting and fishing citations, including tickets for poaching and invalid licenses, dropped by an even steeper 32.6 percent.\n\n\"A second main finding was that enforcement of game laws was declining while enforcement of boat/snowmobile laws was rising,\" Loving explained.\n\n\"Given that the DNR had been working with a frozen budget and staffing level for decades, it looked as if they were making a conscious choice to re-direct resources away from hunting/fishing and toward motorized vehicles. Anderson's reporting confirmed that hypothesis.\"\n\nLoving said he used Access 97 to establish the general trends.\n\nHe grouped hunting and fishing and motorized-vehicle violations together, grouped them again by year, and pasted his query results into Excel. He then used Excel to build a series of summary tables and charts highlighting his findings.\n\nThe database also gave Anderson a good idea of which officers to interview.\n\nUnfortunately, Loving said most refused to talk on the record.\n\nBut those who spoke to Anderson gave him the rest of the story, detailing how low morale, budget shortfalls, and a rift between officers and the enforcement director played a hand in the citations plunge.\n\nFew problems\n\nThe biggest challenge of the project was coordinating schedules with Anderson. This took months.\n\nAfter the two found time to work together, Loving said the story took only a couple of weeks to complete.\n\nThere was only one other slight glitch in the data that Loving was able to work around. \"They initially withheld the cases in which juveniles had been ticketed,\" he said. \"But I later persuaded them to give me those records, too, but with the juveniles names deleted.\"\n\nIn all, Loving was pleased with the story, which he sees as a good example of CAR on the beat.\n\n\"We didn't uncover any major scandals, but it was an inside look at a public agency that most people don't know much about,\" Loving said. \"And given the extreme importance of natural resources and outdoor recreation in Minnesota, it shed light on some real concerns about how the resources were being managed.\"\n\nLoving recently compiled his own database using DNR violations to analyze hunting accidents, just in time for start of deer-hunting season.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-poll-producer","slug":"1997-november-poll-producer","url":"https://dwillis.github.io/uplink/article/1997-november-poll-producer.html","year":1997,"month":"November","monthNum":11,"headline":"Poll producer","kicker":"","author_name":"Griffin Palmer","author_title":"Sports/Data Editor","affiliation":"The Daily Oklahoman","summary":"The Daily Oklahoman uses FoxPro to automate sports statistics processing, reducing the time spent on prep football standings and basketball reports from eight hours to 30 minutes per week through a system that imports scores and generates typesetter-ready charts.","keywords":["FoxPro","statistics","automation"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["FoxPro","SII","Internet"],"full_text":"Much of meat-and-potatoes of day-to-day sports reporting is driven by statistics. FoxPro ideal for processing statistics and getting them ready to put in the paper. At The Daily Oklahoman, we use FoxPro to generate typesetter-ready prep football standings charts. During the basketball season, sports reporters use FoxPro to generate win-loss reports for prep teams. Sports staffer Penny Soldan says she used to devote an eight-hour day each week to calculating prep football standings and keying them into the system. Soldan said she used to spend more than eight hours a week wrangling numbers for girls' basketball. She now does the same tasks in about 30 minutes.\n\nComputer rankings\n\nBecause The Daily Oklahoman's newsroom network is so crotchety, it has been impossible for us to create a system where agate clerks can key scores into a central database. We've worked around the network's limitations by transferring weekly scores files out of our SII typesetting system, importing them into FoxPro, and using FoxPro's relational capabilities to write scores from the weekly scores files to a master schedule. Then, we use FoxPro's report generator to create a weekly standings file, including typesetting codes, from the master schedule. We transfer that report back to the SII system. It's a crude solution, but it works. I've accomplished all this with the FoxPro training I've gotten at NICAR conferences, what I've picked up off NICAR-L and other Internet lists and newsgroups, and what I've been able to dope out by reading the FoxPro manuals. Each summer, The Oklahoman publishes a master schedule of all prep games. That forms the nucleus of our weekly standings system. The sports staff uses SII macros to convert the schedule into a comma-delimited file, which we import into FoxPro.\n\nOnce a week, we take the scores file published in the Saturday paper, convert it to comma-delimited, and import it into FoxPro.\n\nThen, we run a program that compares the team, opponent and date fields in the weekly file with the same fields in the master schedule, and writes the appropriate scores into the master schedule.\n\nThis gets a little tricky. In the master schedule, every game involving two in-state teams is listed twice: Once in each team's schedule. The program has to write each score into the table twice - in reversed order. Another twist is the fact that in the weekly scores table, the winner always appears in the left-hand column; the loser, in the right.\n\nOur program accounts for all this, and writes all the scores into the proper columns.\n\nThe master schedule contains district win-loss columns, overall win-loss columns, a column for a weighted scoring system developed by our sports staff, a column to indicate district and non-district games, and an overtime column.\n\nAfter writing scores into the correct columns, the program writes 1s into the appropriate win and loss columns and calculates weighted scores. It sums in-district wins and losses, overall wins and losses and weighted scores for each team, ordering by win-loss and weighted score within each class and district.\n\nIt stores the results to a separate table, from which FoxPro generates the report.\n\nI figured out which of FoxPro's special characters correspond to the correct SII typesetting codes by opening an SII-formatted standings file into FoxPro's text editor.\n\nI then selected the necessary characters from FoxPro's menu of special characters and inserted them into the report profile as literals. FoxPro's report generator allows you to build expressions with FoxPro functions.\n\nI used the iif() function to vary insertion of SII spacing codes, depending on the number of bytes written into the win-loss columns and weighted-score columns.\n\nWe sometimes get errors when a team's name is written differently in the weekly scores file than in the master schedule.\n\nI've written queries that checks the values in the master schedule, after the program has run, against the values in the weekly scores file. Another query tests for mismatched team names when the two files don't balance out.\n\nGriffin Palmer can be reached at (405) 475-3311 or send e-mail to palmer@qns.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-playing-with-statistics","slug":"1997-november-playing-with-statistics","url":"https://dwillis.github.io/uplink/article/1997-november-playing-with-statistics.html","year":1997,"month":"November","monthNum":11,"headline":"Playing with Statistics","kicker":"Stats from the Road","author_name":"Sarah Cohen","author_title":"","affiliation":"NICAR","summary":"NICAR offers computer-assisted reporting classes designed specifically for sports reporters, focusing on data analysis techniques like understanding averages versus medians and applying statistical significance testing to sports statistics.","keywords":["statistics","reporters","sports"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["spreadsheet"],"full_text":"It's always surprising that sports reporters avoid computer-assisted reporting classes offered by NICAR in newsrooms around the country.\n\nOf course, sports reporters keep strange hours, are often out of the office, and take month-long breaks to make up for the grueling hours they put in during the season. But no other beat depends so much on framing numbers in an entertaining and insightful way.\n\nFirst, NICAR is looking to help. We're searching for a good time of year to hold two-day classes in computer-assisted reporting just for sports reporters.\n\nInstead of government budgets and school reports, these would focus on covering high school football and stadium issues.\n\nLet us know when you think you could fit it in.\n\nThat said, sports reporters have an instinctive understanding of statistics. Here are a few more tips for sports reporters that can make those numbers-laden stories just a little more insightful:\n\nUnderstand Averages\n\nThe single biggest problem in sports reporting, particularly on salaries, but also on other player statistics, is a confusion between the words, \"average\" and \"typical.\"\n\nGet to know the concept of a median rather than a simple average, and use it in your stories. A median is often called the typical value, because, for example, it's the middle salary rather than the total paid divided by the number of players. That has a huge effect on what most readers view as astronomical player salaries. If one player makes $7.2 million, but the other eight make closer to $500,000 then the median is half-a-million. The average is something like $1.3 million. Which one more accurately portrays how much players get? The median. Which one says more about the finances of the team? The average. Pick yours wisely.\n\nPay attention to the database\n\nEveryone understands in their guts that batting .333 is very different when a player bats 1-for-3 versus one who bats 33-for-100. The first could obviously be a fluke. The second shows consistent performance. That's easy. But what happens when it's 2-for-6? Or 3-for-10? In this space in September, John Sullivan discussed the use of significance testing in the newsroom. Here's an obvious case for a chi-square statistic. It tells you whether the batting average is likely to have occurred by chance. You can use a spreadsheet, a piece of statistical software, or a pocket calculator to test it. You'd set up your test like Table A.\n\nThe .449 says that a difference as big as the one shown could happen about 45 times out of 100 just by pure, dumb luck. In other words, if a player hits 3-for-10, he might or might not be better or worse than random chance (or 5-for-10). We don't need to set it up to expect hits to happen randomly. We can make our test more realistic by setting them up proportionately. This is usually a fairer way to test. Look at Table B. This time, I calculated \"expected\" hits by figuring the team's average of 15 percent hits (or 15 of 100). That percentage was applied to our star player's at-bats to compute the expected 2 of 10 hits, and to the rest of the team to figure the 14 of 90 hits. Now the 0.000 means that we'd rarely find a player to do this differently from the average purely by chance, or 0 of 1,000 times. Yes, it's a little work to get. But if your sports story revolves around a star's performance in a limited number of games, it's worth checking to see whether this could happen without any particular talent.\n\nEvaluate players salaries\n\nAs many of my colleagues know, I'm not a big fan of gratuitous use of statistical techniques in the newsroom. But player salaries may be one area in which a regression can help. The problem is, it's a classic case of a difficult variable to test using simple linear regression.\n\nIn fact, it's one of the best examples of why reporters have to pay attention to the kind of data they're trying to predict or explain using regression.\n\nIn a linear regression, reporters try to guess a salary, a school's score or any other value based on a series of other figures that they think will explain its performance.\n\nIn this case, you'd try to guess a salary based on the idea that better players get paid more. The trick is to figure out whether it's consistent - if you get 40 percent more hits, do you almost always get a 10 percent higher salary?\n\nHow can you get a model?\n\nRather than reinventing the method needed to accurately predict or explain salaries, borrow from another field. Studies abound in explaining chief executive salaries in major corporations.\n\nIn fact, one consultant uses a method that helps tell shareholders whether the executive is worth the money he or she is paid.\n\nWhy not try adapting this model to sports? These techniques argue strongly for better use of the information that most sports reporters keep in books, in nightly reports and in their heads.\n\nIf they'd just type some data into a spreadsheet instead of writing it on paper, they'd be able to work through any of these problems on the fly. It's worth it.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-one-to-many-how-to","slug":"1997-november-one-to-many-how-to","url":"https://dwillis.github.io/uplink/article/1997-november-one-to-many-how-to.html","year":1997,"month":"November","monthNum":11,"headline":"One-to-many how-to","kicker":"Tech Tip","author_name":"Jo Craven","author_title":"","affiliation":"The Washington Post","summary":"A step-by-step guide to using FoxPro programming to match and merge data from two database tables, specifically matching neighborhood names from a lookup table to loan records based on matching Census tract numbers.","keywords":["database","programming","matching"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["FoxPro","dBase"],"full_text":"One-to-many how-to\n\nBy Jo Craven\nThe Washington Post\n\nRecently, I had a large Home Mortgage Disclosure Act table called master.dbf that included a field with Census tract numbers. The various tract numbers appeared repeatedly, indicating the numerous loan applications that came from each area.\n\nI wanted to identify these tracts by their common neighborhood names.\n\nTo do this, with information provided by a local Census Bureau, I created a table called lookup.dbf with two fields: one called \"tract\" with the tract numbers, and the second called \"hood\" with the neighborhood names. Master.dbf already contained a tract field; I added a second corresponding field called \"hood.\"\n\nMy task was to use lookup.dbf to fill in neighborhood names in the newly created and blank \"hood\" field in master.dbf To do this, I wrote the following FoxPro program, which works by advancing through each table, plucking neighborhood names from lookup.dbf and dropping them into master.dbf when the tract numbers from the two tables match.\n\nGetting technical\n\nIn geek speak, this is known as a one-to-many relationship: One record in lookup.dbf corresponded to many records in master.dbf. Here is the program; an explanation follows:\n\nclear\nclear all\nclose all\nuse c:\\master in 1\nuse c:\\lookup in 2\nselect lookup\ndo while not eof ('lookup')\n  scan\n    select master\n    replace master.hood with lookup.hood for master.tract=lookup.tract\n  endscan\n  select lookup\n  skip 1\nenddo\n\nExplanation\n\nclear\nclear all\nclose all\n\nThese introductory commands make sure that you are working with a clean slate - everything is cleared or closed before the program begins.\n\nuse c:\\master in 1\nuse c:\\lookup in 2\nselect lookup\n\nThe two tables are opened with the \"use c:\\master in 1\" and \"use c:\\lookup in 2\" lines. You must still tell the computer which of these files to use first. \"Select lookup\" takes care of this.\n\ndo while not eof ('lookup')\n\nThis acts like a scan loop. It steps through lookup.dbf. The \"not eof\" means \"not end of file\" and instructs the computer to continue working as long as it has not reached the end of lookup.dbf.\n\nscan\n\nselect master\nreplace master.hood with lookup.hood for master.tract=lookup.tract\nendscan\n\nThis nested scan loop then selects the second table, master.dbf, and issues the replace command, which fills in the empty \"hood\" field in master.dbf with a neighborhood name drawn from the \"hood\" field in lookup.dbf when master.tract matches lookup.tract.\n\nselect lookup\nskip 1\n\nThe computer has looked at the first entry in lookup.dbf, then scanned through every record in master.dbf and made the appropriate changes. But the computer still needs to advance to the next record in lookup.dbf and perform the same job. \"Select lookup\" and \"skip 1\" tells the machine to return to lookup.dbf, skip one record, and repeat the procedure. It does this until the computer reaches the end of lookup.dbf.\n\nenddo\n\n\"Enddo\" concludes the \"do while\" command.\n\nJo Craven can be reached at (202) 334-6259 or by e-mail at cravenja@washpost.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-world-wide-sports","slug":"1997-november-world-wide-sports","url":"https://dwillis.github.io/uplink/article/1997-november-world-wide-sports.html","year":1997,"month":"November","monthNum":11,"headline":"World Wide Sports","kicker":"On the Internet","author_name":"Nora Paul","author_title":"Training Instructor","affiliation":"The Poynter Institute","summary":"Nora Paul from Poynter Institute provides a guide to sports websites and resources for journalists, recommending link pages and mega-sports sites to help reporters and editors verify information and cover sports stories that extend beyond the sports section.","keywords":["sports","journalism","internet"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"World Wide Sports\n\nBy Nora Paul The Poynter Institute\n\nIn the interest of full disclosure, I'm not much of a sports fan. I have no special teams I loyally root for. But I am a dedicated fan of Uplink so when the assignment for the column was changed to sports, I didn't try to punt it back.\n\nAfter \"adult entertainment\" sites, there are probably more Internet sites dedicated to sports topics than anything else. I don't go to them. In fact, a friend who runs a big sports server called to tell me all about his new venture when he was starting it up and I said, \"Oh great, just what the Internet needs, another damn sports site.\" I haven't heard much from him lately.\n\nBut then, as often happens, life made me face something that I had avoided. I had to train sports copy editors and reporters at the Minneapolis Star-Tribune on how to use the Internet. I had to explore sports sites. And I came away with an appreciation of some of the services and their meticulous arrangement and wide scope of sports information. I also realized that sports stories aren't just for the sports section, as a big story in Minneapolis, and in a number of other communities, has revealed. The hostage taking of communities by major league sports teams insisting they need a new stadium or they'll leave town is a business story, a community story and a government story.\n\nSo, whether you are a copy editor needing some verification of names and statistics, a reporter covering a big game or the investigative reporter needing to background some rich sport franchises' dealing, here are some sports links that can help.\n\nHope they help you pitch a field goal, or some such thing.\n\nLinks pages\n\nSometimes, finding the site that has the best collection of links is your first, best bet in searching.\n\nRely on \"Web scouts\" who go out and locate good sites rather than wading through the vast listings in large search engine sites. Here are a couple of good ones:\n\nWorld Wide Web of Sports\nhttp://www.tns.lcs.mit.edu/cgi-bin/sports\nA sort of Yahoo of sports sites, compiled by students at MIT. The first page will link you to the listings for specific sports or click the \"Sports Search\" link to get a search box and look for specific items.\n\nHandilinks\nhttp://www.ahandyguide.com/\nCheck out this guide to Internet resources - categorized in broad subject areas. Click on sports and get a wonderful listing by sport, from back-packing to wrestling, with sub-categories for teams and other topics.\n\nA great way to find the specific Web sites of particular sports teams, but be careful that you are looking at the \"official\" Web site, not just a fan's ravings.\n\nTip for finding sports link pages:\n\nGo to the Infoseek search site (www.infoseek.com) and put in the search box: +title:links +title:<name of sport.\n\nYou'll find pages with the title containing the word \"link\" and the name of the sport you are interested in.\n\nMega-sports sites\n\nCBS SPORTSLINE\nhttp://cbs.sportsline.com/index.html\nGet the latest news for free, pay $4.95 per month for other features and functions and access to reference works like the Information Please Sports Almanac.\n\nESPN\nhttp://ESPN.SportsZone.com/\nOK, even for a non-sports nut, ESPN's array of players' statistics and the manipulation and cross comparisons that can be done with the statistics is very impressive. Want to see your favorite player's statistics compared with others, you'll have to subscribe to the service, also at $4.95/month.\n\nSPORTSNETWORK.COM\nhttp://www.sportsnetwork.com/home.asp\nLooking for U.S. and non-U.S. sports action?\n\nThis site specializes in providing sports scores, news and information from around the world from a number of different information services.\n\nRULES OF THE GAME .FAQ\nhttp://ps.superb.net/FAQ/\nA wonderful collection of faq's (frequently asked questions) that has details of the rules for every sport from AFM racing to volleyball. Search tip: use the same technique as for finding link pages but put \"rules\" in the title: search.\n\nOther interesting things\n\nINTERNETS: your global link to 1,000s of databases\nhttp://www.internets.com/ssportlk.htm\nThe category for \"sports\" has databases for auto racing, baseball, football, skiing and several other sports. Some are databases of articles, others are databases of statistics and information.\n\nCalculators On-Line\nhttp://www-sci.lib.uci.edu/HSG/RefCalculators1C.html\nGolf handicap, leader size for fishing, freefall in parachuting, even quarterback passing rating these are just a few of the calculators you are linked to from this site.\n\nNora Paul can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-when-the-team-leaves-home","slug":"1997-november-when-the-team-leaves-home","url":"https://dwillis.github.io/uplink/article/1997-november-when-the-team-leaves-home.html","year":1997,"month":"November","monthNum":11,"headline":"When the team leaves home","kicker":"Handout of the Month","author_name":"Trebor Banstetter","author_title":"Reporter","affiliation":"Nashville Banner","summary":"A guide for journalists covering professional sports team relocations, emphasizing the importance of treating teams as businesses and understanding the financial significance of luxury boxes in new stadiums.","keywords":["relocation","business","luxury"],"topics":["Business & Economy","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"When the team leaves home\n\nBy Trebor Banstetter\nNashville Banner\n\nThe frenzy of big-league sports franchises pulling up the stakes and moving is one of the biggest stories to hit sports and business journalism in years. In Nashville, the relocation of the Houston Oilers was the biggest story to hit Music City in years. I was assigned to cover the story from a business perspective, and it took the bulk of my time for more than eight months. Here are some things to remember when relocation fever hits your city.\n\nIt's a money game\n\nAlways remember that professional sports is a business, and should be covered as such. Very often we don't look at a football team the same way we cover the town's big manufacturer, but its likely the team's revenues are just as high or higher.\n\nDon't worry about the coaches and the players. Find out the names of the administrators, finance people, marketing and human relations people - these will be your real sources.\n\nEarly on, do a story profiling the team as a business. What's its revenue, profit? Who are its employees? What is its business strategy? Does it have any subsidiaries? How about its historical business performance? What else is the team owner into? (Rarely is the team their only business).\n\nSince most pro sports teams are closely held, you'll need to get much of the financial data from the organization. But surprisingly, (probably because they're not used to being covered as a business) they often share a good amount of information. Other data can come from the league, and consultants and academics who track the sports business, and other business sources. Check state corporate records and the Securities and Exchange Commission to find out what other cookie jars the owner has his hands on and how successful he or she has been outside of pro sports.\n\nLuxury, luxury, luxury!\n\nThe most important element in a team relocation that involves a new stadium is the luxury box (or skybox, luxury suite, dream suite, etc.) These suites are gigantic cash generators for the teams, and usually are the single largest source of revenue coming out of the stadium every Sunday.\n\nIn fact, one sports economics expert told me that the ideal stadium of the '90s would be 85 percent luxury boxes and about 15 percent seats to look good on television.\n\nThe reason for the importance of the luxury is twofold. First, they are enormously expensive.\n\nA second reason for the luxury suite mania is the way they are structured in the stadium lease. Generally, the suites are managed by a separate corporation, and the revenues flow directly through that company, rather than the team.\n\nOf course, the suites that create all this cash for the team owners are usually built at taxpayer expense. Nashville taxpayers will be coughing up more than $6 million to pay for the suites in our new stadium.\n\nWho's behind the door?\n\nAfter identifying the importance of luxury boxes, a fantastic story to pursue is identifying all the suite buyers and how much they are paying. Readers might be interested in knowing if luxury box owners include non-profit hospitals, public utilities, companies that have recently laid off workers, and city government political cronies. You can also check to see if anyone is getting a discount or break on their suite and why.\n\nThis is not an easy nut to crack. In most cities, the suite holders list is considered \"super top secret\" and they will not hand it over without a fight.\n\nMake your own list. Determine the obvious suspects from the big companies and rich private citizens in town, and then call around and ask if they've got luxury boxes. Many will own up, especially big companies that see the stadium as an economic development tool. The obvious problem with this is that you will never get a complete list, and many of the really interesting luxury box buyers will never admit they hold suite leases.\n\nDo some footwork. Work your way into the suite level of the stadium and look at the doors. In some stadiums, the suite holders names are printed. In Boston, a Globe reporter was invited to enjoy a game in a PR firm's luxury suite at the city's new Fleet Center, and was able to compile a complete list. He followed up by calling all the buyers, and using the standard city and county documents to identify more obscure names.\n\nGet a list leaked to you. This isn't easy, but if you have been cultivating business sources within the organization, sometimes you can find a lower-level person with access, who will share it with you.\n\nTake the team to court. If your paper or station has the stomach for this, a good case can be made that a luxury suite list in a publicly-funded stadium is public record. In Nashville, we successfully sued the Oilers and they gave us copies of all the leases a week later.\n\nThe political connection\n\nPolitics plays a major role in sports relocations. Look to see what kinds of deals are getting cut to get support from the city's legislators. In addition to just the stadium, lots of \"pork\" can be generated on the city and state level to get widespread support for the project. The paper's political writers can have a bonanza with this.\n\nWatch the neighbors\n\nAlways compare your city to other cities of similar size that also have new sports teams.\n\nDon't forget philosophy. There is a philosophical element to sports relocations that screams to be examined. Talk to philosophy professors, ethicists, and religious leaders about what the issue says about the community's priorities. A school teacher may have a very different perspective that needs to be pursued. Also, don't forget that there are people who oppose professional sports on principal, and see it as violent, exploitive pastime that is a major waster of civic money. Don't forget to examine this point of view.\n\nIt's a zero-sum game\n\nIf your city is getting a team, someone is losing one (unless it is through expansion, but that's only a fraction of sports movement today). Always keep close tabs on the city that the team is leaving. What do they think about it? People there can tell you what kind of a corporate citizen the team was, how active the owner was in the community, how they dealt with employees, unions, etc. See what kind of measures the city is taking to save the team. (Houston, for example, didn't care much at all that the Oilers were leaving. Even business leaders told me they would rather go without football and try to attract a new team than work to keep Bud Adams in town).\n\nPick apart the lease\n\nThe stadium lease is the real treasure trove of information. This document spells out exactly how the facility will be managed, how the revenues will be divided, how contracting will work, etc. This will be the single most important document of a sports relocation. Will the team manage the stadium? In Nashville, a subsidiary of the Oilers will manage the stadium, even for non-football events. That subsidiary will essentially have total control over the taxpayer funded stadium get to choose the events, the scheduling, and keep 100 percent of the revenues for most of the stadium functions.\n\nIf the team is managing the stadium, look into what kind of controls the city has. How often will they be inspected and audited? Who oversees them? Does the city have the authority to fire them if its management is inadequate?\n\nLook to the future\n\nA team relocation is like someone leaving their spouse to marry someone else they recently met and decided they like better. The danger there is that they've done it before, they might do it again.\n\nCheck the lease for an \"escape clause.\" In St. Louis, for example, the city's lease with the Rams has a clause that voids the lease if the stadium is no longer considered \"First Tier.\" The problem here is that \"First Tier\" is not really well defined, and if the Rams decided in five years they want to move, that clause will be a strong tool to persuade a judge that the lease is null and void. The team would then also have the option of holding the city hostage and demanding millions in renovation to the facility.\n\nFinally, one great question you must ask the team owner: Do you plan to permanently move to our city along with the team? (It's surprising how many team owners don't live in their host city.)\n\nTrebor Banstetter can be reached at (615) 726-5968 or by e-mail at trebor.banstetter@nashville.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-database-update","slug":"1997-november-database-update","url":"https://dwillis.github.io/uplink/article/1997-november-database-update.html","year":1997,"month":"November","monthNum":11,"headline":"Database update","kicker":"IRE/NICAR Notes","author_name":"Justin Mayo","author_title":"Staff","affiliation":"NICAR","summary":"NICAR has expanded its database library with new aviation and health datasets including FAA enforcement records and NTSB accident data, while updating existing collections on transportation, safety, and arts funding through 1997.","keywords":["databases","aviation","transportation"],"topics":["Data & Databases","Public Safety","Technology"],"technologies":[],"full_text":"In NICAR's database library we are continually trying to expand our database offerings as well as maintain the most up-to-date existing collection. Here's a run down of the new and updated data sets.\n\nRecent Data Acquisitions\n\nAdding to our expanding aviation databases is the Federal Aviation Administration's Enforcement Information System (EIS) The EIS has current and historical data on FAA enforcement actions against airlines, pilots, mechanics and designees filed from 1962 through mid-1997. The data set includes information on hazardous materials, security, drug testing, and training and maintenance infractions among others.\n\nAnother aviation data set is the National Transportation Safety Board's Aviation Accident Database, 1989-1997. This is the NTSB's version of the FAA's accident and incident database. Both agencies keep crash data but with a slightly different focus. The NTSB investigates civil aviation accidents and incidents within the United States and in international waters and has detailed information on the variables involved in the accident. The database contains information describing the aircraft, operations, personnel, environmental conditions, consequences, probable causes, and contributing factors of accidents. The NTSB also includes a narrative of the accident or incident.\n\nNICAR has just finished the 1996 update of the Center for Disease Control's AIDS database. The data set provides case-by-case demographic information about Acquired Immunodeficiency Syndrome cases reported to state and local health departments. The information can provide a detailed look at AIDS in your area, including demographic breakdowns and risk factors of those with AIDS in your community.\n\nNational Endowment for the Arts, 1987-1996, has information on individuals or organizations receiving money for arts programs, including the type of grant, the total award amount and a description of the project.\n\nUpdated Data\n\nAlong with the two new acquisitions, we've also updated the rest of our aviation data sets. The FAA's service difficulty reports (SDR), accidents and incidents, aircraft registration, and pilot licenses are all current through September 1997.\n\nAdditionally, NICAR can set up a monthly subscription to SDRs for those who want to maintain the most updated information. NASA's air safety reporting system, containing anonymous complaints by pilots and air traffic controllers, is current through August 1997 and is useful for finding near misses and problems at local airports, 1988-1997.\n\nOther transportation data sets we've updated: the National Highway Traffic Safety Administration vehicle recalls and complaints, 1966-1997, includes information about vehicles that have been recalled by certain manufacturers; Federal Railroad Administration data for accidents, casualties, and highway crossings, 1991-1996; U.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971 to 1996; U.S. DOT fatal accident reporting system, includes all roadway accidents from 1975 to 1996; and U.S. Coast Guard boating accidents, 1969-1996.\n\nThe 1996 Home Mortgage Disclosure Act database is also available. This can be used for tracking who gets loans and who gets turned down and for finding redlining patterns.\n\nUp-and-coming Data\n\nData sets in the offing include the FAA's Near Mid-Air Collisions database, which is used to record reports of in-flight incidents where two aircraft closed to an unsafe distance but avoided collision. This database serves as a description of the incident, setting, weather, intended and actual operations, evasive actions (if any) taken, location, flight profile, flight conditions, aircraft and aircrew data for the two involved aircraft.\n\nThe following NICAR databases are presently being updated: the 1996 National Bridge Inventory System data, including inspection grades, will be available in December; Occupational Safety and Health Administration violation data, including worker accidents and exposures to hazardous chemicals by companies, will be also ready in December.\n\nJustin Mayo can be reached at (573) 882-0684 or by e-mail at justin@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-get-your-training","slug":"1997-november-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-november-get-your-training.html","year":1997,"month":"November","monthNum":11,"headline":"Get your training","kicker":"Seminars, boot camps, conferences","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers various training programs including newsroom seminars, workshops, boot camps, and conferences to teach journalists computer-assisted reporting skills and data analysis techniques.","keywords":["training","journalism","workshops"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Get your training\n\nCheck out the NICAR Web site at www.nicar.org to find out times and dates for upcoming on-the-road training and boot camps.\n\nNewsroom Seminars\n\nNICAR provides specialized training in your newsroom. Learn to transfer data from government files into newsroom PCs. Build spreadsheets for insightful stories on the beat. Comprehend documents with database managers. Navigate the Internet and online databases. Cost varies. For information, call Lisa Barnes at (573) 882-8969. December 3, 5 and 8, 1997 - Columbia, Mo. On-the-road training for the Bureau of National Affairs December 15-17, 1997 - Chicago, Ill. On-the-road training for The Chicago Tribune\n\nWorkshops\n\nIRE and NICAR can also bring road-tested workshops to sites around the country. We match regional public records experts and experienced reporters with our own trainers for unique seminars ranging from one-day demonstrations and panels on regional public records laws to three-day hands-on sessions on computer-assisted reporting or advanced techniques for data-intensive beats. Don't wait for someone else to organize a computer-assisted reporting workshop in your area. IRE and NICAR depend on members, news organizations, journalism schools and other journalism organizations to help us bring our seminars to you. Contact Sarah Cohen at (301) 942-2199 or e-mail her at sarah@nicar.org December 11-13, 1997 - Washington, D.C. Computer-Assisted Reporting Workshop Sponsored by NICAR and the Medill School of Journalism January 23-25, 1998 - St. Louis, Mo. NICAR Health Care Workshop May 10-15, 1998 - Chapel Hill, N.C. NICAR Advanced Stats and Maps Workshop\n\nBoot Camps\n\nBoot Camps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo. As with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and online databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size. For information, call Katy Fanning at (573) 882-0684. January 4-9, 1998 - Columbia, Mo. NICAR Regular Boot Camp January 9-11, 1998 - Columbia, Mo. NICAR Advanced Boot Camp May 17-22, 1998 - Columbia, Mo. NICAR Regular Boot Camp July 12-17, 1998 - Columbia, Mo. NICAR Regular Boot Camp August 9-14, 1998 - Columbia, Mo. NICAR Regular Boot Camp\n\nConferences\n\nNICAR will offer training at the following professional conferences, including the IRE and NICAR national conferences. These conferences feature practical tips and story ideas shared by prize-winning journalists in quick-hit sessions. Keep up with our conference schedule through our Web site, at www.ire.org. Costs vary. For more information or to register, call Lisa Barnes at (573) 882-8969. January 31-February 1, 1998 - Washington, D.C. IRE Regional Conference March 5-8, 1998 - Indianapolis, Ind. NICAR National Conference March 27-29, 1998 - New York, N.Y. IRE Regional Conference June 4-7, 1998 - New Orleans, La. IRE National Conference","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-from-the-nicar-library","slug":"1997-november-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-november-from-the-nicar-library.html","year":1997,"month":"November","monthNum":11,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR maintains an extensive collection of federal government databases spanning multiple agencies and decades, covering topics from aviation safety and medical device complaints to campaign finance and census data, providing journalists with raw data for investigative reporting.","keywords":["databases","government","data"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\n\nNEW National Endowment for the Arts, 1987-1996, includes information on individual or organization receiving money.\n\nNEW Immigration and Naturalization Service legal residency, 1980-1995, includes information on the characteristics of aliens who were admitted as immigrants.\n\nNEW National Highway Traffic Safety Administration vehicle recalls and complaints, 1966-1997, includes information about vehicles that have been recalled by certain manufacturers.\n\nU.S. Food and Drug Administration's Medical Devices Reports, detailing complaints about drugs, pacemakers and other medical products, 1974-1996.\n\nU.S. Centers For Disease Control's AIDS database, providing case-by-case demographic information about those with the HIV virus, 1995.\n\nU.S. Census Bureau's Consolidated Federal Funds Reports, showing which communities get how much under various federal programs, 1983-1995.\n\nFederal Aviation Administration's accidents and incidents, including major plane crashes since 1974.\n\nNASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1996.\n\nFederal Election Commission campaign contributions by individuals and political action committees, 1990-1997.\n\nThe Health Care Financing Administration's 1995 database of all Medicare-funded in-patient work in U.S. hospitals.\n\nFederal Railroad Administration data for accidents, casualties, and highway crossings, 1991-1996.\n\nCoast Guard boating accidents, 1969-1994.\n\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration, 1974-1997.\n\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1995.\n\nFederal procurement data, 1992-1996, includes breakdowns by agency.\n\nAlcohol, Tobacco and Firearms gun dealer records, 1993-1996.\n\nNational Bridge Inventory System data, includes inspection grades, 1994-1995.\n\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the 1995 final report.\n\nSocial Security death records, by name and social security number, going back to 1937.\n\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\n\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\n\nU.S. Small Business Administration loan guarantees, 1980-1997. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\n\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\n\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\n\nThe National Inventory of Dams, 1991-1995.\n\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971-1996.\n\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1975-1995.\n\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\n\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-november-bits-bytes-and-barks","slug":"1997-november-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-november-bits-bytes-and-barks.html","year":1997,"month":"November","monthNum":11,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR announces upcoming campaigns on finance information, expanded seminars and conferences for journalists, including an annual March conference in Indianapolis with specialized beat-focused sessions, and invites reader contributions to improve their Uplink newsletter.","keywords":["journalism","training","data"],"topics":["Journalism & Newsroom","Government & Politics","Data & Databases"],"technologies":["World Wide Web","SQL","Access","GIS"],"full_text":"Bits, Bytes and Barks\n\nCampaign Finance\n\nThe Campaign Finance Information Center is in full swing. Data, tips, links and stories are on the center's Web site, www.campaignfinance.org\n\nAlso, coming soon to a mailbox near you will be the Campaign Finance Information Center's newsletter. Stories from the Fall '97 newsletter include reports about states with electronic filing systems, a look at how states are going to inform the public of the connection between money and politics and an explanation of in-kind contributions.\n\nConferences and Seminars\n\nNICAR is forging ahead with new seminars and new approaches to conferences and workshops. The annual conference, March 5-8 in Indianapolis, will not only include the usual panels and hands-on classes for beginners to the advanced, but also a set of special half-day sessions that will take in depth looks at how to use electronic information for particular beats.\n\nWe also are expanding our seminars for the intermediate range. Earlier this year we did a seminar on education and statistics. In the coming months, we plan to hold seminars on database design and data cleaning, use of SQL servers for the newsroom, mapping software, and business reporting. Watch our website at www.nicar.org for dates and times. For more information on seminars and boot camps, call (573) 882-0684.\n\nNICAR Net\n\nRecent topics on the IRE and NICAR listserv have included campaign finance data, troubles with traffic accident data, ethics, medical investigations and Access tips.\n\nTo subscribe to IRE-L or NICAR-L, send e-mail to listproc@lists.missouri.edu In the body of the message, type: subscribe NICAR-L<your name> subscribe IRE-L<your name> Also, check out the IRE-L and NICAR-L mailing list archives on our website at www.ire.org and www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.\n\nReader Response\n\nWe are still looking for readers' ideas of how to improve Uplink. Are there certain columns that you would like to add? Are there certain topics you would like covered? Have you done a story that you think should be featured in an upcoming issue? Do you have something you would like to contribute? A Tech Tip? A column about a CAR story you reported?\n\nHere's your chance. Send comments, critiques and suggestions to Uplink's editor, Jody Sowell, at jody@nicar.org We look forward to hearing from you.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-road-map-for-the-future","slug":"1997-decemberjanuary-road-map-for-the-future","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-road-map-for-the-future.html","year":1997,"month":"December/January","monthNum":12,"headline":"Road map for the future","kicker":"","author_name":"Brant Houston","author_title":"Executive Director","affiliation":"IRE/NICAR","summary":"Brant Houston discusses the future of training journalists in computer-assisted reporting, addressing past challenges and proposing a roadmap for the integration of advanced CAR into newsrooms.","keywords":["training","journalism","technology"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web","spreadsheet"],"full_text":"After nearly four years and more than 175 seminars and conferences, the staff of IRE and NICAR have been assessing what the next step should be in training journalists in computer-assisted reporting. We know that the demand for instruction in the basics effective use of the Internet, spreadsheets and database managers - has not lessened. But we also know that there are hundreds of journalists looking for a higher level of instruction and hundreds of managers and editors trying to figure out how to blend CAR into their newsrooms. When IRE and NICAR began its on the road training at the beginning of 1994 we were confronted by a set of problems that now seem to belong to another century (and soon will). There was no real curriculum or textbook for computer-assisted reporting. Managers expected reporters to learn everything they needed to know about CAR in one day. Reporters assumed they could learn CAR, but avoid math. Many news organizations had no personal computers for their newsrooms and many reporters did not know how to use a mouse. There was no point and click, visually pleasing World Wide Web. The word 'database' was an unfamiliar term to most reporters and editors. The idea of training had only recently come into fashion at most news organizations. As a result, some of our seminars were, at best, a show-and-tell about the electronic world that awaited journalists once they had the equipment, software and databases. Other seminars created frustration because reporters would get a jump-start on using spreadsheets and database managers, but then have to return to a newsroom where there was no time or support for them to continue to learn. While we still teach journalists who have not used a mouse and/or searched the Web, we find that most reporters, editors and news directors are much more aware of the power and potential of electronic information. A journalist with superior computer-assisted reporting skills is assured of a good-paying job and the closest thing to job security in our profession. Nonetheless, we still hear about the hurdles that journalists encounter in newsrooms when trying to take their new skills and turn them into stories. The reporters are frustrated because they sometimes work for managers who still think one day on a keyboard and mouse should be enough. There seldom is a plan for follow-up training. Access to equipment and the Internets are not always guaranteed. In small to medium organizations, the reporters work alone and have no one to talk to about CAR problems. The managers are frustrated because they see reporters spending inordinate amounts of time jockeying with software. The managers don't see enough examples of using electronic documents and databases to save time and produce better stories. The managers seldom get training for themselves. However, none of these problems is insurmountable. Over the past few months, NICAR trainers have been talking with other instructors and journalists who have completed NICAR seminars. Slowly a consensus has emerged. While the plan is not final, it is a start: First, we will continue the basic seminars for those journalists just getting the opportunity to learn CAR. Second, we'll concentrate much more on relating CAR training to particular beats. We've already done some of those kinds of seminars. In the coming year, we will focus on areas such as health care, crime, education, economy, business, environment and city hall. We hope that these seminars will show reporters and editors how to get faster results from training, even if it's only adding a key paragraph exposing an official's spin, finding a source or getting a number or salary that rounds out a story. Third, we'll offer special one-day sessions for editors and newsroom managers who don't have time to learn the details, but need to know how to get a handle on using the new technology for stories. Fourth, we'll work at improving the sophistication of journalists when dealing with numbers so they can avoid the pitfalls of using CAR or statistics carelessly. Fifth, we'll offer seminars on specialized topics, particularly on mapping and visualizing data and on how to write or broadcast stories that aren't buried in numbers. And sixth, we won't forget the leaders and pioneers in CAR in newsrooms who are trying to develop ways to make databases and electronic documents easier to use. In fact, in January we will hold our first advanced seminar on those skills. A look at our program for our national conference this year, Indiana CAR, on our Web site shows our efforts to move in this direction. In the coming months, our Web site also will carry more and more tutorials and tips on everything from learning the basics to covering beats to mastering advanced CAR. As the year progresses, please let us know what is working and what is not and we'll change whatever we need to. Brant Houston can be reached at (573) 882-2042, or send e-mail to brant@ire.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1997-decemberjanuary-error-free-goof-proofing-car","slug":"1997-decemberjanuary-error-free-goof-proofing-car","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-error-free-goof-proofing-car.html","year":1997,"month":"December/January","monthNum":12,"headline":"Error Free Goof-proofing CAR","kicker":"","author_name":"Ronald Campbell","author_title":"reporter","affiliation":"The Orange County Register","summary":"A journalist recounts nearly making a critical mathematical error in analyzing mortgage data, revealing how computer tools can mask journalists' fundamental weakness with numbers and statistical analysis.","keywords":["mathematics","journalism","error"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["FoxPro"],"full_text":"Confession is good for the soul. Last May, I very nearly made the most embarrassing error of my 21-year career. The only thing that saved me from printing the error was a skeptical remark from one expert. The only reason I made the mistake in the first place was because I got careless using a computer. Computers have done wonderful things for journalism. At the same time, however, computers have exposed a basic flaw in journalists: We are, most of us, terrible at math. We became reporters because we couldn't cope with chemistry or economics or, God save us, engineering. On our worst days we struggle to remember how to calculate a percentage or a percentage change; some of us don't know the difference.\n\nYet computers invest us with the illusion that we have mastered numbers. Senator Sleeze collected $5,000,000 from 20 tobacco-industry groups; the computer says so. The crime rate is higher in North Palooka than in Detroit; the computer says so. The city's five top banks made no loans in the inner city; the computer says so.\n\nIn fact, the computer has said none of those things. We have said those things, and any of a number of errors could have led us astray.\n\nError #1:\n\nNominating the wrong denominator. Every percentage implicitly is a comparison of one value with another. Pick the wrong denominator (the number by which the numerator is divided), and chaos results.\n\nPerhaps the most common example of this error in CAR comes each year with the release of FBI crime statistics. Crime rates are calculated by dividing the number of crimes in a city, county or state by the number of people living there; this number is then recast as a rate per 10,000 or 100,000 to allow easy comparison among cities or counties. The problem comes when you've got, say, 1990 population estimates for one city and 1995 population estimates for another.\n\nI stumbled into a subtle version of the same mistake. While analyzing Home Mortgage Disclosure Act data for California last spring, I decided to calculate each major lender's market share of the loans originated for whites, Hispanics, Asians and blacks. After a few hours of coaxing these numbers out of FoxPro, I reached a stunning conclusion: While the top 10 lenders originated 30-40 percent of all mortgages to whites, Hispanics and Asians, they originated just 13 percent of all loans to blacks.\n\nAs a result, my analysis suggested, blacks were unusually dependent on expensive \"sub-prime\" lenders.\n\nLet me pause for a moment and admit that after four or five years of HMDA diving, I had grown quite bored with my annual mortgage story. Here, then, was a discovery worth savoring: a national story buried in a familiar piece of data. Every expert with whom I broached the subject — academics, analysts, even bankers — expressed surprise and enormous interest.\n\nAnd then I ran across an expert at the Mortgage Bankers Association, an economist who spent even more time obsessing on HMDA data than I did. He didn't believe my results. With a certain amount of self-assurance, I offered to redo the analysis while he waited. Within a minute or so I had my numbers: My original analysis was flat-out wrong; the top 10 lenders had the same share of black mortgages as they did of whites, Hispanics and Asians.\n\nLater, I redid my calculations; as near as I could tell, my original mistake occurred because I had divided the top 10 lenders' total volume to black borrowers by the overall loan volume to Asians - the group I had analyzed just before doing the analysis of lending to blacks.\n\nError #2:\n\nWhat's less than nothing? The answer, of course, is nothing. But with just a bit of work you can calculate a percentage change showing that a quantity has dropped 150 percent - a remarkable feat, but one that routinely makes it into the newspaper. The easiest way to do this is to compare a negative and positive number, as when a company reports a profit after a string of losses.\n\nWhile preparing this article, I posted a message on the NICAR list, inviting anonymous confessions. Here's one:\n\n\"Early in my use of spreadsheets, I did a percentage change formula backwards for a 50-state list. The scary part: It sailed right past two copy editors, even though it said some states' pollution had dropped by more than 100 percent. Now I hand-check formulas on a calculator, and people in our group double-check each other before we release calculations.\"\n\nError #3:\n\nComparing the incomparable. A murder is not the same as a car theft or a robbery. So why does anybody ever compare cities' crime rates without at least breaking down the rates between violent and non-violent crime? And why does any editor sit still for such journalistic malpractice?\n\nError #4:\n\nMissing the obvious. I hadn't intended to write a screed about the FBI Uniform Crime Report, but my data is leading in that direction. Another anonymous confession:\n\n\"I was doing the FBI Uniform Crime Report thing, ranking California cities by crime rate. When I was all done, I started calling all the usual suspects for reaction. One cop-management guy said, 'hmm,' or something, then 'What's the top 10?' I read them. He said, 'hmm,' again, then 'Where's Oakland?'\n\n\"Well, the long and short of it is that, as you probably can guess, Oakland didn't get its data in on time and wasn't on the list, and of course the FBI doesn't bother to point this out.\"\n\nSo what is a conscientious reporter to do? It seems to me there are a few good answers:\n\nFirst, if any finding takes your breath away, then breathe in again, hold your breath for a good 30 seconds, let it out - and recalculate your finding from scratch. Do it slowly. Look at your own numbers the way you look at a particularly squirrelly pol's conflict of interest form. Second, widely share your findings before publication. Third, don't use statistical methods that you can't explain while nursing a hangover. I am very comfortable with percentages, percentage changes, medians, means, modes and margins of error. But I seriously doubt that I could explain a chi-square or another sophisticated measure of statistical significance under the best of circumstances. So I avoid using them in my research. I might use them someday - after I've finally finished reading those two big statistics books I've got in my \"to read\" pile at home.\n\nRonald Campbell can be reached at (714) 664-5030 or by e-mail at Ron_Campbell@link.freedom.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-beginners-lesson-in-teaching","slug":"1997-decemberjanuary-beginners-lesson-in-teaching","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-beginners-lesson-in-teaching.html","year":1997,"month":"December/January","monthNum":12,"headline":"Beginner's lesson in teaching","kicker":"CAR Trips: First ventures","author_name":"Charles N. Davis","author_title":"media law instructor","affiliation":"Southern Methodist University","summary":"A first-time computer-assisted reporting instructor shares lessons learned from teaching the course, including the importance of staggered deadlines, building student enthusiasm through examples, and addressing mathematical knowledge gaps.","keywords":["deadlines","teaching","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet"],"full_text":"When my new course, \"Computer-Assisted Reporting and Research,\" met for its final session, my students were in full panic mode, frantically scheduling interviews they should have done two weeks before and double-checking spreadsheets that should have been finished a month earlier.\n\nWhat happened? How did all those weeks go by so fast? And why didn't my students do more of the reporting work earlier in the semester? Easy. I didn't require them to.\n\nStaggered deadlines - having a rough draft due one week, requiring that all primary source interviews be done the - would have saved my students a lot of stress and made my grading life a lot easier. It was one of several important lessons learned by this first-time CAR instructor, who will be fine-tuning his syllabus in the coming months.\n\nHere are the major lessons I learned this semester, in no particular order:\n\nSparking enthusiasm\n\nGet your data at least six months in advance.\n\nUndergraduate students at most institutions don't know what don't share your evangelical glee about doing CAR.\n\nUpon my arrival at Southern Methodist University, I was approached by my departmental chair, a forward-thinking type who saw the obvious need for a computer-assisted reporting course. I'd done a bit of CAR dabbling in my previous life as a financial reporter, so I eagerly volunteered. A grant sent me to NICAR's CAR Boot Camp, where I saw the light.\n\nI spent the summer honing lectures, developing spreadsheet and database exercises and lining up guest instructors from newspapers and television stations. I even invited a private investigator who uses the Internet to track people and assets for a visit. I was pumped. I was ready.\n\nMy students were not.\n\nThe first week, barely half of them showed up. My speaker was uninspired, and I was mad. But then it hit me: in my glee, I had neglected to give my students context. I had scarcely defined CAR, its importance to the reporter, its ability to break great stories. I forgot to start at the beginning, simple as it sounds.\n\nSo I made some changes on the fly.\n\nI made the second week of class a sort of CAR \"show-and-tell.\" Using a CAR videotape of award-winning IRE stories (a bargain at $20), I showed examples of CAR stories. I passed newspaper stories out, and had the students identify the CAR technique used and explain why the stories were newsworthy. This led to a great discussion of story ideas, and before I knew it, my students were fired up and ready to go tell stories.\n\nAnd, just to make sure the enthusiasm lingered, I also came up with some type of homework or in-class assignment for every week the rest of the semester to keep them in the classroom.\n\nMath lesson\n\nStudents don't know math. I will now devote a lecture early in the semester to mathematics. My students had to be reminded how to average! I was stunned, until I thought about my own liberal arts education, when I deftly avoided all math courses.\n\nEarly bird data\n\nAs a media law instructor who conducts research on freedom of information issues, I knew that the struggle for government information would be time-consuming and frustrating. I often made public records requests as a reporter, so I started early in the summer seeking data for my exercises and for student projects I had in mind.\n\nFortunately, I made one wise decision before starting the data hunt: I came up with twice as many data requests as I had students. That way, I could fail half the time and still have plenty of data.\n\nAs predicted, it was a long, frustrating process that left me at times wondering whether I was a participant in a democratic government or a meddling outsider. But I got enough data for the semester, and it became critical to several course assignments.\n\nStudent-led teaching\n\nI have always been a vocal proponent of letting students voice their opinions. My students in CAR were all senior journalism majors. They were eager enough to report, but didn't know what they wanted to report on yet. So the first few assignments were directed toward story origination. I wanted them to find quality story ideas utilizing Internet research and/or government data. They found dozens of great ideas, many self-generated. Some ideas were rejected, others were traded among students, but in the end everyone had a couple of great stories to chase.\n\nProfessional context\n\nLean heavily on the professional community. When I began planning my inaugural CAR course, I knew I had allies. I spent the summer as a visiting professor at The Dallas Morning News, where I learned some data analysis first-hand with Shawn McIntosh. Shawn helped me with the course plan, with my student's projects, and with actually convincing me that I could pull it off. She was my first guest lecturer, wowing my students with a day-by-day account of how reporters used the Internet, dial-up databases and other techniques to chase the McVeigh story in the days after the Oklahoma City bombing.\n\nSheryl Rubinett, WFAA-TV's news researcher, came to class bearing dozens of great examples of how CAR has transformed broadcast reporting.\n\nNot every journalism professor has such a rich media market to lean on, and some instructors will have to look as far as a neighboring state to find an expert. But don't hesitate to call upon the professional community. In fact, the unwavering support of the pros has been one of the most rewarding parts of teaching the class. NICAR, CAR-L, IRE and FACSNET all provide a great deal of support. I recommend them all to aspiring CAR instructors as a source of expert advice and good classroom exercises.\n\nDon't be scared away\n\nCAR is important. So important, in fact, that if no course exists in your program, step right up and volunteer. It appears daunting, but if you don't teach it, who will? Am I a CAR expert? No. Not even close. But neither are my students. We're getting better at spreadsheets and databases every day, with the students often teaching me. It's been an incredible learning experience for me, and it's rekindled my love for reporting.\nCharles Davis can be reached at (214) 768-2775 or send e-mail to cndavis@post.cis.smu.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-what-teaching-is-really-like","slug":"1997-decemberjanuary-what-teaching-is-really-like","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-what-teaching-is-really-like.html","year":1997,"month":"December/January","monthNum":12,"headline":"What teaching is really like","kicker":"Lesson Plan","author_name":"Jeff South and Steve Doig","author_title":"Professors","affiliation":"Virginia Commonwealth University and Arizona State University","summary":"Two former journalists who became university professors describe the demanding reality of academic teaching, revealing that despite its appealing reputation, professorship requires as much work and stress as newsroom journalism, with extensive preparation needed beyond classroom hours.","keywords":["teaching","journalism","professors"],"topics":["Education","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"What teaching is really like\n\nBy Jeff South and Steve Doig, Virginia Commonwealth University and Arizona State University\n\nIt sounds so tempting, to ditch the frenzied pressures of the newsroom for the sedate, ivied halls of academia.\n\nTeach a few courses, spending perhaps 10 hours a week in classrooms filled with eager journalists-to-be. Be part of a respected institution of scholars who prize your research and technology skills. Take summers off, but enjoy a year-round salary. And do all the freelancing and consulting you can - it's considered a credit to the university.\n\nThat glowing job description is legendary. Its allure has encouraged some of the number-crunching propeller-heads among us to consider new careers as journalism professors, teaching computer-assisted reporting. After all, anyone in a newsroom who can calculate percentage change reliably already has been nicknamed \"professor.\"\n\nBut before you surrender to this warm and fuzzy vision of leisurely semi-retirement, take a reality check. Teaching, at least teaching well, is every bit as demanding and harrowing as analyzing and mapping election-night precinct returns on deadline. University life can be as frantic and challenging (and political) as life in a newsroom. And colleges had more on-the-job fatalities than newspapers did in 1995. Oh, and about those \"10 hours a week\"? For each hour in the classroom, plan on spending four or five more preparing your lessons - and praying you won't fall flat on your face like a bad comic on open-mic night.\n\nNot that we want to discourage you. As longtime journalists who recently became professors, we have no regrets about joining The Academy. We see the rewards every day: Students buzzing with energy and enthusiasm learning the skills needed to help the news media survive and prosper in the 21st century.\n\nBut when we switched careers, we had few clues beyond tweed jackets and a whiff of chalk dust what a professor's life would be like. We want to share lessons learned since then, and for You, too, soon may face similar career-change choices.\n\nAs more and more journalism schools realize the importance of computer-assisted reporting, they will be encouraging, even recruiting, CAR-adept reporters. The authors of this article illustrate the trend: We each spent two decades in newsrooms, specialized in CAR, then recently became full-time professors. Here are some things we learned that you should know if you, too, are considering making the jump from newsroom to classroom:\n\n1. Teaching is hard work.\n\nYou're often barely 24 hours ahead of your students, madly grading their work from the last class and trying to create meaningful lessons for the next. Reporters-turned-professors typically spend more time each week doing their new jobs than they had spent doing their old ones.\n\nMany of us learned writing in the newsroom, where a better writer or a tough editor coached us one-on-one. In a typical journalism class, you'll have 20 students who need that kind of tough-love coaching.\n\nIf you don't invest the time, students know they're being cheated - and they'll be rightfully angry: They're losing far more than the cost of a newspaper.\n\n2. Being an expert is necessary - but not sufficient - to make you a good teacher.\n\nWhen standing before a roomful of students, you have to be as much a performance artist as fount of wisdom. You must learn how to engage students, generate discussions, anticipate questions, set and enforce rules, explain, listen, explain again. Remember those boring lectures back in college where the professor droned on and on? Today's students don't like them either.\n\n3. Though we both think that working with students is our most important task, teaching still is only one-third of a professor's functions.\n\nWe're also responsible for research/scholarship, and for service. Research can be doing full-fledged academic studies, developing new training materials, publishing an occasional article, and so on. As for service, we are expected to work with journalism professionals and organizations, advise student groups and publications, and serve on myriad committees. Often, you need to spend about 50 percent of your time on each of those three functions.\n\nAs a CAR expert, you are likely to be appointed to a slew of university advisory committees: on curriculum development, technology, faculty recruitment and other subjects. Many weeks, you'll spend more time in committee meetings than in the classroom. As a faculty member, you also spend considerable time advising students. Sometimes it's on official matters like selecting courses, graduating and getting a job. In addition, you'll often be asked for advice on unofficial topics ranging from immigration problems to apartment hunting to help with a problem in some other class. Walk-in visits from students are fun but time-consuming.\n\n4. Students and reporters have different learning styles.\n\nAs the CAR wizard in your newsroom, you were used to working with reporters focused on learning a particular computer skill so they can get their story in tomorrow's paper. But to journalism students, you must teach a continuum of skills and show how they fit together and could apply to any number of stories. Reporters are quick to make the connection between data crunching and the story. They don't need much direction after you've run that list of school bus drivers against the list of drunken driving convictions. But students inexperienced in journalism may have trouble seeing where the story is headed. As a professor, you'll spend a lot of time helping students conceptualize stories, not just SQL queries.\n\n5. Students defy the stereotypes.\n\nThey're not unmotivated slackers snoring away in the back rows. They're not necessarily computer whiz-kids, either.\n\nThese days, many students work part-time or even full-time jobs. Some are single parents. And they're taking three, four or five other courses as well as yours.\n\nMost aren't willing to learn by rote, simply swallowing what they're told without challenge.\n\nClassroom discussions on ethics, diversity, civic journalism, invasion of privacy and other topics often are as interesting and heated as any we've heard in a newsroom.\n\nCompared to a newsroom, a journalism classroom generally is filled with a much wider range of abilities and weaknesses.\n\nYou'll have some students who haven't learned to conjugate verbs properly, and a few who can write better stories and CGI scripts than you can. It takes time and thought to tailor lessons, assignments and grading strategies for such a diverse mix.\n\n6. Just as newspapers still struggle to mainstream CAR into the reporting process, journalism schools are struggling with how to put it into their curriculum.\n\nThere's no widely-accepted model of how CAR should be taught: Stand-alone courses, integrated into existing courses, or some hybrid. As the new CAR expert on a university faculty, you may be called on to build a program from scratch.\n\nAlso, you may be surprised at the limits on what you can teach. Academic freedom is very real, but your dean will decide what courses you will handle, based on the needs of the school. You can't just barge in, whip together a course in spreadsheet reporting, and require students to take it. Because of accreditation requirements, journalism students take only a fourth of their credits in journalism.\n\nBy the time they have taken the mandatory courses, they'll have room for very few electives. That's why you start looking seriously at ways to work CAR concepts into other existing courses.\n\n7. While some parts of the university may be on the cutting edge of technology, journalism schools often are lagging.\n\nThey've computers, mostly Macs, largely for newspaper design. As a professor, you may be hard-pressed to find the CAR hardware and software you used back in the newsroom. Moreover, school budgets often are tighter even than those in the newsroom; getting that faster CD-ROM, that software upgrade, even that trip to NICAR's next data extravaganza, may require some blood, sweat and tears.\n\nAgain, we hope this doesn't scare you off. The work may be considerable, but the satisfactions are immense. And you'll find it stimulating to learn and practice these new skills at mid-career. If you bring from the newsroom the same dedication to knowledge and pride in doing good work that you practiced there, you'll make a happy - and successful - professor.\n\nSteve Doig can be reached by e-mail at steve.doig@asu.edu. Jeff South can be reached by e-mail at jcsouth@vcu.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-drivers-education","slug":"1997-decemberjanuary-drivers-education","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-drivers-education.html","year":1997,"month":"December/January","monthNum":12,"headline":"Driver's education","kicker":"On the Internet","author_name":"Nora Paul","author_title":"","affiliation":"The Poynter Institute","summary":"A guide to free online resources and tutorials for teaching Computer-Assisted Reporting (CAR) techniques in newsrooms, covering topics from general CAR skills to statistics, databases, spreadsheets, and internet research.","keywords":["training","journalism","resources"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Excel","Lotus 1-2-3","Quattro Pro","FileMaker Pro","Access","Internet","World Wide Web","FTP","Email"],"full_text":"As Jerry Garcia said, \"Somebody has to do something and it's just incredibly pathetic it has to be us.\"\n\nIf you are feeling that way because you have to do something about CAR training in your newsroom, don't get all pathetic, there is a lot of help out on the Internet.\n\nI'm going to break down the types of CAR techniques and resources you might be needing to do training on and will list a few places with good tutorials, handouts, or explanations that you can use.\n\nGeneral\n\nFree Academy http://www.freep.com/jobspage/academy/index.htm The Detroit Free Press FREEP Academy page has a great listing with links to handouts on topics from getting started with computer-assisted reporting to transportation stories to cleaning computer data to why you should know financial journalism. A very nice resource, written by and for working journalists. Thanks, FREEP Academy Dean Joe Grimm.\n\nComputer Assisted Journalism Front Page http://www.nyu.edu/pages/journalism/caj/home.htm\n\nJohn Garcia, formerly a professor at NYU, put this guidebook together as a teaching aid for his classes.\n\nIt will help you in yours, too. He's put together great reporting exercises and examples.\n\nStatistics\n\nStatistics Every Writer Should Know http://nilesonline.com/stats/ Having to teach about mean versus median (when you don't always keep them straight yourself). This tutorial on statistics is geared for journalists. Here's your manual, but contact Robert Niles at robert@nilesonline.com before using it.\n\nAs he says on the site, \"Hey, since he went to the trouble of making this site, it's the least you can do.\"\n\nNews & Numbers: Help for placing \"scientific\" and statistical claims in perspective. http://www.facsnet.org/cgi-bin/facsnet acgi?alpha=Nwr5r.67ujU5E1 Victor Cohn, former science editor at the Washington Post, wrote these chapters, which should be the required reading of anyone teaching, or taking, a class on numeracy.\n\nDatabases\n\nDatabases Basics http://www.indiana.edu/~ucspcc/training/db/database.html A brilliant tutorial on databases, their use and their construction, by Indiana University. It is an Adobe Acrobat file, so you will need the reader installed on your computer to get it. Look, too, at the files on Filemaker Pro and Access.\n\nSpreadsheets\n\nExcel training basics http://www.indiana.edu/~ucspcc/training/db database.html This tutorial, also from Indiana University, includes sample files you can use in your teaching.\n\nThe Spreadsheet Page: http://www.j-walk.com/ss/\n\nAn incredible collection of information about spreadsheets with specific links to pages with information about Excel, Lotus 1-2-3, Quattro Pro, and other programs.\n\nAnd to liven up those training sessions, there is even a section for spreadsheet jokes.\n\nInternet Overview\n\nNora Paul's Computer-Assisted Research: a guide to tapping online information http://www.poynter.org/car/cg_chome.htm OK, I'm allowed to put mine first, aren't I? Anyway, this guide has chapters on the main tools of the Internet and an overview of the uses of computer-assisted research techniques in newsgathering.\n\nThe structure focuses on the whats and the whys of using these resources.\n\nMike Wendland's Wired Journalist http://www.rtndf.org/programs/century/wired/wjforward.html With Mike's focus on broadcast journalists' needs, this book is a terrific guide to the uses of the net for journalists. For you trainers, his chapter V, Training Exercises, is a life-saver with real life-like reporting assignments for which the Internet can provide information.\n\nE-mail/Newsgroups/Listservs\n\nThe Reporter's Guide to Internet Mailing Lists http://www.daily.umn.edu/~broeker/guide.html Timothy Broeker, in nice clear language, covers all the various aspects of Mailing List use. It's focus on use in journalism is particularly helpful.\n\nEverything E-Mail\n\nhttp://everythingemail.net/discussion.html Mary Houton-Kemp's Everything E-mail is just what it is billed, EVERYTHING e-mail! If you are doing a class on e-mail use, this is your one stop shop. There are e-mail tips, a glossary of terms, finding e-mail addresses, getting news by e-mail. The listing of resources about listservs, how to find them, etc. is very useful and great handout material.\n\nSearching the World Wide Web\n\nResearching on the Internet http://www2.dgsys.com/-editors/articles excerpt.html The Internet Newsroom's Tim Maloy put this quick overview of ideas and tips for searching the net together, it would be a good quick read for starting web search instruction sessions.\n\nFinding Data on the Internet http://nilesonline.com/data/ This is a terrific site, particularly the compilation of \"Links to Potential Story Data\", compiled by Robert Niles. Check it out.\n\nFinding Information on the Internet: A Tutorial http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/FindInfo.html\n\nThe librarians at Berkeley have put this very comprehensive self-teaching guide together. Why not try doing some virtual training, have everyone, on their own time, step through this tutorial and then meet for a brown bagger discussion of what they encountered along the way.\n\nEvaluating information on the web\n\nICYouSee A to Z: T is for Thinking http://www.ithaca.edu/library/Training/hott.html\n\nWritten by the librarians at Ithaca College, this site gives a good overview and some directed exercises for looking at and evaluating information found on the World Wide Web.\n\nEvaluating Web Resources http://www.science.widener.edu/~withers/webeval.htm\n\nWritten by librarians at Widener University, this is the ultimate website for critical thinking about evaluation of information. There is even a PowerPoint program: The Web as a research tool: evaluation techniques. The handouts on evaluating different kinds of web sites (advocacy, news, informational, etc.) are great.\n\nThe Online Journalist: Part 1: How \"good\" is information on the Internet? Part 2: What's in a Domain Name? http://www.facsnet.org/cgi-bin/facsnet.cgi?alpha=Nwr5r.67ujU5EL Randy Reddick, of FACSnet and author of the Online Journalist, put these two chapters on the website. They are good explanations and would be wonderful handouts, for helping journalists learn how to check the credibility of information and sources they find on the Web.\n\nNora Paul can be reached at (813) 821-9494, or send e-mail to npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-broadening-a-car-program","slug":"1997-decemberjanuary-broadening-a-car-program","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-broadening-a-car-program.html","year":1997,"month":"December/January","monthNum":12,"headline":"Broadening a CAR program","kicker":"IRE/NICAR Notes","author_name":"Dave Gulliver","author_title":"CAR coordinator","affiliation":"Dayton Daily News","summary":"The Dayton Daily News expanded its use of computer-assisted reporting by hiring a CAR resource and providing training to reporters, enabling them to quickly analyze databases for stories ranging from military doctor qualifications to local campaign finance and crime trends.","keywords":["databases","reporting","training"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Excel","FoxPro","SQL","9-track tape","FTP"],"full_text":"Finding the military's problem doctors took a year for a veteran reporter. Finding problem roads and out-of-town campaign contributors took some computer-assisted reporting novices just a few days.\n\nIn October the Dayton Daily News published a series on how the military can hire unlicensed and unqualified doctors, and how they are rarely held accountable for errors. The series was picked up in papers coast-to-coast and on TV's \"20/20.\"\n\nReporter Russell Carollo, a computer-assisted reporting veteran, spent more than a year finding and acquiring military and state medical databases, then cleaning them and creating more of his own.\n\nThe result was another chapter in the paper's history of long, comprehensive CAR projects. But until recently databases hadn't been a part of regular news coverage, and only a handful of reporters had the know-how.\n\nIn July the paper set a goal of getting more computer-assisted work into smaller projects and everyday reporting.\n\nThe first step was hiring me, to have a newsroom CAR resource. That role entails some heavy lifting, teaming up with reporters on projects, and general cheerleading on computer use.\n\nThe second step was bringing in a veteran for some basic training.\n\nMy background is on PCs and this is a Mac shop, so Griff Palmer of The Daily Oklahoman came in for a three-day Boot Camp.\n\nHe ran four reporters through some basic training, and then he and I worked on advanced SQL for data cleaning and table-building.\n\nUsing CAR every day\n\nWith only an occasional pointer, schools reporter Lynn Hulsey has been using Excel and FoxPro in much of her everyday work. Last week she used Excel to analyze the differences between education plans proposed by the governor, Senate and House.\n\nIn September, we imported state proficiency test data. She analyzed it for a next-day story, then did more analyses for a weekender.\n\nNow, with several tables built, she uses it as a resource in daily work.\n\nNext step: Linking test performance data with spending data.\n\nCity hall reporter Laura Bischoff wanted to check tips about the police department's response time to 911 calls. She negotiated for the dispatches - 53,000-plus of them - and we imported them from a nine-track tape.\n\nThat analysis only led to another question, one that requires the record of all received calls. We expect to receive those records in December.\n\nBut as we worked on that project, the interim campaign finance reporting deadline arrived in the mayoral race.\n\nWe had nine days to analyze the records before the election.\n\nIn the past, the paper had not tried to do more than a cursory analysis. This year, we went for a broader story.\n\nLaura, one of the more computer-savvy reporters, built her own database structure. With help from clerks, we inputted the bulk of some 1,400 records. In four days we had turned a story in four days showing how money from affluent suburbs figured in the race.\n\nFinding the trends\n\nTrends reporter John Keilman and I wanted to look at crime patterns in Dayton and some two dozen nearby cities and towns. So we put FBI UCR data and Census population figures into Excel, and assembled per-capita and percentage change tables for the region.\n\nThat sparked John's interest, so he sought out traffic accident data on a rapidly growing adjacent county.\n\nWe downloaded the 25,000 records from a state FTP site, then imported it into FoxPro.\n\nWith some careful sorting and traditional reporting he was able to pinpoint some trouble spots and surpass the findings in a state report that came out just as we finished the project.\n\nTwo investigative reporters, Jim Bebbington and Steve Bennish, already had some database experience.\n\nBut the three-day in-house boot camp has them aggressively pursuing major state databases for projects.\n\nBusiness reporter Mike Drummond also soloed on a project using FoxPro to figure CEO salaries. Even some longtime reporters are seeing the trend and getting information off the Web faster than they could get it otherwise.\n\nAs the year ends, that's at least a half-dozen projects finished or nearly so, and they've resulted in everyday reference tools for various beats.\n\nThe Dayton Daily News is fortunate to have good technical resources.\n\nBut the key to our progress was in people. We brought in a pro to teach, creating some enthusiasm and showing a commitment to CAR.\n\nWe have a point man to help reporters past sticky problems and point them to resources and ideas, and who has learned enough to train more reporters in the basics.\n\nMost important, we gave people some time to learn and practice. It didn't keep them from working: No reporter took a whole day off.\n\nBut they got at least half-day blocks over a few days, time for them to develop and practice skills.\n\nAnd as they begin to turn more routine stories into full-fledged quick projects, more of their colleagues are going to be looking to them for pointers, or maybe looking for a training session of their own.\n\nDave Gulliver can be reached by e-mail at Dave_Gulliver@coxohio.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-military-hospitals-report","slug":"1997-decemberjanuary-military-hospitals-report","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-military-hospitals-report.html","year":1997,"month":"December/January","monthNum":12,"headline":"Military hospitals report","kicker":"Medical Malpractice","author_name":"Brent Johnson","author_title":"","affiliation":"NICAR","summary":"A Dayton Daily News investigation reveals serious malpractice and safety issues at military hospitals, where inadequately licensed doctors operate with minimal oversight and legal accountability due to systemic loopholes that protect them from lawsuits and state regulation.","keywords":["malpractice","military","hospitals"],"topics":["Investigative Methods","Health & Environment","Government & Politics"],"technologies":[],"full_text":"Military hospitals report\n\nBy Brent Johnson\nNICAR\n\nThe teenage daughter of an Army hospital nurse in Colorado underwent a routine exploratory procedure to determine whether her severe abdominal pain was due to appendicitis or a tubular pregnancy, but emerged permanently crippled in her right leg after a main artery was punctured by a doctor not licensed to practice in the state. That doctor, who had a history of malpractice allegations and disciplinary actions against him, was approved for transfer to the hospital by its chief of gynecology, who himself had a history of allegations and disciplinary actions against him, because the department was short-staffed.\n\nIn Washington, D.C., a Navy lieutenant who had an abnormal grouping of veins and arteries at the base of his brains now suffers from memory problems, mood swings and sleep disorders caused by an unsuccessful catheterization attempt that tore an artery. The new problems have kept the lieutenant from achieving the mandatory promotion needed to continue in the service. The internationally known doctor at a nearby university hospital where the lieutenant was transferred was later unable to catheterize the same artery that had healed but was by then almost completely closed. The doctor did make the following reflection: \"There are two kinds of doctors in the military. They're either fabulous, wonderful doctors, or they're spectacularly incompetent.\"\n\nIt is that spectacular incompetence that a recent series in the Dayton Daily News detailed painstakingly. The series, led by reporters Russell Carollo and Jeff Nesmith and editor John Erickson, was more than a year in the making.\n\n\"Whenever you take on the military, you have to have time on your hands,\" Erickson said. \"The military tries to drag out requests for public records.\"\n\nMalpractice loopholes\n\nThe reporters documented a secretive atmosphere destined for trouble. Military doctors are not required to have either malpractice insurance or licenses from the states in which they practice. By law, they cannot be sued. The defendant in malpractice claims is the U.S. government, not the doctor. Furthermore, claims cannot be filed by active military personnel or personnel stationed overseas. They conclude that the military is \"a magnet for problem doctors,\" a fact only made worse by recent downsizing.\n\nThe team used more than a dozen databases to lead them to the appropriate records and interviewees that shaped the series on care provided by the 115 hospitals and medical centers and 471 clinics that comprise the Military Health Services System. Some databases were acquired only after a federal judge ordered them released by the military; some were built by the newspaper itself based on information from state medical boards; and some were updated and combined from two prior series on the military.\n\nThe Dayton Daily News also went to court to obtain the Department of Defense's TORT2 database, which identifies individual doctors, their qualifications and their links to medical malpractice claims. The paper still has not acquired it.\n\nSimilar records identifying civilian doctors with malpractice claims are required by states and available to the public.\n\nThe National Practitioner Data Bank, too, is used daily by hospitals, HMOs, insurance companies and state medical boards for background checks. But the newspaper found, for instance, that from 1994 to 1995, though the Army paid $66 million of taxpayer dollars in malpractice claims, only one of its doctors was reported to the Data Bank.\n\n\"The military employs about 18 of every 1,000 doctors in America, yet it accounts for just two of every 1,000 malpractice reports in the database,\" Carollo and Nesmith wrote.\n\nThe reporters also discovered that the military's own confidential database contains nine times the number of malpractice cases it has reported to the Data Bank.\n\nUnable to force its release, Erickson said the newspaper instead used the tort databases maintained by each branch of the military.\n\nThose databases do not include the name of the doctor involved as in the TORT2 database but do include the name of the attorney filing the claim and a synopsis of each case.\n\nSorting through the records for leads on the most egregious examples, he said the team would, for instance, isolate claims of brain damage and see whether they occurred at the same base or involved the same attorney. Erickson stressed the importance of creating your own databases. By entering state medical board information, \"we were able to supplement what little we got from the military.\" Erickson said the series has generated a strong response. The newspaper has received hundreds of calls from people wanting to tell their stories. \"The reason is that the problem is there. It's a bad system and many think they're mistreated.\"\n\nBrent Johnson can be reached at (573) 882-0684, or send e-mail to brent@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-taming-crime-statistics","slug":"1997-decemberjanuary-taming-crime-statistics","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-taming-crime-statistics.html","year":1997,"month":"December/January","monthNum":12,"headline":"Taming crime statistics","kicker":"Crime Counts","author_name":"Jere Downs","author_title":"Staff Writer","affiliation":"Philadelphia Inquirer","summary":"The Philadelphia Inquirer discovered errors in Pennsylvania crime statistics after initially misreporting a 17 percent crime decrease in Delaware County, leading them to conduct a comprehensive five-year analysis of crime data across 255 suburbs and interview 100 police chiefs to correct inaccuracies.","keywords":["crime","statistics","reporting"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Excel"],"full_text":"This story began when the Inquirer screwed up crime statistics reporting.\n\nIn July, the Inquirer followed the annual release of the Pennsylvania State Police UCR report with a front page story trumpeting that overall crime fell 17 percent in Delaware County, the Philadelphia area's most urban county. But what the Inquirer missed was that Delaware County looked so good because it's second-most crime ridden town had been completely omitted from the state's crime report.\n\nUpper Darby Township was left out of the annual report, state officials told us, because police there had simply failed to report on a timely, monthly basis. The second-day, front page story was a feature-length correction that began something like \"Delaware County residents, put down the champagne.\" Meanwhile, back in our suburban headquarters, reporters and editors broke out the annual crime reports to better our understanding of crime statistics.\n\nGetting a handle on the stats\n\nIn his spare time, an editor began inputting two years of crime statistics into Excel to try to get some perspective. That effort grew to a five-year analysis of crime in 255 suburbs in seven suburban counties surrounding Philadelphia.\n\nAll the data was entered by hand from printed state reports into Excel. Most \"Part One\" or serious crimes were included: murder, manslaughter, rape, robbery, aggravated assault, burglary, larceny and car theft. Our goals were first to save face with a sound analysis of suburban crime, and secondly to build a permanent five-year database that reporters could use to write breaking news about crime in its true historic context.\n\nWe learned early that crime statistics are rife with error and tricky to interpret. Larceny in one town may mean shoplifting at a major mall. In another, larceny could mean a rash of thefts from unlocked cars. As far as errors went, many crimes appearing in the state's report seemed patently wrong. For example, one suburb showed 92 arsons in one year. Impossible. The state police told us that must have been a data entry error. In addition, about half of our 255 towns had submitted only nine months of reports, and the state police ended up estimating missing months. We printed out all the data and scanned for what seemed like anomalies. Reluctantly, staff writer Larry King and I concluded we would have to conduct phone interviews with local police departments to be sure we got the numbers right. That took about three weeks. In the end, we interviewed 100 police chiefs to correct the data.\n\nI ran into another snag when Jennifer LaFleur was visiting the Inquirer from the San Jose Mercury News. Looking at our data, LaFleur, who is head of computer-assisted reporting in San Jose, correctly pointed out that state police population numbers did not seem to reflect suburban growth over the five year period. When state police numbers were compared to U.S. Census estimates, it turned out LaFleur was right. The state police population estimates were thrown out, and I substituted U.S. Census estimates. Where none were available for certain years, like 1993 and 1995, I calculated the compound average growth rate of each town's population for an estimate.\n\nGetting the story\n\nThe intensive effort to correct the crime data was worthwhile because we also learned about trends in detail along the way. For example, when it was time to do spot reporting to highlight instances of crime, we were able to pinpoint exactly the communities and chiefs we wanted to focus on because of the prior interviewing and data analysis. This cut down a lot on time spent searching out anecdotal trends. Correspondent interns helped out by spot-reporting in towns on their beats.\n\nThe result was an 80-inch story which focused on property crimes as the major offenses of substance in the suburbs. We mapped the average, five-year property crimes of burglary, larceny and car theft throughout the region to show which suburbs were hit hardest. Neill Borowski, the Inquirer's director of computer-assisted reporting, mapped a five-year-average crime rate throughout the suburbs for a large, inside regional map. In addition to the front page story, each of the seven county 'Neighbors' sections ran that day with a unique crime story about trends in that area. Inside each 'Neighbors' section, readers were also provided with extensive graphics listing car theft, burglary and larceny rates in their towns and neighboring suburbs for the last five years. All of the data was posted to Philly Online, the Inquirer's web site.\n\nI hope the story debunked widespread paranoia about violent crime. We learned that old established suburbs near Philadelphia's border were often not the most vulnerable to theft because of tight neighborhoods where neighbors look out for each other, and crammed roads that make poor escape routes for thieves.\n\nThe story also showed that new suburbs have lost the exclusive, protected aspect they once had, because malls and new highways lure thieves to affluent areas.\n\nHowever, suburban dwellers stubbornly cling to the notion they don't have to lock up. Police chiefs know otherwise, and said theft could be cut way down if only homeowners would lock their belongings.\n\nReader response was fantastic. In the week after the story was published and the historical data put on-line, a reporter in our Delaware County bureau looked up one town's rape record to report on a string of rapes in that town.\n\nThe long term usefulness of the data made the project more rewarding than a one-day story. I hope reporters continue to use the on-line database in this way. I have also received many queries requesting the data from local planning boards and agencies such as the Greater Philadelphia Chamber of Commerce.\n\nWhen the Pennsylvania State Police release their 1997 annual crime report, the Inquirer will be ready. Never again, hopefully, will state police crime statistics be reported out of context here.\n\nThe crime report seems to have changed how editors approach data stories in the Philadelphia suburban bureaus. A year ago, it was difficult to interest suburban editors in data analysis stories. They were rightly anxious to fill the book and get stories in by deadline. However, it seems editors are finding that the projects can work to their advantage. Always eager to fill large Neighbors sections, zone editors have embraced the idea of listing large charts of data specific to small towns, so readers can compare communities. When the Census recently released population data showing shifts between 1990 and 1996, I wrote a story about the decline in population in Philadelphia and its older, affluent suburbs. Curious to see if there was indeed an \"inner ring\" of stagnant suburbs around the city, I asked Neill Borowski to map the changes using GIS software. The next day, suburban editors were so wowed by the color maps they commissioned a population treatment in the crime model. That is, big regional map and story accompanied by zoned sections with specific charts and stories. Using data this way has also created some of the most substantial cooperation between our six, far-flung suburban bureaus that has ever existed at the Inquirer. For each data project, editors and reporters in the Pennsylvania and New Jersey suburbs meet and discuss the trends and plot coverage strategies. The result is it enables us to take a broad regional look, and zero in on local angles using the zoned sections - exactly the mandate of suburban editors. I am pleased they view computer-assisted reporting as a way to get there.\n\nJere Downs can be reached at (610) 832-8328, or send e-mail to Jdowns@Philly.Infi.Net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-math-for-every-reporter","slug":"1997-decemberjanuary-math-for-every-reporter","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-math-for-every-reporter.html","year":1997,"month":"December/January","monthNum":12,"headline":"Math for every reporter","kicker":"Stats on the road","author_name":"Sarah Cohen","author_title":"","affiliation":"NICAR","summary":"Journalists must overcome their fear of mathematics and learn essential numerical skills like calculating percentages, percent changes, and per capita rates to accurately report on statistics and avoid misrepresenting data in their stories.","keywords":["mathematics","percentages","statistics"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":[],"full_text":"\"We do not expect reporters to be mathematical geniuses. But we do expect them to sidestep their mind-numbing fear of mathematics long enough to ask, 'Does this make sense?' 'What would I conclude from these numbers?'\"\n\n- A.K. Dewdney, in 200% of Nothing: From Percentage Pumping to Irrational Ratios, an Eye-Opening Tour through the Twists and Turns of Math Abuse and Innumeracy\n\nAt a recent seminar, an editor recalled a tax issue in which the funds raised would amount to a two-thirds jump in spending on a project. She dutifully reported every relevant number her sources gave her, acknowledging much later that she had no clue what any of them meant.\n\nShe mentioned that numbers meant nothing to her, other than a quite effective way to mess up a perfectly good story.\n\nIt wasn't until years later that she knew enough about numbers, about perspective, and about selectively using statistics in stories, to make sense of the budget issue.\n\nSo when it comes time to write about the math that every reporter should know, it's also time to step back and attack the journalist's fear of numbers.\n\nThis month, I'll give you a mercifully short list of techniques you must know, and your editors expect you to know.\n\nNext month, I'll give you a couple of tricks you can play on yourself to help overcome the panic, or glazed eyes, that come when you have to deal with numbers or statistics.\n\nFractions and percents\n\nA percent is just a fraction, expressed as 100ths. You know one-quarter is 25 percent. You may not remember why.\n\nIt's because 1 divided by 4 is 0.25. So think of the words, \"out of\" when you have to figure a percent of total.\n\nFormula: (Portion/Total) X 100 = Percent\nExample: If 250 people voted for Councilman Jones, and 270 voted for Councilwoman Smith, then the percentage of people voting for Smith is 270 out of (270+250), and the formula is: 270 / (270+250) X 100, or 52 percent.\n\nPercents\n\nThis is just the change over time, divided by the starting level. Remember, it's NOT the little number divided by the big number. Changes can be greater than 100 percent.\nFormula: (Difference / Old value) X 100 = Percent difference, or ((New - Old) / Old) X 100\nExample: A budget item rose from $1,500,000 to $1,600,000. Rounding off, the percent change is: (16-15)/15, or 1/15th, or 0.06667 or 6.7 percent. Notice that you can reverse the process: 1/ 0.06667 = 15, so the fraction is 1/15th.\n\nPer capita, and other rates\n\nYou often need to level the playing field. It's not fair to say that New York has more crime than other areas, or that drivers of Fords die more frequently than Hyundai owners.\n\nYou need to figure a fraction, and often raise it to a bigger base than a percentage, to make a fair comparison.\n\nThis is no different than percentages, only the \"out of\" word gets changed while you change the base to 1,000, 100,000 or something else.\n\nThere's nothing magic about 100 as a base. It's convenient, and lots of people think that way. But sometimes, that turns into too small of a number, like in the number of infant deaths each year.\n\nThe hard part is figuring out the base. Often, it's the corresponding population of a county, a state or the nation.\n\nFor instance, crime rates are computed by dividing the number of crimes by the number of people living in the area, and multiplying by 100,000.\n\nOther times, it's something else that levels the playing field: Infant deaths divided by live births, driver deaths divided by the number of cars on the road.\n\nFormula: (Number / Some base) X whatever will give you a round number, usually 1,000 or 100,000.\n\nExample: There were 80,000 crimes in Pittsburgh out of a population of 2.5 million. There were 60,000 crimes in Tucson out of a population of 700,000. How can you compare the two numbers?\n\nPittsburgh = 80,000/2,500,000 X 1,000 = 32 crimes per thousand residents\nTucson = 60,000 / 700,000 X 1,000 = 85 crimes per thousand residents.\n\nDifferent kinds of averages\n\nYou almost never hear anyone talk about average incomes or average home values. There's a reason they focus instead on the median value. It's because medians ignore the effects of very high and very low values.\nAverage: (Sum / Number of items)\nMedian: The middle item, in a sorted list\nMode: The most common value.\nSo in a list of home values [$60,000; $65,000; $70,000; $80,000; $85,000; $130,000; $250,000; $250,000] here are the three possible answers:\nAverage = $148,125, or $1,185,000 / 8\nMedian = $82,250 (average of the 2 middle values)\nMode = $250,000\nMost people would agree that the median is the more \"typical\" value. That's often how you can express it in a story, too.\n\nAdjusting for inflation\n\nIt may not be necessary for you to ever know how to adjust for inflation, only that you know you should adjust for inflation. There are plenty of tools available on the Internet for you to use. Many newsrooms keep on hand an inflation calculator. But should you want to try it yourself, make one decision first - what year's dollars do you want to use? I usually choose the most recent, since our audiences understand what a dollar buys today.\nTo do this:\n\nFormula: CPI New / CPI Old X Value Old\n\nExample: Teachers made $28,000 in 1985. How much would that be in today's dollars? The Consumer Price Index was 107.6 then and stands at 157.3 (for year-end 1996): (157.3 / 107.6) X 28,000 = About $41,000.\n\nCompound changes over time\n\nAgain, you may never have to do this, but you should know about it. If politicians promise that spending will rise only 3 percent a year, after inflation, it doesn't rise 3 X 10 or 30 percent over 10 years. It rises much more. It's the same principle that makes your own nest egg grow. To compound, convert the percentage back to a multiplier. For 3 percent, it's 1.03, or (3/100) + 1. Then take it to the 10th power: 1.03^10 = about 1.35. Now reverse the process, subtracting 1 and multiplying by 100 to get about 35 percent. Going backwards just reverses the process and use 1 divided by the number of years as your exponent. To get an annual rate for a 35 percent jump in spending over 10 years, use this formula: 1.35^(1/10) = about 1.03, or 3 percent a year.\n\nA look ahead\n\nA far bigger problem that I see on the road, though, is the phobia many reporters have of numbers.\n\nThose who aren't afraid learn the formulas above quite easily, if they don't already know them.\n\nThose who are can get all the cheat sheets in the world, but they simply can't focus their attention on the process.\n\nSo getting over the fear is the first step toward actually understanding numbers, rather than memorizing formulas. We will discuss ways to overcome that fear by rounding off, developing a sense of scale, watching television and asking your best sources for help.\n\nThese tips will help reporters get over their math phobia and will help readers better understand their news and their world.\nSarah Cohen can be reached at (301) 942-2199, or send e-mail to sarah@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-car-down-under","slug":"1997-decemberjanuary-car-down-under","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-car-down-under.html","year":1997,"month":"December/January","monthNum":12,"headline":"CAR down under","kicker":"International Report","author_name":"Stephen Quinn","author_title":"","affiliation":"Deakin University (Australia)","summary":"Computer-assisted reporting (CAR) has been slow to adopt in Australia compared to the United States, with cost, management attitudes, and training time cited as barriers, though a few pioneering journalists like Thom Cookes at The Age are beginning to use CAR methods for investigative journalism.","keywords":["reporting","technology","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"One way to improve the caliber of newspaper reportage would be to teach reporters better ways to obtain and analyze data. Computer-assisted reporting is becoming recognized as one way to do that in the United States.\n\nWhat is happening in Australia? CAR has not evolved in Australia to the extent it has in America. In an article in Australian Studies in Journalism in 1994, journalism lecturer Kerry Green predicted that the Australian news media would eventually adopt CAR. The adoption would occur \"probably sooner than later,\" he said.\n\nThree years later, he was still optimistic but he accepted that CAR had not developed as quickly as he expected. Costs, management attitudes and time for training were the major reasons for the delay, he said. \"Someone in control of the purse strings at a publication must support CAR developments.\"\n\nGiven these difficulties, have working journalists in Australia ventured their CAR onto the information highway? Key journalists at Melbourne's two dailies (the Sun Herald and The Age), the Brisbane Courier Mail, the Sydney Morning Herald and The Australian (based in Sydney) were interviewed.\n\nSelection of newspapers was based on the theory that news organisations can be divided into \"A\" and \"B\" categories. \"A\" level organizations are willing to devote resources to reporting complex issues in the community, nation and the world. In Australia, the most likely candidates are the papers at the quality end of the spectrum among the major capital city dailies, and the Australian Broadcasting Corporation.\n\nStarting CAR\n\nIn June and July, I conducted a national study of Australia's 49 dailies to discover journalists' use of the Internet for newsgathering. Two of the 27 questions covered the issue of deep CAR training, that is the use of databases and spreadsheets for investigative journalism. Only one paper, The Age in Melbourne, said it had offered any CAR training.\n\nAn inquiry to the CARR-L mailing list in August this year asking for the names of any Australian journalists actively involved in using CAR methods received no replies. An inquiry to the same list last year produced one reply, from Thom Cookes. Cookes, a senior reporter at The Age, is one of a handful of reporters in the country who employ CAR methods. He has been producing stories based on deep CAR since February last year. CAR was \"mostly virgin territory in Australia,\" he said. It could easily be sold to news executives because it produced exclusives. \"News executives love exclusives because by their very nature they are not on the news agenda, they come out of the blue. It's a major selling point for CAR.\" Cookes said journalists must learn how to apply technology to the reporting process. \"The organizations and people that we are reporting on are now more often than not keeping their records electronically. Sometimes there is no paper document to be FOI'd [obtained via Freedom of Information legislation] or leaked. Government is scrutinizing itself - and is being scrutinized by others - with the aid of computers and we should be learning how to do the same.\"\n\nBeginning training\n\nThe Herald Sun, also in Melbourne, has begun some training of journalists, but at a basic level. As of the end of last year, 12 journalists had attended a one-day introduction to computers and a further two-hour session on the Internet.\n\nKeith Moor, editor of the paper's investigative section, \"Insight,\" said some of the paper's reporters were using basic CAR to arrange interviews via e-mail and conduct research via the Internet. \"CAR is another tool for journalists, like the phone book.\"\n\nThe paper has one Internet connection but only a handful of journalists know how to use it. Moor maintains that journalism courses must introduce students to the online world. \"It's a vital tool that journalism students need to know how to use.\"\n\nRodney Chester, computer and technology writer for the Courier-Mail in Brisbane, said his paper employs CAR methods. Since 1995, the Courier-Mail had taken examination results from schools around Brisbane, dropped them into Excel and analyzed them to produce stories. Chester said his paper had not provided CAR training but one staff member, who used spreadsheets prior to joining the paper, had conducted informal sessions. Jon Casimir, editor of the Sydney Morning Herald's online edition, said journalists at the Herald were beginning to use the Internet for newsgathering. But no-one was doing any deep CAR work. \"I would argue that there isn't a media outlet in Australia committed to investigative journalism. They've all been stripped back too much. This is just my opinion, of course.\" No Herald journalists had ever been sent to any of the CAR training organizations in America, he said. Malcolm Schmidtke, editor of The Australian until July this year, said his paper hoped to send one or two people to such a course in the next couple of years. Schmidtke said CAR would be adopted in Australia much more slowly than had been the case in the U.S. \"Sophisticated research-based journalism has not been part of the culture in Australia in the way it has in the United States. Editors have not had the resources - nor probably the inclination - to pursue long-term and complicated projects that may in the end produce very little.\n\n\"Apart from a brief flirt with [UK] Sunday Times style investigative journalism, Australian papers have tended to rely on the dump of documents or the whistleblower to provide the impetus for investigation. However, times are changing and editors - certainly this one - are becoming more aware of the rich pickings in databases.\"\n\nA few television journalists use online journalism methods. Peter McEvoy, a former researcher with the Australian Broadcasting Corporation's flagship current affairs program Four Corners, said he had used the Internet \"a bit\" for research.\n\n\"It just seems sensible to have a look around on the Internet to see what groups are active in one particular story or issue. A lot of scientific information, for example, is now posted on the Internet.\"\n\nA handful of enthusiasts on the nation's major dailies are using CAR in their work. Some newspaper owners are realising the opportunities that the Internet offers for newsgathering.\n\nFor example, all journalists at the Fairfax organization (one of the three main newspaper groups in Australia, and owners of the Sydney Morning Herald, the Australian Financial Review and The Age) will have e-mail and Internet access from their desks by the end of the year.\n\nI plan to conduct a similar study next year to the inquiry I conducted this year. The results could be fascinating. Stephen Quinn can be reached by e-mail at squinn@deakin.edu.au","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-car-training-tips","slug":"1997-decemberjanuary-car-training-tips","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-car-training-tips.html","year":1997,"month":"December/January","monthNum":12,"headline":"CAR training tips","kicker":"Handout of the Month","author_name":"Anne Saul","author_title":"","affiliation":"Gannett Co. Inc.","summary":"This guide provides strategies for training newspaper staff in computer-assisted reporting (CAR), emphasizing the importance of mastering the skill first, finding internal allies, and establishing a gradual buddy system rather than company-wide rollout.","keywords":["training","reporting","newsroom"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Paradox","spreadsheet","Internet","CompuServe","PC"],"full_text":"Many newspapers and televisions rely on one byte head to do the operation's computer work. For CAR to survive and flourish, however, the knowledge must be shared and taught to others. The following handout offers some tips of how to train staffers.\n\nBefore you start\n\n1) Master CAR yourself. You can't train others if you don't know how to do it. Make sure you feel comfortable with the programs.\n2) Produce one good story using what you learned the first week. It needn't be a big project; it just needs to be something that proves the value of computer-assisted reporting.\n3) Find a compatriot in the newsroom to work with you - possibly someone who attended a conference with you. Two people can accomplish much more than one, and if one of you is an editor he or she is in a better position to convince your newspaper's top editor what is needed in the way of resources and additional training.\n4) Create a Buddy system. Initially, it might just be the two or three or four of you. But once you start getting converts, team them up with someone who has more expertise.\n5) Don't worry about trying to get everyone in the newsroom involved. Many of them never will. Other folks will want a piece of the action when they see what you can do.\n6) A word of warning - it will be slow going for a while. Be patient, focus on your own work and that of your buddy. Watch for new converts, show them how it's done and build from there.\n\nTraining others\n\nFirst, look for resources in your building. You can't do it all alone and you might find help in places you never thought of.\n\nIn-house training:\n\n1) The Human Resource Director. They don't know CAR, but they do know training. Talk to him or her about how to set up a training program or class. He or she may even be able to do a lot of the legwork for you.\n\n2) Look for other in-house resources. They may be where you least expect to find them:\n\na) One newspaper columnist who writes about online services, teaches staffers how to access the Internet, Compuserve, etc.\nb) Someone from the marketing department may know a lot about what data means since that department has to make sense of marketing data.\nc) Systems editors - if you have one - can provide training in PCs and sometimes spreadsheet and database programs. If you don't have a systems editor, a systems technician can sometimes help - particularly with basic computer training. Just make sure he or she speaks English and not \"technobabble.\"\nd) The newsroom librarian. If you are fortunate to have a good librarian, enlist his or her help in finding data sources and in teaching a session on what's available and how to get it.\n3) Set up a regular training schedule. Focus on those reporters and editors who have shown an interest in what you are doing and make the schedule realistic say once every two weeks in the beginning.\n4) Keep it small. Forget the 2 1/2-day seminars, in the beginning. Announce a brown-bag lunch. Post the topic in the in-house newsletter or on the front-end system. Don't worry if the turnout for the first few sessions is small. If you get only one or two people, you have potential converts and ambassadors.\n5) Focus on story ideas - particularly those from the reporters' beats - and talk about how a spreadsheet or database program can enhance those stories.\n6) Start with the basics. Participants will have a difficult time learning Paradox for Windows if they've never used Windows or a mouse. Encourage them to play Solitaire to enhance their mouse skills.\n7) Divide up the expertise. Rather than forcing trainees to learn several different programs, have one person become the database expert, another the spreadsheet expert, and so on. When a story or project needs to be done, the experts can team up to take advantage of their expertise.\n8) Dump a city budget or other data from one of those beats into a PC and show the other staffers what can be done with the information. (Avoid using the examples that come with the program's tutorials; they aren't relevant to reporting.)\n\n9) Remember to start with the story idea, not the computer. (Yes, I said that before, but it bears repeating.) Reporters and editors have to see the relevance of what you're doing. Budgets, by the way, are an excellent way to start. Almost every beat reporter has to deal with them at one time or another.\n10) Assign \"homework\" to make sure participants use the skills they have learned. Give the participants a list of questions that can be answered only if they use the program. (What was the percentage increase in the police department's budget over the previous year, for example, if you used a city budget in the spreadsheet class.) Let them use any resource they like (manuals, etc.) to figure out the answer except asking a fellow staffer for the answer.\n\nRegional training\n\n1) Solicit participation from neighboring (non-competitive) newspapers and pool the expenses to bring in a trainer for a group session. You may be able to get an expert from another newspaper simply by paying his or her expenses. CAR reporters and editors are always willing to share their expertise and spread the gospel.\n2) APME and state AP bureaus conduct regional CAR training utilizing AP and NICAR trainers. These have been held in several cities in the past year. The per-person cost is minimal (often less than $50). Contact your state AP bureau to inquire about a session in your area. It helps if you've already solicited interest from other newspapers in your state or region.\n\nLocal colleges\n\nCheck out local colleges and universities for computer training - classes in Lotus or Paradox or computer basics if they don't offer anything specially in computer-assisted reporting. You might even find someone with CAR experience. The Free Press in Burlington, Vt., for example, found a former reporter-turned-student instructor who used to be president of IRE.\n\n(A word of caution: Be careful of training by non-journalists - such as at local business schools or training institutes.\n\nThe trainer may know Lotus, but tracking widget profits is hardly applicable to newspapers. If you do use an outside source, see if you can talk to the instructor before hand about what you do, bring a sample - such as the city budget.)\n\nTraining tools\n\n1) Don't worry if your only available PC is a 386 with not much memory and an old version of Lotus. It can handle the small projects you'll want to start with. And once you produce some stories, it will be easier to convince your editor that a faster machine and more up-to-date software are warranted.\n2) By all means steal and copy everything you received - except the licensed software, of course.\nDon't duplicate everything in the beginning though; it will be too overwhelming. It's better to copy only the handouts that are pertinent to your particular training session.\n3) Manuals/Cheat sheets: Put the handouts, cheat sheets and other documents in a loose-leaf binder for participants. That way, they'll have the beginning of a CAR manual to which they can add later.\nInclude lists of information/databases available local and at the state and federal level, how to access DMV on-line, etc.\nIf possible, keep the handout files in a computer so that you can update them easily. (Long lists kept in the CAR computer also make searching for data easier.) Encourage users to add to the lists as they discover new sources of information.\n4) Form a public records committee and have the members find out what local and state public records are available in computer-format. Keep others informed as to what's available; start a database of databases. (The librarian should be on the committee.)\n5) Create a data center - even if it's a single PC and a bookshelf in a corner of the newsroom. Keep manuals and public record information. Create a catalog listing what's in the center. Don't segregate it in a locked room that doesn't invite people to use it.\n\nThis is an excerpt from the CAR Rock National Conference. To get a copy of the full handout, call the IRE Resource Center at (573) 882-3364.\n\nOne great way to find story ideas is by reading IRE's \"100 Computer-Assisted Stories,\" which includes ideas for both features and hard-hitting investigations. To order, call (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-get-your-training","slug":"1997-decemberjanuary-get-your-training","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-get-your-training.html","year":1997,"month":"December/January","monthNum":12,"headline":"Get your training","kicker":"Seminars, boot camps, conferences","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR and IRE offer various computer-assisted reporting training opportunities including workshops, boot camps, and conferences throughout 1998 at different locations. Interested journalists can contact organizers or visit websites to register for hands-on sessions ranging from one-day demonstrations to week-long intensive training.","keywords":["training","journalism","workshops"],"topics":["Journalism & Newsroom"],"technologies":["spreadsheet","Internet","World Wide Web"],"full_text":"Get your training\n\nCheck out the NICAR Web site at www.nicar.org to find out times and dates for upcoming on-the-road training and boot camps.\n\nWorkshops\n\nIRE and NICAR can also bring road-tested workshops to sites around the country. We match regional public records experts and experienced reporters with our own trainers for unique seminars ranging from one-day demonstrations and panels on regional public records laws to three-day hands-on sessions on computer-assisted reporting or advanced techniques for data-intensive beats.\n\nDon't wait for someone else to organize a computer-assisted reporting workshop in your area. IRE and NICAR depend on members, news organizations, journalism schools and other journalism organizations to help us bring our seminars to you. Contact Sarah Cohen at (301) 942-2199 or e-mail her at sarah@nicar.org\n\nJanuary 23-25, 1998 - St. Louis, Mo. Risky Behavior Workshop on health care databases and issues\n\nBoot Camps\n\nBoot Camps are week-long, intensive training sessions offered at NICAR's headquarters at the Missouri School of Journalism in Columbia, Mo.\n\nAs with on-the-road training, you will learn to transfer data from government files into newsroom PCs, build spreadsheets for stories on the beat, comprehend documents with database managers, and navigate the 'Net and online databases - but you'll be drilled all day, every day for a full week. Tuition ranges from $500-$1,000 depending on circulation or market size.\n\nFor information, call Katy Fanning at (573) 882-0684. January 4-9, 1998 - Columbia, Mo. NICAR Regular Boot Camp January 9-11, 1998 - Columbia, Mo. NICAR Advanced Boot Camp May 10-15, 1998 - Chapel Hill, NC NICAR Advanced Boot Camp\n\nMay 17-22, 1998 - Columbia, Mo. NICAR Regular Boot Camp July 12-17, 1998 - Columbia, Mo. NICAR Regular Boot Camp August 9-14, 1998 - Columbia, Mo. NICAR Regular Boot Camp\n\nConferences\n\nNICAR will offer training at the following professional conferences, including the IRE and NICAR national conferences. These conferences feature practical tips and story ideas shared by prize-winning journalists in quick-hit sessions. Keep up with our conference schedule through our Web site, at www.ire.org. Costs vary. For more information or to register, call Lisa Barnes at (573) 882-8969.\n\nJanuary 31-February 1998 - Washington, D.C. IRE Regional Conference March 5-8, 1998 - Indianapolis, Ind. NICAR National Conference March 27-29, 1998 - New York, N.Y. IRE Regional Conference June 4-7, 1998 - New Orleans, La. IRE National Conference\n\nJournalism Conference Tour\n\nIRE and NICAR will be appearing at several upcoming conferences of other journalism organizations. IRE and NICAR will participate in panels and provides hands-on computer-assisted reporting training. For more information on these conferences, contact the organization sponsoring or check out the www.reporter.org Web site.\n\nJuly 24-27, 1998 - Miami, Fla. On-the-road training for the National Association of Hispanic Journalists Convention '98\n\nJuly 29-August 3, 1998 - Washington, D.C. On-the-road training for the National Association of Black Journalists Convention '98\n\nAugust 5-8, 1998 - Chicago, Ill. On-the-road training for the Asian American Journalists Association Convention '98","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-from-the-nicar-library","slug":"1997-decemberjanuary-from-the-nicar-library","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-from-the-nicar-library.html","year":1997,"month":"December/January","monthNum":12,"headline":"From the NICAR library","kicker":"Growing collection of federal databases","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to a comprehensive collection of federal government databases covering topics from safety and health to finance and crime, enabling journalists to investigate government programs, corporate records, and public safety issues.","keywords":["databases","government","investigation"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"From the NICAR library\n\nNICAR offers a number of federal government databases. Here is a list of our growing collection:\nNEW National Endowment for the Arts, 1987-1996, includes information on individual or organization receiving money.\nNEW Immigration and Naturalization Service legal residency, 1980-1995, includes information on the characteristics of aliens who were admitted as immigrants.\nNEW National Highway Traffic Safety Administration vehicle recalls and complaints, 1966-1997, includes information about vehicles that have been recalled by certain manufacturers.\nU.S. Food and Drug Administration's Medical Devices Reports, detailing complaints about drugs, pacemakers and other medical products, 1978-1996.\nU.S. Centers For Disease Control's AIDS database, providing case-by-case demographic information about those with the HIV virus, 1995-96.\nU.S. Census Bureau's Consolidated Federal Funds Reports, showing which communities get how much under various federal programs, 1983-1995.\nFederal Aviation Administration's accidents and incidents, including major plane crashes since 1974.\nNASA's air safety reporting system, including anonymous complaints by pilots and air traffic controllers. Useful for finding near misses and problems at local airports, 1988-1997.\nFederal Election Commission campaign contributions by individuals and political action committees, 1990-1997.\nThe Health Care Financing Administration's 1995 database of all Medicare-funded in-patient work in U.S. hospitals.\nFederal Railroad Administration data for accidents, casualties, and highway crossings, 1991-1996.\nCoast Guard boating accidents, 1969-1996.\nFederal Aviation Administration data, including airplane maintenance work documented in the service difficulty report, pilot licenses and grades, and aircraft registration, 1974-1997.\nHome Mortgage Disclosure Act records, for tracking who gets loans and who gets turned down, and finding redlining patterns, 1992-1996.\nFederal procurement data, 1992-1996, includes breakdowns by agency.\nAlcohol, Tobacco and Firearms gun dealer records, 1993-1996.\nNational Bridge Inventory System data, includes inspection grades, 1994-1996.\nFBI Uniform Crime Reports, a detailed compilation of crime data that includes statistical breakdowns of individual murders. This includes the 1995 final report.\nSocial Security death records, by name and social security number, going back to 1937.\nOccupational Safety and Health Administration violation data includes worker accidents and exposures to hazardous chemicals by companies, 1974-1996.\nU.S. Department of Transportation truck accident and census data. It includes accidents by company and road.\nU.S. Small Business Administration loan guarantees, 1980-1997. This includes the name of the business, address, amount covered by the SBA, and status, including whether the loan went bad.\nU.S. Small Business Administration disaster loan guarantees, 1989-1995. This includes individuals and businesses, the amount covered by the SBA, and the status, including whether the loan went bad.\nU.S. Small Business Administration's list of minority companies certified for SBA assistance in seeking federal contracts. It includes the name of the company, its address, the owner, type of business and phone number.\nThe National Inventory of Dams, 1991-1995.\nU.S. Department of Transportation hazardous materials accidents database, a collection of roadway, rail, air and waterway accidents from 1971-1996.\nU.S. Department of Transportation fatal accident reporting system. It includes all roadway accidents from 1975-1995.\nU.S. Coast Guard directory of U.S. merchant vessels. It includes the name of the ship, the managing owner, home port and various descriptive information.\nHandouts describing how to use these databases and others are available online at www.ire.org. Also, check out the job postings recently put on this page.\nFor up-to-date prices and more information, call (573) 882-0684, or send e-mail to info@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1997-decemberjanuary-bits-bytes-and-barks","slug":"1997-decemberjanuary-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1997-decemberjanuary-bits-bytes-and-barks.html","year":1997,"month":"December/January","monthNum":12,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The Campaign Finance Information Center launches with a new website and newsletter, while NICAR expands its conferences, seminars, and online resources for journalists interested in computer-assisted reporting and investigative journalism.","keywords":["campaign","journalism","data"],"topics":["Journalism & Newsroom","Government & Politics","Technology"],"technologies":["Access","SQL","World Wide Web","Internet","Email"],"full_text":"Campaign Finance\n\nThe Campaign Finance Information Center is in full swing. Data, tips, links and stories are on the center's Web site, www.campaignfinance.org\n\nAlso, coming soon to a mailbox near you will be the Campaign Finance Information Center's newsletter. Stories from the Fall '97 newsletter include reports about states with electronic filing systems, a look at how states are going to inform the public of the connection between money and politics and an explanation of in-kind contributions.\n\nConferences and Seminars\n\nNICAR is forging ahead with new seminars and new approaches to conferences and workshops. The annual conference, March 5-8 in Indianapolis, will not only include the usual panels and hands-on classes for beginners to the advanced, but also a set of special half-day sessions that will take in depth looks at how to use electronic information for particular beats.\n\nWe also are expanding our seminars for the intermediate range. Earlier this year we did a seminar on education and statistics. In the coming months, we plan to hold seminars on database design and data cleaning, use of SQL servers for the newsroom, mapping software, and business reporting. Watch our Web site at www.nicar.org for dates and times. For more information on seminars and boot camps, call (573) 882-0684.\n\nNICAR Net\n\nRecent topics on the IRE and NICAR listserv have included campaign finance data, troubles with traffic accident data, ethics, medical investigations and Access tips.\n\nTo subscribe to IRE-L or NICAR-L, send e-mail to listproc@lists.missouri.edu\n\nIn the body of the message, type: subscribe NICAR-L<your name> subscribe IRE-L<your name>\n\nAlso, check out the IRE-L and NICAR-L mailing list archives on our Web site at www.ire.org and www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.\n\nChanging of the editor\n\nThis is the final issue for Uplink's current managing editor, Jody Sowell. Taking over for next year will be Brent Johnson. Johnson is looking for new ideas and new themes for the new year.\n\nAre there certain columns that you would like to add? Are there certain topics you would like covered? Have you done a story that you think should be featured in an upcoming issue? Do you have something you would like to contribute? A column about a CAR story you reported?\n\nSend comments, critiques and suggestions to Johnson at brent@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-inspection-du-jour-restaurants-revealed","slug":"1998-february-inspection-du-jour-restaurants-revealed","url":"https://dwillis.github.io/uplink/article/1998-february-inspection-du-jour-restaurants-revealed.html","year":1998,"month":"February","monthNum":2,"headline":"INSPECTION Du JOUR Restaurants Revealed","kicker":"INSPECTION Du JOUR","author_name":"Joel Grover","author_title":"Reporter","affiliation":"KCBS-TV","summary":"Joel Grover discusses the impact of an investigative series on restaurant conditions in L.A., revealing widespread sanitary issues and prompting legal changes.","keywords":["restaurant","inspection","L.A."],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["World Wide Web"],"full_text":"When I first proposed the idea of a series on restaurant conditions, it seemed like such a basic investigation. I knew similar investigations had been done for years in other cities. I would have never thought a series like this would attract a firestorm of attention, be the subject of dozens of newspaper articles, and result in major changes in the law. But that's just what happened when KCBS-TV aired 'Behind the Kitchen Door' last November. The story began with a tip: a former L.A. County health inspector told me the county was doing nothing to force filthy restaurants to clean up. A quick computer search showed me that no other local media had ever investigated L.A. County's restaurant inspection program, so it seemed like a good subject to probe. I began my research with what seemed like a simple request to the L.A. County Health Department for its computer database of restaurant inspections. At first, the county said such a database didn't exist and later said certain restaurant data wasn't public. Fortunately, my station was willing to fight for this data. CBS lawyers battled L.A. County back and forth for 8 months. Only when we told them we were taking them to court did the health department hand over its database. KCBS hired NICAR to crunch the data. Clearly, we could not have done this ourselves. L.A. County's database contained hundreds of thousands of records on 20,000 restaurants. The county tried to send us the data in a form that would have made it virtually unusable. One phone call from NICAR to the health department solved the problem. Our producer, Adam Symson, and I brainstormed with Andy Lehren and Jack Dolan at NICAR about all the ways we could crunch the data. We wanted to know how often restaurants were inspected. We wanted to see what percentage of restaurants were averaging poor to failing scores on their inspections. And if restaurants were failing every inspection, were they penalized or shut down? Our analysis revealed that L.A. County's restaurant inspection system was not doing its job of protecting the public health. We found that many restaurants hadn't been inspected in over 2 years, even though the health department had previously insisted every place gets at least 3 inspections a year. We found that nearly 30 percent of L.A. area restaurants averaged poor to failing scores on health inspections. The data showed that most of these places were never penalized or closed down. Finally, NICAR ranked all 20,000 restaurants from the lowest to highest scoring. We were amazed when we examined that list. Some of the most respected and popular restaurants in Los Angeles were also getting the worst inspection scores. We then decided to conduct an undercover investigation. Using the list that NICAR produced for us, we chose several low-scoring restaurants and sent researchers to work there. Our computer analysis and our undercover tapes gave us enough material to do many stories. What was originally intended to be a 2-part series turned into an 11-part series. Our first story exposed filthy conditions at many popular L.A. restaurants and used facts from our computer analysis to show the problem was widespread. On the second day, we exposed the failures of the inspection system to keep restaurants clean. We then did several stories using the rankings of restaurants by average inspection score. One story focused on the lowest-scoring restaurants in town; another highlighted the cleanest places. KCBS was deluged with calls from viewers wanting to see the entire list of restaurant scores. We put that list on our Web site, www.Channel2000.com. In the following weeks, it received an unprecedented one million hits. Two days after our first report, the Los Angeles County Board of Supervisors publicly grilled health department officials about the problems KCBS uncovered. The department then started cracking down. Our initial reports generated hundreds of tips, which gave us material for many solid follow-ups. We reported that a grand jury had warned the health department years earlier that there could be an epidemic of food poisoning if it didn't get tougher on unsanitary restaurants. (The County ignored that warning). We also reported how the health department had a secret surplus of millions of dollars that it could've used to hire more restaurant inspectors. Until we broke that story, health officials had been saying they were short on money to beef up their inspection program. What amazed us most was the fallout and public outcry from our series. Two days after our first report, the Los Angeles County Board of Supervisors publicly grilled health department officials about the problems KCBS uncovered. The department then started cracking down, closing more than 300 unsanitary restaurants, including one owned by Los Angeles Mayor Richard Riordan. Three weeks later, the board adopted a sweeping overhaul of Los Angeles County's restaurant inspection system. Because of our reports, restaurants must now post letter grades reflecting their latest inspection scores. And restaurant workers will be required to get training and certification in safe food-handling practices. Three months later, we are still doing follow-up stories from what we all thought would be just a basic investigative story.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1998-february-free-boat-to-china-all-expenses-paid","slug":"1998-february-free-boat-to-china-all-expenses-paid","url":"https://dwillis.github.io/uplink/article/1998-february-free-boat-to-china-all-expenses-paid.html","year":1998,"month":"February","monthNum":2,"headline":"FREE BOAT TO CHINA All Expenses Paid","kicker":"","author_name":"Kara G. Morrison","author_title":"","affiliation":"NICAR","summary":"Gannett News Service conducted a database investigation revealing that private groups spent nearly $8 million on over 5,100 congressional trips between 1996-1997, with findings showing China as the top destination and the House Commerce Committee as the most frequent traveler.","keywords":["Congress","travel","lobbying"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Paradox"],"full_text":"After Congress tightened rules on member travel funded by private groups in late 1995, Gannett News Service decided to check if the new rules had made a difference. 'We wanted to assess how much travel members of Congress and their staffs have taken from private groups with a stake in Capitol Hill legislation,' said GNS special projects editor David Milliron. The result was a GNS special report titled 'Congress: Have invitation, will travel.' The special report, which ran predominantly in many of Gannett's 91 daily newspapers in November, included an overall analysis and story on congressional travel, along with stories detailing which special interests paid the highest travel tabs, which destinations were most common, which committees traveled most frequently, and which Congressional spouses earned the most frequent flyer miles. The new rules require members of Congress and their staffs to explain how such travel relates to their work and to disclose its sponsor, destination, cost and purpose within 30 days.The GNS database investigation found that private industry and non-profit groups spent nearly $8 million in 16 months on 5,100 trips. Other key findings included: China was the hottest congressional travel destination; the Nuclear Energy Institute spent $123,000 on congressional travel while lobbying to send radioactive waste to Nevada during 1996-97; the tobacco industry spent $70,000 on 27 trips for congressmen during the same period; and the House Commerce Committee, which oversees telecommunications and electricity deregulation, topped the list of well-traveled congressional committees, taking nearly 200 paid trips in 16 months. GNS chief political correspondent Chuck Raasch and regional correspondent Fredreka Schouten, together with Milliron, spent three months on the project. They collected the paper records on the more than 5,100 trips between January 1996 - when the new reporting requirements kicked in and early May 1997. The data from the reports was then keypunched by two interns into six Borland Paradox databases. Milliron said The Center for Responsive Politics subsequently keypunched the 1997 travel records, which are now available on its Website. 'I was pleased to see they picked up on this. We feel this is an important thing for the public to know,' Raasch said. 'This is the type of information Congress doesn't want in electronic form because it's too easy to see patterns among trips when it's in a computer database,' Milliron said. Milliron explained there were a few pitfalls during the project. Because the majority of the reports were handwritten, GNS had to painstakingly decipher the name of each traveler on the forms using various congressional directories. And many of the travel records identified sponsors not by company names but by acronyms, requiring additional research. Raasch said members of Congress also did a poor job of filling out the forms. Often, writing was illegible or total expenses (including transportation, lodging, meals) were added incorrectly. And in one instance, Milliron said, House Speaker Newt Gingrich reported the cost of his trip in Swiss francs, which GNS converted to U.S. dollars using the exchange rate at the dates of his trip. 'If that's not trying to abate the system, I don't know what is,' Milliron said. Included in the database were the trip sponsors, names of travelers, destinations and the cost and dates of travel. GNS created a separate database for travel related to committee work. Once the analysis was complete for the national package, Milliron worked with the bureau's regional reporters, who wrote about three dozen stories on the travels of elected representatives serving their regions. Overall, Raasch said they were happy with the play the series received. 'As we tried to point out, there are a lot of new ways or different ways for money to get into this process,' Raasch said.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1998-february-behind-the-ratings","slug":"1998-february-behind-the-ratings","url":"https://dwillis.github.io/uplink/article/1998-february-behind-the-ratings.html","year":1998,"month":"February","monthNum":2,"headline":"Behind the Ratings","kicker":"INSPECTION Du JOUR","author_name":"Stuart Watson","author_title":"","affiliation":"WRAL","summary":"A North Carolina TV station analyzed restaurant health inspection databases and found significant inconsistencies in grading systems, with restaurants receiving 'A' ratings despite serious violations like maggots and raw sewage backups due to flawed point deduction rules.","keywords":["restaurants","inspections","violations"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["spreadsheet"],"full_text":"Most of us eat out hundreds of times a year - sometimes more often than we eat at home. While some CAR veterans look down their noses at reports on restaurant health inspections as elementary or overdone, few topics strike closer to home with the public. Many newspapers have long posted weekly sanitation inspections as part and parcel of the news of record. Now, databases of restaurant inspection reports are ready-made for the Web. They give the public fingertip access to raw information about local eateries.\n\nWRAL's series began when a customer at a McDonald's restaurant in Raleigh, N.C., took a close look at its sanitation inspection report posted on the wall. Surprised, she called us to say that, while the local health department had awarded the McDonald's an \"A\" sanitation grade, the inspector had noted maggots growing in an unused ice cream machine.\n\nMaking the grade\n\nSo what does an \"A\" rating mean, anyway? We filed an Open Records Act request to get our county's database of sanitation inspection reports to find out. Rules, however, vary from state to state. As we say in TV, check your local listings.\n\nNorth Carolina, South Carolina and Nevada are the only states that use A-B-C ratings. Most states still use variations of a 100-point scale. Some are moving toward the Food and Drug Administration's latest \"model\" food inspection rating system, which calls for a satisfactory/unsatisfactory approach that is obviously less conducive to computer-assisted reporting.\n\nNorth Carolina's database (including only one fourth of all counties) includes fields for type of violation, numbers of points deducted from a 100-point scale, final score and - most valuable of all - the inspector's comments. We found restaurants reportedly having maggots, raw sewage backups, and rats breeding in the dumpster that still received \"A\" ratings.\n\nInspectors routinely cited restaurants for less sensational - though more hazardous - violations like food served at improper temperatures and raw meat contaminating cooked food, yet gave them \"A\" ratings as well. The inspectors aren't to blame. The state's 100-point system allows a maximum deduction of only five points for each type of violation - not each individual violation. So a restaurant with a series of relatively innocuous but different violations could score lower than a restaurant with a series of threatening but similar ones. As is the case with almost all computer-assisted reporting, the database was not the final story only the beginning of questions. We adopted the methodology used by Mike Wendland at WDIV-TV in Detroit: culling the lowest and highest scores, then looking at files at the health department to build a much smaller spreadsheet with at least 3 inspections for the group of low-scoring restaurants.\n\nExpecting surprise\n\nThis removed the \"just-had-a-bad-day\" defense and showed restaurants that scored poorly over time.\n\nIt also contained a surprise. Seven of the 10 lowest scoring restaurants posted \"A\" ratings. Because inspectors are required to re-inspect within 15 days, some restaurants would routinely score low on surprise inspections then clean up their act. The lower and arguably more representative grade from the surprise inspection hangs in public view for two weeks tops; the higher grade from the not-so-surprise inspection hangs on the wall for public consumption for months until it's time for another surprise inspection. Our report aired just as the state was debating changing from letter grades to number grades. So we were able to follow the debate with new information and context. Our experience is by no means unique. But in this instance, just because a topic has been covered well in another market does not mean your audience would not benefit from another treatment of it locally - especially if you can augment the on-air story with relevant information online.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-processor-violations","slug":"1998-february-processor-violations","url":"https://dwillis.github.io/uplink/article/1998-february-processor-violations.html","year":1998,"month":"February","monthNum":2,"headline":"Processor violations","kicker":"INSPECTION Du JOUR","author_name":"Brent Johnson","author_title":"Database Editor","affiliation":"NICAR","summary":"Investigative reporters analyzed USDA inspection data and found that meat and poultry plants repeatedly cited for critical violations are rarely shut down by the agency, with one Tyson Foods plant accumulating 1,700 violations. The reporters used database analysis techniques to organize and examine the massive USDA dataset to expose weak enforcement of food safety violations.","keywords":["violations","inspection","enforcement"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["FoxPro","CD-ROM"],"full_text":"Processor violations\n\nBy Brent Johnson NICAR\n\nHundreds of meat and poultry plants operating throughout the country are repeatedly issued citations following inspections by the U.S. Department of Agriculture. But the enforcement record against these gutting operations has most often been gutless. An investigative story by Elliot Jaspin and Scott Montgomery of Cox News Service found that few plants ones cited hundreds of times - are ever forced shut by the USDA.\n\nCritical condition\n\nJaspin and Montgomery used FoxPro to analyze the massive \"Feedback File\" database provided by the USDA following a FOIA request. Although they requested files from 1990-1996, the USDA only released information from 1996. The reporters wanted to know how many \"critical violations\" - defined by the USDA as a plant condition \"certain\" to cause both food contamination and to reach and then harm consumers - occurred at given plants. \"We were expecting 200 tops for a particular plant,\" Jaspin said. However, they found a poultry plant in Arkansas operated by Tyson Foods that had amassed 1,700 violations. That's when they knew they had a story.\n\nFile me up, file me down\n\nThe USDA database arrived on CD-ROM organized by a series of subdivisions. The data were divided into quarters of the year. Each quarter was divided into 26 regions. Each region was divided by week. \"So there were 13 zip files for each region, and within those, when unzipped, were different 'circuits,' or strings of meat and poultry plants,\" Jaspin said. \"We're talking in the neighborhood of 5,000-6,000 little databases.\"\n\n\"When you have large databases, the whole idea is to divide and conquer, to be able to work in a feasible fashion,\" Jaspin said.\n\nTransferring the data from the CD-ROM to the hard drive using a batch file presented the first challenge. Then appending the zip files into databases for each quarter awaited. \"We had to smash (the unzipped files) into one large database actually four. One database would've been too unwieldy - over 25 million records.\"\n\nDivide and conquer\n\nDirty data always lurks. Jaspin and Montgomery had to work around an identification problem. They thought each plant would have a unique number but then found individual plants entered multiple times. By cleaning the unique number problem up, they could link violations from inspection records to particular plants by joining tables.\n\nFinally ready to analyze the data, they created \"subset databases\" to answer particular questions, such as the frequency of critical violations occurring at plants.\n\n\"When you have large databases, the whole idea is to divide and conquer, to be able to work in a feasible fashion,\" Jaspin said.\n\nJaspin and Montgomery also analyzed the data by geographic distribution of violations and plant size. Smaller databases were created and sent to individual Cox newspapers so they could pursue local stories.\n\nUSDA puts up, but doesn't shut down\n\nThrough their database analysis of 1996 inspections, the reporters discovered 299 plants with 100 or more \"critical violations\" and 7 with more than 1,000. That year, the USDA shut down only 6 plants.\n\n\"Heavens,\" an epidemiologist at the Centers for Disease Control and Prevention told the reporters upon hearing those numbers. \"The burning question is why is a plant with that many violations - why has the (federal) inspector not been pulled from that plant? And I have no idea.\"\n\nNICAR IS IN THE PROCESS OF ACQUIRING THE SUMMARY DATA OF THE USDA's PROCESS DEFICIENCY REPORTS, WHICH INCLUDE A LIST OF VIOLATIONS - CRITICAL, MAJOR OR MINOR - BY PLANT.\n\nBrent Johnson can be reached at (573) 882-0684 or by e-mail at bjohnson@nicar.org\n\nA PROFILE OF PAUL ADRIAN, AN INVESTIGATIVE REPORTER AT WBNS-TV IN COLUMBUS, OHIO, WHO HAS ALSO USED THE SUMMARY DATA IN A STORY, WILL APPEAR IN NEXT MONTH'S ISSUE.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-dirty-kitchens","slug":"1998-february-dirty-kitchens","url":"https://dwillis.github.io/uplink/article/1998-february-dirty-kitchens.html","year":1998,"month":"February","monthNum":2,"headline":"Dirty kitchens","kicker":"","author_name":"","author_title":"","affiliation":"KCBS-TV Los Angeles","summary":"KCBS-TV Los Angeles conducted a computer-assisted investigation of restaurant health inspections, discovering that many establishments hadn't been inspected regularly and that low-scoring restaurants faced minimal penalties, leading to an 11-part series and public accountability.","keywords":["restaurants","inspection","investigation"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"ACCORDING TO STUART WATSON, HIS STATION'S RESTAURANT INSPECTION SERIES \"DEMONSTRATED BETTER THAN ANY OF OUR PREVIOUS INVESTIGATIVE PROJECTS THE ABILITY OF WRAL ONLINE TO SUPPLEMENT ON-AIR REPORTS.\" HE PROVIDED WEB SITES FOR SIMILAR PROJECTS AT OTHER ORGANIZATIONS:\n\nWRAL ONLINE HTTP://www.WRAL-TV.COM/NEWS/WRAL/5INVESTIGATES/19971REST-EVAL-DATA\n\nWDIV-TV DETROIT HTTP://www.WDIV.COM/DD4.HTML\n\nWSB-TV AND THE ATLANTA JOURNAL-CONSTITUTION HTTP://WWW.ACCESSATLANTA.COM/LOCAL/INSPECTIONS\n\nTHE DESERET NEWS HTTP://199.104.95.5/MISC REST/RESTFORM.HTM\n\nKCBS-TV LOS ANGELES HTTP://www.CHANNEL2000.COM\n\nscores on their inspections. And if restaurants were failing every inspection, were they penalized or shut down?\n\nOur analysis revealed that L.A. County's restaurant inspection system was not doing its job of protecting the public health. We found that many restaurants hadn't been inspected in over 2 years, even though the health department had previously insisted every place gets at least 3 inspections a year. We found that nearly 30 percent of L.A. area restaurants averaged poor to failing scores on health inspections. The data showed that most of these places were never penalized or closed down.\n\nFinally, NICAR ranked all 20,000 restaurants from the lowest to highest scoring. We were amazed when we examined that list. Some of the most respected and popular restaurants in Los Angeles were also getting the worst inspection scores.\n\nGoing undercover\n\nWe then decided to conduct an undercover investigation. Using the list that NICAR produced for us, we chose several low-scoring restaurants and sent researchers to work there.\n\nOur computer analysis and our undercover tapes gave us enough material to do many stories. What was originally intended to be a 2-part series turned into an 11-part series. Our first story exposed filthy conditions at many popular L.A. restaurants and used facts from our computer analysis to show the problem was widespread. On the second day we exposed the failures of the inspection system to keep restaurants clean.\n\nWe then did several stories using the rankings of restaurants by average inspection score. One story focused on the lowest-scoring restaurants in town; another highlighted the cleanest places.\n\nAn activated audience\n\nKCBS was deluged with calls from viewers wanting to see the entire list of restaurant scores. We put that list on our Web site, www.Channel2000.com. In the following weeks, it received an unprecedented one million hits.\n\nTwo days after our first report, the Los Angeles County Board of Supervisors publicly grilled health department officials about the problems KCBS uncovered. The department then started cracking down.\n\nOur initial reports generated hundreds of tips, which gave us material for many solid follow-ups. We reported that a grand jury had warned the health department years earlier that there could be an epidemic of food poisoning if it didn't get tougher on unsanitary restaurants. (The County ignored that warning). We also reported how the health department had a secret surplus of millions of dollars that it could've used to hire more restaurant inspectors.\n\nUntil we broke that story, health officials had been saying they were short on money to beef up their inspection program.\n\nWhat amazed us most was the fallout and public outcry from our series. Two days after our first report, the Los Angeles County Board of Supervisors publicly grilled health department officials about the problems KCBS uncovered. The department then started cracking down, closing more than 300 unsanitary restaurants, including one owned by Los Angeles Mayor Richard Riordan. Three weeks later, the board adopted a sweeping overhaul of Los Angeles County's restaurant inspection system. Because of our reports, restaurants must now post letter grades reflecting their latest inspection scores. And restaurant workers will be required to get training and certification in safe food-handling practices. Three months later, we are still doing follow-up stories from what we all thought would be just a basic investigative story.\n\nJoel Grover can be reached at (213) 460-3284 or by e-mail at lax@primenet.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-demystifying-car","slug":"1998-february-demystifying-car","url":"https://dwillis.github.io/uplink/article/1998-february-demystifying-car.html","year":1998,"month":"February","monthNum":2,"headline":"Demystifying CAR","kicker":"QUICK HITS","author_name":"Neil Reisner","author_title":"","affiliation":"The Miami Herald","summary":"The Miami Herald's Broward County edition demonstrates how computer-assisted reporting (CAR) can be applied quickly to everyday journalism and breaking news stories without requiring elaborate databases or complex technology. The article provides practical examples of how their newsroom used simple data analysis tools to add value to routine coverage, including an analysis of school board election results.","keywords":["reporting","analysis","data"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Excel","Access","modem/BBS","ASCII"],"full_text":"Okay. You've heard one too many stories about the yearlong CAR investigation using umpity-two databases, way-too-sophisticated statistics and a Pentium II 300 with 556 megabytes of RAM and a 200-gigabyte hard drive.\n\nThat just ain't the reality you live in. But you've also heard one too many stories from one too many CAR wizards wearing a pointed cap with stars and crescents on it about the turn-on-a-dime quick hits that they produced merely by blinking their eyes.\n\nThey can be done. With just a little bit of forethought, it's possible - even easy - to apply CAR to your everyday work, even to breaking news.\n\nThat's what we've been doing at The Miami Herald's edition in Broward County, which, despite its connection to a major metro, is really not very different from a medium-sized daily.\n\nThat's because we put out our own edition of the paper in Broward with a remade front page, a local news section, bi-weekly neighbors sections, independent calendar listings, local sports - the whole package. We struggle along with the same limitations of staff and resources that confront most other papers. And we compete fiercely with the Sun-Sentinel of Fort Lauderdale, with whom we're at war.\n\nHere are four examples of the ways we've used CAR in recent months- to produce mega-stories but to add value to the routine. (And, for spice, examples of how we could have done better.) None took more than a day, and most were finished in several hours.\n\nElecting school boards\n\nEducation is the big political issue here in Broward, where the local Board of Education has about the same credibility as used car dealers and the schools are overcrowded despite the fact that they've spent $2 billion on construction over the last 10 years. The action involved an election last fall deciding whether to scrap the traditional system of electing school board members at large for a system in which most members would represent specific districts.\n\nThe change won overwhelming approval, much to the surprise of some pundits (and school board members) who figured what we call the \"condo vote\", mostly elderly voters loyal to the locally ruling Democrats, would rally to the status quo. The question was: How did this happen?\n\nWe covered election night as usual, downloading overall results from a BBS run by the Superintendent of Elections. The day after the election, though, precinct-by-precinct results were available. Usually, they'd been provided on paper, but last year we got them on floppy disk in a tab-delimited ASCII format.\n\nFrom there, it was a simple matter to import the data using Excel's import wizard and then to move it over to Access. Teaming up with the political reporter, who knows the precincts like the back of his hand, we were able to isolate model precincts and analyze which way the vote went. The result: Young families and minorities came out in force to support a move they deemed important. The condo vote stayed home. With turnout only about 5 percent, it didn't take much to turn the election.\n\nBoating accidents\n\nIt was a real tragedy. A powerboat - the kind dope smugglers and men in midlife crisis love - was speeding up the Intracoastal Waterway in the dark of the night when it ran over a small cabin cruiser out for a midnight cruise, just about cutting it in half. Six people died, and more were critically injured- ing it the worst boat accident in years and one that we covered intensely.\n\nWe already had about ten years' worth of data on recreational boating accidents from the Florida Marine Patrol. The dirty little secret was that it was still in the fixed-field ASCII format it arrived in. That meant writing import specifications for four tables with lots and lots of fields. And on deadline, in a competitive situation, that wasn't practical. (Remember what I said about \"a little bit of forethought?\" I should've imported it ahead of time.)\n\nNICAR to the rescue. A quick phone call to Missouri and the good folks at NICAR extracted five years' worth of Florida data for us from the national data set they have. And may God bless the inventors of the Internet - within an hour it was winging its way into my e-mail box.\n\nOnce in-house, it became clear that we'd only be able to use two years' worth of data because of changes in the field layout. But, with only a few straightforward queries, we were able to find out how many boating fatalities there had been during that time statewide and along our stretch of the coast, how many accidents, the boats most commonly involved in accidents and the most common cause of accidents. We printed out the information, turned it over to a graphic artist and put the data in the next day's paper. We could have done better, though. As is too often the case, there was insufficient communication between the data guy (me), the reporter and the editor on the story. That meant the reporter didn't get the data soon enough and was insufficiently briefed on what it meant. Keeping everyone on the same page is real important.\n\nMultiple births\n\nA local woman gave birth to quadruplets a few months ago. And, of course, the hospital PR folks were on the phone with us all agog and, of course, we prepared to go out and do something on the blessed event. (This was before the \"Magnificent Seven\" in Ohio made mere quadruplets passé.)\n\nDiscussing this in the morning news meeting, an editor said words to the effect of: \"Geez. Every time quads are born we take the same picture and write the same story. Can we do something different this time? I wonder how much it will cost to raise these kids?\"\n\nWe went out on the Web and found, oddly enough, a college professor's class lectures noting the USDA's annual cost estimate of child-rearing. A quick trip to the USDA site produced a press release announcing the most recent estimate and a full report on the estimates in .pdf format. The USDA estimates how much it costs to raise one child. But you can't just multiply that by four to get the cost or raising quads.\n\nA call to the contact named on the press release got us to a USDA economist who told us the formula we needed. Plugged it all into a spreadsheet and handed it off to the graphics guy.\n\nReligious migration\n\nWith around 270,000 Jewish residents, Broward County makes up the fourth largest Jewish community in the country. That's why we pay a fair amount of attention to Jewish issues. So, when a long-established synagogue decided to sell its building to a charter school and move a few miles north to a more densely populated area, it was news. The local Jewish federation had recently released a demographic study of the local community with a table showing the number of Jewish residents in each ZIP code. They faxed it to us, and a few minutes later it was typed into a database table. We'd recently started playing with mapping software and were fortunate enough to have a copy of ArcView. It didn't take much to hang the ZIP code table onto a base map of ZIP codes we'd gotten from the county. (Forethought worked here - the base map was all ready, completed by my colleague Dan Keating.) Once we drew the map, we saw clearly that the two ZIP codes surrounding the synagogue were among the densest Jewish areas in the county. And the area to which the congregation was moving was among the least populated. That let us ask much more focused questions and explain that, even though the synagogue was located in a Jewish area, most of its members didn't live there. The area where the congregation was moving was closer to where most congregants lived and, more importantly, near a neighborhood where young Jewish families were moving. Were any of these stories prize-winners? Hardly. And, with the possible exception of mapping software, they didn't use any software other than the standard spreadsheets, database managers and web browsers we all have. It's just a matter of thinking about the stories we're writing and how a little bit of data can make them better.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-clientserver-solutions","slug":"1998-february-clientserver-solutions","url":"https://dwillis.github.io/uplink/article/1998-february-clientserver-solutions.html","year":1998,"month":"February","monthNum":2,"headline":"Client/server solutions","kicker":"ADVANCED SEMINAR","author_name":"George Landau","author_title":"NICAR instructor","affiliation":"NICAR","summary":"NICAR held an inaugural advanced seminar teaching experienced computer-assisted reporting specialists how to build web-based, client/server data warehouses using Microsoft technologies (Windows NT, SQL Server, and Internet Information Server) to enable newsroom-wide data access without network slowdowns.","keywords":["databases","newsroom","technology"],"topics":["Technology","Investigative Methods","Journalism & Newsroom"],"technologies":["Windows","SQL Server","IIS","Access","Approach","Paradox","Excel","FoxPro","ODBC","BCP","UniQuery"],"full_text":"Take a dozen experienced CAR jockeys from around the nation, lock them in a NICAR computer lab for three days at the mercy of two hard-core data wonks, and what do you get?\n\nFor starters, you get a dozen CAR jockeys who can install and maintain a Web-based, client/server data warehouse for their newsrooms. And you get to brag about the successful inaugural run of NICAR's advanced seminar.\n\nThe advanced seminar arose from a basic question that our crowd has been wrestling with for quite a while: how can CAR specialists manage their valuable data so that the newsroom can perform basic queries without bugging the specialist, and perform fancier queries without a lot of logistical hassles and network slowdowns?\n\nThe answer is a client/server database manager with a Web interface. For this particular seminar, the teachers - myself and Tom Torok of the Philadelphia Inquirer - instructed our colleagues in the 100% Microsoft solution that consists of Windows NT as the operating system, SQL Server as the database manager, and Internet Information Server (IIS) as the Web server. This all-Microsoft lineup is certainly not the only way to go, but Torok and I believe it provides a strong combination of low-cost, high-performance and predictable results. Besides, what journalist really wants to learn UNIX?\n\nDay One\n\nWe began with an hour-long discussion of client/server databases and Web technology - why they make sense for sharing data within a newsroom. The idea behind client/server is that the database queries run on a beefy PC (the server), which accepts commands over a network in the form of SQL statements. The server executes the SQL statement and passes only the results back over the network to the user (the client). The client software can be a Web browser or a desktop data tool like Access, Approach, Paradox or Excel. The client software can use a graphical interface, like a Web form or the Access query builder, to generate the SQL statements; your users don't have to know SQL.\n\nClient/server is generally a better solution than simply putting an Access or FoxPro database on a server and letting your users open the database over the network. While this might work for simple queries on indexed fields, for anything more ambitious it means pumping the entire database through the narrow (and typically congested) pipes of your network. This tends to make the network crawl for everyone else in your newsroom, especially the folks in photo who inevitably need to move several large images at the same time your query is scanning a 1-gigabyte table.\n\nSo that they could repeat the process in their own newsrooms, led the participants through the entire drill of installing Windows NT Server, SQL Server, IIS and the numerous bug fixes (\"Service Packs\") that Microsoft provides subsequent to any product's release. Fortunately, we hit just enough snags in this process to make for an authentic experience, and by day's end each participant was the new administrator of a fully-loaded NT Server.\n\nDay Two\n\nWe spent the morning learning about security, configuration and administration in NT Server, SQL Server and IIS. We sailed through the creation of SQL Server devices, databases, tables, views and indexes. We dabbled with various settings and discussed ways to maximize performance.\n\nOnce we had a grip on SQL Server, we used ODBC (Open Data Base Connectivity) to connect from Microsoft Access to SQL Server over the network. We transferred data from Access to SQL Server using ODBC alone and in combination with SQL Server's BCP (Bulk Copy Program).\n\nWith some data on the server, it was time to install a Web interface. NewsEngin provided all seminar participants with an application called UniQuery and a license to deploy it in their newsrooms. UniQuery's greatest value lies in its ability to perform global searches of the databases on your server. A user can enter a name, address, birthdate, etc.; regardless of what those fields are actually called, in the particular tables being searched, the user gets back search hits in the form of hyperlinked data to follow to related or detailed data.\n\nUniQuery is written as an Active Server Pages (ASP) application, which is essentially a version of Visual Basic for Microsoft's Web server. Microsoft has positioned ASP as the language of choice for NT applications (like UniQuery) that do most of their work on the Web server, as opposed to the browser.\n\nDay Three\n\nUsing an online workbook he'd written as an ASP application, Torok guided everyone through the key steps in building a Web application that can query or collect data. ASP applications are just Web pages that contain ordinary HTML and Visual Basic Script (the text aspect of VB - the \"If...Then,\" \"For...Next\" kind of stuff). Among the topics he covered were the use of conditional logic and loops, input forms, the \"post\" and \"get\" HTTP actions, input validation, URL redirection, dynamically generated hyperlinks, submit buttons, the \"request,\" \"response\" and \"session\" objects, string functions, numeric functions, database querying and database navigation. It was an awesome performance that left folks wishing for a fourth day spend more time applying it all. Alas, most of us had a plane to catch. But there's always e-mail...","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-risky-behavior","slug":"1998-february-risky-behavior","url":"https://dwillis.github.io/uplink/article/1998-february-risky-behavior.html","year":1998,"month":"February","monthNum":2,"headline":"Risky Behavior","kicker":"HEALTH CARE WORKSHOP","author_name":"Seth Hemmelgarn","author_title":"","affiliation":"NICAR","summary":"Journalists and epidemiologists gathered at the first Risky Behavior conference to share expertise in health care reporting, learning how to use statistical analysis tools and CDC data to produce accurate health stories.","keywords":["journalism","statistics","health"],"topics":["Health & Environment","Investigative Methods","Journalism & Newsroom"],"technologies":["SPSS"],"full_text":"Risky Behavior\n\nWhen you're talking about things like margins of error, chi-squares and weighted data, it's easy to sound esoteric. But journalists and epidemiologists from around the country transcended the stereotype Jan. 23-25 at the first-ever Risky Behavior health care conference. Sponsored by IRE and NICAR and the Centers for Disease Control and Prevention, the workshop gave journalists and epidemiologists the chance to share ideas and concerns face-to-face. \"Workshops Risky Behavior put beat-specific, analysis-oriented sessions over geekism,\" said David Heath, director of computer-assisted reporting at the St. Louis Post-Dispatch. \"I think that they serve to make CAR more relevant in the newsroom.\"\n\nPeople on both sides came away from the conference with a new appreciation for each other. \"I think the researchers learned just as much about how reporters are using CAR as we learned about the rigors of social science,\" said Paul Garber, CAR specialist at the Post-Dispatch.\n\nThe two groups have at least one common goal: giving people accurate information. Both sides use extensive research to do this.\n\nA large part of the seminar was spent discussing the CDC's Behavioral Risk Factor Surveillance System. The survey questions in the system cover topics ranging from seatbelt use to AIDS testing. Answers are compiled in a database that the CDC makes available.\n\nSarah Cohen, NICAR training director, taught participants how to get statistics from the data using SPSS, a statistics program that lets users compile statistics with the push of a button.\n\n\"The statistics-speak was pretty understandable,\" said Diane Targovnik, news editor of Managed Health Care News. \"All the speakers emphasized that you can't look at data and run to a conclusion.\"\n\nOne researcher clarified for Heath that with enough interest and perseverance, anyone can do fairly sophisticated analysis. \"Not that I expect many reporters to start doing double-blind trials,\" Heath said.\n\nThe conference also included information on other types of health-care data, where to get it and how to use it. A Web site with follow-up information is in the works.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-evaluating-educators","slug":"1998-february-evaluating-educators","url":"https://dwillis.github.io/uplink/article/1998-february-evaluating-educators.html","year":1998,"month":"February","monthNum":2,"headline":"Evaluating educators","kicker":"FIRST VENTURES","author_name":"Jim Ludwick","author_title":"database editor","affiliation":"The Missoulian","summary":"A new journalist uses database management skills learned at NICAR training to help organize and analyze standardized test score data from Montana schools, enabling the newspaper to publish a comprehensive comparison package of school performance across their circulation area.","keywords":["data","schools","database"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["Lotus 1-2-3","Excel","Access"],"full_text":"My first project after boot camp began on the day I returned to the office. Rod Murphy, the associate editor, told me that our education reporter had been trying to determine how various schools had compared in their standardized achievement-test results, that had just become public under a new state law.\n\nThe reporter, Gary Jahrig, had been overwhelmed by paperwork he'd received from the state Office of Public Instruction, nearly 200 pages, covering many of the schools in our circulation area, and wasn't sure how to make sense out of the data. I suggested that we could get the Office of Public Instruction to give us the file for the entire state of Montana in computerized form.\n\nTraining results\n\nI'd attended training at NICAR in early January. I had previous experience, but the NICAR training had been useful for several reasons: it helped me learn to use the latest software; it gave me experience in combining databases; and it provided valuable advice about obtaining and using data from government sources.\n\nWhen I contacted the Office of Public Instruction, I negotiated an appropriate price for the material: no charge.\n\nThe data was on a spreadsheet file, for Lotus 1-2-3. I moved the material to Excel, and from there to Access.\n\nI wanted to identify the schools from particular counties, but the Office of Public Instruction had used code numbers instead of county names. I got a list of the code numbers for each county, and quickly created a database with three fields: the code number, the county name and a designation that indicated whether the county was in the circulation area of our newspaper.\n\nProducing in Access\n\nOnce I linked my database to the material supplied, I could easily identify schools in our circulation area and compare their results to the state as a whole. The data covered test results from the fourth, eighth and eleventh grades, showing an average score for each school in each of five subject areas: reading, language arts, math, science and social studies.\n\nUsing the Access database manager, I produced tables for each subject at each grade level. My tables listed every school in our circulation area, ranked in order of scores from highest to lowest. The tables included four fields: the school, its county, the score, and the number of students who had taken the test.\n\nPackage delivery\n\nA package was published in the Sunday paper. Jahrig wrote the wrap-up story. After interviews with the state superintendent of public instruction and others, he wrote about the test results, the reaction and controversy relating to the use of standardized testing. Another reporter, John Stromnes, wrote a sidebar focusing on one elementary school that had been especially successful. I helped deal with the table and wrote explanatory articles to accompany them. I also made numerous checks and re-checks to make sure I had not asked Access the wrong questions.\n\nThe package was published on Jan. 25. My last day at boot camp was Jan. 9.\n\nI already have three other database projects underway. Two involve data I brought home from NICAR.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-soups-on","slug":"1998-february-soups-on","url":"https://dwillis.github.io/uplink/article/1998-february-soups-on.html","year":1998,"month":"February","monthNum":2,"headline":"Soup's On","kicker":"ON THE INTERNET","author_name":"Nora Paul","author_title":"News Library Director","affiliation":"Poynter Institute","summary":"A guide to online resources and reference materials for journalists covering food inspection and food-borne illness stories, featuring links to FDA databases, university resources, and research materials on foodborne pathogens and outbreaks.","keywords":["food","inspection","safety"],"topics":["Health & Environment","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","Listserv"],"full_text":"To keep up with this month's food inspection stories, this column will link you to good sites with background, glossaries, and resources food inspections, food-borne illnesses and other things that will help you cover these important topics and help those of us who made dieting resolutions for New Year's very glad to never eat anything ever again!\n\nThe Bad Bug Book\nhttp://vm.cfsan.fda.gov/-mow/intro.html\nThe handbook on food-borne pathogenic microorganisms and natural toxins.\n\nIowa State University Extension: Food Safety Project\nhttp://www.exnet.iastate.edu/Pages/families/fs/homepage.html\nThis has got it all. Read Food Safety News daily for story tips and leads and use their list of links to other food safety news sources. Learn how to join the Food Safety Netlistserv for daily updates. Use their excellent compilation of other food safety sites to jump to for background and explainers on different food safety issues.\n\nU.S. Food and Drug Administration: Center for Food Safety and Applied Nutrition\nhttp://vm.cfsan.fda.gov/list.html\nA well organized collection of pamphlet type material on all aspects of food safety and handling. Good consumer background and tips as well as technical explanations of some food-borne illnesses and concerns.\n\nUSDA/FDA Food-borne Illness Education Information Center: Food Safety Index\nhttp://www.nal.usda.gov/fnic/foodborne/fbindex/index.htm\nGet the full-text of the FDA 1997 Food Code and a nice list of listservs to join.\n\nGlossary of Epidemiologic Terms\nhttp://www.cvm.uiuc.edu/courses/vp350/glossary.html\nIf terms like \"Extrinsic incubation period\" and \"gold standard\" and \"horizontal transmission\" are being thrown at you during the interview, be sure you know what they are referring to by using this glossary.\n\nCheck out a couple of the Poynter Institute's Hot News / Hot Research items on food-borne illnesses and outbreaks, researched by David Shedden:\n\nHong Kong Bird Virus\nhttp://www.poynter.org/research/hr/hr_010598.htm\n\nTainted Strawberries Served in School Lunch Lead to Hepatitis Outbreak\nhttp://www.poynter.org/research/hr/hr_040397.htm\n\nMad Cow Disease\nhttp://www.poynter.org/research/hr/hr_032696.htm","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-upcoming-events","slug":"1998-february-upcoming-events","url":"https://dwillis.github.io/uplink/article/1998-february-upcoming-events.html","year":1998,"month":"February","monthNum":2,"headline":"Upcoming Events","kicker":"IRE/NICAR NOTES","author_name":"","author_title":"","affiliation":"","summary":"A comprehensive listing of computer-assisted reporting (CAR) and journalism training workshops offered throughout 1998 by NICAR, IRE, and other organizations at various locations including boot camps, specialized workshops, and convention-based training.","keywords":["training","workshops","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"March 23 - 24, 1998 - Washington, DC NICAR/Medill Intermediate CAR Workshop March 27-29, 1998 - New York, NY IRE Broadcast Workshop May 10-15, 1998 - Chapel Hill, NC NICAR Advanced Boot Camp May 17-22, 1998 - Columbia, MO NICAR Basic Boot Camp June 47, 1998 - New Orleans, LA IRE National Conference featuring panels for every beat, TV Show and Tell and optional day of CAR June 14 - 16, 1998 - Washington, DC NICAR/Medill Basic CAR Workshop July 12-17, 1998 - Columbia, MO NICAR Basic Boot Camp July 24-27, 1998 - Miami, FL On-the-road training for the National Association of Hispanic Journalists Convention '98 July 29-August 3, 1998 - Washington, D.C. On-the-road training for the National Association of Black Journalists Convention '98 August 5-8, 1998 - Chicago, IL On-the-road training for the Asian American Journalists Association Convention '98 August 9-14, 1998 - Columbia, MO NICAR Basic Boot Camp December 10 - 12, 1998 - Washington, DC NICAR/Medill Basic CAR Workshop","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-wipe-away-math-fears","slug":"1998-february-wipe-away-math-fears","url":"https://dwillis.github.io/uplink/article/1998-february-wipe-away-math-fears.html","year":1998,"month":"February","monthNum":2,"headline":"Wipe away math fears","kicker":"STATS FROM THE ROAD","author_name":"Sarah Cohen","author_title":"","affiliation":"NICAR","summary":"Journalists can overcome math anxiety by using practical techniques like rounding numbers to relatable scales, developing numerical intuition through estimation exercises, and learning from how broadcast journalists selectively use meaningful statistics.","keywords":["numbers","estimation","journalists"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"You probably fear math and numbers far less than you suspect. Most reporters can figure a tip on a restaurant tab. They can split the bill easily. Many invest, buy homes, and work with other complex financial calculations regularly. And, unlike me, most can and do balance their checkbooks.\n\nHere are some tricks to get you started in your quest to conquer your math phobia.\n\nTrick #1: Round off.\n\nIf you're talking about the population of the U.S., don't think of 270 million. Think of 250. Or 300. That makes it easy to get a gut feeling of scale. If 12 percent of the U.S. is African-American, round that off to 10 percent, or 15 percent. Now just figure a tip on a $30 dinner bill - somewhere between $3.00 and $4.50 if you're frugal. So the number you're seeking is somewhere between 30 and 45 million people.\n\nThe lesson: Think in small, round, numbers that correspond to something you'd be able to figure out in real life: a tip, a 25-percent-off sale, a 10-percent raise. You can change the scale of virtually every number you'll see to get to that point.\n\nTrick #2: Develop a sense of scale.\n\nAnother trick is borrowed from John Allen Paulos' book, Innumeracy. The book is filled with what Paulos calls simple exercises, which few of us think are simple. It does, though, include one remarkably effective piece of advice: Learn to guess the numbers you look up before you look them up. Being wrong, as you almost always will be, will help cement in your mind the correct answer.\n\nIf you guessed that the U.S. had an economy of $450 billion, and you learned through research that it is closer to $8 trillion, you'll remember your error every time you need to use the number. It also means you'll no longer think of a billion as the biggest number you can imagine, the way an editor did when she changed the word 'billion' to 'million' in a story because she thought it was simply too big to be true.\n\nI once admitted to my boss, jokingly, that I didn't pay attention to numbers - they all looked the same to me. He exploded. He rightly mentioned, not so politely, that I couldn't possibly know whether something was right or reasonable if I had no frame of reference. If I didn't know that the Nixon administration slapped on price controls when inflation was only 6 percent year, how could I possibly call any inflation rate 'small' or 'large'?\n\nYou'll have different scales that are important to your beat: How long the typical person lives, the number of people in your community, or the market share of General Motors. Developing your instinct of what's big and what's little will cut seemingly complex issues down to size, eliminating the meaningless numbers and focusing on the important ones.\n\nTrick #3: Watch television news.\n\nBroadcasters have far less freedom to bore readers with meaningless numbers. In any story, you may quibble with the number the reporter chose to focus on. But you'll become more selective in your use of numbers and avoid the all-too-common story construction of anecdote, nut graf, then numbers you hope your readers will avoid.\n\nTrick #4: Ask your best sources for help.\n\nPeople love sharing certain kinds of secrets. They usually involve secrets that will show you how much they know or how clever they are.\n\nYou might ask your county budget director to show you the math (and the spreadsheets) she uses when she develops the budget each year. You might ask the academic author of a new study to walk through some of the methodology with you. You'll become more adept at listening to people talk about numbers - something that has to come while you try to overcome your fear.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-converting-dates-in-access","slug":"1998-february-converting-dates-in-access","url":"https://dwillis.github.io/uplink/article/1998-february-converting-dates-in-access.html","year":1998,"month":"February","monthNum":2,"headline":"Converting dates in Access","kicker":"TECH TIP","author_name":"Richard Mullins","author_title":"","affiliation":"NICAR/The Missouri School of Journalism","summary":"This article explains how to convert date information stored as character strings in Access databases into proper date data types using a two-step process involving string functions and the CVDate() function.","keywords":["dates","conversion","Access"],"topics":["Data & Databases","Technology","Investigative Methods"],"technologies":["Access","FoxPro"],"full_text":"One of the most common data cleanup needs is converting date information. The two-step process and the concepts for doing this are the same for FoxPro and Access. Arrange the numerals in mmddyy order and put in legal separators (hyphens or slashes) between month, day and year. This result may look like a date, but it's still a string of characters that you can't use for calendar math or date arithmetic. Step two is required to convert this correctly arranged date string into a date data type.\n\nThe only differences are in the details, which are minor, like the British and the American ways spelling \"realisation.\" Other Tech Tips have explained the FoxPro steps. This one will show the Access how-to.\n\nThe word string is used a lot when talking about computer databases. But it's not a technical term; it's a good metaphor that helps you think about what you can do with strings of characters - cut them apart and tie them together. String functions are what you use for the cutting apart. The ampersand symbol is how you tie pieces together.\n\nYou only need two string functions: one that picks out a certain number of characters starting from the left, and another that picks out a certain number of characters starting anywhere in the string.\n\nAssume a table with date of birth information, stored in a character field. I'll call that field cDOB, for \"character date of birth.\" The new field to hold the date data-type I'll call DOB.\n\nIf the cDOB column has data like 112184 then step one involves arranging it into 11/21/84.\n\nHere are the string functions to do that. Left (cdob, 2) & \"/\" & Mid (cdob, 3, 2) & \"/\" & Mid (cdob, 5, 2)\n\nIf the date information looked like 841121, you'd rearrange the pieces accordingly to get the same result of mm/dd/yy.\n\nStep two is converting this character representation of a date into a date datatype.\n\nIn Access, this means wrapping CVDate() around the entire command above. Since there are already lots of parentheses, I spread out the two parentheses belonging to CVDate() to make things clearer. (In Access SQL, anywhere you can put one space, you can put several, or even a new line.)\n\nCVDate ( Left ( cdob, 2) & \"/\" & Mid ( cdob, 3, 2) & \"/\" & Mid ( cdob, 5, 2) )\n\nPutting this fragment together in an update query for our example table (called Employees) looks like this:\n\nUPDATE employees SET dob = CVDate ( Left ( cdob, 2) & \"/\" & Mid ( cdob, 3, 2) & \"/\" & Mid ( cdob, 5, 2) )\n\nThis works, but know that data isn't perfect. Not all combinations of numbers that may have been typed into a government database to represent a real calendar date actually will. We know, and Access knows, that there is no such date as the 30th of February or the 41st of November. If your update query finds any invalid character dates like this, you'll get this message:\n\nMicrosoft Access can't update all the records in the update query.\n\nMicrosoft Access didn't update field(s) due to a type conversion failure.\n\nClick the Yes button to go ahead with the update. All the character dates that couldn't be converted to a calendar date will be set to empty or Null. Run a query to show all the rows where the date column you just updated is Null.\n\nSELECT * FROM employees WHERE dob is Null","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-highlights-from-the-database-library","slug":"1998-february-highlights-from-the-database-library","url":"https://dwillis.github.io/uplink/article/1998-february-highlights-from-the-database-library.html","year":1998,"month":"February","monthNum":2,"headline":"Highlights from the Database Library","kicker":"OSHA","author_name":"Justin Mayo","author_title":"Database Administrator","affiliation":"NICAR","summary":"NICAR has released an updated OSHA database spanning 1974-1997 containing workplace safety information organized into four relatable tables, with experienced computer-assisted reporters sharing practical tips for analyzing the data to uncover workplace safety violations and fatalities.","keywords":["database","workplace","safety"],"topics":["Data & Databases","Investigative Methods","Public Safety"],"technologies":["CD-ROM"],"full_text":"Computer-assisted reporters unite! Now available is a database that NICAR recently updated and that the federal agency overseeing worker well-being maintains.\n\nThe Occupational Safety and Health Administration database is cumulative from 1974 through November 1997. NICAR will slice off information from your state or you can order the entire U.S. database on 4 CDs.\n\nThe OSHA database includes 4 tables that may be analyzed separately or joined using the activity number field appearing in each one. Here are the tables and some of the information they contain:\n\nOSHA: The main table which gives company sites, previous violations, the number of workers employed at the site in the previous 12 months and other OSHA activity at the sites, such as inspections or complaints.\n\nAccidents: Names those injured, identifies the task that was being performed when the accident occurred, indicates whether a hazardous substance contributed to the accident, lists which body parts were injured, and indicates the degree of the injury - for example, whether it was fatal.\n\nHazardous Substances: Describes accidents that involve hazardous substances but that do not involve people.\n\nViolations: Details penalties, types of violations - willful, repeat, serious, the number of employees exposed to the danger.\n\nAlthough this relational database structure can be tricky, it should not scare off CAR neophytes. St. Louis Post-Dispatch reporter, Virginia Baldwin Hick, first used OSHA data in a St. Louis Post-Dispatch series that examined local workplace falls that resulted in fatalities.\n\n\"I cut my teeth in computer-assisted reporting by using OSHA data,\" Hick says. \"OSHA had just put into place new regulations on fall protection. I featured how these regulations were supposed to save lives.\"\n\nHere are some tips Hick, former Post-Dispatch reporter Natalya Shulyakovskaya and reporter Russell Carollo, who used OSHA at the Dayton Daily News, offer for handling the data.\n\nUse ZIP codes. \"Some companies had many, many locations,\" Carollo says. \"If you used company names, like General Motors, you'd end up with a zillion hits. We used ZIP codes to identify locations.\"\n\nLook for deaths. \"One of the more interesting queries is to look for citations that result from fatalities,\" Carollo says. \"Once you've got those, you can get all the paperwork on it because somebody's dead. Then rank those in order of monetary fines.\"\n\nFind the survivors. \"Look through the data field of fatalities, find the names and look up the survivors early,\" Hick says. \"It helped me focus my story and focus my interviews. We're not talking about meaningless data; we're talking about lives. If you zero in on that story, you see how other things fit.\"\n\nExamine penalty fields carefully. \"There are actually three different amounts of penalties that you will see for each inspection,\" Shulyakovskaya says. First comes from the OSHA table; it's called 'total current penalties' and represents the initially assessed fine. Second is the 'current penalty reflecting changes and modifications', which indicates the fine negotiated between the company and OSHA. It appears in the violations table. Third is the 'total penalty dollars remitted for the inspection, which what was actually paid. This appears in the OSHA table.\n\nTake advantage of SIC codes. \"We didn't get into looking at OSHA for dangerous occupations because the occupation field is rarely filled in,\" Hick says, \"but standard industrial codes (SIC) are listed. These are standard codes used throughout federal government. SIC codes indicate how the industry is classified in the federal bureaucracy. You can find out which types of industries have the most fatalities.\"\n\nJustin Mayo is Database Administrator at NICAR. He can be reached at (573) 884-7711 or by e-mail at justin@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-health-care-hints","slug":"1998-february-health-care-hints","url":"https://dwillis.github.io/uplink/article/1998-february-health-care-hints.html","year":1998,"month":"February","monthNum":2,"headline":"Health care hints","kicker":"HANDOUT OF THE MONTH","author_name":"Sarah Cohen","author_title":"Training Director","affiliation":"NICAR","summary":"Health care reporters have access to numerous public data sources including hospital discharge databases, financial records, Medicare reports, insurance payouts, and physician disciplinary records that can reveal important stories about hospital operations, doctor performance, and insurance company practices.","keywords":["healthcare","databases","records"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["9-track tape","Internet","World Wide Web"],"full_text":"Whether you care about doctors, managed care companies, insurance or employee benefits, there's no shortage of data sources for stories in the beat. Each piece of the industry is either heavily regulated or heavily supported by taxpayers, making the records readily available. Here's a taste of what's waiting for you.\n\nHospitals and nursing homes\n\nIt's easiest to start with the biggest, most subsidized and often still regulated industries. Hospital Discharge Databases: The motherlode of data for health care reporters is often in what's known as the \"discharge database\" for each state. Not all states release it. An alternative is the federal Medicare database of hospital discharges, which includes less detail about doctors but adds an important element - whether the patient was readmitted or died soon after release. Discharge data has been used to document the shift toward shorter hospital stays, to find doctors who would talk about reasons for leaving certain hospitals, to identify list prices, or to figure out which procedures managed care companies will cover.\n\nHospital financial records: Don't think these are only for business stories. They include important information on the staffing levels hospitals, the amount of charity care given, and you can estimate how much a nonprofit actually spends-rather than charges-on the care it gives away in return for its special status. Often available through your state health agency or a hospital association in a simpler form.\n\nMedicare cost reports: Any Medicare service that's based on costs rather than fixed prices has become a windfall for some companies. They get paid on the honor system. Then they submit the costs to justify it. Medicare sells these records.\n\nMedicare labor rates: Average wages in each area are used to determine how much hospitals in each area are paid under Medicare. You can use them to figure out how much (or little) employees in your area are making in hospitals.\n\nInsurance records\n\nThese are all under state law. You'll often have to type in the simpler records, like financial reports, yourself. But others are available on tape or database in many states. Malpractice payouts: This is a list of every malpractice claim paid through insurers. In Tampa, reporters found a doctor who had been sued repeatedly over a controversial orthopedic procedure, which included washing bones he used in the operation with a garden hose in his backyard.\n\nWorker's compensation claims: Some private vendors sell access to workers' comp claims by employer.\n\nFinancial records: Any Medicare HMO and many Medicaid HMOs are required to maintain a certain number of private-pay patients, usually through employers. As one official put it, \"private industry will demand standards\" that regulators can't enforce. Look for HMOs that depend too heavily on Medicare payments. If nothing else, Medicare can cut them off from the program.\n\nPhysician and other professional records\n\nYes, the National Practitioners Databank is closed to us. But some companies are compiling records held in it and selling them to anyone who wants to check on disciplinary actions against their doctors in other states. Other databases that may come in handy are:\n\nUPIN numbers: These are the key identifiers that doctors get so they can get paid by Medicare. Having UPINs will open a world of information to you, especially if you get billing records. The tape costs about $200 from the Health Care Finance Agency. Corporate records and partnerships: As physicians form associations to vie for managed care contracts, you can learn more about their business dealings by looking through your state's corporate records. Check the 990 summaries at www.irs.ustreas.gov and look under Statistics of Income for the Exempt Organization master list.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-february-bits-bytes-and-barks","slug":"1998-february-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-february-bits-bytes-and-barks.html","year":1998,"month":"February","monthNum":2,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"A journalism newsletter covering IRE membership information, NICAR listserv discussion forums, staff changes at news organizations, and updates to the Campaign Finance Information Center's database of state campaign contribution data.","keywords":["membership","journalism","campaign"],"topics":["Journalism & Newsroom","Government & Politics","Technology"],"technologies":["World Wide Web","Adobe Acrobat","Internet","Email"],"full_text":"Bits, Bytes and Barks\n\nMembership Message\n\nYou can join IRE via our Web server, or you can download a copy of our membership form in Adobe Acrobat format from the Web site and submit it by fax to (573) 882-5431. For more information, check out http://www.ire.org/membership.html IRE Memberships are $40 for professional journalists and journalism educators, $25 for students and $55 for international journalists. For a detailed account of what IRE has to offer, contact the IRE main office at (573) 882-2042.\n\nNICAR Net\n\nTalk in never cheap on the IRE and NICAR listservs. A wealth of topical discussion awaits you. To subscribe to IRE-L or NICAR-I, send e-mail to listproc@lists.missouri.edu In the body of the message, type: subscribe NICAR-L<your name> subscribe IRE-L<your name> Also, check out the IRE-L and NICAR-L mailing list archives on our website at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in html or in plain text format.\n\nMoving on\n\nIRE board member Shawn McIntosh has moved from The Dallas Morning News to become managing editor at The Clarion-Ledger in Jackson, Miss. If you have recently switched locales, let us know of your whereabouts.\n\nCampaign Finance\n\nJack Dolan, director of the Campaign Finance Information Center, is continuing to update the CFIC Web site at http://www.campaignfinance.org. Campaign contribution data is now available from 6 states - Illinois, Indiana, Kansas, Minnesota, Ohio and Wisconsin. The data can be easily downloaded, and comparisons can be made by joining tables between the state files. CFIC hopes to enable such comparisons to be made online once common fields are established and standardized. CFIC's overriding goal is to acquire campaign contribution data from as many states as possible to allow national contribution patterns to be discerned. The CFIC Web site also has added new stories to its story library, links to related Web sites and the inaugural issue of the newsletter Tracker. To subscribe to the newly formed CFIC listserv, send an e-mail to majordomo@campaignfinance.org In the body of the message, type subscribe CFIC-L yourname <your e-mail address>.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-making-the-waves-tv-transformed","slug":"1998-march-making-the-waves-tv-transformed","url":"https://dwillis.github.io/uplink/article/1998-march-making-the-waves-tv-transformed.html","year":1998,"month":"March","monthNum":3,"headline":"MAKING THE WAVES: TV transformed","kicker":"MAKING THE WAVES","author_name":"Stuart Watson","author_title":"Reporter","affiliation":"WRAL","summary":"A journalist describes his conversion from skepticism to enthusiasm for computer-assisted reporting, illustrating how database analysis of ambulance response times in Durham, North Carolina revealed systemic delays that caused patient deaths, transforming anecdotes into evidence of broader problems.","keywords":["databases","journalism","investigation"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["FoxPro","Access","World Wide Web"],"full_text":"At the 1993 national conference on computer-assisted reporting, my baptism by fire, I counted about 500 participants. According to the roster, exactly ten were from local TV stations. It didn't take a relational database to tell me this computer-assisted thing wasn't exactly sweeping the television world. Surveys tell us a large percentage of Americans get their news primarily - in many cases only - local television. Yet CAR has been slow to catch on. Why?\n\nThe Disclaimer\n\nLet's tell the simple truth: computers can be a pain. The 20th century has endowed us with a legacy of wonderful technology - the car, the television, the remote control garage door opener. Yet no technology has ever demanded as much of the user as the computer. It's easier for this Luddite to assemble toys for all four of my children at 3 o'clock Christmas morning with instructions entirely in Japanese than it is to navigate relational query by example or structured query language in FoxPro. Maybe that's why the rest of you have moved on to Access - and why I'll never leave Fox. I worked too many years huffing and puffing as I climbed that learning curve.\n\nThe Conversion\n\nBut I've since joined the clan. I worship at the altar of Phil Meyer, Richard Mullins and Elliot Jaspin. I like using the phrase \"comma delimited\" in FOIA requests. And, most important, I get a thrill out of finding - even producing - original information. Information that government agencies hadn't thought to look for in their own data. Information that will never be in a press release.\n\nThis time last year a tipster told me it was taking far too long for ambulances to reach patients in Durham, N.C., \"the City of Medicine.\" Duke University boasts some of the finest doctors in the world, but it doesn't do any good if the ambulance is too late. I immediately asked whether a database of ambulance responses existed. It did.\n\nAfter months of negotiation, request, re-request, calculation, conversion of a Julian calendar field, conversion of time in one-thousandths of a second, recalculation for runs that began before midnight and ended after midnight the next day, bugging reporters at WTVT-TV and the Charlotte Observer who had done similar stories, reviewing death records and knocking on doors, I wound up in a neighborhood within two miles of the ambulance headquarters where at least four patients had suffered cardiac arrests, waited for longer than the recommended eight minutes for an ambulance, and died.\n\nThe computer can point you to the anecdote and put it in context. Micro and macro. Individual and system. In the ambulance database, it not only pointed me to the cases but also allowed me to draw conclusions about overall delays. It showed me which neighborhoods had the longest response times. Before the computer I would have had the tip and a collection of anecdotes. Too often these descend into a \"this is an isolated incident\" quagmire. Databases can set this to rest.\n\nOnline enhancement\n\nDone correctly, CAR allows us to draw conclusions independent of spin doctors and point us to great stories. Thanks to the Web, we also can pass along that original information to viewers in a form they can use. Information like:\n\na list of charities that use professional solicitors and what percentage of the donation really goes to the charity\na county-by-county database of school violence reports for every school in the state\nan alphabetical, searchable database of restaurant inspection reports including inspectors' comments\n\nI get a thrill out of finding original information. Information that government agencies hadn't thought to look for in their own data. Information that will never be in a press release.\n\nWhat we used to send in a self-addressed, stamped envelope to viewers, we can now simply post on WRAL Online. What might take hundreds of pages to print out, we can post in an easily searchable format. Computer-assisted reporters can place public information directly in the hands of the public. I can tell you from my e-mail that viewers love it.\n\nA CAR-powered litany\n\nWRAL has used databases and spreadsheets in many other reports, including:\n\na judge-by-judge comparison of drunk driving conviction rates to accompany a series of reports on court delays produced by defense attorneys shopping for lenient judges\nan analysis of state traffic accidents that confirmed anecdotes of motorists dying of fatigue on a certain stretch of I-95\na state database of cell phone calls showing that top state employees were using state phones to place personal calls\na database of gas station inspections showing which stations were charging for a gallon while not pumping a gallon and which stations were charging for premium octane while pumping regular octane\na school-by-school \"report card\" with adjusted test scores for poor students that looked beyond raw scores to see how well schools taught the students they were assigned\nan evaluation of school violence statistics that found dubious reporting methods and raised questions about supposedly declining school violence - a position trumpeted in front page headlines of the state's largest newspapers\nan analysis of ambulance response times which revealed patients waiting too long for help to arrive\na review of speeding tickets to show the likelihood state troopers would write a ticket at less than ten miles an hour over the posted limit\na comparison of a military malpractice claims database to the National Practitioners Databank that revealed how the military withheld claims information (with the Dayton Daily News)\na comparison of Public Citizen's list of 13,000+ disciplined doctors to databases of military doctors (also with the Dayton Daily News)\nan analysis of restaurant sanitation inspections that found 95 percent of restaurants in the area get an \"A\" grade even though some had maggots, rats, or sewage backup\nan analysis of funeral homes pointing to the increase in corporate ownership\n\nI could go on at length with other examples of computer-assisted reporting for television - many of which you can find in IRE publications or on the first IRE Feed - available on VHS for a pittance.\n\nCAR bloopers\n\nIt might be more useful - though embarrassing - to tell of the bigger mistakes I've made. Mistakes peculiar to television. Mistakes perhaps you can avoid.\n\nMost can be summed up in the well-worn phrase: \"The data is not the story.\" Take the ambulance response tale. Yours truly placed the gripping 911 tape and the grieving relative at the end of the story after several minutes of big-picture overview, documents and babble.\n\n(I know. I know. I heard TV remotes clicking all over eastern North Carolina. I violated TV 101 and may be forced to turn in my hair dryer and secret decoder ring.)\n\nIn our series on judge shopping and drunk drivers, we placed all the anecdotes in the overview - separate from the conviction rates instead of sprinkling them throughout the other pieces. As a result, the material dried up in most parts.\n\nKeeping the customer satisfied\n\nWe know better. But after weeks of doing your own data entry, you tend to become so attached to the data that you want to show them off to viewers. I don't think they care. Especially if \"How I Got This Story\" is interfering with telling the story properly.\n\nThen there's the eternal question: How much do we mention Mr. Computer? I remember hearing a TV reporter six or seven years ago at an IRE seminar say we should praise the computer in our reporting at every turn. I've since come to view this as a mistake. We never referred to telephone-assisted reporting or FOIA-assisted reporting. Good reporting has always just been good reporting.\n\nThen there's the eternal question: How much do we mention Mr. Computer? Good reporting has always just been good reporting. I hope the computer will become so commonplace soon that calling attention to its use will become gratuitous. It may be already.\n\nI hope the computer will become so commonplace soon that calling attention to its use will become gratuitous.\n\nIt may be already. I also say this because TV is a visual medium, and I've used pictures of the computer screen too often. Too many of us have to stare into computer screens all day long at work. We don't want to turn on the TV at home and see them again. There's almost always a better picture - namely, the picture of the subject of the report.\n\nSharpest tool in the shed\n\nAfter producing several computer-assisted reports, I got a message from our promotions department: \"Can we get away from the computer and use more, say, hidden cameras?\" (If you watch KCBS-TV's report on restaurant inspections, you'll find the two are not mutually exclusive.) Promotions saw all the computer references and scrolling the data as death on toast.\n\nI think they have a good point. We don't have to tell viewers every phone call we made to get a story. We don't have to refer to every database or every FOIA request, denial, and battle.\n\nComputers are an appropriate and extremely powerful tool. And as the major source of information for a significant slice of America, we as local TV reporters should know how to use them.\n\nI hope the computer will become so commonplace soon that calling attention to its use will become gratuitous. It may be already.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-air-apparent-update","slug":"1998-march-air-apparent-update","url":"https://dwillis.github.io/uplink/article/1998-march-air-apparent-update.html","year":1998,"month":"March","monthNum":3,"headline":"AIR APPARENT Update","kicker":"AIR APPARENT","author_name":"","author_title":"","affiliation":"","summary":"This journalism newsletter showcases computer-assisted reporting (CAR) initiatives across broadcast and print newsrooms, featuring award-winning journalists who use data analysis to investigate stories about paramedics, traffic violations, and campaign finance.","keywords":["journalism","reporting","data"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Don't touch that dial. With increased frequency, CAR is airing it out at broadcast stations across the country. Tune in.\n\nIn this issue, former boot camper and 1997 IRE award winner Stuart Watson details a CAR-prolific WRAL and offers tips on keeping the audience interested in the story, not the computer work. Two authors relate their search to find the compelling example to illustrate the importance of their computer analysis: former boot camper Chris Heinbaugh on paramedics lying about their criminal backgrounds and Nancy Amons on abusers of speeding and parking ticket loopholes. Plus there are reviews of stories by former boot camper Paul Adrian and Mark Lagerkvist.\n\nLongtime CAR-user Ray Robinson of The Virginian-Pilot also checks in with a CAR-intensive campaign finance story prompted and assisted by the Virginia Public Access Project.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-campaign-finance-access-granted","slug":"1998-march-campaign-finance-access-granted","url":"https://dwillis.github.io/uplink/article/1998-march-campaign-finance-access-granted.html","year":1998,"month":"March","monthNum":3,"headline":"CAMPAIGN FINANCE: Access granted","kicker":"CAMPAIGN FINANCE","author_name":"Ray Robinson","author_title":"","affiliation":"The Virginian-Pilot","summary":"The Virginian-Pilot investigated campaign contributions from a nondescript Virginia Beach office building and uncovered how businessman Edward S. Garcia Sr. funneled nearly $200,000 in contributions through multiple corporate entities to hide the true source of his political donations. Through database analysis and investigative reporting, journalists traced the money back to Garcia and found he had contributed $257,000 total to various political candidates while concealing his involvement.","keywords":["contributions","investigations","politicians"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["Nexis","Dialog","Dun's Market Identifiers","World Wide Web"],"full_text":"The small, one-story office building nestled along a major thoroughfare in Virginia Beach couldn't be more nondescript. But when The Virginian-Pilot decided to use a database of state campaign contributions to see which addresses were generating the most money for political candidates, we found something astonishing.\n\nThe tiny building at 1120 Laskin Road had generated nearly $200,000 in contributions to candidates for statewide and legislative offices in Virginia over the past four years. That made it easily the richest stop in our area for politicians raising cash. What, we wondered, was going on? Reporter Toni Whitt and news researcher Diana Diehl teamed up to find out.\n\nAfter six weeks of hard reporting and research, they were able to tell our readers more than simply who the money was coming from. Through interviews, database searches of obscure business publications and public records, they were able to determine what the contributions were buying. Their story was a classic example of using good, old-fashioned investigative reporting to transform a good CAR story into a great story about what politicians mean when they say that big contributors are only buying \"access\" - and nothing more.\n\nHidden agenda\n\nIn the case of Edward S. Garcia Sr., the man behind the contributions, that access helped him bring in $6 million for one of his many business ventures.\n\nVirginia is an Eastern but is the Wild West of campaign finance. Anything goes. There are no limits on campaign contributions from individuals or companies, but Garcia funneled the money through half a dozen corporations, his relatives or his employees. Four of those companies were located in the Laskin Road building, setting off the alarm bells when we ran our address analysis. We can't say why Garcia chose to contribute his money that way. But the effect was clear: the real source of the money was hidden from anyone looking over candidates' campaign finance reports.\n\nGarcia has long had a reputation as something of a swashbuckler, involved in controversial real estate developments and indicted on gambling charges. At one point, U.S. Sen. Charles Robb of Virginia refused to accept campaign contributions from Garcia. (Robb later changed his mind, accepted $8,000 and helped Garcia with his business venture.)\n\nSeeking the source\n\nPinpointing Garcia as the source of the money was relatively easy. Using property tax and incorporation records, Whitt and I quickly found that all of the Laskin Road companies had connections to Garcia.\n\nWe began assembling a list of other companies Garcia controlled and running them through the campaign contribution database, turning up even more contributions to state politicians. When we ran the same entities through the Federal Election Commission database, we found contributions to members of Congress, such as Robb, and to congressional leadership PACS. The total over four years came to $257,000. And of that, a whopping $110,000 went to one candidate: a state senator from Virginia Beach named Kenneth Stolle.\n\nWhere's the catch?\n\nFrom the Virginia General Assembly Web site, we downloaded and plowed through copies of bills sponsored by Stolle. We retrieved and read through 20 years' worth of clips on Garcia. We used Nexis and Dialog to search newspapers nationwide for references to Garcia and his companies. We used online financial databases such as Dun's Market Identifiers to pull up information on the finances of Garcia companies. Whitt called those we knew were plugged into the power network in Virginia Beach. We came up empty. Diehl returned to online databases and drilled down to the next level, searching the thousands of obscure magazines and newsletters that cover specific industries in detail.\n\nFrom one covering the insurance industry, Diehl found that United Benefit Administrators Inc., a Garcia company, was embroiled in a dispute with insurance regulators in several states who claimed it was operating an unlicensed health insurance plan. The article opened the door.\n\nWhitt found that Garcia, who employed a handful of police officers at his amusement park, had actually launched the health insurance venture in partnership with the officers' union. By claiming to be a union health plan, rather than regular insurance company, Garcia and the union could sell policies to anyone they wanted without undergoing the scrutiny of state insurance regulators. At least 10,000 signed up in five states from 1992 to 1997. To our knowledge, none were actually police officers.\n\nEnter politicos, stage left\n\nHow the plan stayed alive for so long is where the politicians came in. Garcia had received an entrée to the office of then-Gov. George Allen from Stolle, the single biggest beneficiary of the businessman's campaign contributions.\n\nRegulatory relief\n\nWe can't be sure exactly what happened after that. But employees in Virginia's legislative liaison office in Washington - basically the Commonwealth of Virginia's official lobbyist - were soon squiring a Garcia employee around the capital, introducing her to politicians and officials at the Labor Department to get federal relief from troublesome state regulators.\n\nAmendments were tacked onto bills making their way through Congress. Virginia's U.S. senators wrote letters to federal regulators. A lawsuit was filed to force the Labor Department to rule that Garcia's venture was in fact a legitimate union health plan.\n\nWhen Garcia's health plan was sued by Virginia insurance regulators, he was represented by Norfolk lawyer Tom Moss, who moonlights as Speaker of the Virginia House of Delegates and received $20,000 in campaign contributions from Garcia.\n\nA Pyrrhic victory?\n\nIn the end, the regulators won. The insurance plan shut down late last year. But not before Garcia and the union made a pile of money. The best part of any story is the one tiny detail that by itself means nothing, but speaks volumes when placed in context. Whitt found it while plowing through the records of Virginia insurance regulators' case against Garcia's health plan: After bringing in $800,000 in dues and $6 million in management fees for Garcia's company, the plan faced a fine for operating in violation of Virginia's insurance laws. The amount of the fine: $30. Ray Robinson can be reached by email at rsquared@pilotonline.com\n\nmore than simply who the money was coming from. Through interviews, database searches of obscure business publications and public records, they were able to determine what the contributions were buying. Their story was a classic example of using good, old-fashioned investigative reporting to transform a good CAR story into a great story about what politicians mean when they say that big contributors are only buying \"access\" - and nothing more.\n\nHidden agenda\n\nIn the case of Edward S. Garcia Sr., the man behind the contributions, that access helped him bring in $6 million for one of his many business ventures.\n\nVirginia is an Eastern but is the Wild West of campaign finance. Anything goes. There are no limits on campaign contributions from individuals or companies, but Garcia funneled the money through half a dozen corporations, his relatives or his employees. Four of those companies were located in the Laskin Road building, setting off the alarm bells when we ran our address analysis. We can't say why Garcia chose to contribute his money that way. But the effect was clear: the real source of the money was hidden from anyone looking over candidates' campaign finance reports.\n\nGarcia has long had a reputation as something of a swashbuckler, involved in controversial real estate developments and indicted on gambling charges. At one point, U.S. Sen. Charles Robb of Virginia refused to accept campaign contributions from Garcia. (Robb later changed his mind, accepted $8,000 and helped Garcia with his business venture.)\n\nSeeking the source\n\nPinpointing Garcia as the source of the money was relatively easy. Using property tax and incorporation records, Whitt and I quickly found that all of the Laskin Road companies had connections to Garcia.\n\nWe began assembling a list of other companies Garcia controlled and running them through the campaign contribution database, turning up even more contributions to state politicians. When we ran the same entities through the Federal Election Commission database, we found contributions to members of Congress, such as Robb, and to congressional leadership PACS. The total over four years came to $257,000. And of that, a whopping $110,000 went to one candidate: a state senator from Virginia Beach named Kenneth Stolle.\n\nWhere's the catch?\n\nFrom the Virginia General Assembly Web site, we downloaded and plowed through copies of bills sponsored by Stolle. We retrieved and read through 20 years' worth of clips on Garcia. We used Nexis and Dialog to search newspapers nationwide for references to Garcia and his companies. We used online financial databases such as Dun's Market Identifiers to pull up information on the finances of Garcia companies. Whitt called those we knew were plugged into the power network in Virginia Beach. We came up empty. Diehl returned to online databases and drilled down to the next level, searching the thousands of obscure magazines and newsletters that cover specific industries in detail.\n\nFrom one covering the insurance industry, Diehl found that United Benefit Administrators Inc., a Garcia company, was embroiled in a dispute with insurance regulators in several states who claimed it was operating an unlicensed health insurance plan. The article opened the door.\n\nWhitt found that Garcia, who employed a handful of police officers at his amusement park, had actually launched the health insurance venture in partnership with the officers' union. By claiming to be a union health plan, rather than regular insurance company, Garcia and the union could sell policies to anyone they wanted without undergoing the scrutiny of state insurance regulators. At least 10,000 signed up in five states from 1992 to 1997. To our knowledge, none were actually police officers.\n\nEnter politicos, stage left\n\nHow the plan stayed alive for so long is where the politicians came in. Garcia had received an entrée to the office of then-Gov. George Allen from Stolle, the single biggest beneficiary of the businessman's campaign contributions.\n\nRegulatory relief\n\nWe can't be sure exactly what happened after that. But employees in Virginia's legislative liaison office in Washington - basically the Commonwealth of Virginia's official lobbyist - were soon squiring a Garcia employee around the capital, introducing her to politicians and officials at the Labor Department to get federal relief from troublesome state regulators.\n\nAmendments were tacked onto bills making their way through Congress. Virginia's U.S. senators wrote letters to federal regulators. A lawsuit was filed to force the Labor Department to rule that Garcia's venture was in fact a legitimate union health plan.\n\nWhen Garcia's health plan was sued by Virginia insurance regulators, he was represented by Norfolk lawyer Tom Moss, who moonlights as Speaker of the Virginia House of Delegates and received $20,000 in campaign contributions from Garcia.\n\nA Pyrrhic victory?\n\nIn the end, the regulators won. The insurance plan shut down late last year. But not before Garcia and the union made a pile of money. The best part of any story is the one tiny detail that by itself means nothing, but speaks volumes when placed in context. Whitt found it while plowing through the records of Virginia insurance regulators' case against Garcia's health plan: After bringing in $800,000 in dues and $6 million in management fees for Garcia's company, the plan faced a fine for operating in violation of Virginia's insurance laws. The amount of the fine: $30. Ray Robinson can be reached by email at rsquared@pilotonline.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-the-disclaimer","slug":"1998-march-the-disclaimer","url":"https://dwillis.github.io/uplink/article/1998-march-the-disclaimer.html","year":1998,"month":"March","monthNum":3,"headline":"The Disclaimer","kicker":"","author_name":"Stuart Watson","author_title":"","affiliation":"WRAL-TV","summary":"The author reflects on the complexity and difficulty of using computers compared to other 20th-century technologies, explaining why despite the challenges of learning FoxPro, they remain committed to it rather than switching to easier alternatives like Access.","keywords":["computers","technology","learning"],"topics":["Investigative Methods","Technology"],"technologies":["FoxPro","Access"],"full_text":"Let's tell the simple truth: computers can be a pain. The 20th century has endowed us with a legacy of wonderful technology - the car, the television, the remote control garage door opener. Yet no technology has ever demanded as much of the user as the computer. It's easier for this Luddite to assemble toys for all four of my children at 3 o'clock Christmas morning with instructions entirely in Japanese than it is to navigate relational query by example or structured query language in FoxPro. Maybe that's why the rest of you have moved on to Access - and why I'll never leave Fox. I worked too many years huffing and puffing as I climbed that learning curve.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-the-conversion","slug":"1998-march-the-conversion","url":"https://dwillis.github.io/uplink/article/1998-march-the-conversion.html","year":1998,"month":"March","monthNum":3,"headline":"The Conversion","kicker":"","author_name":"Stuart Watson","author_title":"","affiliation":"WRAL-TV","summary":"A journalist investigates ambulance response times in Durham, North Carolina by obtaining and analyzing a government database, demonstrating how data journalism can uncover information that agencies haven't considered themselves.","keywords":["ambulance","database","journalism"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":[],"full_text":"But I've since joined the clan. I worship at the altar of Phil Meyer, Richard Mullins and Elliot Jaspin. I like using the phrase \"comma delimited\" in FOIA requests. And, most important, I get a thrill out of finding - even producing - original information. Information that government agencies hadn't thought to look for in their own data. Information that will never be in a press release.\n\nThis time last year a tipster told me it was taking far too long for ambulances to reach patients in Durham, N.C., \"the City Medicine.\" Duke University boasts some of the finest doctors in the world, but it doesn't do any good if the ambulance is too late. I immediately asked whether a database of ambulance responses existed. It did.\n\nAfter months of negotiation, request, request, calculation, conversion of a Julian calendar field, conversion of time in one-thousandths of a second, recalculation for runs that began before midnight and ended after mid-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-columbus-discoveries","slug":"1998-march-columbus-discoveries","url":"https://dwillis.github.io/uplink/article/1998-march-columbus-discoveries.html","year":1998,"month":"March","monthNum":3,"headline":"Columbus discoveries","kicker":"MAKING THE WAVES","author_name":"Curt Poff","author_title":"","affiliation":"NICAR","summary":"An investigative reporter used data analysis to expose that Ohio's Bureau of Worker's Compensation significantly understated the average time between injury claims and payment, discovering the real lag time was 106 days instead of the claimed 42 days, with contested claims taking even longer at 272 days.","keywords":["compensation","claims","delays"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":[],"full_text":"A hostage standoff in the Bureau of Worker's Compensation offices in Columbus, Ohio, focused the media's sights on perceived bureaucratic apathy of the agency. An armed man, angry because the bureau ignored his injury claim, stormed into the government building and took several hostages. Though the situation ended without gunfire or injury, it set off a firestorm of similar complaints taking aim at the bureau.\n\nPaul Adrian, an investigative reporter for WBNS-TV who attended NICAR boot camp in May 1996, engineered a data-intensive segment involving worker's compensation claims and the delay between claims and payment.\n\nAdrian pounced on a BWC fact sheet that claimed the average time between an injured worker's claim and initial payment was 42 days. Analysis of the state's records indicated the average statewide lag time was, in fact, 106 days.\n\nThe discrepancy came from the time frame the BWC used in their assertion. The bureau's figures were based upon an average time that spanned only four months. If they had used a larger sample period, their numbers would have changed dramatically. They also based their figures on claims actually paid and disregarded the pending claims that languish away in the appeals process.\n\nThis led Adrian to ask why there was such long wait between injury claims and compensation. The answer seemed simple: employers dispute the claims for financial reasons.\n\nA fact sheet claimed the average time between an injured worker's claim and initial payment was 42 days. Analysis indicated the average statewide lag time was, in fact, 106 days.\n\n\"While that's one of the reasons, it's not the only one,\" Adrian said. \"If you pulled out all the contested claims, I think you would find that the lag was still much longer than claimants want.\"\n\nContested claims\n\nA second trip into the data showed that when claims are contested, the average waiting period for the first worker's compensation payment shot up to 272 days. \"The 272-day figure is for the top 12 companies. If you look at all the companies that appeal claims, the average is 152 days.\"\n\nAcross Ohio, Adrian located the companies that contest the highest percentage of claims. The list included well-known entities like the City of Cleveland, United Dairy Farmers, Raytheon Appliances and Ameritemp.\n\nAdrian interviewed the attorneys handling Ameritemp's worker's injury claims. They simply said that with the amount of claims filed, the company can't afford to let every case go unchallenged - a high number of claims translates into high insurance premiums for employers.\n\nUSDA graded\n\nAdrian recently beefed up another story with data analysis.\n\nUsing meatpacking and slaughterhouse data maintained by the USDA, he discovered an area plant with multiple violations. The data, known as PDR (process deficiency report) summary data, contain a list of violations by individual plants. Although detail of the violations doesn't extend beyond classification as either a minor, major or critical violation, the total number of inspections performed is listed. The seriousness of the plant's violations can be discerned by examining the number of critical or major violations compared to the number of inspections performed. Adrian used the data to confirm that a processing plant in nearby Bowling Green was one of Ohio's major violators. After finding that the plant supplied meat products to several local food chains, he employed basic reporting skills to add background. He found the plant's USDA inspector, who was responsible for filing most of the critical reports - from there the story took on a life of its own.\n\nSlaughterhouse jive\n\nThe inspector was embroiled in a lawsuit filed by an employee of the processing plant who alleged sexual harassment by the inspector. Adrian says he's still sorting through \"some juicy PDR's.\" Curt Poff can be reached by email at curt@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-pension-plunder","slug":"1998-march-pension-plunder","url":"https://dwillis.github.io/uplink/article/1998-march-pension-plunder.html","year":1998,"month":"March","monthNum":3,"headline":"Pension plunder","kicker":"MAKING THE WAVES","author_name":"Joe Stange","author_title":"","affiliation":"NICAR","summary":"Mark Lagerkvist of News 12 Long Island conducted a six-month computer-assisted investigation into pension plan theft, analyzing 6 gigabytes of government data to uncover systemic weaknesses in the 401(K) system and produce an award-winning three-part series that led to multiple indictments.","keywords":["pension","investigation","database"],"topics":["Investigative Methods","Business & Economy","Data & Databases"],"technologies":["Nexis","9-track tape","PC"],"full_text":"Mark Lagerkvist of News 12 Long Island was on a plane home from Los Angeles in 1995 when he got the idea for a computer-assisted story that would later consume six months of his life.\n\nThe USA Today he was reading contained a one-paragraph blurb about a 401(K) plan that had been pilfered by an employer.\n\n\"Is this just an isolated event or is there something systemic?\" Lagerkvist asked himself. \"I became more curious. Once I started pulling clips, doing Nexis searches, and started looking at reference materials, it became apparent there were weaknesses in the pension system.\"\n\nNo plan's grand\n\nLagerkvist, grappling with truly massive amounts of data, began squeezing out those disparaging weaknesses:\n\nMost pension nest eggs are being watched by the employers. As Lagerkvist says, this allows \"the fox to guard the henhouse.\"\n\nFewer than 10 percent of plans are legally required to have audits on any kind of regular basis. Most are not insured by the government.\n\nThere's no way the resources of federal agencies can effectively protect the nation's 750,000 private pension plans from piracy.\n\nEventually \"Stolen Dreams\" was broadcast in a three-part series on News 12, followed by a half-hour documentary. Following the broadcasts, several indictments were handed down to Long Island-area 401(K) embezzlers. One such case turned out to be the largest known case of pension theft in the country.\n\nData behemoth\n\nHowever, the road was long, rough and littered with dirty data. Lagerkvist had to put up with what he called a \"computer-assisted reporting nightmare.\" He went mano a mano with 6 gigabytes of raw government data, produced after a much-delayed FOIA request. Lagerkvist was not initially equipped to deal with the enormity. \"The database basically included an abstract for all 750,000 pension plans in the U.S. on 27 nine-track tapes. At the time I was using a 486 with a 400-meg hard drive. I had to go out and buy a 2-gig hard drive just to handle this project. Even then I had to be very selective what kind of data I imported.\"\n\nField selection\n\nThe next step: the harrowing narrowing process.\n\n\"I think the key is to find out what data is important and what fields of data and information are important for your research. You can always go back later and create a second database. It's important to start basic and add the things you think are critical.\"\n\nLagerkvist advises any reporter with an urge to work with a massive database to do some homework first in order to get a solid idea of what oddities to look for.\n\n\"When you get a massive amount of data like this, you don't go into a project like this without a clue. I didn't go out and get those tapes just because they were there.\"\n\nThe sheer size of the project was unlike anything he had encountered in his years of investigative reporting. And while he asserts his insistence on being the only one to \"get his hands dirty\" with the data, he still recommends finding a mentor who knows more about computer-assisted reporting.\n\nLagerkvist acknowledged his indebtedness to Drew Sullivan, once with NICAR and now with the Associated Press.\n\n\"It's helpful to have someone to bounce the technical questions off of. Of course NICAR would tell me that's why they exist, and that's fine, too.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-problem-paramedics","slug":"1998-march-problem-paramedics","url":"https://dwillis.github.io/uplink/article/1998-march-problem-paramedics.html","year":1998,"month":"March","monthNum":3,"headline":"Problem paramedics","kicker":"MAKING THE WAVES","author_name":"Chris Heinbaugh","author_title":"reporter","affiliation":"KNXV-TV","summary":"A Phoenix TV station conducted a four-month computer-assisted investigation that uncovered paramedics and EMTs who falsified their criminal histories on job applications, revealing that Arizona's state authorities failed to verify applicant backgrounds. The investigation required innovative database matching techniques and extensive manual verification work to identify approximately 60 cases worthy of further journalistic pursuit.","keywords":["paramedics","investigation","database"],"topics":["Investigative Methods","Public Safety","Crime & Justice"],"technologies":["FoxPro","Excel","9-track tape","CD-ROM"],"full_text":"\"Who's in the Ambulance,\" four-month, computer-assisted investigation, involved trusted public servants: paramedics and emergency medical technicians. While at KNXV-TV in Phoenix, we discovered dozens of paramedics who had lied about their criminal history on their applications, despite laws requiring disclosure. News 15 discovered they were getting away with it because the state wasn't bothering to check.\n\nOn the surface, our report appeared to be a simple relational database story. Instead, it ended up being an extended exercise in improvising computer investigations with imagination and persistence.\n\nOur first challenge was finding a \"criminal\" database to run against the EMT/paramedic database I had obtained from Arizona Emergency Medical Services. Unfortunately, after nine months of trying, efforts to get the state's corrections and jail database weren't going anywhere.\n\nWe figured the next best place would be the courts. After we acquired the index for the Maricopa County Court on nine-track, NICAR processed it into a FoxPro database format on two CD-ROMs.\n\nA paperful society\n\nHaving separated the criminal from the civil cases, we were still faced with other challenges. In the index, for example, the first, middle and last names were all in one field. Not understanding string functions, we used Excel to parse the million names, about 26,000 at a time. Once separated, we joined the databases by first and last name, which brought thousands of matches. Normally, one major identifier would help narrow the matches down, but the index did not include dates of birth.\n\nSo photographer Jon Taylor, associate producer Matt Goldberg and I literally read a thousand cases, checking not only dates of birth but also charges and outcomes. We could access some cases online, but many others were only available the old-fashioned way: in files or on microfilm. After two months we finally had pinpointed about 60 cases to begin working with.\n\nReady to go with the story? No way. All this work merely got us to the starting gate. We still needed to cull the great stories from the stack of cases, verify facts, make more records requests, and track down offenders and victims. Most daunting, we had to find the video and graphics to make it all great TV news.\n\nMining for gold\n\nCAR is another tool in your bag of tricks to flesh out the best stories. And that's ultimately what this is about. You can compile the facts you want with CAR, but if you can't or won't take the time to mine that information for compelling stories - especially TV stories - why should your audience care?\n\nWhile our print colleagues can rely on documents and phone interviews to bring these stories to life, we must also have video. We need people talking on camera, must conduct long, drawn-out stakeouts, and use graphics when the video just isn't there.\n\nThe emotion is what makes our medium unique. Without these elements, you won't have a compelling story.\n\nStruggling to find a compelling criminal to convince our viewers that the problem was real, I learned a great lesson about CAR: Don't let all your hard work blind you. Be willing to walk away. Only by admitting this did I find our \"story.\"\n\nThe original database we had was of certified paramedics and EMTs. The worst cases would probably no longer be certified and so wouldn't show up in the data. While this seems obvious now, I was so determined to find the story in \"my\" data that I ignored my instincts.\n\nInterstate hideaway\n\nA friendly source within the agency rescued me by suggesting I request a list of revoked certifications. With that data, we uncovered a paramedic - not on my list - who had a criminal history, lied about his past, and got certified. And he then went on to steal from patients. He violated his probation, left Arizona and got certified in Florida, where he stole from patients again. We later learned he lied on the Florida application as well. On top of that, we tracked him to California, where he was doing it all again.\n\nWe already had the hard facts from the CAR work. And now we had the compelling story to make people care, to convince people that the system needed repair, that no one was checking the backgrounds, that people were slipping through, and that people were being hurt. Because we went beyond the CAR, beyond just the facts, it prompted action. legislators announced they would address the issue in the next session. Those voices included Arizona's House Speaker, who had opposed fingerprint background checks for these workers in the past.\n\nIt's hard to spend this kind of time on a story, but you don't have to drop everything to do it. You have to work smarter and be organized. For us, what made the tedium worth it was that we were pretty damned sure our competitors would never put this kind of work into a project and what we'd end up with would stand out from the pack. And it did.\n\nChris Heinbaugh can be reached at cheinbaugh@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-car-hits-the-road","slug":"1998-march-car-hits-the-road","url":"https://dwillis.github.io/uplink/article/1998-march-car-hits-the-road.html","year":1998,"month":"March","monthNum":3,"headline":"CAR hits the road","kicker":"MAKING THE WAVES","author_name":"Nancy Amons","author_title":"Journalist","affiliation":"WSMV","summary":"A Nashville newsroom's data analysis of speeding tickets uncovered a vehicular homicide that had fallen through the cracks, leading to the indictment of chronic speeder Corey Jenkins and revealing systemic failures in reporting traffic violations to state authorities.","keywords":["speeding","investigation","homicide"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["CD-ROM","Access","PC"],"full_text":"The daughter of the late Robert Woodson said, \"we've been calling and calling and you're the first person to call us.\" She was talking about her family's efforts to prosecute the young, chronic speeder who killed her father, a retired widower. What we thought would be a simple database project to profile the worst speeder in Nashville ended up uncovering a vehicular homicide that had fallen through the cracks. The Woodson family was waiting for the district attorney's office to prosecute Corey Jenkins, but no one there had heard of the fatal accident until our investigation.\n\nOur four-part, week-long \"Speed Demons\" series began when we requested a police department database of three years of speeding and parking tickets. For a $450 computer-time charge, the police downloaded a CD-ROM with 500,000 records of very clean data. The data also included several disposition fields added by the clerk's office. Cleaning and analyzing the data took just a few days for each project.\n\nLicensed to kill\n\nUsing Access 7.0 and a 486 computer (both have since been upgraded, thank goodness), a simple totals query named our most prolific speeder: 20-year-old Corey Jenkins. He'd had 13 tickets in three years and still had a valid license. When we took a closer look at his accident reports on file with the Department of Safety, we found out about Woodson, who had been fatally injured when Jenkins sped through a red light. Then we discovered the police department had failed to forward the case to the district attorney's office. Jenkins was indicted shortly after our story aired.\n\nWe found some other unexpected surprises in the speeding data. One field indicated whether speeding tickets had been reported to the state drivers' license office. About one-fourth of them had not. We found a long-forgotten law was allowing people to keep speeding convictions off their driving record as long as they were going less than 20 miles an hour above the speed limit on the interstate.\n\nAccess helped us find the people with the most unreported tickets. We profiled one of them, a man who would have lost his license if not for this law. Appropriately, his license plate read \"ISPEED.\" One legislator wanted the law just the way it was, admitting it helps him keep tickets off his driving record.\n\nTo round out the \"Speed Demons\" series, we profiled the officer who had written the most tickets, named the locations where the most tickets are written, and looked at traffic-calming measures being tried in other cities.\n\nStart to finish, the project took three weeks with a team of three: myself, an intern, and a photojournalist. Here's some advice to others considering a similar project: Pull paper records to back up every important ticket and accident. Find the victims. When you hit a data-analysis snag and can't find your way out, post a message to the NICAR listserv. Time after time, journalists came to my rescue. Finally, use a faster computer. Some of my queries took hours.\n\nTickets to hide\n\nThe parking ticket database and three-part \"Meter Hogs\" series was fun. Again, we looked for chronic violators. We found the worst offenders parking overtime on a meter were state employees. Worse, they had free parking a few blocks away and a shuttle bus to get there. We profiled a woman in the Department of Finance and Administration who accumulated so many thousands of dollars' worth of unpaid tickets that she declared bankruptcy. Yet she continued to park illegally just outside her office. The database gave us her tag number, so we found her car and were waiting for her when she left work one afternoon. She resigned when our story aired.\n\nA related series also profiled people parking illegally on handicapped placards. One of our violators was the assistant to the director at the DEA. Our viewers said they especially appreciated our holding such violators accountable.\n\nNancy Amons can be reached at (615) 353-2284 or by email at amons@wsmv.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-plug-in-to-the-net","slug":"1998-march-plug-in-to-the-net","url":"https://dwillis.github.io/uplink/article/1998-march-plug-in-to-the-net.html","year":1998,"month":"March","monthNum":3,"headline":"Plug in to the 'Net","kicker":"ON THE INTERNET","author_name":"Jonathan Oatis","author_title":"","affiliation":"Reuters America","summary":"This article surveys popular Internet browser plug-ins and software tools, including both free and shareware programs like Adobe Acrobat Reader, Winzip, RealAudio, and Shockwave that enhance web browsing and multimedia experiences.","keywords":["software","browser","multimedia"],"topics":["Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","Netscape","Internet Explorer","Adobe Acrobat","WinZip","RealAudio","RealVideo","VDOLive","QuickTime","Shockwave"],"full_text":"Netscape Communications Corp.'s Navigator browser and rival Microsoft Corp.'s Internet Explorer are the main tools for navigating the Internet, but there are various accessories, called plug-ins, that further enhance the experience. These computer programs, and a treasure trove of other software, are available on the Internet.\n\nSome of this software is \"freeware,\" i.e., free. Some is shareware. You download it and try it out. If you decide you want to keep it, you register it, paying the author's price. Most shareware is based on the honor system. Some is set up in such a way that, if you don't pay for it, it is disabled after a set time.\n\nAdobe Acrobat Reader: Adobe Acrobat is a program that enables users of different types of computers to convert their files into a form readable by all computers, so long as those computers are using the free Adobe Acrobat Reader software. Adobe makes its money from the software used to convert the files into \".pdf\" (for \"portable document format\") files. Many Web sites, including government and corporations, offer documents in \".pdf\" form.\n\nYou can find Acrobat Reader at Adobe's Web site: http://www.adobe.com\n\nWinzip: One of the most useful shareware programs to have. Winzip compresses software and data, saving space on your hard drive and allowing you to put massive data files and programs onto a few floppy disks. The newest version also encodes binary files like sound, video and picture files into text, making them easier to send over the Internet.\n\nYou can find Winzip at http://www.winzip.com\n\nRealAudio: Allows you to hear audio sent over the Internet, whether it be a radio story you may have missed when it was first played, a replayable recording of a statement at a news conference or music from an \"Internet broadcast station\" that uses the Net instead of the airwaves to transmit its programing.\n\nYou can find the free RealAudio program at http://www.realandio.com The site has a link to RealAudio's guide to audio programming on the Internet, http://www.timecast.com\n\nProgressive Networks, the folks who make RealAudio, also make RealVideo, which allows you to view video transmitted over the Internet. This program, however, requires a fairly powerful computer and a speedy Internet connection.\n\nVDOLive: Another free program that allows users to view real-time video. It can be found at http://vdolive.com Like RealAudio, VDOLive's site also features a guide to sites that produce programming that can be viewed with the software.\n\nQuicktime: Apple Computer's free program enables you to play movies on your computer, either downloaded from a Web site or while online. You can find it http://www.apple.com\n\nShockwave: Adds interactivity, including games, interactive maps and animation, to Web sites. The free \"viewer\" can be found at http://macromedia.com\n\nThere are scores of other plug-ins and useful auxiliary programs. If you use Netscape Navigator, many are available at Netscape's Web site: http://home.netscape.com\n\nNetscape and its Internet software archrival, Microsoft, are constantly upgrading their own programs in their war over market share. Netscape's site also contains its products. For Microsoft's wares, go to http://www.microsoft.com\n\nOther software can be found at Shareware.com (http://wwww.shareware.com), a nice site maintained by the C-Net computer news service.\n\nAnother great site is TUCOWS (The Ultimate Collection of Winsock Software), which features a rich, categorized selection of software and is notable for its opening picture of two cows. TUCOWS has mirror sites scattered around the world. Enter the main site (http://www.tucows.com), then look for the one closest to you.\n\nIf those don't do the trick, try Jumbo (http://www.jumbo.com)\n\nGood luck, but remember. If you are downloading at work, have everything you download, no matter how good the source, checked by a technician for viruses.\n\nThis column is excerpted from a handout provided at Indiana CAR, the 1998 National Computer-Assisted Reporting Conference.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-sex-criminals-exposed","slug":"1998-march-sex-criminals-exposed","url":"https://dwillis.github.io/uplink/article/1998-march-sex-criminals-exposed.html","year":1998,"month":"March","monthNum":3,"headline":"Sex criminals exposed","kicker":"FIRST VENTURES","author_name":"Matt Goldberg","author_title":"Investigator","affiliation":"KPRC-TV Houston","summary":"A Houston newsroom used computer-assisted reporting to analyze Texas sex offender registration data, creating a searchable database and pamphlet to help viewers identify registered offenders in their neighborhoods.","keywords":["database","offenders","investigation"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Excel","Access"],"full_text":"It's been done in almost every market. They're your neighbors. They teach your children. And they're convicted sex offenders.\n\nSo when we were working on the sex offender notification process for Texas a few months back, the question of how to do the story in a different way came up. I sent an open records request to the Department of Public Safety to find out exactly how many sex offenders were living in the greater Houston area. In Harris County alone, it turned out we had 1,058 registered offenders since 1995. Then the bells went off. Why not get the state's database containing the names and locations of all registered sex offenders? We sent our request but found out the state only had the information on paper because they had only recently started converting to computer records. So we decided to test our computer-assisted skills.\n\nOffender overload\n\nWe called the state and told them to send us each sex offender's registration form. That would have been 20,000 documents, so we settled for Harris County records. A week later we had more than 1,000 forms.\n\nWe entered the information into an Excel worksheet. With the help of some associate producers and interns, we spent about a week entering each person's name, date of birth, street of residence, city of residence, ZIP code, and nature of offense. It took us about a week to enter and double-check the data for typos.\n\nThe next step was moving the data to Access. God, Microsoft made this easy! I imported the data into a new database using the column headings as the field names.\n\nScanning for specifics\n\nNow that we had a solid database, we had to find out exactly where these criminals were living. We wanted to find specifics for the story, so we ran a count of the number of sex offenders in each ZIP code. We were even able to figure out the total numbers living on any given street in a ZIP code.\n\nWe also wanted to learn which crime occurred most frequently. We ran a count of each type of crime and discovered that indecency with a child was the most prevalent. Using the techniques from before, we also had the most common sex crime in a given location.\n\nWith the data needed for the broadcast version of the story already in hand, we wanted to take the story further and give viewers something to use on an \"as-needed\" basis. So we created a pamphlet based on the database. Figuring viewers would want to know exactly how many sex offenders were living in their neighborhood, we sorted the data by ZIP code. How to make the list reader-friendly? Access is great for creating reports. We designed a standard report with the Access wizard, used the reports design mode to change its look, and added an explanation for how to read the data, a legal disclaimer, and our investigator's logo.\n\nBut what about viewers wanting to check whether a particular person was on the list but didn't know where he or she lived? No problem. We saved the database as an *.html file and posted it on our Web site. We repeated sorting the data by different fields. Now viewers could on and run a search by name, city, ZIP code, offense or date of birth. Mission accomplished.\n\nThe Aftermath\n\nThe sex offender story was a great success. To date we have received numerous requests for a copy of the pamphlet and our Web site receives thousands of hits. The database continues to be updated. With each new registered sex offender, we simply add their information into a new record. In addition, I created a viewing for the rest of the newsroom. Our staff can run searches of the database in a read-only environment without damaging the data. The whole project took about three weeks. And the hardest part was the data entry!\n\nMatt Goldberg can be reached at (713)778-4843 or by email at news2matt@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-upcoming-boot-camps","slug":"1998-march-upcoming-boot-camps","url":"https://dwillis.github.io/uplink/article/1998-march-upcoming-boot-camps.html","year":1998,"month":"March","monthNum":3,"headline":"UPCOMING BOOT CAMPS","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"The article announces upcoming investigative journalism boot camps and workshops scheduled for May through August 1998 at various locations, with both basic and advanced training options available.","keywords":["bootcamp","training","journalism"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"UPCOMING BOOT CAMPS: MAY 10-15, 1998 - ADVANCED BOOT CAMP IN CHAPEL HILL, N.C. MAY 17-22, 1998 - BASIC BOOT CAMP IN COLUMBIA, Mo. JULY 12-17, 1998 - BASIC BOOT CAMP IN COLUMBIA, Mo. AUGUST 9-14, 1998 - BASIC BOOT CAMP AUGUST 14-16, 1998 - ADVANCED BOOT CAMP ON INTRANETS IN COLUMBIA, Mo. FOR A COMPLETE LISTING OF BOOT CAMPS AND WORKSHOPS, ALSO VISIT OUR WEB SITE AT WWW.IRE.ORG/RESOURCES/CONFERENCES/TRAINING","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-cleaning-tables","slug":"1998-march-cleaning-tables","url":"https://dwillis.github.io/uplink/article/1998-march-cleaning-tables.html","year":1998,"month":"March","monthNum":3,"headline":"Cleaning tables","kicker":"TECH TIP","author_name":"Andrew Lehren","author_title":"Journalist","affiliation":"Dateline NBC","summary":"A tutorial on writing a FoxPro program to automatically clean up data by standardizing character columns to uppercase and left-aligned formatting, using a scan loop and lookup table approach.","keywords":["FoxPro","data","programming"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["FoxPro"],"full_text":"I don't remember data scrubbing one of the allures of journalism, but it is a perennial topic on the NICAR listserv. All those 'replace all... FoxPro commands can be tiresome, particularly on lengthy tables. But this needs to be done - especially if we want to make sure queries work right.\n\nLast year, at one of NICAR's programming seminars, someone asked whether it's possible to write a FoxPro program that would clean up data automatically with commands that make sure every character column is uppercase and flush left. After a few minutes, I came up with this.\n\nIt's pretty straightforward. For those just learning FoxPro programming, it shows one way to use a scan loop. It relies on just one memory variable. The logic is pretty simple: After some stage setting, the program creates a lookup table so it knows the character fields. Next, the program goes through each character field, one at a time, and runs the 'replace all' commands.\n\nIf you want everything in proper case (with the first letter uppercase and the rest lowercase), you can easily alter this program. This is a stripped-down version to focus on the essence of the program, but you can spruce it up if you're so inclined.\n\nWithout further adieu, let's step through the program:\n\nTo make sure nothing else is open, and that FoxPro starts with a clean slate, we'll begin with these three lines. The first line clears the screen. The second closes any open tables, programs and just about anything else lying about. The third flushes out memory variables that could mess up what follows. Now FoxPro should be pretty clearheaded.\n\nclear close all clear all\n\nWe'll tell FoxPro where to work. When you use this for real, you'll have to adjust by putting in the path for the folder that holds your tables. So change c:\\YourFolder to the path you want.\n\nset directory to \\YourFolder\n\nNext, we'll tell FoxPro what table to clean. You'll need to change TableName. The phrase 'in 1' is just a spot on FoxPro's imaginary desk - like saying I'll put this stack of papers on the left corner so that later, when I open another stack of papers, I'll have some free space on the right side. This makes it easier to go between the two stacks. The 'alias ToClean' means that, from here on out, I'll be calling the table ToClean. This is important because it saves us the trouble of having to edit the rest of the program. You shouldn't need to make any more changes to make this program work for you.\n\nuse TableName in 1 alias ToClean\n\nNext, we create the lookup table. It's just a record layout in a Fox table. That's done with 'copy structure extended.' If this seems bewildering, just run this command on your favorite table and look at the result. It may help you better understand what will follow.\n\ncopy structure extended to Lookup\n\nFoxPro will now open the lookup table in another spot on its imaginary desk.\n\nuse Lookup in 2\n\nHere's the promised memory variable. It will remember the name of every character field.\n\nStore to NameOfField\n\nLet's start going through the lookup table line by line. That's what happens with a scan loop.\n\nselect lookup scan\n\nWe'll check for character fields, and remember their names. Let's say the field name is \"Address,\" and it's a character field. FoxPro now knows that.\n\nif field_type = 'C' Store Lookup.Field_name to NameOfField\n\nTime to flip to our table to be scrubbed.\n\nSelect ToClean\n\nNow we'll make FoxPro run the 'replace all.' The ampersand, by the way, is needed so FoxPro knows we're referring to a memory variable, not a column called literally NameOfField.\n\nreplace all &NameOfField; with alltrim(upper(&NameOfField))\n\nWe're done with that column. Let's see what character columns come next. Will it be a city column? A state column? Whatever it is, we'll be ready, looping through our lookup table one row at a time.\n\nSelect Lookup\n\nThe next two lines close our condition and our scan loop. They're the required periods at the end of our programming thoughts.\n\nEndIf EndScan\n\nBy this point, FoxPro has cleaned all the fields. The next two lines just get our processed table ready to check.\n\nselect ToClean use\n\nWe're done. Open it up and revel in all the typing you've avoided.\n\nAndrew Lehren can be reached at 212-664-7739, or by email at alehren@nbc.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-a-bridge-to-car","slug":"1998-march-a-bridge-to-car","url":"https://dwillis.github.io/uplink/article/1998-march-a-bridge-to-car.html","year":1998,"month":"March","monthNum":3,"headline":"A bridge to CAR","kicker":"HIGHLIGHTS FROM THE DATABASE LIBRARY","author_name":"Justin Mayo","author_title":"Database Administrator","affiliation":"NICAR","summary":"A guide to using the National Bridge Inventory database for investigative journalism on bridge conditions, explaining key data fields like sufficiency ratings and status codes while cautioning that the data should serve only as a starting point for further reporting.","keywords":["bridges","database","inspection"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["National Bridge Inventory database"],"full_text":"A bridge to CAR\n\nBy Justin Mayo NICAR Database Administrator\n\nLike water under a troubled bridge.\n\nWhen a bridge collapsed and killed 10 people in 1987, Jonathan Salant, then with the Syracuse Post-Standard, became interested in the subject of bridge conditions. In 1993 he completed a series titled \"The Real Bridges of Madison County,\" which found that New York's bridge maintenance and inspection systems were insufficient.\n\nSalant, now an AP reporter in Washington, D.C., described the inspection system and generated a list of the 10 worst bridges in New York from the National Bridge Inventory database of the U.S. Department of Transportation, which comprehensively details the condition of the nation's bridges. The Federal Highway Administration maintains and releases the database each year. NICAR has the years 1994 through 1996 and can send you the data by state or for the whole country.\n\nThe database structure is quite simple - one table with about 120 fields. Some of the information includes: location, type of structure, maintenance responsibility, owner, year built, average daily traffic, dimensions, inspection dates, and structural evaluations.\n\nTwo fields that can be used for an overall indicator of a bridge's quality are \"sufficiency rating\" and \"status.\" The sufficiency rating is derived from a complex formula explained in the Recording and Coding Guide you'll receive when you order the data. Essentially, a score of 80 percent or less means the bridge probably needs some rehabilitation. A score of 50 percent or less means the bridge may need to be replaced. The status field contains a code to tell you whether the bridge is structurally deficient, functionally obsolete, or not deficient.\n\nThe scoring procedure is complicated. Although the formula for arriving at the sufficiency rating is uniform, the people and the agencies applying the formula are not. The sufficiency and status fields help to begin evaluating bridges but do not tell the whole story. When analyzing the data, Salant found that many bridges in the state had low sufficiency ratings but were not being repaired. However, he stresses that the data are only a starting point. \"You really need to talk to your department of transportation folks,\" he said. \"The data can be old and you need to make sure of the status of the bridges. Don't rely on the data too much. It will only tell you where to look.\"\n\nJustin Mayo can be reached at (573) 882-0684 or by email at justin@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-march-bits-bytes-and-barks","slug":"1998-march-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-march-bits-bytes-and-barks.html","year":1998,"month":"March","monthNum":3,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The 1998 IRE National Conference in New Orleans will focus on investigative journalism and computer-assisted reporting, while the organization updates its resources including NICAR listservs and campaign finance databases covering nine states.","keywords":["conference","investigative","database"],"topics":["Journalism & Newsroom","Investigative Methods","Government & Politics"],"technologies":["World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nIRE National Conference 1998\n\nThe heat may be rising off of Bourbon Street, but inside the Fairmont Hotel will be the cool musings of investigative journalists discussing their craft.\n\nIt's not too soon to start making plans to get to The Big Easy for the 1998 Investigative Reporter & Editors National Conference, scheduled for June 4-7.\n\nVisit IRE's Web site at www.ire.org/resources/conferences/neworleans for information and a registration form. June 4 has been set aside as the day to focus on computer-assisted reporting.\n\nNICAR Net\n\nTalk may be cheap, but it's always valuable on the IRE and NICAR listservs. A wealth of topical discussion awaits you.\n\nTo subscribe to IRE-L or NICAR-L, send an email to listproc@lists.missouri.edu\n\nIn the body of the message, type: subscribe NICAR-L<your name> subscribe IRE-L<your name>\n\nAlso, check out the IRE-L and NICAR-L mailing list archives on our website at http://www.ire.org and http://www.nicar.org. You can see posts to both lists organized by thread, author and date. The list archives are available in *.html or plain text format.\n\nMoving on\n\nDavid Milliron has moved from Gannett News Service to become database editor at The Atlanta Journal-Constitution in Atlanta, Ga. If you have recently switched locales, let us know of your whereabouts.\n\nCampaign Finance\n\nJack Dolan, director of the Campaign Finance Information Center, continues to update the CFIC Web site at http://www.campaignfinance.org.\n\nCampaign contribution data from three more states has now been added. Data is now available from nine states - Idaho, Illinois, Indiana, Kansas, Kentucky, Michigan, Minnesota, Ohio and Wisconsin. The data can be easily downloaded, and comparisons can be made by joining tables between the state files. CFIC hopes to enable such comparisons to be made online once common fields are established and standardized. CFIC's overriding goal is to acquire campaign contribution data from as many states as possible.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-voter-fraud","slug":"1998-april-voter-fraud","url":"https://dwillis.github.io/uplink/article/1998-april-voter-fraud.html","year":1998,"month":"April","monthNum":4,"headline":"VOTER FRAUD","kicker":"","author_name":"Dan Keating","author_title":"","affiliation":"The Miami Herald","summary":"The Miami Herald investigated unusual voting patterns in Miami's 1997 mayoral election, uncovering over 300 fraudulent or suspicious ballots including votes from dead people, non-residents, and felons, using both traditional reporting and computer-assisted data matching against multiple public records.","keywords":["election","fraud","investigation"],"topics":["Crime & Justice","Government & Politics","Investigative Methods"],"technologies":["SAS"],"full_text":"When the election results for mayor of Miami came in on the night of Nov. 4, it seemed obvious that something strange had gone on in the voting, the first for the new \"strong mayor\" form of government that would give the city's chief executive a new level of power.\n\nIncumbent Joe Carollo won a majority of the votes cast at the polls. But he was trounced by a 2-1 margin in absentee ballots by one of his challengers, former mayor Xavier Suarez. The absentee tide stopped Carollo 155 votes short of a outright election, forcing a runoff that he then lost.\n\nThe strange voting pattern and unprecedented number of absentees—more than 10 percent of the ballots cast—prompted Carollo, the Florida Department of Law Enforcement and The Miami Herald to launch investigations into what had gone on.\n\nThe Herald has tracked down and published more than 300 \"problem\" votes so far:\n\nA dead guy.\nPeople who don't live in the city or live in different city commission districts than where they voted.\nPeople who swore they didn't vote, despite a record showing a ballot cast in their names. Many said their names had been forged.\nPeople who said ballots were taken from them.\nPeople who said they were paid $10 to vote absentee.\nPeople who were solicited to vote by a former food stamp eligibility officer who still worked as a facilitator for elderly immigrants.\nPeople whose absentee ballots were \"witnessed\" by people who never saw the voter or by false signatures.\nMore than 100 felons who had lost their civil right to vote.\n\nContacting more than 1,400 voters required a unique shoe-leather effort that lasted more than two months. But the project also involved a heavy dose of computer-assisted reporting to set the plate for the reporters fanning out into the community, organizing what they found and painting the emerging portrait of fraud.\n\nThe essence of the computer work - to find potentially suspicious activity - was matching different lists against the list of people who voted in the election. Though Florida law specifically forbids releasing the voting list to the public, The Herald obtained it. With that voter roll, we could wrest great value out of the other public records that we keep on hand.\n\nMovers and matchmakers\n\nTo start off, we matched the voter list against the Social Security Master Death List we had obtained from NICAR, though that list is somewhat spotty by having fields for ZIP codes but not full addresses. Since the voter list did not include a social security number, we had to match by name, date of birth and ZIP code. To broaden our search, we matched name and date of birth to anyone whose last ZIP code was in our county.\n\nIt looked exciting - we had eight exact matches. But, much to our disappointment, our field reporters found the purportedly dead voters quite alive and quite well: each proved that the rumors of their deaths had been greatly exaggerated. Most told tales of fighting with the Social Security bureaucracy about being listed as deceased.\n\n(We learned from reporters around the country that such errors aren't at all unusual. And the one dead man we did identify didn't appear in the Master Death List.)\n\nPacked voting houses\n\nOur next step was more straightforward. With the list of 44,000 voters, we simply grouped by address and sorted descending to find the addresses housing the most voters. Since addresses included apartment or unit numbers, the search aggregated individual living units rather than apartment buildings and showed many residences with six, seven, eight or more voters.\n\nTo make the list of voters per home more useful, we merged the addresses with the county's property appraiser roll, which we obtain each year for real estate analysis. Matching the addresses didn't go smoothly because the two agencies are inconsistent when it comes to \"3 Ave.\" or \"3rd Avenue,\" and \"S.W. 29th Ter.\" or \"SW 29 Terr.\"\n\nUsing SAS for Windows, we cleaned the addresses from both lists. We removed all punctuation and letters (\"th,\" \"st,\" and \"rd\") from the fields with numeric street names and scanned to replace variations of \"ave,\" \"avenue,\" \"av,\" and so on. Matching with the property roll allowed us to identify single-family homes with one or two bathrooms and more than five voters. It also showed us vacant and non-residential properties with voters. When we visited the properties with many voters, we found out-of-town and phantom voters stashed there. To confirm that people lived elsewhere, we used driver's license and vehicle registration information through AutoTrak. We also used homestead exemptions, a tax deduction that can be taken only on your primary residence. We found people whose driver's license, vehicle registration and homestead records indicated that they lived outside the city. In every case, we spoke directly to the person or, if that was impossible, sent a letter asking for proof of being a legitimate voter.\n\nThe cell bloc\n\nOne seemingly straightforward match proved tricky: matching with the list of people sentenced to prison or probation in Florida. The list, available on CD-ROM from the state Department of Corrections, doesn't include those sentenced to time or probation locally, but it captures a good population of convicted felons, who have lost their right to vote.\n\nBecause the prison list doesn't have an address field and the voter roll doesn't have a social security number field, we matched by first name, last name, middle initial and date of birth for people convicted in our county or neighboring counties. Of the more than 400 matches, almost none were false matches caused by two people having identical names and birthdays. So it looked like we had a lot of illegal voters.\n\nComplications arose when we checked court files for our first batch of interesting felons. Almost none, it turned out, had been convicted. Miami's overloaded court system is notorious for convicting rarely. Almost everyone can reach a plea agreement with \"adjudication withheld.\" About three-quarters of the people sentenced to state probation had not been convicted. People who would probably be convicted elsewhere in the state and lose voting rights had not lost them in our county. We accessed online the local criminal court docket to find which of the 400 offenders had been actually convicted. We determined that more than 130 had been, but the search still wasn't over. We ran those 130 names past the state's Executive Clemency Board that handles restoration of rights. Sorting through the index cards where they keep their records, we found more than 100 felon voters who did not have their rights restored. We also compared the list of people who voted with the entire list of registered voters and looked for people who had changed addresses into the city or between city commission districts near the election. Under Florida's implementation of the motor-voter law, you can announce on election day that you've moved into the district. And you're allowed to vote.\n\nAvailable absentees\n\nComputer matching helped us draw webs of conspiracy involving the organizers of bad votes. A database of people who witnessed absentee ballots showed that many campaign workers witnessed dozens of ballots. If any witnessed ballots that we knew were bad, we checked other ballots handled by them.\n\nWe checked bad ballots against a database of campaign contributions and expenditures we had made from paper records for stories before the election. We found people who used their real, out-of-city address to contribute but an in-city address to vote.\n\nMapping uncovered the operation that paid people $10 to vote absentee in the runoff election. Using ArcView 3.01, we found participants and proved the scope of the operation being run from a parking lot in a poor neighborhood.\n\nIn that neighborhood, the primary election included the mayor's race and a contested city commission race involving a popular candidate.\n\nPeople had months to obtain and fill out an absentee ballot for the election. In the runoff, voters had less than a week to obtain and return an absentee ballot. By mapping absentees within three-quarters of a mile of the parking lot in both races, we discovered unlikely results: Dozens more absentees were in the runoff than in the main election. By comparing the list of voters from the two elections, we found people who had troubled themselves to vote absentee in the runoff but had not bothered to vote at all in the primary election. We targeted that list in our search for paid voters. Several confessed, and others admitted knowing of the operation but denied that they were personally involved.\n\nMaiden Intranet voyage\n\nTo keep track of our contacts of more than 1,400 voters, reporters carried paper forms to collect consistent information. Results were then entered in an Access database. As the project progressed, the Access front-end was replaced by a Web-based version that christened our Intranet.\n\nWe used the Intranet to set up Web-based search forms so everyone could look up reporter notes on voter contacts. Reporters and editors could search lists of people who had voted or witnessed absentee ballots. Empowering team members to handle their own look-up chores was instantly popular and prevented a bottleneck at the geek's office.\n\nAfter a three-week, non-jury trial, a judge in March overturned the election and called for a new vote. An appellate court then went a step further by putting Carollo back into office and declaring that the voting-fraud participants should be punished rather than simply forced to compete in another election.\n\nThe Florida Department of Law Enforcement has followed our tracks frequently, arresting the man we identified as running the paid-voter scam and several of the illegal voters we uncovered. More arrests are pending.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-tech-tip-enter-intranets","slug":"1998-april-tech-tip-enter-intranets","url":"https://dwillis.github.io/uplink/article/1998-april-tech-tip-enter-intranets.html","year":1998,"month":"April","monthNum":4,"headline":"TECH TIP Enter Intranets","kicker":"TECH TIP","author_name":"George Landau","author_title":"","affiliation":"NewsEngin Inc.","summary":"A guide to setting up an Intranet system in newsrooms to enable reporters to search public records efficiently, covering essential infrastructure requirements including TCP/IP networking, Web browsers, and server setup recommendations.","keywords":["Intranet","newsroom","server"],"topics":["Technology","Investigative Methods"],"technologies":["TCP/IP","PC","Macintosh","Netscape","Internet Explorer","Windows","IIS","Personal Web Server","Pentium","Internet"],"full_text":"Who among us wouldn't want a newsroom where reporters can search public records in a flash, without a lot of training and without having to leave the comfort of their ergonomic chairs? All it takes to fulfill the promise of an Intranet are a newsroom full of modern PCs or Macs, a network with Transport Control Protocol/Internet Protocol (TCP/IP), a Web server and a big chunk of your time. There's no way around it: setting up an Intranet for CAR is a lot of work. But if you make the right choices at the outset, your work can pay large dividends. As someone who has been wrestling with this stuff for a few years, I can offer some advice on getting started.\n\nThe network\n\nSince an Intranet uses the same technology as the Internet, your network needs to use the Internet's TCP/IP networking protocol. When your Intranet users fire up their Web browsers, they'll use TCP/IP to connect your internal Web server. Chances are pretty good that your network already has TCP/IP, but you should make sure before diving in.\n\nThe Web browsers\n\nRegardless of which Microsoft operating system you use, make sure the server has plenty of RAM, especially if you intend to design Web applications that search databases. I'd start with at least 32 megs of RAM. If you plan to use databases, go for 64 megs or more. RAM is cheap these days.\n\nIf possible, make sure that all of the potential users of your Intranet are using the same kind of Web browser and that it's Version 3 or later. Netscape 3 and 4 and Internet Explorer 3 and 4 are all fine choices. Both companies' browsers are free these days, but my favorite on both Windows and the Mac OS is IE 4.\n\nIt's not essential that everyone use the same browser software, but it'll make your job easier if they do. That's because when you design a Web page, you'll discover that it can look different depending on the browser with which you view it. While it's best to design pages that look great in both Navigator and IE, this can mean more work for you.\n\nThe Web server\n\nAssuming you already have a TCP/IP network and that everyone on it already has a decent Web browser, the real work will be in setting up your Web server and putting some useful data on it.\n\nIf you're not just dabbling and want a robust Web server, I strongly recommend you use Windows NT Server for the operating system and Microsoft's Web server software, Internet Information Server. Microsoft released IIS 4.0 a few months ago, and it's an excellent product. It's fast, powerful, widely supported and free. You can download it as part of the Windows NT Option Pack from http://www.microsoft.com/ntserver.\n\nIf you can't afford NT Server (about $500), the next best thing is NT Workstation (about $200). NT Workstation has the same underlying code as NT Server, so it's just as fast and just as dependable. You can download a version of IIS for NT Workstation called Personal Web Server, also part of the NT Server Option Pack.\n\nIf your newsroom can't afford NT Workstation and you can't afford to get a job in another newsroom, you can install Personal Web Server for Windows 95. This will work fine in the beginning, but Windows 95 is simply more prone to crashes and does not run as fast as NT. If you want to get your colleagues hooked on your Intranet, it's important that you build a system that runs well with a minimum of downtime.\n\nYou should plan to put all of your Web-searchable data on the Web server's local hard drives. Do not keep the data on a network drive, as this will lead to pokey searches that bog down the entire network.\n\nIn terms of the speed of your CPU, I'd recommend at least a Pentium 133 for your Web server. Faster is definitely better. Fast hard drives are also a good way to boost the performance of your server.\n\nIn terms of the hard disk space need, this depends on how much data you plan to share. Set aside about 500 megs for the operating system and applications (planning ahead for the inevitable code bloat in Microsoft's future offerings).\n\nUnless you'll be using a client/server database like Microsoft SQL Server or Oracle, you should plan to put all of your Web-searchable data on the Web server's local hard drives. Do not keep the data on a network drive, as this will lead to pokey searches that bog down the entire network.\n\nBuilding the Web pages\n\nWhen it comes to building Web applications that search a database, there are basically two popular ways to go. Which one you choose depends on your budget, ambitions and prior experience. Either approach will require that you learn the basics of HTML forms and tables.\n\nApproach #1: Allaire's Cold Fusion (http://www.allaire.com)\n\nCold Fusion (about $500) extends HTML with special tags that support straightforward SQL queries. It includes a well-designed wizard that does most of the work in building basic search forms and result tables. You can tweak the wizard's offering using any text or HTML editor.\n\nCold Fusion works in partnership with Microsoft's IIS (and other Web servers) by acting as a broker for database requests. When a user submits a query, the Web server passes the request to Cold Fusion, which runs the query using your database software and passes the HTML-formatted results back to the Web server, which in turn sends it back to the user. The whole process is pretty fast.\n\nApproach #2: Microsoft Active Server Pages\n\nASP, which is built into IIS and thus costs nothing, is essentially a version of Visual Basic (VB) that runs on the Web server. To use it, you write Web pages that contain a combination of Visual Basic programming and HTML. Your users and their browsers never see the VB part - they get just the HTML. Just as with regular VB, you can write ASP pages that query databases and manipulate the results. If you intend to build a moderately complex Web application with conditional logic and loops, ASP might be a better choice than Cold Fusion. If you already are comfortable with VB or another flavor of BASIC, you'll be able to scan the ASP documentation and get up to speed pretty quickly.\n\nASP works in a similar fashion to Cold Fusion, but since ASP is more tightly integrated with the IIS Web server, it runs a little faster. For normal newsroom use, though, this speed difference isn't noticeable. The speed of a Web database application depends largely on your choice of hardware and database software, not on your choice of ASP or Cold Fusion.\n\nIf you go the ASP route, it's probably worth spending about $200 for Microsoft Visual InterDev, which includes wizards that build some basic database applications for ASP. InterDev does a nice job of managing the various components of a Web application, and it includes a version of the FrontPage HTML editor, so it's worth the money. A new version of InterDev is due soon and is supposed to provide much-needed improvements in the visual design tools (in the current version, \"visual\" pretty much means you're allowed to see your Visual Basic and HTML code as you write it).\n\nManaging the data\n\nActive Server Pages and Cold Fusion both use ODBC (Open Database Connectivity) to access your data. If your database software provides an ODBC driver, these tools should work for you. Access, FoxPro, SQL Server and Oracle all have ODBC drivers that work well for basic Web queries. It's a good idea to make sure you have the latest ODBC driver for your software by downloading it from the database vendor's Web site and installing it on your Web server.\n\nBut if you want folks to be able to use more than a Web browser (Access or Excel, for example) to work with large databases on the server, you should consider setting up a client/server system like SQL Server.\n\nIf you'd rather focus on Web users and don't want to mess with client/server, FoxPro is probably faster than Access and has fewer limitations on database size. Tom Boyer of The Seattle Times has succeeded with an Intranet based on FoxPro, Cold Fusion and Windows NT.\n\nAs for which database software is best suited for newsroom Intranets, that's a question for another Tech Tip.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-ire-resource-center-voter-fraud-stories","slug":"1998-april-ire-resource-center-voter-fraud-stories","url":"https://dwillis.github.io/uplink/article/1998-april-ire-resource-center-voter-fraud-stories.html","year":1998,"month":"April","monthNum":4,"headline":"IRE RESOURCE CENTER VOTER FRAUD STORIES","kicker":"","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"The IRE Resource Center maintains a library of over 12,000 investigative stories, with examples showcasing investigations into voter fraud including felons voting, duplicate votes, deceased voters, and ineligible registrations.","keywords":["fraud","voting","investigation"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"THE IRE RESOURCE CENTER MAINTAINS A LIBRARY OF MORE THAN 12,000 INVESTIGATIVE STORIES. OTHER EXAMPLES OF VOTER FRAUD INCLUDE: \"FELONS VOTING,\" A 1996 WCCO-TV (MINNEAPOLIS) SERIES ON CONVICTED FELONS VOTING WHILE ON PROBATION (#13530 IN THE RESOURCE CENTER) \"VOTER LISTS INVITE FRAUD,\" A 1995 CHICAGO SUN-TIMES LOOK INTO A SYSTEM ALLOWING DUPLICATE VOTES AND VOTES BY THE DECEASED (#I 1059) \"FELONS CAST VOTES DESPITE SYSTEM,\" A 1995 THE HARTFORD COURANT STUDY OF CONVICTED FELONS STILL REGISTERED TO VOTE (#12342) \"DEAD OR ALIVE,\" A 1990 ST. LOUIS POST-DISPATCH RECORD ANALYSIS OF REGISTERED VOTERS REVEALING THOUSANDS OF INELIGIBLE VOTERS, SOME DEAD FOR YEARS (#7284) FOR COPIES OF STORIES, CONTACT THE CENTER AT (573) 882-3364 OR BY E-MAIL AT RESCNTR@IRE.ORG","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-newsroom-boost","slug":"1998-april-newsroom-boost","url":"https://dwillis.github.io/uplink/article/1998-april-newsroom-boost.html","year":1998,"month":"April","monthNum":4,"headline":"Newsroom boost","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"A guide for setting up an intranet in a newsroom, covering the hardware and software requirements including TCP/IP networking, Web browsers, and Web server options using Windows NT and Microsoft's Internet Information Server.","keywords":["intranet","server","database"],"topics":["Technology","Journalism & Newsroom"],"technologies":["TCP/IP","Internet Explorer","Netscape","Windows","IIS","Personal Web Server","Pentium","SQL Server"],"full_text":"WRANGLING TOO LONG WITH A PARTICULAR DATABASE PROBLEM? WOULD YOU LIKE TO SEE YOUR FRUSTRATIONS ALLEVIATED IN THESE PAGES? SEND SUGGESTIONS FOR FUTURE TECH TIPS TO BRENT JOHNSON AT BJOHNSON@NICAR.ORG\n\nTo DISCUSS TOPICS RELATED TO RUNNING AND DEVELOPING CAR INTRANETS, JOIN THE INTRACAR MAILING LIST. SEND AN E-MAIL TO: MAJORDOMO@NICAR.ORG IN THE BODY OF THE MESSAGE TYPE: SUBSCRIBE INTRACAR <YOUREMAILADDRESS\n\nLIST TOPICS INCLUDE MICROSOFT'S INTRANET PRODUCTS, NETSCAPE'S SERVER PRODUCTS, LOTUS NOTES APPLICATION DEVELOPMENT, PROGRAMMING LANGUAGES, AND STRATEGIES TO SHARE DATA WITH THE NEWSROOM AND THE PUBLIC. CONTACT TED PETERSON WITH ANY LISTSERV PROBLEMS AT TED@NICAR.ORG\n\nneeds to use the Internet's TCP/IP networking protocol. When your Intranet users fire up their Web browsers, they'll use TCP/IP to connect your internal Webserver. Chances are pretty good that your network already has TCP/IP, but you should make sure before diving in.\n\nThe Web browsers\n\nRegardless of which Microsoft operating system you use, make sure the server has plenty of RAM, especially if you intend to design Web applications that search databases. I'd start with at least 32 megs of RAM. If you plan to use databases, go for 64 megs or more. RAM is cheap these days.\n\nIf possible, make sure that all of the potential users of your Intranet are using the same kind of Web browser and that it's Version 3 or later. Netscape 3 and 4 and Internet Explorer 3 and 4 are all fine choices. Both companies' browsers are free these days, but my favorite on both Windows and the Mac OS is IE 4.\n\nIt's not essential that everyone use the same browser software, but it'll make your job easier if they do. That's because when you design a Web page, you'll discover that it can look different depending on the browser with which you view it. While it's best to design pages that look great in both Navigator and IE, this can mean more work for you.\n\nThe Web server\n\nAssuming you already have a TCP/IP network and that everyone on it already has a decent Web browser, the real work will be in setting up your Web server and putting some useful data on it.\n\nIf you're not just dabbling and want a robust Web server, I strongly recommend you use Windows NT Server for the operating system and Microsoft's Web server software, Internet Information Server. Microsoft released IIS 4.0 a few months ago, and it's an excellent product. It's fast, powerful, widely supported and free. You can download it as part of the Windows NT Option Pack from http://www.microsoft.com/ntserver.\n\nIf you can't afford NT Server (about $500), the next best thing is NT Workstation (about $200). NT Workstation has the same underlying code as NT Server, so it's just as fast and just as dependable. You can download a version of IIS for NT Workstation called Personal Web Server, also part of the NT Server Option Pack.\n\nIf your newsroom can't afford NT Workstation and you can't afford to get a job in another newsroom, you can install Personal Web Server for Windows 95. This will work fine in the beginning, but Windows 95 is simply more prone to crashes and does not run as fast as NT. If you want to get your colleagues hooked on your Intranet, it's important that you build a system that runs well with a minimum of downtime.\n\nYou should plan to put all of your Web-searchable data on the Web server's local hard drives. Do not keep the data on a network drive, as this will lead to pokey searches that bog down the entire network.\n\nIn terms of the speed of your CPU, I'd recommend at least a Pentium 133 for your Web server. Faster is definitely better. Fast hard drives are also a good way to boost the performance of your server.\n\nIn terms of the hard disk space need, this depends on how much data you plan to share. Set aside about 500 megs for the operating system and applications (planning ahead for the inevitable code bloat in Microsoft's future offerings).\n\nUnless you'll be using a client/server database like Microsoft SQL Server or Oracle, you should plan to put all of your Web-searchable data on the Web server's local hard drives. Do not keep the data on a network drive, as this will lead to pokey searches that bog down the entire network.\n\nBuilding the Web pages\n\nWhen it comes to building Web applications that search a database, there are basically two popular ways to go. Which one you choose depends on your budget, ambitions and prior experience. Either approach will require that you learn the basics of HTML forms and tables.\n\nApproach #1:\n\nAllaire's Cold Fusion (http://www.allaire.com) Cold Fusion (about $500) extends HTML with special tags that support straightforward SQL queries. It includes a well-designed wizard that does most of the work in building basic search forms and result tables. You can tweak the wizard's offering using any text or HTML editor.\n\nCold Fusion works in partnership with Microsoft's IIS (and other Web servers) by acting as a broker for database requests. When a user submits a query, the Web server passes the request to Cold Fusion, which runs the query using your database software and passes the HTML-formatted results back to the Web server, which in turn sends it back to the user. The whole process is pretty fast.\n\nApproach #2: Microsoft Active Server Pages ASP, which is built into IIS and thus costs nothing, is essentially a version of Visual Basic (VB) that runs on the Web server. To use it, you write Web pages that contain a combination of Visual Basic programming and HTML. Your users and their browsers never see the VB part - they get just the HTML. Just as with regular VB, you can write ASP pages that query databases and manipulate the results. If you intend to build a moderately complex Web application with conditional logic and loops, ASP might be a better choice than Cold Fusion. If you already are comfortable with VB or another flavor of BASIC, you'll be able to scan the ASP documentation and get up to speed pretty quickly. ASP works in a similar fashion to Cold Fusion, but since ASP is more tightly integrated with the IIS Web server, it runs a little faster. For normal newsroom use, though, this speed difference isn't noticeable. The speed of a Web database application depends largely on your choice of hardware and database software, not on your choice of ASP or Cold Fusion.\n\nIf you go the ASP route, it's probably worth spending about $200 for Microsoft Visual which includes wizards that build some basic database applications for ASP. InterDev does a nice job of managing the various components of a Web application, and it includes a version of the FrontPage HTML editor, so it's worth the money. A new version of InterDev is due soon and is supposed to provide much-needed improvements in the visual design tools (in the current version, \"visual\" pretty much means you're allowed to see your Visual Basic and HTML code as you write it).\n\nManaging the data\n\nActive Server Pages and Cold Fusion both use ODBC (Open Database Connectivity) to access your data. If your database software provides an ODBC driver, these tools should work for you. Access, FoxPro, SQL Server and Oracle all have ODBC drivers that work well for basic Web queries. It's a good idea to make sure you have the latest ODBC driver for your software by downloading it from the database vendor's Web site and installing it on your Web server.\n\nAs for which database software is best suited for newsroom Intranets, that's a question for another Tech Tip.\n\nBut if you want folks to be able to use more than a Web browser (Access or Excel, for example) to work with large databases on the server, you should consider setting up a client/server system like SQL Server.\n\nIf you'd rather focus on Web users and don't want to mess with client/server, FoxPro is probably faster than Access and has fewer limitations on database size. Tom Boyer of The Seattle Times has succeeded with an Intranet based on FoxPro, Cold Fusion and Windows NT.\n\nGeorge Landau can be reached by e-mail at george@newsengin.com\n\nUPCOMING IN THE MAY ISSUE OF UPLINK IS A REVIEW OF COLD FUSION BY RAY ROBINSON OF THE VIRGINIAN PILOT AND TOM BOYER OF THE SEATTLE TIMES.\n\nSIGN UP FOR THE ADVANCED BOOT CAMP, SCHEDULED FOR AUG. 14-16 IN COLUMBIA, Mo., TO LEARN ABOUT NT, IIS, SQL SERVER, ODBC AND ASP. YOU'LL ALSO GET A FREE COPY OF NEWSENGIN'S UNIQUERY GLOBAL SEARCH SOFTWARE, AN ASP APPLICATION. CONTACT NICAR AT (573) 882-0684. IF YOU CAN'T ATTEND THE SEMINAR, YOU CAN DOWNLOAD ITS HANDOUTS AND ASP EXERCISES FROM THE \"FREE TOOLS\" SECTION OF NEWSENGIN'S WEB SITE, HTTP://WWW.NEWSENGIN.COM.\n\nCONTACT TED PETERSON WITH ANY LISTSERV PROBLEMS AT TED@NICAR.ORG","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-indiana-car-learning-to-share","slug":"1998-april-indiana-car-learning-to-share","url":"https://dwillis.github.io/uplink/article/1998-april-indiana-car-learning-to-share.html","year":1998,"month":"April","monthNum":4,"headline":"INDIANA CAR Learning to share","kicker":"INDIANA CAR","author_name":"Justin Mayo","author_title":"","affiliation":"NICAR","summary":"Computer-assisted reporters at Indiana CAR conference are tackling the challenge of implementing Intranets in newsrooms to enable data sharing among journalists, with discussion focused on client/server systems and the need for collaboration between technology experts and newsroom staff.","keywords":["Intranet","database","newsroom"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["TCP/IP","ODBC","ASP","Intranet","Power Point"],"full_text":"Judging from the activity at Indiana CAR, the next challenge facing computer-assisted reporters will be navigating the quagmire of Intranets. Sounding more like a spelling bee than a journalism conference, terms such as TCP/IP, ODBC and ASP were heard throughout the halls of the Hyatt. And although it's easy to get lost in the technical aspects of setting up an Intranet, there is a simple, journalistic motivation driving CAR specialists and database editors to immerse themselves in acronym hell - sharing data with the rest of the newsroom.\n\n\"A reporter has easier access to a government computer thousands of miles away than a colleague's computer three feet away,\" said Tom Torok of The Philadelphia Inquirer. \"In such an environment, each computer and journalist is an island. However, the information could be made available to all, quickly and securely, through client/server arrangements.\"\n\nAt the conference, Torok, George Landau of NewsEngin Inc., Tom Boyer of The Seattle Times, and Bill Miller of the Battle Creek Enquirer led roundtable discussions on the choices available in setting up an Intranet.\n\n\"Journalism is about answering questions quickly and authoritatively, and a well-stocked client/server system helps reporters do just that,\" Landau said. He pointed out that these systems provide the best way to keep large amounts of clean, reliable data-such as voter registration, property, and corrections records - readily available for deadline queries.\n\nIndiana CAR marked the first time a NICAR conference has offered such extensive coverage of Intranet technology. The response was enthusiastic. \"I was shocked at the number of people we had in our session,\" said Boyer. \"This field is changing so rapidly. As the demand for client/server systems in newsrooms grows, so too will the necessity for CAR specialists to learn the technology. \"The classes and panels gave folks a taste, but I think they wanted more,\" Torok said. \"In the future, I'd like to see an all-day workshop where people can bring in samples of their databases and walk out with a finished application.\"\n\nBut will the trend toward Intranets push CAR specialists into the role of full-time propeller heads? Most people argue that although the impetus for setting up an Intranet will have to come from the newsroom, the long-term goal should be to get the information-systems experts involved.\n\n\"I think the ideal situation might be that you have technology-savvy journalists who can oversee these projects, but full-time database wonks, whose skills are way beyond journalists', to set up and maintain the systems,\" Boyer said.\n\nLandau emphasized that cooperation between the two departments is possible as long as (information-systems) people are working for, not against, the newsroom.\" Client/server realms such as data cleanup and verification, he said, should remain newsroom chores because they involve journalistic judgment.\n\nIn the long term, assuming the technical hurdles of setting up an Intranet don't consume all your energy, you should have more time for the real work of journalism. If all reporters can quickly and easily access the information they need, then CAR specialists may spend more time on hard-core analyses. \"Even if the resident CAR guru has to do most of the work, setting up the server can still save time in the long run because it allows reporters to do more digging on their own,\" Landau said.\n\nOnce a client/server system is in place, Boyer and Torok stress that all kinds of information can shared. The Seattle Times, for instance, is experimenting with free-text queries over its Intranet. Not only can structured databases be shared throughout the newsroom, but Boolean searches of huge text blocks can be run.\n\n\"Don't just think of 'data' as in databases,\" Torok said. \"Just about any form of information - spreadsheets, text, graphics, even Power Point presentations - can be manipulated and made searchable to suit your staff's needs.\"\n\nIn the long term, assuming the technical hurdles of setting up an Intranet don't consume all your energy, you should have more time for the real work of journalism. If all reporters can quickly and easily access the information they need, then CAR specialists may spend more time on hard-core analyses.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-foxpro-to-access-and-back-again","slug":"1998-april-foxpro-to-access-and-back-again","url":"https://dwillis.github.io/uplink/article/1998-april-foxpro-to-access-and-back-again.html","year":1998,"month":"April","monthNum":4,"headline":"FoxPro to Access and back again","kicker":"INDIANA CAR","author_name":"Seth Hemmelgarn","author_title":"","affiliation":"NICAR","summary":"The article examines the shift from FoxPro to Microsoft Access in newsrooms for computer-assisted reporting, with journalists debating the merits of each database software for data analysis and cleaning.","keywords":["database","Access","FoxPro"],"topics":["Data & Databases","Technology","Journalism & Newsroom"],"technologies":["FoxPro","Access","SQL","SQL Server"],"full_text":"FoxPro to Access and back again\n\nBy Seth Hemmelgarn NICAR\n\nLooking at the class schedule at Indiana CAR, one trend stood out:\n\nMost classes were taught using Access, not FoxPro. This reflected the shift in newsrooms across the country; use of Access has become more widespread.\n\n\"Ever since Access 2.0 came out and proved to be fairly stable, there's been a steady drift toward deploying it for general newsroom use,\" said NewsEngin's George Landau. As part of the Microsoft Office Package, Access is readily available, and newsrooms just starting CAR programs - even those already having them - are using Access more frequently.\n\nIf you don't know anything about Structured Query Language, Access' query grid can be simpler way to build queries. You can point and click your way through them. But if you do know SQL, the grid can make analysis harder.\n\n\"Still resisting,\" said The Cleveland Plain Dealer's Beth Marchak, who prefers FoxPro's more direct approach to analysis. \"I think in SQL. Learning Access is like being forced to walk around the barn to gain entry when I'm already standing at the front entrance.\"\n\n\"Learning basics in Fox makes it easier for anyone to apply basic concepts and database skills no matter what program is used.\" - Dave Fallis, Tulsa World\n\nMarchak also shies away from Access because she doesn't want it choking on her data, as previous versions of the software have been notorious for doing. \"What good is a database manager that can't handle more than 250,000 records?\" Marchak said. \"All my databases are much bigger than that.\"\n\nThough maybe not the most fashionable software, FoxPro is still worth learning. \"Learning basics in Fox makes it easier for anyone to apply basic concepts and database skills no matter what program is used,\" said the Tulsa World's Dave Fallis.\n\nFoxPro's library of string functions, downloadable utilities, and easily understood programming/scripting language should keep the program from getting too antiquated, Tom Boyer of The Seattle Times said.\n\nLandau uses Access to keep track of queries and build front-ends, but he still prefers FoxPro for processing and cleaning. \"Even Access97 continues to lack many of the important data-cleaning functions that are built into FoxPro,\" he said. \"It does seem, however, that Access97 is considerably faster than its predecessors at running large queries.\"\n\nAccess does have its advantages. It has more export options. You can save an Access file as a FoxPro file, but not vice versa. Fallis also finds the import wizard handy for quick and dirty imports into blank database structures. Working with relational databases is where Access has the most obvious advantage over other software. \"FoxPro is not and never will be a purebred relational database,\" Landau said. \"Access, in contrast, has relational integrity and validation rules hard-wired into its database engine.\"\n\nLandau assumes this partially explains why Access is slower than FoxPro at some tasks: it takes a no-frills approach to data manipulation.\n\nWith the proliferation of Access, what will become of FoxPro and others?\n\nLandau, for one, isn't sure. \"Once SQL Server is made less scary and more affordable, I bet they'll try to wean everyone off of FoxPro and onto a single platform.\" Landau's not thrilled with this idea, since SQL Server is slower than FoxPro at reformatting large amounts of data. But, he said, SQL Server beats both Access and FoxPro at allowing lots of people to query the same database simultaneously.\n\nDespite Access' popularity, those who prefer FoxPro may never give it up completely. As Boyer said, \"I'll probably still have FoxPro on my machine when I'm old and gray.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-newsworthy-contributions","slug":"1998-april-newsworthy-contributions","url":"https://dwillis.github.io/uplink/article/1998-april-newsworthy-contributions.html","year":1998,"month":"April","monthNum":4,"headline":"Newsworthy contributions","kicker":"INDIANA CAR","author_name":"Jack Dolan","author_title":"","affiliation":"CFIC","summary":"Campaign contributions become newsworthy when journalists dig deeper than donation amounts alone, examining financial disclosures, committee assignments, and industry-wide spending patterns to reveal potential conflicts of interest and quid pro quo relationships between donors and lawmakers.","keywords":["contributions","campaigns","disclosure"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["Excel"],"full_text":"Newsworthy contributions\n\nBy Jack Dolan CFIC\n\nIf the CEO of a casino company gives $1,500 dollars to a state senator running for reelection, is it news? Giving money to a candidate you believe in is not undemocratic. For most busy people, it is more practical to promote their ideals by writing a check than by stumping door-to-door with brochures. So when do campaign contributions become news?\n\nThe consensus at Indiana CAR was that determining which contributions lead to good stories is difficult to do with contribution data alone. Experienced campaign data crunchers, like David Poole of the Virginia Public Access Project, suggest taking the analysis further. First, Poole says, get the candidate's financial disclosure forms. If the candidate has two kids working for a casino or owns stock in a casino, noting a few hundred dollars' worth of gaming company contributions barely scratches the surface of the story.\n\nNext, check the committee assignments of incumbents. Kevin Murphy of The Kansas City Star recently wrote about contributions of $1,500 from gaming executives to the co-chair of the gaming and wagering committee of the Missouri Senate. The senator recently proposed abolishing hourly loss limits for casino customers.\n\nThen there are the aggregate stories: Which industries gave the most? What did they get in return? These stories are the real data challenge because summing contributions from a single contributor can be difficult if the name is entered in different ways. The difficulty grows exponentially when you try to sum contributions from an entire industry.\n\nBut if you have the patience and data skills to standardize contributor names, the reward can be staggering. With technical help from the Center for Public Integrity, The Indianapolis Star/News chronicled how a consortium of businesses aligned with the Indianapolis Chamber of Commerce went on an unprecedented spending spree and, coincidentally, got its entire legislative agenda passed. According to the Star/News report, \"When some lawmakers tried to protest or even debate the issues, they were reminded in private meetings to stick with the the money that got them elected.\"\n\nAnother good aggregate story, by Shirish Date of The Palm Beach Post, clawed its way to the heart of one of the sadder ironies of the modern legislative process. The Post requested records of contracts with registered lobbyists from every public agency in the state and then entered those records into an Excel spreadsheet. According to the resulting story, \"Last year, Florida taxpayers paid more to lobbyists to influence their own legislators than they paid in salaries to those same lawmakers.\"\n\nWendell Cochran of American University reminds reporters to understand the system the data describes. \"For every complex problem, there is a simple answer wrong.\" Cochran says one oversimplified approach to campaign finance reporting is the bribe paradigm: rich businessmen bought candidate X's vote. Cochran says we should not ignore the possibility that candidate X solicited the rich businessman's contribution. He calls this the \"blackmail paradigm.\" Cochran advises asking contributors how they came to give the money and getting on the candidate or committee mailing lists so you know when they are directly soliciting from industry-wide groups.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-stats-from-the-road-causes-and-effects","slug":"1998-april-stats-from-the-road-causes-and-effects","url":"https://dwillis.github.io/uplink/article/1998-april-stats-from-the-road-causes-and-effects.html","year":1998,"month":"April","monthNum":4,"headline":"STATS FROM THE ROAD Causes and effects","kicker":"STATS FROM THE ROAD","author_name":"Sarah Cohen","author_title":"","affiliation":"NICAR","summary":"Reporters using statistics must go beyond finding correlations to establish causation, but this is challenging without controlled experiments and complete data sets. The article provides guidelines for validating research claims of cause-and-effect relationships.","keywords":["statistics","causation","reporters"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"One stumbling block that reporters face when they begin working with statistics is the timeless cause-and-effect debate.\n\nWhen virtually every book on numeracy and statistics lectures us about our constant confusion between correlation and causation, it may seem wise to avoid any analysis that will thrust you into the debate.\n\nBut complaints are often a little hysterical. I doubt many of us would write stories ascribing (a) more drinking to more clergymen in town, or (b) higher stock prices to sun spot activity.\n\nEven so, the point is well taken. It's all very nice to say that the failure to get a mortgage or the amount of bail set are associated with a person's race. Those only become stories when we can say why. We aren't interested in just linking two events; we want to show why they're related.\n\nStatistical tests such as chi-squares and regression results can tell you whether a relationship between two events could have happened by dumb luck. But they can't tell us whether one event caused another.\n\nMore bad news: We rarely conduct controlled experiments, the only kind of research that can tell you cause and effect. Instead, we work from a mass of public records that were collected for some other purpose. These records often miss the key elements and defy our attempts to control for even simple and obvious confounding events.\n\nSchool report for instance, lack the single biggest indicator of poor performance - a definitive measure of poverty. Home mortgage records lack any information on credit records. And health records lack medical histories.\n\nValidating research\n\nAt least one author is ready to help us through the steps that others take when they want to say that one event may cause another. In Statistics: A Spectator Sport, author Richard M. Jaeger reveals the ways researchers decide whether they've found a cause and effect.\n\nWe usually only care about what researchers call \"internal validity\" of our work. External validity would mean that we care about generalizing from our specific work to a bigger world - something we rarely do. So here are some of the guidelines:\n\nDoes the cause come before the effect? This seems obvious, but it isn't always. Test scores may depend on the number of teachers in a classroom. The timing, though, is relative. Spending may, in turn, depend on performance just last year. Performance in a school doesn't usually change that quickly.\n\nIs there a third event that causes the relationship? Again, a school example is worth noting. Most reporters would find that high minority populations in schools are associated with low test scores. But a third factor - poverty and education of the parents - plays a role in both the cause and effect in this case. Although ethnicity comes before just about anything else, few (outside The Bell Curve crowd) would argue that it causes any of the difficulties minorities face in school.\n\nIs there another explanation? Can you think - or have your sources thought up - of rival theories that are equally sound? Look to your reporting effort rather than statistics for this step.\n\nDoes anything refute the theory? Are there other studies that fail to show the expected causal relationship? Why might your theory only work locally? Why might it fall apart in another year?\n\nThis is an uncomfortable position for us. It involves an about face in our normal reporting process - faced with disproving our own theories rather than proving a tip. And it leaves us open to criticism we've never faced when collecting quotes or uncovering documents.\n\nBut there is good news. One way to figure out cause is to watch and listen for anecdotes that are described by the numbers.\n\nNow that's something we're happy to do. In fact, we prefer it. We'll search for weeks to find the people who are described by our statistics. It's there - not in the numbers - that we find cause. Statistics can document a pattern and tell us how consistent it is. Reporting tells us why it happens.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-handout-of-the-month-required-reading","slug":"1998-april-handout-of-the-month-required-reading","url":"https://dwillis.github.io/uplink/article/1998-april-handout-of-the-month-required-reading.html","year":1998,"month":"April","monthNum":4,"headline":"HANDOUT OF THE MONTH Required reading","kicker":"HANDOUT OF THE MONTH","author_name":"Richard Galant","author_title":"Reporter","affiliation":"Newsday","summary":"This article outlines best practices for investigative journalism, emphasizing the importance of identifying a single overarching theme to organize complex reporting, crafting compelling human-centered leads to engage readers, and presenting findings with narrative structure rather than data dumps.","keywords":["storytelling","narrative","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["FoxPro","Pentium"],"full_text":"From all the data you accumulate, from all the interviews you conduct, from all the documents you inspect, you should come up with one single, simple theme supported strongly by the data. Think of it as the skeleton on which everything else will hang or the classical music theme that will serve as the basis for all the variations. Everything in the story - the data, the interviews, the documents, the examples and anecdotes - will in some way relate to the theme, either by supporting it, expanding it, explaining its limitations or reacting to it.\n\nEnsuring the story gets read\n\nWrite a lead that exemplifies the theme. If you're writing about a pattern of discrimination against minorities by a local hospital, don't begin with somebody who's the exception to the pattern. Instead, find the most interesting and dramatic case of discrimination you've got. The more you reduce the story to its essential elements, stripping away extraneous detail, the greater the impact on the readers. The more you begin with a story about people rather than numbers, the more likely you are to hook readers.\n\nYou should try to summarize your theme in a sentence or at most a lead paragraph. Think of it as the hard news lead you would write if you chose not to begin with an anecdote.\n\nKeep it simple\n\nYou spent three weeks on the story, interviewed 50 people, examined 212 documents and put 6,000 records into a FoxPro database. You used a Pentium 200 with 64 megs of RAM to crunch the data. That's impressive and important, but why should the reader care? Don't let it get in the way of telling the story.\n\nStory elements\n\nWhy do people read 500-page novels but complain they don't have time to read newspaper stories? Often, we try to give readers stories that are long on detail but short on compelling narrative that drives people through to the end. Narratives people reading by introducing believable characters and describing what happens to them over time. The best stories of this type have a beginning, a middle and an end. Reporting these is tough. You need to nail down the details of the characters, the color, the dialogue that you might not gather in reporting a more traditional survey story. There's no excuse for embellishing the details or creating \"composite\" characters. But if you do the reporting painstakingly and craft the story carefully, there can be a big dividend in more readership.\n\nAvoid large, uniform sections-chunks of data or extended interview segments - in the story, if possible. To keep readers interested, create a \"rocky road\" mix of anecdotes, key findings, data and quotes.\n\nProtect your findings\n\nBe conservative in interpreting the data. Try to imagine all the potential arguments that could be made against your central theme. Bounce it off experts. You may have to go back to the data to make sure your findings and methodology are bulletproof. Give prominent play to the point of view of those who can make substantial arguments against your central finding.\n\n\"Killer comparisons\"\n\nStrive for ways to derive statistical findings that can be stated fairly in the most concrete terms. In Newsday's series on racial discrimination in mortgage lending, an analysis of nearly 100,000 mortgage loans found that African-American applicants are rejected nearly three times as often as whites are. That was stated prominently. But the story went on to derive another point from the same data set in a powerful way: \"The gap is so wide that a black applicant making more than $150,000 a year stands a greater chance of being rejected than does a white person making less than $35,000 a year.\"\n\nHome to papa\n\nBring it back home at the end of the story. Explain what ultimately happened to the people you wrote about at the top. The last few grafs of a story can be the most powerful and can leave readers with a lasting impression.\n\nRichard Galant can be reached at (516) 843-3274 or by e-mail at rgalant@newsday.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-on-the-internet-search-for-news-not-noise","slug":"1998-april-on-the-internet-search-for-news-not-noise","url":"https://dwillis.github.io/uplink/article/1998-april-on-the-internet-search-for-news-not-noise.html","year":1998,"month":"April","monthNum":4,"headline":"ON THE INTERNET Search for news - not noise","kicker":"ON THE INTERNET","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute","summary":"This article profiles Web sites and search services that provide free access to newspaper archives and online news searches, helping librarians and researchers find newspaper articles without using expensive databases like Lexis/Nexis.","keywords":["newspapers","archives","search"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Nexis","DIALOG","Internet","World Wide Web","Infoseek"],"full_text":"\"Hi all. We do not yet have access to Lexis/Nexis, DIALOG, etc. in our library yet but plan on (I am pushing for it) getting at least one of them soon. In the interim, however, I'd like to have some access to newspaper articles online (preferably free). I'm trying to avoid having to go to individual newspaper archives to search. General searches on search engines bring back some good results but often have to weed through a lot of junk. Any suggestions from others out there who are in the same situation? Thanks.\"\n\nCould you have written this message? Do you wish there was a way to get at the archives of newspapers or search their Web sites for articles easily? To help you accomplish that very thing, this column profiles Web sites with news search services or listings of news archives that enable to cast that net out to the 'Net and snag news stories only.\n\nSmall Hours / News Archives\nwww.aa.net/-rclarklarchives.html\n\nRod Clarke, the creator of the list, includes weekly and monthly newspapers and the archives of news agencies and newsletters from organizations such as Amnesty International or U.N. Daily Highlights, and even some television station sites.\n\nList Organization: Organized by regions of the U.S., Canada, Western Europe, Eastern Europe and Russia, Mideast Africa, South Asia, Asia, Oceania, Central and South America. The links take you to the homepage of the newspaper or to the search page for the archive.\n\nNews Hunt\nwww.newshunt.com\n\nList Contents: News Hunt is a hybrid-part \"scout page,\" part search engine facilitator. It gives you a listing of news publications with free searchable archives. Click on one of the 66 listings and you get a screen with a search box into which you can type your request.\n\nList Organization: News Hunt is organized by categories of news needs: general archives arts & entertainment papers, breaking news business, popular, technology, and weather. Another section - newspapers by state provides a listing of all the publications with archives in each state, creating a very useful compendium of both mainstream daily publications and niche and university publications.\n\nSpecial Libraries Association/News Division Newspaper Archives on the Web\nsunsite.unc.edu/slanews/internet/archives.html\n\nThis classic \"scout page\" was the brainchild of Margot Williams, metro desk researcher at the Washington Post. She was teaching an Internet course at George Washington University and turned her students on to the task of locating daily newspapers with available archives. The resultant list was augmented with additional links.\n\nList organization: This long chart, expanding to include non-U.S. archives, includes links to the news sites' homepages, specific links to the archives within the site, database start-up date, and the cost of searching the site.\n\nNews Index\nwww.newsindex.com\n\n\"The most comprehensive, up to the minute News Search Engine on the planet.\" Begun in April 1996 by the entrepreneurial Sean Peck, this innovative service attempts to cull current articles from news sites from around the world.\n\nSite contents: Including the major news sites and Reuters, it is an index of current articles only, not an archive. In fact, the service supposedly re-indexes the target sites every 1-2 hours. As such, it is a good way to get a cross section of contemporary coverage but it is not a good way to research old stories.\n\nOther features: The custom search service delivers to your e-mail box articles fitting the profile you've selected. You can create up to five profile searches.\n\nNews Works\nwww.newsworks.com\n\nA nice service from the ill-fated New Century Network will be around, but use it until it dies a lamentable death. The feature most useful to journalists and others seeking content from a variety of news sources was the Search Service, which indexed and searched the contents of about 120 newspaper Web sites using Infoseek's (www.infoseek.com) spider technology.\n\nTotal News\nwww.totalnews.com\n\nSoon after its launch, Total News got itself into hot water with news site publishers, who objected to the frames because they made the retrieved stories stay within the Total News frame. It looked, the publishers said, as if it were a Total News story.\n\nAccording to a New York Law Review story in June, 1997, the media companies brought a litany of claims, alleging that the defendants' practice of framing constituted misappropriation, trademark dilution, trademark infringement, copyright infringement, false designations of origin, false representations and false advertising, unfair competition, deceptive acts and practices and tortious interference with contractual relations.\" Total News lost the lawsuit. This is actually an advantage to the searcher. Unlike News Works' search, which keeps the results screen within its page and makes it difficult to determine which site you are on, the whole page from the news site remains when using Total News.\n\nSite contents: Total News' search robot scans about 1,200 news sites and updates the database three or four times a day.\n\nSearching: The system uses a Web crawler called Paradigm News Search. Be sure to read the search help files-you'll see some interesting things, including the fact that one or two letter (or numeral) words are ignored in the search. (So forget using Total News to find reviews about the band U2's recent tour!)\n\nNewsBot\nwww.newsbot.com\nwww.wired.com/newbot/\n\nUsing HotBot's search engine, this site is all about time. The search box lets you put in a search term and then select how current you want the items retrieved to be: 6 hours, 12 hours, 24 hours, 48 hours, 4 days, 7 days, whenever. You can also select certain types of news you want retrieved: all news, business, politics, entertainment, health, sports, technology, world. Each of the retrieved items has a date and time stamp on it in red to help the searcher select the most recent postings. The display of results is crisp and clean. The bad news, though, is there is no clear listing of what sites are searched or how often. More bad news is the fact that the database is not purged of links to URLs no longer containing the news story. This is a big problem with Web-published news search databases. NewsBot, like its big brother HotBot, does provide some of the most innovative ways to focus searches through its search option boxes.\n\nNews Tracker\nnt.excite.com\n\nExcite's news-only search service grants a number of ways to quickly scan coverage of particular events or specific features from a variety of news sources. Do a search and then have the results sorted by relevancy, with the lead of the story displayed. Select \"Show Titles Only\" for a listing of relevancy-ordered articles with only the headline and publication name displayed.\n\nClick on \"View by Publication\" to see results grouped by publication and then sorted by relevancy. Or click on \"View by Date\" to get the results grouped chronologically first and then by relevancy.\n\nYAHOO\nheadlines.yahoo.com/Full_Coverage\n\nGet a quick list of top news stories and how they are covered by papers around the world. Click on one of the top stories they are following and you'll get a grouping of news stories by news, magazines & features, related Web sites, live coverage, or multimedia coverage. This is another of the clean, organized, reliable resources put together for 'Net citizens by the good people at Yahoo. Thank you, Yahoo. But there is a bit of bad news, too. When you use the \"Search\" box to find stories in the news area of Yahoo, you get a listing of stories but no indication of where they come from.\n\nNora Paul can be reached by e-mail at npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-first-ventures-government-felons","slug":"1998-april-first-ventures-government-felons","url":"https://dwillis.github.io/uplink/article/1998-april-first-ventures-government-felons.html","year":1998,"month":"April","monthNum":4,"headline":"FIRST VENTURES Government felons","kicker":"FIRST VENTURES","author_name":"Jeff Harris","author_title":"Reporter","affiliation":"WEWS-TV","summary":"A three-month investigation by WEWS-TV reporter Ted Hart uncovered hundreds of convicted felons employed in local government positions, including 375 in Cleveland and 151 in Cuyahoga County, raising questions about hiring practices and background check policies.","keywords":["felons","investigation","government"],"topics":["Crime & Justice","Government & Politics","Investigative Methods"],"technologies":[],"full_text":"After a three-month investigation that combined a massive CAR project with old-fashioned, pen-in-hand research, reporter Ted Hart and WEWS-TV aired a two-part report in March that uncovered hundreds of convicted felons working in local government. Briefly, we reported:\n\nThere were 375 convicted felons working as City of Cleveland employees. Another 90 individuals had been charged with felonies but eventually pled guilty to first-degree misdemeanors. One in every 20 city employees, we learned, was a felon.\nOf the 375 felons, 99 were convicted of violent felonies.\nOf the 28 city employees that served hard time in an Ohio prison, seven had claimed on their job application that they had never been convicted of a crime.\nCuyahoga County has hired 151 convicted felons - 20 for violent crimes. An additional 17 were charged with felonies but pled guilty to misdemeanors.\nWe also explored the quandary convicted felons face filling out job applications. We found that many are eager to find good, steady work but fear they won't be hired if they reveal their criminal background. We included interviews with the director of Community Re-Entry, a nonprofit group in Cleveland that assists convicted felons with finding jobs, and an ex-con who had found work through the group and put his life back together.\nMany local and state government entities have developed policy initiatives that encourage the hiring of convicted felons and work with groups like Community Re-Entry. This was not the case with the hiring practices in Cleveland. No initiatives were in place to recruit or hire ex-offenders.\n\nBrainstorming\n\nOhio's prison system, we learned, maintained a database on every individual incarcerated in an Ohio reformatory. It included the name, date of birth, date of admission, term of sentencing, name of prison, county of conviction, description of crime(s) and projected release date of the prisoner. After $200 and a few days of tinkering and retooling by the brilliant NICARians in Columbia, we had our grubby hands on the database.\n\nAs we began to hash out our story, we addressed a number of issues and research strategy questions. For those felons on the workforce, is there a correlation between their crime and their position? Did these ex-offenders lie on their applications? What are the total numbers and how do they compare from city to city? Is there a pattern? What is a felon to do? Are there instances of convicted felons committing crimes after having been hired? Are there policies in place regarding the hiring of felons? What are the application policies on criminal background checks.\n\nHunting and gathering\n\nWe decided to broaden the scope of our investigation beyond Cleveland employees. In late 1997, WEWS began submitting public record requests for names of employees, job titles, departments, dates of hire, and dates of birth from a broad range of local government agencies. For the most part, the cities turned the FOIA requests around within reason and at reasonable costs. I did encounter the common line of defense employed by bureaucrats: \"That's not public record. We can't do that. Well, we may have to do it, but we don't have to do it on disk. You'll have it by 5 o'clock.\"\n\nRealizing the vast majority of convicted felons never see the inside of a prison, we concluded our developing story would be most thorough and accurate if we were able to cross employee databases with county clerk of court records - in which we would pick up those sentenced to county jail time and probation.\n\nThus began the tale of two county court databases. WEWS submitted written requests to the clerks of Cuyahoga County, Ohio's largest, and Summit County, nearly one third the size. We sought the name, date of birth, case number, criminal charge and disposition of every criminal defendant in their courts. Within three days, Summit County e-mailed a database containing the exact information we asked for for a total of $86.\n\nCuyahoga County handled the FOIA much differently. A month after our FOIA request, we were informed it could be fulfilled for $10,000. News director Jim LeMay pounced on the phone and called a county commissioner. In his tactical tirade, he used words such as \"gouge\" and \"outrageous\" and \"lawyers\" and \"court costs\" and \"in the public interest.\" The next day the price dropped to $4,000. And if we would settle for just name, date of birth and case number, the database would be free. We settled for the free one. Three months after our initial request, the county delivered the database to WEWS on CD-ROM.\n\nConfounding fields\n\nThe major stumbling block in getting the information into one database was that every city recorded employee names differently in their tables. Most complicated were tables that included middle initials and Jr./Sr. designations. I used Excel's Text to Column command to parse the confounding fields, often pasting only the last name column back into the original employee spreadsheet. I knew my Access cross match of the court and prison databases would entail last names and dates of birth, so the last name column would be all that I needed.\n\nLast names and dates of birth are constants. First name matches leave out a large chunk of true matches because nicknames or initials may appear in one table while an \"official\" name appears in another. They may appear in a first name field or a field of their own. And first names are misspelled more often than last names.\n\nMatching first names by eye allows for the perfect integrity check and for the opportunity to truly examine the results of the query record by record.\n\nWeeding through several hours of queries and the resulting tables, we found more than 700 separate last name/date birth matches, with first name \"eyeball matches\" totaling more than 1,000 separate cases.\n\nReality check\n\nOur remaining task was confirming that the employee and the ex-con were one. The county tables contained 700 individuals who had been indicted in felony court over the past 30 years, but we had no idea whether these people had felony convictions, and if so, when, and for what crimes.\n\nTo confirm convictions, we took our list to the public computer terminal at the clerk of court and punched in case numbers. Initially, we selected the case numbers of employees who worked in positions with a high rate of contact with either the public or money, or who were involved in safety. Our goal was to find potential subjects to highlight for our story. Our first sweep yielded a host of compelling cases: a deputy treasurer convicted for sexual battery; a recreation center director with drug trafficking convictions; and a park maintenance supervisor convicted of involuntary manslaughter for his part in a drug store robbery and murder.\n\nUnfortunately, it took us weeks to narrow our list of subjects, find them, and then get interviews and pictures. One week before airing the story, we didn't have final numbers. We should have completed confirming charges on the entire list and compiled our final numbers before finding the subjects for our story.\n\nOur news desk received dozens of calls after our story led the 11 p.m. newscast on March 2. All were negative. Viewers thought we were picking on people who had paid their debt to society. Many complained that we had needlessly ruined the lives of people who had reformed. After part two ran the next night - discussing the difficult employment problems faced by ex-offenders - we received calls asking just which side we were on.\n\nThe three employees we highlighted as having lied about their felony convictions were suspended by the city pending termination. After requesting our list of 375 felons, the city has opened its own investigation. They will review the job application and convictions in each case to determine whether disciplinary action is warranted. Cuyahoga County has also opened an investigation and fired the deputy treasurer convicted of sexual battery.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-bits-bytes-and-barks","slug":"1998-april-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-april-bits-bytes-and-barks.html","year":1998,"month":"April","monthNum":4,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This newsletter announces updates to computer-assisted reporting resources, including new CAR project listings, upcoming IRE and NICAR training events, recently acquired databases for investigative journalism, and expanded campaign finance data from multiple states.","keywords":["journalism","databases","training"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nOnline CAR Projects\n\nTo view a listing of links of recent computer-assisted reporting stories posted on the Web, point your Internet browser to www.ire.org/resources/conferences/training/carprojects.ht The site includes a description of the stories as well as links to IRE award winners. If you would like to see a story added to the list of links, send Jack Dolan an e-mail at jack@nicar.org\n\nUpcoming IRE & NICAR Events\n\nBesides the bustle of investigative journalists discussing their craft in the Big Easy this June, there's a slew of other IRE and NICAR events coming soon, including on-the-road training, seminars, regional conferences and boot camps. For a full listing, see the IRE Web site at www.ire.org/calendar.html\n\nDatabase Library Additions\n\nNICAR has recently acquired updates to several of its databases: the U.S. Coast Guard's boat registration database, the Immigration & Naturalization Service's database of naturalized citizens, the Government Services Administration's database of federal procurement contracts, and the Bureau of Alcohol, Tobacco & Firearms' database of gun dealer licenses.\n\nFor a description of available databases or a downloadable order form, see the Web site at www.nicar.org/data.\n\nCampaign Finance\n\nThe Campaign Finance Information Center continues its pursuit of acquiring campaign contribution data from as many states as possible to allow national contribution patterns to be discerned.\n\nDownloadable data from ten states-Florida, Idaho, Illinois, Indiana, Kansas, Kentucky, Michigan, Minnesota, Ohio and Wisconsin- has been added to the CFIC Web site, located at www.campaignfinance.org Comparisons can be made by joining tables between these state files. CFIC hopes to enable such comparisons to be made online once common fields are established and standardized.\n\nAlso available at the Web site are links to online databases maintained by various states themselves, though not all can be downloaded.\n\nDirect your browser to campaignfinance.org/cficmap.html.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-april-cfic-mailing-list","slug":"1998-april-cfic-mailing-list","url":"https://dwillis.github.io/uplink/article/1998-april-cfic-mailing-list.html","year":1998,"month":"April","monthNum":4,"headline":"CFIC Mailing List","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Instructions for subscribing to the newly formed CFIC listserv by sending an email to majordomo@campaignfinance.org with specific subscription commands.","keywords":["listserv","subscribe","email"],"topics":["Government & Politics"],"technologies":["Internet"],"full_text":"To subscribe to the newly formed CFIC listserv, send an e-mail to majordomo@campaignfinance.org. In the body of the message, type subscribe CFIC-L yourname <your e-mail address>.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-may-wrestling-giants","slug":"1998-may-wrestling-giants","url":"https://dwillis.github.io/uplink/article/1998-may-wrestling-giants.html","year":1998,"month":"May","monthNum":5,"headline":"Wrestling giants","kicker":"HEALTH CARE INVESTIGATION","author_name":"Josh Barbanel","author_title":"","affiliation":"The New York Times","summary":"New York Times journalists investigate Columbia/HCA Hospital Corp.'s questionable business practices and use computer-assisted analysis of 30 million medical billing records to determine whether the for-profit health care giant represents a unique problem or reflects broader systemic issues in American health care.","keywords":["investigation","healthcare","analysis"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"We were gathered in a tiny windowless conference room at The New York Times noodling over what was undoubtedly going to be an epic tale of the troubled American health care system. But which epic would it be?\n\nFor weeks, Martin Gottlieb, Kurt Eichenwald and Tamar Lewin had traveled around the country meeting with sources, reviewing court and agency records, exploring the troubling business strategies that turned Columbia/HCA Hospital Corp. into a $20 billion health care giant, the world's largest health care provider.\n\nAfter a few days on the road in places like Tallahassee and Baltimore, I stayed back at the office staring at my computer screen. I was trying to make sense of a carton filled with 89 computer cartridges containing more than 30 million medical billing records perched on the edge of my desk.\n\nAs we reviewed our findings - tales of hard-ball takeover practices, questionable incentives for doctors to switch hospitals and bring their patients with them - we struggled over fundamental questions of fairness.\n\nWas this a story about a greedy for-profit company that went way too far? Or a story about the sorry state of the entire American health care system? Was Columbia/HCA a special case or were not-for-profit hospitals, facing the same pressures, doing exactly the same things? Were we picking on the wrong guy?\n\nAnalytical evidence\n\nAt this pivotal moment in the project, I was able to lean forward on the sagging couch and pull out the computer-assisted reporting analysis of medical billing records. I argued that they provided the strongest and clearest evidence yet that Columbia/HCA was simply different from other hospitals and needed to be treated that way.\n\nThe analysis showed that Medicare recipients who were treated at Columbia/HCA hospitals in Texas cost the government millions of dollars more than patients treated at other hospitals. Their costs were higher than costs at all other hospitals, particularly when the entire cost of care included the hospital stay through skilled nursing, rehabilitation and home health care. Their costs were far higher than the costs at comparable not-for-profit hospitals.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-may-sudden-seizures","slug":"1998-may-sudden-seizures","url":"https://dwillis.github.io/uplink/article/1998-may-sudden-seizures.html","year":1998,"month":"May","monthNum":5,"headline":"Sudden Seizures","kicker":"INTERSTATE TRAPPINGS","author_name":"David Washburn","author_title":"journalist","affiliation":"NBC","summary":"Dateline NBC's award-winning investigation uncovered an illegal scheme by Louisiana sheriff deputies who misused drug forfeiture funds for personal expenses, including a luxury ski trip billed as training. The investigation revealed widespread financial misconduct totaling over $200,000 in missing or improperly spent seized money.","keywords":["forfeiture","corruption","investigation"],"topics":["Crime & Justice","Investigative Methods","Government & Politics"],"technologies":["Access"],"full_text":"Dateline NBC's IRE award-winning investigation of a Louisiana sheriff department's illegal use of drug forfeiture money - \"Probable Cause\" - started with a ski trip in Aspen. It ended with a cardboard box filled with photocopied checks and a Microsoft Access database.\n\nIt was a week-long luxury ski trip. Four Jefferson Davis Parish police deputies rented a condo, bought ski clothing in ritzy boutiques and took to the slopes. Police billed the trip as \"deputy training.\"\n\nAll expenses were paid with money police seized from alleged drug dealers in Louisiana, mostly along Interstate 10.\n\nLouisiana Legislative Auditor Daniel Kyle investigated this 1990 ski trip and found that not only was it bogus but an illegal use of drug forfeiture money. An additional $200,000 was missing. One [continued on page eight]\n\n[Continued from page eight]\n\nin ways they haven't. David Washburn can be reached by e-mail at david.washburn@mcall.com or by phone at (610) 820-6584.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-may-system-protecting-military-malpractice","slug":"1998-may-system-protecting-military-malpractice","url":"https://dwillis.github.io/uplink/article/1998-may-system-protecting-military-malpractice.html","year":1998,"month":"May","monthNum":5,"headline":"System protecting military malpractice","kicker":"HEALTH CARE IN THE ARMED FORCES","author_name":"Stuart Watson","author_title":"","affiliation":"WRAL-TV","summary":"A journalist recounts the two-and-a-half-year investigation into the death of an infant at a military hospital, using databases and FOIA requests to uncover systemic issues with military medical care.","keywords":["investigation","military","database"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["9-track tape","FTP"],"full_text":"Normally I'm terrible at phone tips. To my detriment. If Deep Throat called, I'd put 'em on terminal hold, then cut 'em off. Maybe that's why my colleagues rib me about being more at home with a stack of documents or a database. But it was hard to hang up on Kristie Piittmann. She called March 22, 1995, to say Army medicine killed her baby.\n\nHaving firsthand witnessed labor and delivery four times, her story hit home. You could see it happening. The fever. The racing fetal heartbeat. The antibiotics administered too late. The awkwardness of telling the mother when Cole Piittmann was born a few weeks before Christmas 1994. Then the rush to a civilian hospital and the parents' ultimate choice: handing over the infant to be airlifted to Duke University with little hope of survival or unhooking all the tubes and letting their only son die in their arms. They chose the latter.\n\nIt was a heart-rending story, all right, the news director said, but how was it different from allegations of malpractice in a civilian hospital among civilian doctors? In other words, you have a story - now what is the report? What is the meaning, the context? The full answer would take two and a half years of reporting.\n\nAlley cats\n\nWe went down a number of blind alleys before finding the lessons of Cole Piittmann's death. First we checked the death rates at the military hospital against death rates in the neighboring civilian hospital. Fewer infants died at the military hospital. Then we checked the status of infants transferred from the military hospital to civilian hospitals using a database of death records downloaded from the Web. Finally we checked the number of infants at various hospitals who died of Group B strep infections like Cole Piittmann had. The alleys led nowhere.\n\nThe next set of blind alleys concerned the doctors. We filed Freedom of Information Act requests asking whether Kristie Piittmann's doctors had ever been sued before. The Army denied us saying it couldn't find such information, that doctors' names aren't linked to malpractice claims, and even if they were, the information was exempt from public inspection.\n\nWe also asked whether the doctors had ever been disciplined. Again denied. An Army doctor's disciplinary history-short of court-martial - is exempt from public disclosure. But the U.S. Army Claims Service at Ft. George Meade in Maryland mentioned a database. Did somebody say 'database'? We filed an FOI request.\n\nThe Army withheld names and identifying information for claimants. But after one appeal, the Army included the names, addresses and phone numbers of attorneys. Some attorneys proved helpful in tracking down old clients. The U.S. Army CLAIMS database arrived in the form of a tiny cassette not much bigger than an audio cassette. We didn't have a player. After consulting CAR gurus on the NICAR listserv, we hired a local data house to transfer the cassette to nine-track tape. Though we didn't have a nine-track tape drive in house, our corporate data processing officer did. We begged him to clean it up and FTP it to us.\n\nAll of this took months. The corporate data processors would ask little questions like, \"Do you want us to clean this Army data or do you want your paycheck this week?\" In the end, we had a complete database of every tort claim filed against the Army for about eight years.\n\nThis database could have told us the type and rates of malpractice claims at various military hospitals. But there were all sorts of caveats. Some hospitals were much larger. Some might have been in areas where more aggressive trial attorneys recruited and encouraged military families to sue. In the end, we primarily used the data to track down similar malpractice cases at the same facility.\n\nCivilian differences\n\nThen I bumped into Russell Carollo, an investigative reporter at the Dayton Daily News. I had called Carollo in the summer of 1996 to ask how he got information about missing and stolen military weapons. He had shared his data, provided that we credited his news organization, which we did.\n\nI mentioned that I was interested in medical malpractice in the military. He said, \"Stuart, the stars have aligned. We've been working on this for months.\" Carollo and Jim Stewart at CBS News helped sharpen our focus. They said they didn't think we would ever be able to fairly compare malpractice rates at military and civilian facilities. They said the systems were too different.\n\nThe story, they said, was the differences between the two systems and how those systemic differences left military families unprotected against malpractice. Foremost, active duty soldiers, sailors, airmen and marines could not even file malpractice claims. Doctors were not named individually in claims. Military doctors didn't carry their own malpractice insurance. And disciplinary records were hidden. (We knew this last fact from our FOIA experience.)\n\nCarollo and Jim Nesmith at the Cox News Service had already done considerable work. Carollo offered to share if WRAL didn't air the report before the Dayton Daily News went to print.\n\nCarollo had accumulated databases of military doctors from all branches. We later split the cost of hiring a data entry house recommended by a NICAR listserv member to input the Public Citizen list of more than 13,000 \"questionable\" doctors and run it against the databases of military doctors.\n\nWe then shared the grunt work of filing open records requests with various medical boards to ensure the doctors mentioned in the Public Citizen list were the same ones in the military databases.\n\nResearch reaction\n\nThe years of research paid off in October 1997 when WRAL-TV, the Dayton Daily News and the Cox News Service reported how the military protects bad medicine.\n\nMilitary doctors do not carry their own malpractice insurance like civilian doctors, are not sued like civilian doctors and are not disciplined like civilian doctors.\n\nThe Pentagon does not report to the National Practitioners Data Bank the same information as the civilian world does.\n\nAnd, unlike civilian doctors, military doctors can continue to practice after repeatedly failing state medical exams.\n\nThe Army Times, Navy Times, Air Force Times and wire services all ran Carollo and Nesmith's series. WRAL aired a five-part series and a half-hour special report.\n\nThe Assistant Secretary of Defense for Health Affairs responded with a 13-point plan for improving military health care. The Surgeons General of the Army, Navy and Air Force were called to a closed-door meeting with the National Security Subcommittee of the House Appropriations Committee. The subcommittee later set aside $5 million for an expert panel to evaluate the quality of military health care. The Pentagon pulled doctors who failed medical exams from clinical positions. And it all started with a phone call, a massive database, and lots of teamwork with NICARians!","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-may-undeserved-overtime","slug":"1998-may-undeserved-overtime","url":"https://dwillis.github.io/uplink/article/1998-may-undeserved-overtime.html","year":1998,"month":"May","monthNum":5,"headline":"Undeserved overtime","kicker":"FALLING OUT OF THE BLUE","author_name":"Dan Keating","author_title":"Reporter","affiliation":"The Miami Herald","summary":"The Miami Herald's 1997 investigation into Miami police officers fraudulently adding their names to arrest reports to collect court overtime led to the largest disciplinary action in Metro Dade Police history, with the Pulitzer finalist project demonstrating how data analysis of court records combined with investigative reporting exposed the widespread scam.","keywords":["corruption","investigation","data"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["SAS"],"full_text":"In late April, Metro Dade Police - the largest agency in south Florida - punished 36 officers, the largest disciplinary effort in its history. Supervisors were demoted and officers were suspended without pay for a widespread scam in which officers, even if they had not performed any police work, added their names to arrest reports in order to collect court overtime.\n\nThe department's nine-month internal investigation was prompted by The Miami Herald's 1997 investigation, \"Collars for Dollars.\" In April, the project was named a Pulitzer finalist in the investigative category.\n\nThe story pulsed with rich anecdotes produced by the shoe-leather effort of three reporters on The Herald's investigative team. Its foundation lay in ironclad assertions drawn from data analysis, which involved nine different criminal justice datasets from a variety of sources.\n\nThe key ingredient was the court's electronic docket that lists officers subpoenaed to court. We obtained records of 704,077 officers subpoenaed in 143,055 felony, misdemeanor, juvenile and traffic cases in Dade County from 1994 through 1996. Our main goal was to tally how many officers were coming to court and then calculate by case, agency, officer and charge.\n\nThe hard work wasn't analyzing the data - it was organizing it.\n\nUnfortunately, the raw data was structured in a format that couldn't be used for effective analysis. The database arrived in two tables. One table listed the case number, charge and disposition. With separate records for every charge, each case could have a dozen records or more. The second table had one record for each subpoena, listing the case number, agency, officer and hearing date. It, too, could have a dozen records per case.\n\nAlthough the tables were related by case number, any attempt to join them produced a mishmash of up to 100 records for an individual case.\n\nThe multiplicity challenge\n\nBefore we could do anything, we had the tricky chore of reshaping the data. We needed a single record to summarize each case, but we couldn't lose the ability to pinpoint specific officers or charges.\n\nFor the charges, we made a single summary record for each case. The record included the first eight charges (if that many were included), a count of the total number of charges, and dispositions.\n\nTo be able to quickly find particular kinds of cases, we created yes/no flags for several types of charges: from murder and robbery to prostitution and drunk driving. If a case involved any of those charges, its flag was set to \"yes.\" Since the project concentrated on misdemeanors, the flags let us quickly run statistics on those cases.\n\nWe also created a single record for each case in the subpoena database. It included the total number of officers subpoenaed, the names of up to three departments and how many officers were subpoenaed from each. It also noted the officer and department that initiated the case. That showed us how groups of officers piled onto cases.\n\nIt's usually hard to turn multiple records into individual summaries with such a wide variety of information. We used SAS because it can manipulate a group of records to produce a single summary record.\n\nTo be able to find specific officers, we kept a dataset listing one subpoena per record. A name search would draw up all the case numbers for a given officer. The case numbers could then be linked to the charge summary or subpoena summary to pull statistics for that officer. Once we had the SAS program to reshape the felony/misdemeanor data, we could change file names to run the same operation on the juvenile data and then the traffic data. That ability to recycle SAS code and repeat a lengthy series of steps came in handy since we ended up getting four sets of the traffic data before the court folks finally produced a database including all police witnesses on a case.\n\nGone fishin'\n\nWith the data reshaped, it was time to actually analyze it. First, we cast a sweeping net through the data to catch tips (cases and officers) for the reporters to track down.\n\nWe specified a kind of case using the yes/no flags and then grabbed the cases with the most witnesses. We also pulled the officers whose cases averaged the highest number of witnesses.\n\nTo see who was in the habit of joining with a friend, we checked how many times any two officers appeared together on a case. To do that, we made a temporary duplicate of the subpoena dataset in order to compare it with itself.\n\nWith help from my predecessor and mentor, Professor Steve Doig of the Cronkite School of Journalism at Arizona State University, we used the following code. It's SQL code, but for this story run under SAS. You might find it useful anytime you're trying to match a dataset against itself to pull out pairs.\n\nThe original dataset was named subp1 and the duplicate subp2. In the duplicate, the officer identification was renamed copid2 and the case number was renamed casenum2.\n\nThe code uses a \"less than\" on the officer identification number so that each pair can be listed only one way (you won't have both \"Keating-Doig\" and \"Doig-Keating\"):\n\ncreate table witness.subp3 as select subp1.copid, subp1.casenum, subp2.copid2, subp2.casenum2 from subp1, subp2 where subp1.casenum = subp2.casenum2 and copid lt copid2 order by 1, 2;\n\nThe code produced a table of matched pairs with case numbers. We then grouped by the pairs and counted to find which officers appeared together the most. We could also see all the cases of any given pair.\n\nUsing those techniques, we found plenty of cases worth checking out. At least six police witnesses appeared in 128 misdemeanor prostitution cases. Two officers (close friends) were listed 238 times as witnesses in each other's drunken-driving cases, even though they worked for different departments in different jurisdictions.\n\nPublication precision\n\nTo produce statistics for publication, our analysis had to be more precise. We considered other complicating factors, such as officers subpoenaed only to testify about the calibration of the breathalyzer, the tiny proportion of cases with guilty pleas at arraignment, or cases with multiple defendants.\n\nIn any analysis, a baseline for comparison is the most important part: What can we use to define what's normal or abnormal? In our case, we had four major police departments, three that pay for court overtime and one that does not.\n\nThat made it easy to draw comparisons. For instance, the department that doesn't pay for overtime was 100 times less likely to have at least five officers on a DUI case. The three departments that paid overtime had 6,445 routine DUI cases with at least five officers in three years. Experts said three officers would be the maximum needed to prosecute a case.\n\nOne piece of telling evidence: the conviction rate for a DUI declined as the number of officers involved increased. That finding made it impossible for police to claim that extra officers ensured a conviction.\n\nWe created another database from the logs of the breathalyzer machines. We punched in the names of defendants whose breath-alcohol readings were below the legal limit, then merged the defendant names against the DUI court cases. More than half of those cases had been dismissed outright, meaning that hundreds of people were being charged with DUI even though tests showed they were not legally intoxicated. Although it's hard to prove that police are intentionally arresting innocent people to drum up court overtime, our finding from the data certainly hinted that drivers were being unfairly victimized.\n\nMatching payments\n\nOnly one of the departments we examined kept electronic payroll records that matched pay with specific cases. For the other departments, reporters pulled paper files and read time slips.\n\nI-team reporter Lisa Getter, using Microsoft Access, did much of the matching of the overtime database with databases of arrests and off-duty police overtime jobs. We found officers in multiple places at the same time. One year, there were 85 cases of officers who were in court for less than 10 minutes but still received the union-specified minimum of three hours overtime pay.\n\nTo detail how officers \"piggybacked\" onto cases, we built a supplemental database for one subset of cases. Using paper records, the reporters identified exactly which officers performed which tasks on each case. Using an Access front end, reporters entered the case number or defendant name. The data-entry form then automatically put the officers' names into drop-down boxes to fill in each task on a case. That way reporters could point-and-click very quickly and avoid typos.\n\nThe computer analysis was lengthy and challenging. But it was rewarding. The largest local police union, which was placing blame elsewhere and claiming officers were being victimized, never even attempted to deny or contradict our findings. In police memos before our series, the problem was downplayed as isolated or exaggerated. The data proved them wrong.\n\nDan Keating can be reached at (305) 376-3476 or by e-mail at dkeating@herald.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-may-car-with-microdata","slug":"1998-may-car-with-microdata","url":"https://dwillis.github.io/uplink/article/1998-may-car-with-microdata.html","year":1998,"month":"May","monthNum":5,"headline":"CAR with microdata","kicker":"STATS FROM THE ROAD","author_name":"Paul Overberg","author_title":"","affiliation":"USA Today","summary":"Microdata refers to individual-level records from surveys that allow journalists to perform unique cross-tabulations and uncover news stories, though working with it requires proper understanding of statistical weights and file structures. The article explains why microdata is valuable compared to summary data and provides guidance on how to properly handle hierarchical survey data like that from the Current Population Survey.","keywords":["microdata","survey","analysis"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["SPSS","SAS"],"full_text":"CAR with microdata\n\nBy Paul Overberg USA Today\n\nMicrodata is a word from the social sciences. Loosely, it means data at the level at which it's collected. More specifically, it often means the individual records that result from a carefully designed survey where the unit is often a single person, household, incident or something similar. Microdata is usually contrasted with macrodata, a generic term for various levels of summarization and cross-tabulation.\n\nWhy bother?\n\nIf you're thinking, \"Microdata! Fancy word for something I already do!\" you're wrong. True microdata can be much more rewarding. You can do cross-tabulations that no one else has ever done - and find news. But it's also trickier. It's often produced by a survey or statistically random selection, so each record represents a larger number of its type in the entire population. But we have to preserve and use each record's weight correctly. Depending on the survey's precision, weights can vary a little or a lot from record to record. It's often collected in a hierarchical file, something akin to interleaving the tables of a relational database. For instance, the Current Population Survey, done monthly by the Census Bureau for the Bureau of Labor Statistics, collects information about 50,000 households into one big file with three record types for their personal, family and household characteristics. Often, a researcher will pull data from the people and household records into a new working file. This is called rectangularizing.\n\nHousehold 1 - all data Family 1 - all data Person 1 - all data Person 2 - all data Person 3 - all data\n\nbecomes in a working file\n\nPerson 1 - some data and Household 1 - some data Person 2 - some data and Household 1 - some data Person 3 - some data and Household 1 - some data\n\nThis turns the hierarchical file into a flat (rectangular) file that's smaller and easier to analyze. If you've ever analyzed the Fatal Accident Reporting System (FARS) data from the National Highway Traffic Safety Administration, you've faced this. For each fatal accident record, there's one or two or more related vehicle records, each of which has one, two or more related person records.\n\nBefore using microdata\n\nDon't - if you don't have to. Often, summary data is all you need. Start there. Read the standard reports published by the keeper of the data. Look in the back for detailed cross-tabulations if the standard summaries aren't enough. For instance, median household income appears right up front in the Census Bureau's annual report on income. Dig a little and you can find median income by race (all whites). Dig deeper and you can find median income by race by age group (whites 45-54). Dig deeper and you can find median income by race by age by education (whites 45-54 with a bachelor's degree), deeper still for median income by race by age by education by work status (whites 45-54, bachelor's degree, work full-time) and even deeper still for median income by race by age by education by work status by region (whites 45-54, bachelor's degree, work full-time, Midwest).\n\nTalk to whoever keeps the data before you spend a week plowing into a microdata maze. They may have done what you're attempting or have good advice on how to do it (or why not to).\n\nLearn the data thoroughly. This is especially true for microdata based on surveys or hierarchical files. Read the appendix on the sampling design. Read the file list and values. Try to duplicate published calculations. Learn SPSS or SAS. Either will do, but knowing neither leaves you fumbling with spreadsheets and database managers. They may work, but they're sort of like using a regular screwdriver on a Phillips head screw: You have to work harder and still may get inferior results.\n\nPaul Overberg can be reached at (703) 276-5427 or by e-mail at poverberg@usatoday.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-may-pulitzer-profiles","slug":"1998-may-pulitzer-profiles","url":"https://dwillis.github.io/uplink/article/1998-may-pulitzer-profiles.html","year":1998,"month":"May","monthNum":5,"headline":"Pulitzer profiles","kicker":"ON THE INTERNET","author_name":"Nora Paul","author_title":"","affiliation":"Poynter Institute","summary":"This column highlights Pulitzer Prize-winning journalism entries across multiple categories, including the Grand Forks Herald's disaster coverage, the Los Angeles Times' breaking news reporting on a bank robbery, and the Baltimore Sun's investigative series on shipbreaking, while providing relevant web resources for journalists covering similar stories.","keywords":["Pulitzer","journalism","reporting"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Welcome to the \"On the Internet\" Pulitzer Prize column. We'll look at each of the prize winners and point you to versions of their stories (if available on the Web) and related Web sites that might help you if you are covering a similar type of story.\n\nPublic Service: Grand Forks Herald\n\nCoverage of the flood, blizzard and fire that devastated much of the city, including the newspaper plant. Read about the win: http://www.gfherald.com/news/daily/415/415front.htm Read some of the stories: http://newslibrary.infi.net/ The Infinet NewsLibrary site has the archive of Grand Forks Herald stories back to 1994. Searching is free, but if you want to read an article, it's $1.00 each. If you're covering a similar story: Disaster Resources\n\nFederal Emergency Management Agency: http://www.fema.gov\nEmergencyNetNews: http://www.emergency.com/ennday.htm - a resource of articles, reports and backgrounders on every kind of disaster and emergency, worldwide.\nDisaster Finder: http://ltpwww.gsfc.nasa.gov/ndrd/disaster/ \"A complete index to the best disaster Web sites on the Internet. Period.\" Well, sort of. Good subject organization of the links but little annotation of the information to be found.\nDisaster Relief: Worldwide Disaster Aid and Information Via the Internet: http://www.disasterrelief.org/Links/ - great compilation of resources on the Internet regarding disasters. Links to Red Cross sites, disaster relief agencies, and reference sites.\nYahoo's Disaster Links: http://www.yahoo.com/Society_and_Culture/Environment_and_Nature/Disasters/ - links to the following disaster categories: airplane accidents, avalanches, blizzards, Chernobyl, drought in the Midwest, Dust Bowl, earthquakes, fires, floods, hurricanes, landslides, nuclear disasters, spills, organizations, tornadoes, typhoons, volcanoes.\n\nBreaking News Reporting: Los Angeles Times\n\nCoverage of a botched bank robbery and subsequent police shootout in North Hollywood.\nRead about the win: http://www.latimes.com/HOME/NEWS/REPORTS/PULITZER_98/ See and hear the newsroom's reaction: http://www.latimes.com/MULTIMEDIA/VIDEO/shootout.980415.mov Read some of the stories: http://www.latimes.com/HOME/NEWS/REPORTS/PULITZER_98/N_HOLLYWOOD/ If you're covering a similar story: Crime Resources\n\nCopnet: http://police.sas.ab.ca/ - great list of links to resources and to city and state police sites.\nThe Police Officer's Internet Directory: http://www.officer.com/\nArmed Robbery Page: http://www.ior.com/~jdmoore/ - interesting little page by an entrepreneur wanting to train employees in high-risk jobs about how to deal with an armed robbery. Based out of Spokane, Wash.\n\nInvestigative Reporting: Baltimore Sun\n\nSeries on the international shipbreaking industry, what happens to workers and the environment when discarded ships are dismantled. Read about the win: http://www.sunspot.net/archive/search/ - You'll have to use their archive of stories going back to 1990 to read about the win - and it will cost you $1.95. Read the stories: http://www.sunspot.net/news/special/shipbreakers/ - They have packaged the series on their Web site, with photos.\n\nExplanatory Reporting: Chicago Tribune\n\nProfile of the Human Genome Diversity Project Read about the win: http://www.chicago.tribune.com/news/metro/chicago/article/0,1051,ART-7040,00.html Read the stories: http://www.chicago.tribune.com/news/nationworld/article/0,1051,ART-6989,00.html\n\nIf you're covering a similar story: Human Genome Diversity Project, genetics\n\nHuman Genome Project Information: http://www.ornl.gov/TechResources/Human_Genome/home.html\nHuman Genome Program information from the D.O.E.: http://www.er.doe.gov/production/ober/hug_top.html\nThe Human Genome Organisation: http://hugo.gdb.org/\nThe National Human Genome Research Institute: http://www.nhgri.nih.gov/\n\nBeat Reporting: New York Times\n\nCoverage of the U.S. Supreme Court Read about the win and click to the stories: http://www.nytimes.com/library/national/041498greenhouse.html If you're covering a similar story: U.S. Supreme Court\n\nSupreme Court Decisions: http://supct.law.cornell.edu/supct/\nRules of the Supreme Court of the United States: http://www.law.cornell.edu/rules/supct/overview.html\n\nNational Reporting: Dayton Daily News\n\nDisclosure of flaws and mismanagement in the military health care system Read the stories: http://www.ActiveDayton.com/doctors/i/index.html If you're covering a similar story: Military hospitals\n\nDepartment of Defense, Health Affairs: http://www.ha.osd.mil/index.html\nDepartment of Veteran Affairs: http://www.va.gov\nListing of Defense Department health care policies: http://www.ha.osd.mil/ppc/num_poli.html - many with links to the documents themselves\n\nInternational Reporting: New York Times\n\nSeries on drug corruption in Mexico Read about the win and click to the stories: http://www.nytimes.com/library/national/041498mexico.html If you're covering a similar story: Mexico, DEA, drug smuggling\n\nDrug Enforcement Administration: http://www.usdoj.gov/dea/\nOffice of National Drug Control Policy: http://www.whitehousedrugpolicy.gov/ - use \"search\" and find items related to Mexico.\nUnited Nations International Drug Control Programme: www.undcp.org/\nMediaAwareness Project: Drugnews Search: http://www.mapinc.org/crime.htm/ - locate articles about drug policy and enforcement from a variety of sources.\n\nFeature Writing: St. Petersburg Times\n\nPortrait of a mother and two daughters murdered during a Florida vacation and the investigation of the murders. Read the stories: http://www.sptimes.com/Pulitzer98/awards.html\nIf you're covering a similar story: murder\nWebgator: http://www.inil.com/users/dguss/wgator.htm - when you're doing an investigation, these are great investigative resources.\nThe Crime Library: http://www.crimelibrary.com/ - links and text to other famous/infamous murder cases. The sources of the reports seem credible, but none are crime writers.\n\nCommentary: New York Daily News\n\nCoverage of a Haitian immigrant brutalized by police officers at a Brooklyn station house. Read about the win and click to the stories: http://search.mostnewyork.com/most/manual/news/mcalary/machome.htm If you're covering a similar story: police brutality\n\nThe Police Complaint Center: http://www.policeabuse.com/ - a non-profit seeking to compile incidences of police abuse. Located in Tallahassee, Fla.\nYahoo's Police Brutality links: http://www.yahoo.com/Society_and_Culture/Crime/Crimes/Police_Brutality/ - Most of the sites listed are decidedly biased Websites. Use with caution, but the sites might be interesting sources for personal stories.\n\nNora Paul can be reached by e-mail at npaul@poynter.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-may-steps-to-data-freedom","slug":"1998-may-steps-to-data-freedom","url":"https://dwillis.github.io/uplink/article/1998-may-steps-to-data-freedom.html","year":1998,"month":"May","monthNum":5,"headline":"Steps to data freedom","kicker":"HANDOUT OF THE MONTH","author_name":"Michael J. Berens","author_title":"","affiliation":"Chicago Tribune","summary":"Before requesting government data, journalists must understand file layouts, data formats, and delivery mediums to avoid costly delays and technical obstacles. The article outlines three essential steps: studying file structure and codekeys, determining compatible data formats and delimiters, and specifying the physical medium for data delivery.","keywords":["data","journalism","formats"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["FoxPro","Access","Excel","FileMaker Pro","ASCII","DBF","9-track tape","JAZ Cartridges","ZIP Cartridges","FTP"],"full_text":"Ancient warrior wisdom is based on the belief that the battle is won or lost before the first sword is drawn. Likewise, the fight for data must be plotted before the first request is filed. Understanding the buzz words the warrior language, if you will - will go far in negotiations.\n\nStudy the file layout\n\nSkip this vital first step at your peril. Trust me. Take time to do this or risk major setbacks in time and money. The file layout is a document that shows data organization and structure. It serves as a blueprint for obtaining, importing and analyzing data. Codekeys are a list of standardized abbreviations, acronyms or numbers that represent information. For example, states may be represented as numbers in the database.\n\nStudy file layouts before making requests. For example, the database layout of the National Highway Traffic Safety Administration lists hundreds of fields involving fatal vehicle crashes. One field, called the \"related factors\" field, uses a code number to designate if the fatality was related to a police pursuit. An entire series can and has been built on one field.\n\nLayouts also reveal a major pitfall for the unwary: packed fields.\n\nTypically, packed fields hold information that is encrypted and condensed as a space-saving technique. Unless fields are unpacked, you may not be able to read the information.\n\nOften, government data is stored in multiple files, called a relational database. Examine layouts for a common field, which is necessary for linking files.\n\nHandling Formats\n\nThis is the step where some bureaucrats love to leave reporters in the dust. Learn about the types of formats your software can handle. However, most any database can be copied as an ASCII file. This is an acronym for standardized computer format that is recognizable by all mainstream software (FoxPro, Access, Excel, FileMaker Pro, etc.)\n\nMost software also can handle multiple formats, such as DBF or text format or formats managed by software applications.\n\nNext, you must determine how the data is delimited - how it will be separated. Many reporters will ask for data in ASCII format (an industry standard) and delimited by commas. During importing, the computer will separate the data into new fields at each comma. You can have the data delimited by spaces, or tabs, or most anything, if preferred.\n\nMedium messengers\n\nYou've followed the first two steps flawlessly, the agency quickly delivers your data - and it's on a 4mm tape cartridge. This happened at the Tribune when a reporter forgot to request the medium, which is the all-important third step. Needless to say, few newspapers have 4mm tape drives to read the data. Determine what formats your paper can handle, such as disks or cartridges or tapes. Besides disks, here are some other choices:\n\nNine-track tapes - Used by many government agencies, but you will need a special nine-track reader attached to your computer.\nJAZ Cartridges - These tapes hold 1 gig or more of data. JAZ drives are common and increasingly more inexpensive.\nZIP Cartridges - Similar to JAZ, but typically these cartridges hold less information.\nFTP (File Transfer Protocol) - This a storage medium, but it's a dream way to obtain data. In short, the agency temporarily places the requested data on its Web page, allowing you to transfer the data via the Web into your computer. No tapes, no disks, no special readers. And it's more efficient for agencies.\n\nRoadblocks and solutions\n\nThe pursuit of data is no laughing matter, but government response can border on the ridiculous. Here are actual government objections encountered nationwide. The successful strategy centers on getting past the flak phalanx and talking directly to the agency's data processing employees.\n\nOur computer can't do that.\nYou'll never understand the data. It's far too complex. We're afraid you'll make mistakes and draw wrong conclusions.\nIt's too big. You'll never have a computer to handle it.\nOur software is proprietary, so you can't have the data.\n\nData negotiations are fought with knowledge and confidence. More than once I've had a new term thrown at me during a meeting. Don't panic, don't flinch. Keep smiling, say no problem, then later run to someone in the newspaper's data processing department to find out what the term meant - and whether it's actually going to be a problem. Don't let the agency bluff you, but don't be afraid to do a little bluffing, too.\n\nDemand specifics, then demand proof. IRE and NICAR have excellent resources for those confused by techno-babble such as megs and gigs (terms related to data storage).\n\nDon't be afraid to ask basic questions. Ask the agency computer people how they relate multiple files: you may learn their method is more complex - but more accurate - than your plan.\n\nAnother psychological negotiating ace is to assure the agency that you plan to provide them a copy of all analysis (we're talking aggregate conclusions and totals) prior to publication. I love the moment I get to show the agency what their own data reveals. And more than once, I've been saved from publishing incomplete conclusions. Remember, the potential for errors is exponentially higher in CAR reporting. You're just one keystroke away from an error, always.\n\nMichael Berens can be reached at (312) 222-4798 or by e-mail at mberens@tribune.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-may-cold-fusion","slug":"1998-may-cold-fusion","url":"https://dwillis.github.io/uplink/article/1998-may-cold-fusion.html","year":1998,"month":"May","monthNum":5,"headline":"Cold Fusion","kicker":"TECH TIP","author_name":"Ray Robinson and Tom Boyer","author_title":"","affiliation":"The Virginian-Pilot and The Seattle Times","summary":"Cold Fusion is middleware software that allows journalists to query databases through web browsers instead of specialized database software, making data more accessible to newsroom staff on deadline.","keywords":["middleware","database","journalists"],"topics":["Technology","Data & Databases","Journalism & Newsroom"],"technologies":["ColdFusion","Access","FoxPro","CFML","ODBC","HTML","dBase","Excel","Paradox","SQL Server","Oracle","DB2","Netscape","Internet Explorer"],"full_text":"Not long ago, we all dreamed of the day when everyone in our newsrooms would be able to fire up Access or FoxPro, craft a simple query and retrieve a vital nugget of information on deadline.\n\nCold Fusion is one of a growing class of software products known as \"middleware.\" On a computer that's been set up as a Web server, it takes queries submitted from a user's Web browser, submits them to a database, places the results in HTML and sends them back to the browser.\n\nWell, it's not going to happen, is it? It's not that journalists can't learn these skills. But unless they're using them every day, the skills won't be there when they need them. If we're really going to make the data easily accessible to them, we need another solution.\n\nCold Fusion is one of a growing class of software products known as \"middleware.\" On a computer that's been set up as a Web server, it takes queries submitted from a user's Web browser, submits them to a database, places the results in HTML and sends them back to the browser.\n\nThe technical advantages of such a setup are too numerous to list here, but a few are worth mentioning:\n\nReporters on deadline use Netscape or Internet Explorer instead Access or FoxPro to retrieve data.\n\nAll the heavy lifting is done on the Web server, not the user's computer. So the person using a 486 with 8 megs of RAM can query a 500,000 record database. All they need is a browser.\nEven Mac users can query the database as long as they have a browser.\n\nIf you're comfortable with databases and have a passing knowledge of HTML, you can use Cold Fusion.\n\nThe software operates from \"templates,\" which are HTML-like pages with bits and pieces of CFML (Cold Fusion Markup Language) embedded in them. To put it simply, the HTML controls the formatting and style of the page the user sees. The CFML dictates what is retrieved from the database and how it's displayed.\n\nScript studies\n\nAs an example, let's say you have a database of names and addresses you want to put on the Web. And rather than putting each record into HTML and displaying it, you want to let users query the database for individual names.\n\nThe following code would create a text box where the user could enter a person's last name:\n\n<FORM 1ACTION=\"RESULTS.CFM\" METHOD=\"POST\"> <b>Enter last name here:</b> <INPUT TYPE=\"text\" NAME-\"Last name\" SIZE=\"50\" MAXLENGTH=\" <INPUT TYPE=\" submit\" value=\"submit\"></CENTER></FORM\n\nThat's where Cold Fusion comes in.\n\nWhen the user submits a last name, the Web server kicks off the Cold Fusion script (\"RESULTS.CFM\"), which processes the query and returns the information.\n\nCold Fusion takes the information off the form and runs the query, with the following code from \"RESULTS.CFM\" <CFQUERY NAME= GetResults\" DATASOURCE=\" MYDATABASE\"> SELECT * FROM NAMELIST WHERE NAME = Last Name order by Name </CFQUERY>\n\nEnhanced access\n\nThe query is actually processed by a snippet of computer code called the ODBC (Open Database Connectivity) driver. The results are passed back to Cold Fusion, which sticks them into HTML pages that the server sends back over the network to the user.\n\nThat part of the script looks like this: <CFTABLE =\"GetResults\"> <CFCOL HEADER=\"Name\" WIDTH=\" 20\" TEXT= #Name#\"> <CFCOL HEADER=\" Address\" WIDTH=\" TEXT=\" #Address#\" \"></CFTABLE>\n\n\"GetResults\" is the name of the query that has just run. The Name and Address inside the pound signs are references to fields in the database table. Users, viewing the results with a browser, would see something like this:\n\nName Address\nBOYER 3022 NW 66th\nROBINSON P.O. Box 449\n\nLike many other Web middleware products, Cold Fusion talks to databases through ODBC, a database query standard. That's good. ODBC drivers can access a variety of data formats on a local machine, including Access, dBase/FoxPro, Excel and Paradox. ODBC drivers can also access higher end client-server databases such as Microsoft SQL Server, Oracle or IBM's DB2.\n\nThe bad news is that the speed and quality of the drivers varies greatly. Older Access \"Jet\" drivers can be slow and prone to crash, for example. The Visual FoxPro driver you can download from Microsoft is much faster than the FoxPro driver that generally comes with ODBC packages.\n\nEnabling middleware\n\nSo what do you need in the way of hardware to put Cold Fusion to work?\n\nYou need a Web server. But that doesn't necessarily mean a monster machine with 128 megs of RAM and an 8-gigabyte drive.\n\nIn fact, a Web server is nothing more than a PC with Web server software - such as Website or Microsoft Internet Information Server - installed on it.\n\nIf you want to start out small, you can probably get away with setting up one of the better PCs on your network as a Web server.\n\nAt The Virginian-Pilot, we test Web database applications on a Pentium 200 with 32 megs of RAM. The setup has so far worked fine, even with more than one user accessing the data at the same time.\n\nAfter testing, we move the applications to a bigger server maintained by our online edition for two reasons: (1) It gets them outside the corporate fire wall where our bureaus, which still access the Web through dial-up connections, can get to the data; and (2) It allows us to offer the general public access to some of the data.\n\nBut for those considerations, we could probably get by just fine running them on a makeshift Web server in our newsroom.\n\nKeep in mind, though, that more RAM is always better. And with 32 megs of RAM now selling for about $50, don't short your machine on memory unless you have to. Extra RAM also saves wear and tear on more expensive hard drives.\n\nAnd give your server the best hard drive you can afford. It will dramatically increase performance. Look into an Ultra Wide SCSI hard drive running at a high RPM (7,200 to 10,000). Probably the best available for Web servers right now is the Seagate Cheetah series.\n\nWhen you begin to develop Web database applications, you'll find yourself confronted with a lot of products with exotic - and baffling - names like Active Server Pages, Visual Interdev, Cold Fusion and countless others.\n\nSome Web site building software, such as Microsoft Front Page 98, comes with tools for building dynamic applications via ODBC connections to databases. The growing list of products makes for a pretty confusing picture. We can't say that Cold Fusion is the best solution, but we can say that it has worked well for us at The Virginian-Pilot and The Seattle Times. You can waste a lot of time searching for the perfect product. Or you can pick one that you're comfortable with, learn it, stick with it - and get your data on the Web. That's what we did with Cold Fusion.\n\nRay Robinson can be reached by e-mail at rsquared@pilotonline.com Tom Boyer can be reached by e-mail at tboy-new@seatimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-may-bits-bytes-and-barks","slug":"1998-may-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-may-bits-bytes-and-barks.html","year":1998,"month":"May","monthNum":5,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE offers audio tapes from Indiana CAR conferences, maintains multiple mailing lists for journalists, and provides access to databases including IRS exempt organization records and FBI crime statistics. The Campaign Finance Information Center provides free access to campaign finance data from thirteen states with plans to release a universal search engine.","keywords":["databases","mailing","journalism"],"topics":["Journalism & Newsroom","Data & Databases","Government & Politics"],"technologies":["World Wide Web","FTP","Internet"],"full_text":"Indiana CAR Audio Tapes\n\nAudio tapes from the sessions at Indiana CAR are now available for ordering. For more information or an order form, point your browser to www.ire.orgiresources/nicari conferences/indianalaudio.html\n\nMailing Lists\n\nIRE maintains a number of mailing lists, some with longstanding presence and some just getting started. To subscribe to IRE-L or NICAR-L, send an e-mail to listproc@lists.missouri.edu. In the body of the message, type: subscribe NICAR-L <yourname> subscribe IRE-L <your name>\n\nTo subscribe to CFIC-L, the newly formed Campaign Finance Information Center listserv, send an e-mail to majordomo@campaignfinance.org. In the body of the message, type: subscribe CFIC-L <your e-mail address>.\n\nTo subscribe to the INTRACAR mailing list, which discusses running and developing Intranets, send an e-mail to majordomo@nicar.org. In the body of the message, type: subscribe intracar <youre-mailaddress>.\n\nAlso, check out the IRE-L and NICAR-L mailing list archives on our Web site at www.ire.orgfresources/ire-U index.,html and www.nicar.org/resources/nicarinicari.html You can see posts to both lists organized by thread, author and date. The list archives are available in .html or plain text format.\n\nData Library Additions\n\nNICAR's database library has recently added two databases to its collection. The IRS Business Master File database contains a listing of exempt organizations. Some fields include the organization name, address, exempt classification category, primary function of the organization, asset amount and income amount.\n\nAs well, the library has acquired the 1996 FBI Uniform Crime Report databases: six separate databases gathered from law enforcement agencies around the country. They are arranged by reporting agency and broken down by month. The library can be reached at (573) 884-7332.\n\nCampaign Finance Information Center\n\nCampaign finance data from thirteen states is freely downloadable from the CFIC at www.campaignfinance.org. We also have links to ten online search engines hosted by other non-profits and state boards of election. We will soon release our \"universal\" online search engine so you can type in a contributor from your state and see where else they are giving.\n\nThe more inclusive this database, the better for everyone. So if you have state or local campaign data you want included, contact CFIC Coordinator Jack Dolan at jack@nicar.org or (573) 882-1982. The CFIC credits everyone who contributes data.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-moneyed-interests","slug":"1998-june-moneyed-interests","url":"https://dwillis.github.io/uplink/article/1998-june-moneyed-interests.html","year":1998,"month":"June","monthNum":6,"headline":"Moneyed interests","kicker":"CFIC UPDATE","author_name":"Jack Dolan and Aaron Rothenburger","author_title":"","affiliation":"CFIC","summary":"An investigation by Milwaukee Journal Sentinel reporters uncovers a link between campaign contributions and preferential treatment by Wisconsin Governor Tommy Thompson.","keywords":["campaign","finance","favoritism"],"topics":["Government & Politics","Investigative Methods","Business & Economy"],"technologies":[],"full_text":"A direct link between campaign contributions and favoritism is the Holy Grail of campaign finance reporting. In the 1997 series \"Money and Influence,\" Milwaukee Journal Sentinel reporters Steve Schultze and Daniel Bice connected campaign contributions to preferential treatment by Wisconsin Governor Tommy Thompson. The two reporters examined records from eight state agencies, state contracts, campaign contributions and expenditures, and Thompson's phone records. The eight-month investigation revealed a startling correlation between those who gave money to Thompson's campaign and those who got lucrative state contracts. In one instance, Schultze and Bice found that Ron Van Den Heuvel, a Wisconsin businessman, donated $10,000 to Thompson's campaign fund the day before a state agency awarded him $24 million in tax-free bond financing to build a paper-manufacturing plant. Van Den Heuvel says a Thompson fundraiser asked him for the donation. The newspaper's inquiry forced Thompson to return the contribution. Shultze and Bice stated the Van Den Heuvel affair was not an isolated incident, describing donors and well-connected firms enjoying a close and mutually beneficial relationship with the Thompson administration. Readers can investigate this claim by downloading Wisconsin campaign finance data from the CFIC or by searching the Wisconsin Democracy Campaign website.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1998-june-homes-for-sale","slug":"1998-june-homes-for-sale","url":"https://dwillis.github.io/uplink/article/1998-june-homes-for-sale.html","year":1998,"month":"June","monthNum":6,"headline":"Homes for sale","kicker":"STATS FROM PHILLY","author_name":"Neill A. Borowski","author_title":"Reporter","affiliation":"The Philadelphia Inquirer","summary":"Real estate journalism offers significant opportunities for impactful local reporting, as home values directly affect Americans' largest financial investment and can influence consumer spending and economic trends. Understanding local property value trends through data analysis provides readers with relevant information about their most valuable asset.","keywords":["housing","values","trends"],"topics":["Business & Economy","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Your editor mumbles \"estate story.\" You immediately head for the door. Many reporters have little or no interest in writing about homes. Yet relevant news about the typical family's most valuable investment can make for a front-page story that has impact. This doesn't mean the filler material that tends to populate many newspaper real estate sections. The emphasis is on stories that are local and tell readers something they want to know. What can be more relevant than a story that explains trends in values of their home? Let's consider the role of the home in the United States, according to the American Housing Survey for the United States in 1995. The house is the central asset, the biggest debt, and the greatest monthly expense for countless Americans. Out of 106 million year-round housing units in the nation, 60 percent are owner-occupied. The median value of owner-occupied homes is $92,507. About six out of 10 homes have one mortgage outstanding. Seven percent have two or more mortgages and 13 percent have a home-equity loan. The median number of years to repay the mortgage is 20, with a median principal of $48,466 to go. The median monthly cost is $593. The median ratio of home value to income is 2:3. However, the home value for about 25 percent of owners is four times their income or more. Perceptions—and misperceptions—about trends in real estate prices can influence consumer spending and the economy. Inquirer surveys conducted study trends from 1989, and the conclusions revealed new insights when paired with Realist Inc.'s comprehensive property data. This allowed the creation of matched pairs to observe specific property price changes, thereby providing a clearer picture of real estate value changes in the region.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1998-june-trial-of-the-century","slug":"1998-june-trial-of-the-century","url":"https://dwillis.github.io/uplink/article/1998-june-trial-of-the-century.html","year":1998,"month":"June","monthNum":6,"headline":"Trial of the century","kicker":"MILLENNIUM AD NAUSEUM","author_name":"Wendell Cochran","author_title":"","affiliation":"American University","summary":"As the year 2000 approaches, companies and governments worldwide are spending billions to fix the Y2K bug that could cause computer systems to malfunction, though computer-assisted journalists are relatively insulated from the problem and should still test their systems.","keywords":["millennium","compliance","systems"],"topics":["Technology","Journalism & Newsroom"],"technologies":["spreadsheet"],"full_text":"Nanosecond by nanosecond, the clock ticks toward the millennium and the biggest beta test in computer history - the rollover from 1999 to the year 2000. By some accounts, that first day of the new century is going to bring a silicon meltdown because millions of computers, literally, won't know what day it is. Stock exchanges, banks, airlines, government agencies, and major corporations are spending billions to make their systems compliant. The Information Technology Association of America estimates it will cost more than $50 billion to fix computers in the United States, perhaps $600 billion worldwide. Some economists have predicted that the costs of fixing computer hardware and software are so high that the U.S. economy will be thrown into a recession. Some companies and governmental bodies, apparently hoping the problem will just go away, have barely begun to figure out what the Y2K issue means to their operations. A member of Congress recently reported that some \"mission-critical\" Pentagon systems won't be fixed until 2009. Thankfully, those of us who work in computer-assisted journalism probably don't need to panic. Most of our hardware is relatively new; most of the applications we use regularly have tried to address the Year 2000 problem at least after a fashion; and most of our databases and spreadsheets don't depend on precise date calculations. However, CAR specialists are suggested to test their systems, experiment with date handling, and assess older software applications which may face more complications due to Y2K.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1998-june-two-web-sites-to-check-out-for-year-2000-computer-issues-include","slug":"1998-june-two-web-sites-to-check-out-for-year-2000-computer-issues-include","url":"https://dwillis.github.io/uplink/article/1998-june-two-web-sites-to-check-out-for-year-2000-computer-issues-include.html","year":1998,"month":"June","monthNum":6,"headline":"Two WEB SITES TO CHECK OUT FOR YEAR 2000 COMPUTER ISSUES INCLUDE:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article recommends two websites for researching Year 2000 computer issues and provides links to searchable archives of IRE and NICAR mailing lists where these topics have been discussed.","keywords":["Y2K","websites","archives"],"topics":["Technology"],"technologies":["World Wide Web"],"full_text":"Two WEB SITES TO CHECK OUT FOR YEAR 2000 COMPUTER ISSUES INCLUDE:\n\nwww.Y2K.COM\nWWW.YEAR2000.COM\n\nTHESE ISSUES HAVE FOSTERED LENGTHY DISCUSSIONS ON THE IRE AND NICAR MAILING LISTS. TO SEARCH THE ARCHIVES OF THESE MAILING LISTS BY AUTHOR, SUBJECT OR DATE, POINT YOUR BROWSER TO THE FOLLOWING WEB ADDRESSES.\n\nIRE-L: WWW.IRE.ORG/RESOURCES/IRE-L/INDEX.HTML\n\nNICAR-L: WWW.IRE.ORG/RESOURCES/NICAR/NICARL.HTML","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-date-handling-in-popular-car-applications","slug":"1998-june-date-handling-in-popular-car-applications","url":"https://dwillis.github.io/uplink/article/1998-june-date-handling-in-popular-car-applications.html","year":1998,"month":"June","monthNum":6,"headline":"Date handling in popular CAR applications","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A reference table comparing how popular computer-aided reporting (CAR) applications handle date ranges and two-digit year defaults, showing variations in start dates, end dates, and century interpretation rules across programs like Excel, Access, FoxPro, Quattro Pro, Paradox, Lotus 1-2-3, and Lotus Approach.","keywords":["dates","software","reporting"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["Excel","Access","FoxPro","Quattro Pro","Paradox","Lotus 1-2-3","Lotus Approach"],"full_text":"Date handling in popular CAR applications\n\nPROGRAM\tSTART DATE\tEND DATE\tTwo-DIGIT YEAR DEFAULTS\nExcel 97\tJan. 1, 1900\tDec. 31, 9999\t00-29=2000\n\t\t\t30-99=1900\nExcel 98 for Macintosh\tJan. 2, 1904\tDec. 31, 9999\t\nAccess for Windows 97\t\tDec. 31, 9999\tSame as Excel\nFoxPro\t\tDec. 31, 9999\t\nQuattro Pro 8\tJan. 1, 1600\tDec. 31, 9999\t00-50=2000\n\t\t\t51-99=1900\nParadox 8\t9999 BC\tDec. 31, 9999\t00-50=2000\n\t\t\t51-99=1900\nLotus 123\tJan. 1, 1900\t\t\nLotus Approach\t\t\t\n\nSources: Microsoft, Corel, Lotus. Note: Earlier versions of these programs handle dates differently. Check your documentation.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-wendell-cochran-provides-the-following-web-sites-of-computer-companies-addressing-year-2000-computer-issues","slug":"1998-june-wendell-cochran-provides-the-following-web-sites-of-computer-companies-addressing-year-2000-computer-issues","url":"https://dwillis.github.io/uplink/article/1998-june-wendell-cochran-provides-the-following-web-sites-of-computer-companies-addressing-year-2000-computer-issues.html","year":1998,"month":"June","monthNum":6,"headline":"WENDELL COCHRAN PROVIDES THE FOLLOWING WEB SITES OF COMPUTER COMPANIES ADDRESSING YEAR 2000 COMPUTER ISSUES:","kicker":"","author_name":"Wendell Cochran","author_title":"","affiliation":"","summary":"Wendell Cochran compiles a resource list of major computer companies and their websites addressing Year 2000 computer issues, including Microsoft, IBM, Apple, Dell, and others.","keywords":["computers","Y2K","resources"],"topics":["Technology"],"technologies":["Internet","World Wide Web"],"full_text":"WENDELL COCHRAN PROVIDES THE FOLLOWING WEB SITES OF COMPUTER COMPANIES ADDRESSING YEAR 2000 COMPUTER ISSUES:\n\nMICROSOFT: www.MICROSOFT.COM/CIO/ARTICLES/YEAR2000FAQ.HTM\nCOREL: www.CoREL.COM/2000.HTM\nLOTUS: wWW.LOTUS.COM\nINTEL: HTTP://SUPPORT.INTEL.COM/YEAR2000/PAPER.HTM\nIBM: www.IBM.COM/IBM/YEAR2000\nAPPLE: www.APPLE.COM/MACOS/INFO/2000.HTML\nCOMPAQ: www.COMPAQ.COM/YEAR2000\nHEWLETT-PACKARD: www.HP.COM/YEAR2000/INDEX.HTML\nDELL: www.DELL.COM/YEAR2000\nGATEWAY2000: wWW.GATEWAY.COM HOME/SUPPORT/c%5FTECHDOCS/Y2K/DEFAULT.HTM\nNOVELL: www.NOVELL.COM","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-tales-from-the-vault","slug":"1998-june-tales-from-the-vault","url":"https://dwillis.github.io/uplink/article/1998-june-tales-from-the-vault.html","year":1998,"month":"June","monthNum":6,"headline":"Tales from the vault","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A detailed investigation by The Dispatch reveals how Ohio's campaign finance laws have been exploited by both state and national political parties, with examples including Senator Robert Gardner receiving over $900,000 in party support and a custom-built Republican headquarters designed to maximize in-kind contributions.","keywords":["campaign","loophole","contributions"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["World Wide Web"],"full_text":"Continued from page one:\n\nat http://www.campaignfinance.org/stories/ divert.html.) According to The Dispatch, the biggest beneficiary so far of this loophole is Cleveland area state Senator Robert A. Gardner. State Republicans funneled more than $900,000 into Gardner's last campaign, or more than 90 percent of Gardner's total reciepts. A sudden, last-minute $384,000 advertising campaign paid for by the state party and \"loaned\" to Gardner may have been the deciding factor in Gardner's narrow victory. The loan was especially generous since there is no time line for Gardner to repay it, no state law that prohibits the party from simply forgiving it, and no requirement to report who repays it if it is in fact repaid. Rowland also reports that a new headquarters for the Ohio Republican Party was \"custom-designed around the state's campaign finance laws.\" The building was financed by a party trust fund that allowed corporations to exceed normal contribution limits. Rowland calls the new Republican headquarters veritable factory for in-kind contributions\" since the TV and radio ads, computer services, and direct mailings that originate in the new headquarters count as in-kind contributions to party candidates. Rowland's series also details how the Democratic and Republican national parties used Ohio's less restrictive campaign finance laws to their benefit. In 1996, the Democratic National Committee directed out-of-state contributors to send unsolicited contributions to the Ohio Democratic Party. The money is then transferred, or even sold back, to the national party. The biggest single contributor to the Ohio Democratic party in 1992 was Indonesian businessman James T. Riady, who gave $75,000 to the state party six days before President Clinton's election. The latest Ohio campaign finance data can be searched online at the Ohio Open Elections Project on the Web site of the Center for Responsive Politics (http://www.crp.org/ooepl).\n\nWhen the bureaucrats in charge of computerizing state campaign finance records settle into their customary glacial rhythm, reporters have two choices: cover well-rehearsed sound bytes and photo opportunities while waiting for an information thaw, or start a fire by building their own campaign finance database. A consortium of 29 newspapers and two TV stations in New York lit their own fire under local politicians this year when they set aside traditional rivalries to join forces and turn over 10,000 paper records into the most comprehensive campaign finance database in New York history. The result, according to Newsday's Ford Fessenden, was blizzard of stories\" putting state politicians on notice that the public is interested in their campaign finances. A telling example of the impact: A couple of years ago, a Newsday reporter noticed that the paper contribution reports from Governor Pataki's office came sorted alphabetically by the contributor's first name. Suspecting a computer was used to generate that order, the reporter called Gov. Pataki's press secretary, Zenia Mucha, to ask for the governor's campaign contributions in electronic form rather than on paper. Mucha laughed at the reporter. Today, after hundreds of stories on state campaign finances generated by consortium members, Pataki files contribution reports electronically.\n\nCFIC offerings\n\nThe CFIC library, located online at www.campaignfinance.org/stories/index.html, is an archive of local, state and federal articles that use campaign contribution reports to \"follow the money.\" The library also serves as a complement to the CFIC's online databases of state campaign finance contributions. The three investigative series described above are just a few of the excellent examples of how campaign finance data can enhance state and local political reporting. If you have a relevant article or series to submit to the CFIC's growing repository of stories, or if you know of an article or series that should be submitted, contact Jack Dolan at jack@nicar.org or Noemi Ramirez at noemi@nicar.org, or call (573) 884-1802. Jack Dolan can be reached by email at jack@nicar.org. Aaron Rothenburger can be reached by email at aaron@nicar.org\n\nSEE PAGE 15 FOR TIPS ON CREATING CAMPAIGN CONTRIBUTION DATABASES FROM PAPER RECORDS. THE TIPS ARE WRITTEN BY DIANE RENZULLI OF THE CENTER FOR PUBLIC INTEGRITY.\n\nTHE WEB SITE FOR THE CAMPAIGN FINANCE INFORMATION CENTER IS A VALUABLE RESOURCE FOR ANYONE INVESTIGATING CAMPAIGN FINANCES.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-a-more-perfect-union","slug":"1998-june-a-more-perfect-union","url":"https://dwillis.github.io/uplink/article/1998-june-a-more-perfect-union.html","year":1998,"month":"June","monthNum":6,"headline":"A more perfect union","kicker":"TECH TIP","author_name":"Andrew Lehren","author_title":"","affiliation":"Dateline NBC","summary":"Visual FoxPro 2.6 users can benefit from upgrading to Visual Fox to access a dedicated datetime data type with built-in functions for easier time and date handling. The article demonstrates how to convert character data to datetime format using the ctot() function and explains how to handle missing or invalid entries.","keywords":["datetime","conversion","functions"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["FoxPro"],"full_text":"For FoxPr 2.6 fans wondering whether to jump to Visual Fox, here's one reason: you can handle time without any fancy programming.\n\nThat's because Visual Fox includes a date time data type. It's separate from the old date data type, which remains in the newer versions. And along with the new category come some handy functions. Here's an overview with tips to get things done faster and avoid mistakes.\n\nConverting your data\n\nLet's assume you want to convert data stored as character. You can create a new date time field by taking your mouse to Table Properties and modifying the structure. After inserting a new column, go to data type and pick date time. That's a choice you didn't have in earlier versions of Fox. Now go back to your command window. To bring in data from a character column, you'll need to use the character-to-time function ctot(). Let's look at how it works.\n\nLet's say you're starting with three separate character columns, one with dates, another with times, and a third designating a.m. or p.m. Pretend this is part of a database showing the times that city officials made telephone calls:\n\nCALLDATE\tCALLTIME\tAMorPM\n12/31/95\t12:45\tAM\n09/14/60\t7:35\tPM\n01/10/60\t7:40\tAM\n10/8/96\t11:35\tAM\n\n(Notice the data is a little inconsistent. Sometimes you've got a leading zero in your day or time. Sometimes you don't. It doesn't matter - Fox will cut you slack. You also won't need to trim any leading spaces.) In this example, we'll pretend you've created a new date time column called CALLTIME.\n\nNow we're ready to use the ctot function. Have no fear. It should be familiar territory to anyone who's messed with Fox functions. To fill in the times in the new column, run this command:\n\nReplace all CALLTIME with ;\nctot(CALLDATE + CALLTIME + AMorPM)\n\nThe result should look something like this:\n\n12/31/95 12:45:00 AM\n09/14/96 07:35:00 PM\n01/10/97 07:40:00 AM\n10/08/96 11:35:00 PM\n\nAnd you're done converting your data.\n\nMissing or bizarre entries\n\nIf, for some reason your data includes a time that cannot exist, like 25:00, the replacement for that record won't work. Nothing will be inserted. The same goes for an invalid date, even if you have a valid time. Nothing will be inserted. However, if you have a valid date and nothing in your time field, Fox will insert the date plus 00:00:00, which is midnight. And, if you have a valid time but no date, Fox will insert 12/30/1899. (If you forgot how to see what century it is, simply type in your command window: SET CENTURY ON.) If these seem like boring details, keep in mind that this can mean the difference between a query that gets a right or wrong answer.\n\nHow do I look?\n\nThe default setting shows time with seconds. If you don't like this - your data doesn't have seconds and you don't like looking at all those zeros - type SET SECONDS OFF in your command window. Away go all those pesky :00s.\n\nIn addition, you may find it easier to ask some questions by using military time. That's where the clock goes all the way to 24:00 and you no longer need a.m. or p.m. To do that, type SET HOURS TO 24. Now everything will be in military time. Then again, if you hate seeing things in military time, the opposite command is SET HOURS TO 12.\n\nA query sampler\n\nLet's say you want to find all calls after 2 p.m. on Sept. 12, 1996, when a big piece of legislation was introduced:\n\nSelect * ;\nFrom PhoneCalls ;\nWhere CALLTIME > { 9/14/96 14:00}\n\nHere you need to use the curly braces, the same way you needed them for the original date data type. Let's say you want to find all calls between p.m. on Sept. 12, 1996, and 10 a.m. on Jan. 15, 1997, when the big piece of legislation finally came up for a vote:\n\nSelect * ;\nFrom PhoneCalls ;\nWhere between (CALLTIME, {9/12/96 14:00}, {1/15/97 10:00}\n\nAgain, you need to use the curly braces. Now let's say you want to find out who's so important that the mayor is calling after 10:00 at night. If you have things set to military time, the query is simply:\n\nSelect * ;\nFrom PhoneCalls ;\nWhere hour(CALLTIME) > 22\n\nNotice that when you use the hour() function, the criteria is numeric. You don't need any punctuation, like quotes or curly braces.\n\nTime differences\n\nLet's say you have the starting time of a phone call and the ending time, and you want to know how long a city council member called that 900 number. Remember that the date time data type answers with the number of seconds - even if you've set seconds off. So if you want your answer in seconds, don't do a thing. If you want it in minutes, divide by 60. In hours, by 3600. And so on. For example:\n\nSelect , (ENDCALL - BEGINCALL) / 60 as CallLength ;\nFrom PhoneCalls\n\nThis will tell you the call's length, in minutes. Notice that you need to put the EndCall - BeginCall in parentheses. The reason that must be done is to first generate the difference a result that will be numeric and can then be divided by 60.\n\nLet's say that, for some reason, you have one column as a date field, and another as a date time field and you want to find the difference between them. Let's say it's an arrest database. The date field is Birthday, and the date time field is Arrested, noting when someone was busted. For this mixing and matching, you can use the ttod() function. Here's an example. (This time, dividing converts our answer into years, to see how old they were when arrested):\n\nSelect (ttod(Arrested) - Birthday) /365.25 as ArrestAge ;\nFrom Crimes\n\nCatching nulls\n\nRemember in the section on missing and bizarre entries? Let's say you want to find an instance when no data is in the field. There are two ways to do that: with the isblank or empty() functions. The isnull() function will not work because Fox has yet to allow true null designations for these cases. So here's an example to catch them:\n\nSelect * ;\nFrom PhoneCalls;\nWhere empty(EndCall)\n\nAnd, since you know what Fox inserts when you have a valid time but no date, or vice versa, you can run filters to catch or ignore those entries.\n\nWatch out! A caution about averages\n\nRemember how Fox will convert proper days with no time (by adding midnight) or no days with proper time (by assuming the day is 12/30/1899)? Well, don't forget that can screw up any summary calculations. For some reason, Fox will consider empty entries when running averages unless you tell it otherwise. Regardless, be careful to filter for these kinds of values if you don't want to include them.\n\nAndy Lehren can be reached by e-mail at alehren@nbc.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-battling-for-access","slug":"1998-june-battling-for-access","url":"https://dwillis.github.io/uplink/article/1998-june-battling-for-access.html","year":1998,"month":"June","monthNum":6,"headline":"Battling for access","kicker":"ADVENTURES IN DATASITTING","author_name":"Jennifer LaFleur","author_title":"","affiliation":"San Jose Mercury News","summary":"A journalist argues that news organizations must demand access to government records, highlighting obstacles like privatization, escalating costs, and privacy concerns, while also questioning whether the media adequately covers public records issues and holds government accountable.","keywords":["records","access","government"],"topics":["Government & Politics","Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Jennifer assumed that the press needs access to government databases. seems a little cocky.\n\nI recently spoke at a conference for the organization for federal FOIA Officers (called ASAP). Although my evaluations varied, the above comment stood out for me:\n\nThe battle for records access is heating up. It is no longer a given that government records are easily available. The hurdles are many, but three key problems that are more frequent and of grave concern include:\n\nPrivatization of records: Government agencies are more frequently having private companies maintain their databases. It makes access more difficult and muddies the definitions of what is owned by the public. Sometimes government agencies themselves have problems getting data in these situations.\n\nEscalating costs for records, particularly electronic records, which prevent or greatly delay a news organization's ability to gain access to records.\n\nViolating public records laws in the name of protecting the privacy of citizens. In many states, the courts have flipped public records laws on their heads. They make it the burden of news organizations to show public interest over privacy, which was not the intention of those laws.\n\nAlthough there is a lot of hype about how the Internet is causing everyone's privacy to be invaded (just like cable television was when I was younger), the public doesn't always agree with government's protection. At a recent Mercury News brown bag with readers on a totally different topic, readers spoke up about how they want their newspaper to get access to government records. They even volunteered to get a packet of letters together in support of public records access. If not for ourselves, we owe it to our readers to demand access to government records.\n\nIs it our fault?\n\nRather than dwell on the usual ways government agencies bamboozle reporters into thinking they shouldn't have access to data, let's think about some of the things we do or don't do when it comes to public records.\n\nCover citizen's access battles. We don't do very many stories about how citizen's can't get records or how one government agency can't get information from another.\n\nCover how candidates stand on public records. How many know how your local candidates stand on public records? Did your legislators vote for the Drivers Privacy Protection Act?\n\nDo we follow legislation that affects open records? Do a search of legislation in your state to find those affecting your state's open records act - you might be surprised.\n\nProvide information for readers about how to get information. Once a year the American Society of Newspaper Editors does a project called \"Your Right to Know\" where they provide information to newspapers (and encourage papers to do their own work) on open records. What else could you do?\n\nDo reporters follow through with requests and make agencies accountable? Or do they get tired of asking and move on to something else? We are all guilty of giving up some fights - but that sends a message to government agencies that they've \"won.\" If you don't have legal help at your newspaper, and you've tried every means yourself - contact IRE's access committee or your local SPJ chapter.\n\nDuplicate the Indiana Project. Seven newspapers in Indiana did an audit of government records in their state. Search the Indiana CAR www.ire.org information on this project.\n\nGuidelines to request records\n\n1. Know the law. Know how FOIA/open records law treats the information you're requesting and what the exemptions are. If there are exemptions, the agency must be able to cite them.\n\n2. Know what information you want. Don't ask an agency to provide you everything they have. Make your request specific. It's likely that a reporter may need the actual data that was used for a report. Before asking for the data, request the record layout for that database. Documentation is available under the law.\n\n3. Know how the information is kept. Try to find someone in the information systems department at an agency who knows. This is not to avoid the public relations person you're dealing with, but sometimes it's necessary to talk nerd-to-nerd. Reporters using computer databases are actually quite computer-savvy.\n\n4. Know what the appropriate cost should be. You really should have to pay only duplication costs. Ask for an itemization for those costs. This is an area that gets difficult when an agency has a set price for databases.\n\n5. Know the source of the data - be familiar with how the records are created.\n\n6. Know who administers the data. The person in charge of the database can be helpful for understanding the data.\n\n7. Get hard copy summary reports. This will give you a way to check your data.\n\n8. Know how many records or pieces of information are in the database. When you get the database, make sure you have the right number of records.\n\n9. Know how large the database is. You don't have to know a lot of technical jargon. Most PCs will hold at least 250 megabytes of information or roughly 250 million characters. If they say the file is 900 megabytes, you won't be able to do much with it unless you find a computer a bigger hard drive. Most newsrooms have the capability of reading nine-track tapes from mainframes and many other data formats.\n\n10. Confidential information. A government agency may claim that certain pieces of the information are confidential. There very well may be confidential information in the file, but that doesn't mean you shouldn't get the rest of it.\n\n11. Involved with local group meetings of software users. These groups bring together folks from all different disciplines who work together to solve problems and share tips.\n\nNon-viable denial\n\nThe previous guidelines are all dandy, until you actually go to ask for the data. Many times you'll run into reasons why an agency give you information, probably because 8 out of 10 reporters will just say \"oh, okay\" and go away. Here is a sample of reasons I have received for why an agency cannot fill my request:\n\nListen Missy (yes, this really happened): Our database is very complicated. You probably won't understand it. Most newsrooms have the capacity to handle very large and complex databases. If you're not dealing with someone who knows the data, you need to.\n\nOur computer system can't do that. This is unusual for any database system. It may take talking with the software vendor to figure out the best option for both sides.\n\nThe person that knows how to do that is on vacation for two weeks/doesn't work here anymore. This is a problem.\n\nIt will cost you $20,000. Ask for an itemized estimate of charges. Remember that the list of other recipients of the data and what those recipients paid is available under FOIA.\n\nThe database is not public record. The burden is on the agency to show where in the law that information is not public record. If you have done your homework, this should be less of a problem.\n\nWe don't like what you plan to do with it. Interesting. On the other hand, when you're able to cooperate with an agency, they frequently can help point you in the right direction.\n\nThe database contains confidential information. Everyone needs to do their homework on this one. Redacting with a computer is possible just like on paper with a black marker.\n\nWe don't keep that on computer. Although this is true in some cases with smaller jurisdictions or small offices federal organizations, it's uncommon.\n\nIf we give it to you, we'll have to give it to everyone. So what's the problem with this.\n\nThat uses proprietary software. You don't want software, you want the data. This may require getting together with the vendor.\n\nWe don't mind giving you a few records, we just don't want to give you the whole database. To which the judge asked: You mean if they wanted one record from one person it would be okay? Their answer: depends on the person.\n\nVisit these relevant Web sites:\n\nAmerican Society of Newspaper Editors \"Your Right to Know\" project: www.asne.org/ideas/rtk/rtkmain.htm\n\nFreedom of Information Center: www.missouri.edu/-foiwww\n\nReporters Committee for Freedom of the Press: www.rcfp.org\n\nJennifer LaFleur can be reached by e-mail at jlafleur@sjmercury.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-beyond-plain-boolean","slug":"1998-june-beyond-plain-boolean","url":"https://dwillis.github.io/uplink/article/1998-june-beyond-plain-boolean.html","year":1998,"month":"June","monthNum":6,"headline":"Beyond plain Boolean","kicker":"ON THE INTERNET","author_name":"Debbie Wolfe","author_title":"","affiliation":"St. Petersburg Times","summary":"Advanced web search techniques using Boolean operators and specialized search engine features can significantly improve search results and save time by reducing irrelevant hits. The article provides examples of supercharging searches through phrase matching, proximity searches, and weighted terms, along with recommendations for major search engines of the era.","keywords":["search","Boolean","techniques"],"topics":["Technology","Journalism & Newsroom","Investigative Methods"],"technologies":["Internet","World Wide Web","Nexis","Dialog"],"full_text":"Searching the Web with pizzazz and aplomb requires going beyond (yawn) plain Boolean.\n\nBy some estimates, the Web contains more than 150 million pages-some 50-60 billion words. (Let's not get too picky over the \"exact\" size. The point is the Web is huge and growing exponentially. It's time to learn advanced searching tips and tricks and to keep current with searching advancements.)\n\nIf you're still doing searches only with plain \"and\" and \"or\" connectors/operators, you must love to plow through junk hits. Since search engines and directories offer some of the same advanced Boolean logic options available in Nexis, Dialog and DowJones, why not take advantage to save time and frustration?\n\nSerendipity has its place in any searcher's toolkit, but when you're on deadline-or just not in the mood to plow-try focusing your search with supercharging techniques.\n\nSample supercharges\n\nMaking sure \"and\" means AND!\nAdd a (plus) sign in front of each word you absolutely want to include in the results: +pet+care finds both pet+care finds 'care\" and may find \"pet\"\nProximity: \"phrase\" and \"near\" searches Add quote marks around words you want to search as a phrase: \"pet care\" looks for both as a phrase \"+pet+care\" finds both as a phrase pet near care looks for \"pet\" within a certain number of words of \"care\"\n\nOrder can make a difference Change the order of your search terms to adjust the weight assigned to each part: breeders and title:\"maltese\" may find different results than title:\"maltese\" and breeders\n\nTest your searching savvy\n\nAlta Vista (http://www.altavista.digital.com)\nI'll either use this or Yahoo (http://www.yahoo.com) to start a Web search quest. The advanced search page \"Help\" area reveals one of the most comprehensive tip sheets out there. Run the same query on the simple and advanced pages and you'll get different results. Search words are automatically weighted on the simple page-you're in charge on the advanced page. Caution: If you don't specify a word to sort by, the advanced results will be in no particular order.\nFindLaw - Internet Legal Resources (http://www.findlaw.com)\nDejaNews (http://www.dejanews.com) The best place to search Usenet newsgroup postings, including complete threads.\nCalifornia State University at Stanislaus (http://www.library.csustan.edu)\nLinks to search engines maintained by librarians.\nLycos (http://www.lycos.com) Search for pictures or sound files through the drop-down menu. Pro search help area has a link to \"Web Guides.\" Choose \"Education\" and browse the \"Reference\" link to locate electronic favorites.\nHotBot (http://www.hotbot.com) The form-based interface looks simple to use, but spend some quality time checking out the options since documentation is lacking. The \"Browse by Category\" area has a unique cascading interface. Check out the \"Reference\" link to see the effect.\n\nInfoSeek (http://www.infoseek.com)\nThe tip sheet is a fast scan, the interface is simple, and the results come back at rocket speed.\nThe Virtual Reference Desk (http://thorplus.lib.purdue.edu/reference/index.html)\nI know this is not a search engine or directory, but I couldn't help myself. This page is so wonderful, it will convince any jaded journalist that the Web is a tool that belongs at everyone's desk!\nMagellan (http://www.mckinley.com) Simple to use. What's best about this site is the \"Find Similar\" link on the results page. This feature helps you modify your original search with one click. Reviews are also included.\nExcite (http://www.excite.com)\nOnce you get a results page, the \"List by Web site\" feature is a handy way to look the top 40 hits as multiple links from the same site address are grouped together for easy evaluation. The \"More Like This\" link on the results page is a quick way to modify a strong link for further results.\nWebCrawler (http://www.webcrawler.com)\n\nFast and simple to use. The \"Similar Pages\" feature on the summary results page helps to focus the next search. Choose between \"titles\" or \"summaries\" for display options. Don't have enough time to check out all the links? Save the page in HTML format and explore them later!\n\nReference.com (http://www.reference.com)\n\nMetasearch sites\n\nInternet Sleuth (http://www.isleuth.com) Allows you to search one-six engines/directories at a time and completes the task in seconds, but the results display takes time. The interface also likes the lowest-common-denominator search strategy, which frequently limits what you are looking for to plain Boolean. The service indexes only links to what it considers \"searchable databases.\"\n\nAll4One (http://www.all4one.com)\n\nSearch four engines simultaneously: AltaVista, Lycos, HotBot and Excite. Each engine is displayed in a frame taking up a quarter of the screen.\n\nHighway61 (http://www.highway61.com)\n\nThis is a down-to-earth site, and when you get there, you'll see why. It searches six engines at once: Excite, Infoseek, Lycos, WebCrawler and Yahoo. The display intermingles the results and ranks each hit on a scale of 100. Response time is fast.\n\nMetaCrawler (http://www.metacrawler.com)\n\nSearch six engines at once: Excite, Lycos, Yahoo, WebCrawler, Vista and Infoseek. The display is intermingled, and the results are ranked by confidence level on a scale of 1,000. Check out the \"MetaSpy\" link for a fun, red-light district of what other folks are searching for. Response time is extremely fast.\n\nProFusion (http://www.designlab.ukans.edu/profusion)\n\nSearch nine sites, including Excite, AltaVista, WebCrawler, Lycos, Magellan, Infoseek and Yahoo. You can ask the site to pick the best three, the fastest three, or all nine. The interface is very thorough and user-friendly. Ranked results are displayed by title and URL only.\n\nDogpile (http://www.dogpile.com) Quickly search 13 Web engines and directories, four Usenet newsgroup sources, and 2 FTP archives including Filez (http://www.filez.com). Results are displayed ordered by the engine where they were found. This is my top pick for metasearch sites.\n\nVerio MetaSearch, previously known as OnRamp (http://search.verio.net)\n\nSearch eight sites including Excite, HotBot, Infoseek, Lycos, WebCrawler and Yahoo. Be sure to read \"About\" and \"Origins,\" then go to the advanced search area and experiment with the weight options very cool! BONUS SITE: Search Engine Watch (http://searchenginewatch.com) This is your homework!\n\nDebbie Wolfe can be reached by email at dpwolfe@sptimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-boolean-logic-primer","slug":"1998-june-boolean-logic-primer","url":"https://dwillis.github.io/uplink/article/1998-june-boolean-logic-primer.html","year":1998,"month":"June","monthNum":6,"headline":"Boolean Logic Primer","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A primer explaining how Boolean logic operators (OR, AND, NOT) work to broaden or narrow search results in databases and search engines.","keywords":["Boolean","search","operators"],"topics":["Technology","Investigative Methods"],"technologies":[],"full_text":"Boolean Logic Primer\n\nOR broadens the results AND narrows the results NOT narrows the results\n\ncontain at least one item)* contain all items) (results must contain A & B but no C's)\n\nA or B or C A and B and C (A and B) not C\nhurricane or surge or damage hurricane and surge and damage (hurricane and surge) not damage\n\n* An \"OR\" search would look for \"hurricane\" or \"surge\" or \"damage\" or any combination of the three words. In the illustration above, the results indicate every possible result except the combination of all three words.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-assembly-required","slug":"1998-june-assembly-required","url":"https://dwillis.github.io/uplink/article/1998-june-assembly-required.html","year":1998,"month":"June","monthNum":6,"headline":"Assembly required","kicker":"HANDOUTS OF THE MONTH","author_name":"Jeffrey Meitrodt, Carolyn Tuft, David Washburn","author_title":"","affiliation":"Times-Picayune, St. Louis Post Dispatch, The Morning Call","summary":"A guide to building databases for investigative journalism, emphasizing that reporters can use readily available public records and even small datasets to uncover compelling stories by organizing, analyzing, and carefully verifying the data.","keywords":["database","investigation","records"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["spreadsheet"],"full_text":"\"The Ten Commandments of Building Your Own Database\"\n\n1. Use the computer as a source.\nThe most compelling stories can be put together with readily available information that merely needs to be organized and examined. Such records include standardized test scores available to every parent, traffic tickets, lawsuit records, public contracts, campaign finance reports, city/county school budgets, personnel files, public official voting records, billing records, check files.\n\n2. No amount of data is too small an amount.\nHow big does a story have to be before you plug data into a spreadsheet or database? The answer: \"smaller than you'd think.\" Reporters love to say, \"We analyzed a gazillion records and found\" That is well and good, but you can add meat to almost any story - and improve chances of landing on Page One - by plugging a dozen records into a spreadsheet.\n\n3. Don't be discouraged if you have to do it the hard way.\nSure, it's great when there is a ready-made database waiting to be analyzed for investigative stories and a cooperative public official willing to share it. That's not always the case. Besides, if the agency keeps records on a database, the target is already ahead of you. Chances are, if you're the first person who ever took a look at the numbers, there's a good story to tell. If the agency can't analyze its own data, it has no idea what's really going on.\n\n4. Think about the best way to get data before you go and get it.\nA low profile works best when you're taking on a new target. You don't want to give them time to circle the wagons and take action that would kill your story. So, find out if they release the data to the general public in some fashion and then get those reports acting as Joe Citizen. It's more work, but no one will know what you are up to.\n\n5. Organize your data before you analyze it.\nThink about the questions you want your data to answer before you start working on your spreadsheet, and then customize the columns to fit your needs. Think in terms of yes/no questions. Find ways to create numbers that will allow you to rank your data.\n\n6. The \"sort function\" is your friend.\nThe sort function may sound boring, but it isn't. It is the most powerful key on the computer when it comes to CAR. Do you want to find out who made the biggest campaign contributions? Or which school had the largest test-score swings? By sorting the data, you'll often find your most powerful anecdotes.\n\n7. Keep your data consistent.\nIf you're doing a column of people's names, always put the last name first. Don't use 9 and nine. Don't do anything that will make it harder to rank or sort data or to find data that matches.\n\n8. Double-check the data.\nYou need to be bullet proof. Mistakes are deadly. They give your target a chance to discredit your research.\n\n9. Do it yourself whenever possible.\nSometimes you have no choice about who inputs the data: there is simply too much to handle by yourself or with another reporter. But whenever possible, type in the data yourself. Two advantages: you see stuff you wouldn't have found otherwise, and you eliminate the errors that other people with much less invested in the story are going to make.\n\n10. Buy yourself enough time.\nEditors think CAR is easy. It's your job to make sure they know you're going to spend two weeks doing nothing but punching numbers into a spreadsheet. And then maybe another week checking your work and playing with the data before you're ready to make a single phone call. Tell them up front. You need to sell them on WHY there's no other way to do it.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-paper-riches","slug":"1998-june-paper-riches","url":"https://dwillis.github.io/uplink/article/1998-june-paper-riches.html","year":1998,"month":"June","monthNum":6,"headline":"Paper riches","kicker":"HANDOUTS OF THE MONTH","author_name":"Diane Renzulli","author_title":"","affiliation":"Public Integrity","summary":"This article provides comprehensive guidance on creating accurate campaign databases from paper records, covering essential steps from data gathering and entry through editing, coding, and fact-checking to ensure data quality and integrity.","keywords":["database","campaigns","records"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["CD-ROM"],"full_text":"Tips for Creating Campaign Databases from Paper Records\n\nStart small\n\nDon't take on the whole state legislature on the first try. State legislators in leadership positions, committee chairs, or candidates from local races are much more manageable subsets.\n\nStaffing\n\nMore people does not mean better data quality.\nIf you can, avoid in-house data entry for major projects.\nIf you can't, test potential data-entry personnel for attention to detail before hiring them.\n\nGathering the data\n\nMake friends with the staff at the local elections board.\nCreate an index of the candidates you want to include in your database.\nKeep copies of the records on site.\n\nData entry\n\nUse a double-entry system.\nGive each contribution a unique identification number that allows you to quickly retrieve it in paper form.\nEnter the name of the contributor exactly as it was written on the campaign records, even if it is incorrect.\n\nEditing the data\n\nKeep both an \"edited contributor\" and an \"original contributor\" field in your database.\nTo identify which acronym belongs to which organization, sort your database by address.\nNever guess. If a contributor has two different addresses, confirm that it is the same person or organization.\nIf family members give from the same address, you can combine them by using an \"and/or\" notation in the \"edited contributor\" field.\n\nCoding contributions\n\nCode contributions AFTER the database has been edited.\nUse the Center for Responsive Politics coding system.\nAvoid identifying organizations by name only.\nCreate a field on the database that lists your source for the coding.\nAfter coding, sort by industry code and check for mistakes.\n\nIdentifying individuals\n\nPrioritize. Identify top donors first.\nAlways confirm that an individual at two different addresses is the same person. Call them directly if you need to.\nIf your state or locality doesn't require employer or occupation information, you can match the following with your database:\n- lists of professionals regulated by state agencies\n- information from a CD-ROM telephone directory (for example, ProCD)\n- local companies' board of directors and officers\n- local companies' contributors to state and federal PACs\n- your newspaper's press files\nSort your database by address to find individuals who work at a particular organization.\nCreate a field in the database for the initials of the staff person who identified the individual.\nIf you're in a state or locality that mandates employer or occupation information by law, contact the campaign for more information on unidentified donors.\n\nFact checking\n\nCheck your database with the candidates' cover pages. If the totals for a candidate don't match your database, check each contribution individually.\nTreat your database suspiciously before writing your story: sort, sort, and sort some more.\nIf you are using a campaign database provided by the state, first check it with paper records to ensure its accuracy.\n\nFinal note\n\nWhile you should do everything you can to ensure your database's accuracy, don't let the idea of the \"flawless\" database control you or your project.\nDiane Renzulli can be reached at (202) 783-3900 or by email at drenzulli@publicintegriy.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-june-bits-bytes-and-barks","slug":"1998-june-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-june-bits-bytes-and-barks.html","year":1998,"month":"June","monthNum":6,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"A journalism newsletter covering resource availability for investigative reporters, including audio tapes from conferences, campaign finance databases, staff announcements, and a new international investigative reporting award.","keywords":["conferences","database","award"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web","Internet"],"full_text":"Indiana CAR Audio Tapes\n\nAudio tapes from the sessions at Indiana CAR are now available for ordering. For more information or an order form, point your browser to www.ire.org/recource/ricari conferences/indianalaudio.html Audio tapes from the IRE National Conference in New Orleans are also available from Sound Images, which can be reached at (303) 649-1811 or by email at sitapes@aol.com\n\nCampaign Finance Information Center\n\nCampaign finance data from thirteen states is freely downloadable from the CFIC at www.campaignfinance.org. We also have links to twelve online search engines hosted by other non-profits and state boards of election. We will soon release our \"universal\" online search engine so you can type in a contributor from your state and see where else they are giving.\n\nThe more inclusive this database, the better for everyone. So, if you have state or local campaign data you want included, contact CFIC Coordinator Jack Dolan at jack@nicar.org or (573) 884-1802. The CFIC credits everyone who contributes data.\n\nSubscribe Online\n\nYou can now subscribe to Uplink online. Point your browser to www.ire.org/resources/nicarluplink\n\nMoving On\n\nBill Loving recently moved from the Minneapolis Star Tribune to the Los Angeles Times. He can be reached by email at bill.loving@latimes.com. If you have switched locales, please let us know. Contact Brent Johnson at bjohnson@nicar.org.\n\nNew Deputy Director at IRE\n\nLen Bruzzese has joined IRE as its new deputy director. Bruzzese comes to IRE after serving as the editor of The Olympian, an award-winning state capital daily in Olympia, Wash. Bruzzese will oversee IRE's publications, its World Wide Web site, its resource center and the Campaign Finance Information Center. He also will assist in the administration of IRE and its conferences.\n\nInternational Investigative Reporting\n\nA new $20,000 award recognizing the best international investigative reporting in the world has been announced by the Center for Public Integrity. The award, part of the Center's new International Consortium of Investigative Journalists (ICIJ), will go to a journalist or team of journalists who have demonstrated excellence in transnational investigative reporting during the previous year.\n\nENTRIES MUST BE POSTMARKED BY AUG. 15. For more information on the award, including how to apply, visit the ICIJ Web site at www.icij.org or call (202) 783-3900.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-nursing-home-neglect-unraveled-care","slug":"1998-july-nursing-home-neglect-unraveled-care","url":"https://dwillis.github.io/uplink/article/1998-july-nursing-home-neglect-unraveled-care.html","year":1998,"month":"July","monthNum":7,"headline":"NURSING HOME NEGLECT Unraveled care","kicker":"","author_name":"Mark J. Rochester","author_title":"","affiliation":"The Indianapolis Star and The Indianapolis News","summary":"An investigative series uncovered disturbing levels of neglect and regulatory shortcomings in Indianapolis nursing homes, prompting immediate reforms from state officials.","keywords":["nursing","neglect","reform"],"topics":["Investigative Methods","Health & Environment","Government & Politics"],"technologies":[],"full_text":"Normally, a newspaper pursues an investigative story with the hope that circumstances revealed by the probe eventually will change as a result of the public scrutiny brought by the story's publication. When The Indianapolis Star and The Indianapolis News concluded a year-long investigation of nursing homes with publication of a six-day series in late June, we were confident that the disturbing picture of elder care presented would generate an important public policy debate and could possibly lead to changes in the system. What we never expected was that the series would result in significant changes to the system within days of its publication. Or that some of the state's most powerful legislators and governor would demand reforms and an end to the conditions described by the newspapers. But that's exactly what happened immediately following publication of 'The Tarnished Years,' an investigative series in which computer-assisted reporting helped The Star and The News expose the pervasiveness of poor care in area nursing facilities and shortcomings in the state's nursing home regulatory system. The investigation documented disturbing accounts of neglect, broken bones, exploitation and other instances of poor care in Indianapolis area facilities. The series revealed that one-third of area nursing homes allowed harm to befall one or more seniors in their care. And when poor care is detected, the state makes little use of its own penalty system and instead relies on a federal system that gives facilities weeks or months to make corrections before sanctions are imposed. Nearly nine out of 10 nursing homes statewide were not in compliance with federal regulations, the newspapers found. Closing time. The investigation began simply enough. Two veteran journalists, Bonnie Harris and Joe Fahy, were curious about the same thing at the same time. About a year earlier, the state had closed down a nursing home that had established a hideous care record - some residents had developed bedsores so serious that limbs had to be amputated. The reporters wondered about conditions in other nursing homes. The state closing had triggered a statewide look at patterns of neglect and poor enforcement. The newspapers utilized an online searchable database of nursing home inspection data which prompted immediate calls for reform. State officials, within days of the series' publication announced that efforts would be made to address the deficiencies highlighted by the investigation.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1998-july-spreading-hazards-toxic-two-time","slug":"1998-july-spreading-hazards-toxic-two-time","url":"https://dwillis.github.io/uplink/article/1998-july-spreading-hazards-toxic-two-time.html","year":1998,"month":"July","monthNum":7,"headline":"SPREADING HAZARDS Toxic two-time","kicker":"","author_name":"Duff Wilson","author_title":"Investigative Reporter","affiliation":"The Seattle Times","summary":"An investigative journalist describes how digital distribution via email and the Web amplified her Pulitzer Prize finalist series on toxic waste contamination in fertilizer, allowing the story to gain national attention and prompt regulatory reforms despite initial publication in a remote regional newspaper.","keywords":["investigation","toxins","digital"],"topics":["Health & Environment","Investigative Methods","Journalism & Newsroom"],"technologies":["Microsoft Outlook Express 4.0","World Wide Web","Email","spreadsheet"],"full_text":"It was the revolution in e-mail and World Wide Web publishing-more than the newsprint of The Seattle Times or wires of Associated Press that gave sufficient power and reach to my series of articles on toxic wastes to be named a finalist for the 1997 Pulitzer Prize for Public Service.\n\nWhile I wrote \"Fear in the Fields - How hazardous wastes become fertilizer,\" it could easily have been ignored in the rest of America. After all, the story ran in a remote, rainy corner of the country during the July 4th weekend. It lacked the immediate national media follow-up that has attended more obvious environmental problems.\n\nHeavy industries were saving millions by disposing of hazardous wastes as fertilizer instead of putting them in a landfill or recycling them in a safer manner.\n\nMetal smelters, pulp and paper mills, cement kilns and their allied fertilizer companies would advertise the minor plant foods zinc, iron and lime in their wastes. But they wouldn't tell the ultimate buyers about the arsenic, cadmium, lead or dioxins. Nobody was regulating, testing or disclosing the tag-along toxics. Nobody knew what level was safe. Some experts said the metals build up in soil, plants and animals. Industry controlled the research. Industries were shifting the risk from their wastes onto farmers and gardeners. The industries saved $100 per ton. The unknowing farmers might save dimes per acre. When the mayor of Quincy, Wash., raised questions, the local industry tried to force her to shut up.\n\nBeyond newsprint\n\nBut instead of fading in the drizzle or sinking in Puget Sound, the story kept growing, and growing, spreading by bits, bytes and word of mouth.\n\nThanks to e-mail and the Web (and of course my editor and publisher), the story gained national and foreign attention and, finally, prompted significant reforms in laws, public understanding and industry practice.\n\nLike any self-respecting investigative project nowadays, \"Fear in the Fields\" was documented with help from spreadsheets, a database, the Web and e-mail. More importantly, it gained credibility and impact from what would have to be termed computerized, Web-based, informal publishing strategies following the newsprint version of the story.\n\nStarting from the end: Every time I wrote a follow-up article, sent it by e-mail to 100-200 experts and activists on every side of the issue.\n\nI kept a contact list in Microsoft Outlook Express 4.0 Address Book. This is a fully searchable mini-database that some people say is as good as Eudora Pro for e-mail, and it's free. Outlook Express makes it easy to build groups of people in e-mail lists. Often these experts and activists would get the article in e-mail before they could buy it in the newsstand. I asked these people if the articles were completely fair, accurate and complete - a practice I have previously done by telephone. Knowing you're going to initiate contact with your targets and sources with this question does wonders to make you careful to be fair, accurate and complete.\n\nThe cover note on the e-mail would summarize the new article in a sentence, provide a link to the article, ask for feedback and follow-up ideas, and provide links to source materials.\n\nElectronic contacts\n\nThis added credibility. It also meant the insiders' first response came directly back to me. So I knew the buzz inside industry and government. I harvested the follow-up suggestions. I knew whether any questions were being raised and, in one case, corrected a minor error before the final deadline. The Associated Press and Knight Ridder wire picked up on parts of \"Fear in the Fields,\" but this e-mail list extended the immediate distribution to 20 other states and five foreign countries. It pushed the articles to the desktops of scientists, industry officials, environmental activists, trade journals and government policy makers. They could always look for more information on the Web, too. Our site (http://www.seattletimes.com) linked all the articles from a Special Projects area that got more than 1,000 hits a day for weeks. We did not have time to put source documents on our Web site in Adobe Acrobat format. I wish we had. It adds credibility. It adds punch. The Miami Herald's \"Collars for Dollars\" is a model.\n\nPost-publication tips\n\nThese were some other uses of the computer after publication: \"Fear in the Fields\" generated more comments (most positive) than anything else we've done, with more than 200 e-mails, 100 phone calls, and 15,000 reprints. I sent a mass e-mail to about 100 of those people reporting on the actions that followed the articles.\n\nI published tip sheets for journalists in other states who would like to undertake similar investigations: http://www.seanet.com/~duff/ire/handouts.htm\n\nToward the end of my reporting, I used AskSam, a free text database program, to organize more than 50 *.doc files, 20 .txt files and 300 e-mails in my PC. AskSam lets you search for a word or phrase in different types of documents and returns a list of the hits.\n\nHidden in plain sight\n\nBack at the beginning: When I heard the incredible claim that manufacturing industries were disposing of toxic wastes by turning them into fertilizers, the first thing I did was to go online.\n\nI typed \"fertilizer\" and \"toxic\" in Yahoo, Alta Vista and Hotbot search engines. Nothing came up.\n\nLater I found other, probably better search engines - Northern Lights and Inference and other Web sites where those two words were discussed together. I found hazardous waste recyclers, conferences on recycling waste to fertilizer, and even the \"Industrial Materials Exchange\" that serves as a kind of blind-dating service between generators and recyclers. All hidden in plain sight - on the Web.\n\nAnd I discovered Medline (http://www.ncbi.nlm.nih.gov/PubMed/), which gets my vote for the best site on the Web. Thank you, Al Gore, for making this free. Medline lets you search 9 million medical and scientific studies.\n\nOne of the key breaks in my investigation was the discovery of a company, on its Web site, blasting its competitors' fertilizer products for being recycled from hazardous wastes without having cleaned up the toxic lead and cadmium (http://www.cozinco.com/).\n\nAnother break was the discovery of Australian and European Web sites where researchers talk about the risks of cadmium in fertilizer.\n\nAuthority granted\n\nAs the work went on, I kept a spreadsheet of fertilizer test results in Excel, sortable by column headings such as cadmium or manufacturer name.\n\nThis enabled me to say more than 100 fertilizer products whose analytical results were obtained by The Seattle Times higher amounts of lead than the average farm soil.\n\nI could also say, with authority, that recycled hazardous-waste fertilizer products contain higher amounts of lead and cadmium than the other products on the market. I was happy to share this spreadsheet with anyone who wanted it.\n\nI used e-mail to file public record requests, troll for experts in farm-related and medical discussion groups, and correspond with dozens of experts, farm families and fertilizer manufacturers. To some people, e-mail is a less threatening way to answer questions. They are careful in their responses - given a sensitive topic, that is fine with me.\n\nI also grabbed all the e-mail of one state official who was in the center of the practice. I used the state public records law that makes it clear electronic records are to be treated the same as paper records.\n\nPeople like to say computer-assisted reporting doesn't take the place of old-fashioned boot leather. Now isn't that a cowpie? Naturally I pounded a lot of pavement, corridors and dusty farm fields in my months of reporting. But the Internet pulled the whole world to the undersized computer monitor in my overcrowded cubicle in a dark corner of the newsroom.\n\nCome to think of it, maybe I should have spent more time on the farm. Things slow down there, but you get a lot more sunshine and fresh air.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-wilson-writes","slug":"1998-july-wilson-writes","url":"https://dwillis.github.io/uplink/article/1998-july-wilson-writes.html","year":1998,"month":"July","monthNum":7,"headline":"WILSON WRITES","kicker":"","author_name":"Duff Wilson","author_title":"","affiliation":"The Seattle Times","summary":"Wilson highlights resources for accessing toxics information, including the Right-to-Know Network and Environmental Working Group's report on toxics-to-fertilizer practices, while noting the Fertilizer Institute's denial of the problem.","keywords":["toxics","fertilizer","hazardous"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"WILSON WRITES:\n\nTHE RIGHT-TO-KNOW NETWORK (HTTP://WWW.RTK.NET) DOES A GREAT JOB OF MAKING THE TOXICS RELEASE INVENTORY ACCESSIBLE.\n\nTHE ENVIRONMENTAL WORKING GROUP (HTTP://WWW.EWG.ORG) PUBLISHED A HUGE REPORT ON THE TOXICS-TO-FERTILIZER PHENOMENON.\n\nTHE FERTILIZER INSTITUTE (HTTP://WWW.TFI.ORG) IS TRYING TO DENY IT IS A PROBLEM.\n\nI FOUND A CANDID HAZARDOUS WASTE RECYCLER AT HTTP://MEMBERS.AOL.COM FOURN6/INDEX.HTML.\n\nALSO SEE PAGE 15 FOR AN EXPLANATION OF THE RECENTLY UPDATED DEPARTMENT OF TRANSPORTATION HAZARDOUS MATERIALS INCIDENT REPORTING SYSTEM DATABASE, AVAILABLE FROM THE DATABASE LIBRARY.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-booming-or-busting-rising-bankruptcies","slug":"1998-july-booming-or-busting-rising-bankruptcies","url":"https://dwillis.github.io/uplink/article/1998-july-booming-or-busting-rising-bankruptcies.html","year":1998,"month":"July","monthNum":7,"headline":"BOOMING OR BUSTING? Rising bankruptcies","kicker":"","author_name":"Valerie Lilley","author_title":"Reporter","affiliation":"Peoria Journal Star","summary":"A journalism investigation into bankruptcy filings in the Tri-County area revealed that most filers were low-income individuals burdened primarily by credit card debt, not the wealthy as commonly assumed. The reporters built their own database from court records since complete public data wasn't available, uncovering surprising creditor patterns like Fingerhut and department store charges.","keywords":["bankruptcy","credit","poverty"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Excel","FoxPro","PACER"],"full_text":"The economy is booming and so are bankruptcy filings. What gives?\n\nIn a week-long series, we told readers everything about the Tri-County area's bankruptcy filings. Through computer-assisted reporting, we used numbers to show who the filers were, as well as the causes, losers and winners in the surge of filings.\n\nSurprisingly, we found that most people who filed had been living below or near the poverty level. They weren't the rich - a popular misconception.\n\nThe oddest fact we found was that many filers listed the mail-order company Fingerhut, which made it to the top of the list of creditors. Bills were run up using cellular phones and charge cards at Bergners, a local, upscale department store.\n\nCredit cards were the main cause of the bankruptcies. Medical debt trailed far behind.\n\nIncomplete data\n\nTo gather these statistics, we had to build our own database because we couldn't find an existing database with complete information.\n\nThe U.S. Bankruptcy courthouse had PACER (Public Access to Court Electronic Records), but it didn't list all creditors. The Administrator of the Courts in Washington, D.C., didn't have a complete listing of case contents, either. The courthouse clerks and office manager didn't know of anything else.\n\nWe decided to collect the data ourselves. This was our first big dive into computer-assisted reporting, and naiveté drove it from a mini-project of a few months to a year-long study. Collecting the data took the most time in preparing the project.\n\nThe first lesson we learned is that mining data is not as easy as it looks. From 2,906 cases, we gleaned data such as: filers' names; ZIP codes; residential market value and money owed on it; car market value and money owed; attorneys; assets and liabilities; government support type and income; child support income and expense; income figures from the two years prior to filing up to the present; monthly expenses; and creditors and amounts owed them. (Some cases listed more than 200 creditors.)\n\nInitially, another reporter and I went to the court house armed with laptops. For a week, case by case, we inputted the data in comma-delimited format. Both of us made multiple mistakes. By the end of the week, we only had 90 cases in our database. We ended up hiring someone to input for us.\n\nBecause of the time cases can take to close, we ended up using cases opened and closed between January, 1996, and February 1997.\n\nTo check for errors, we ranked by case number and by the minimum and maximum amounts in each column. The cleanliness of the database was also double-checked by a professor at Illinois Central College.\n\nWe used Excel to find the averages, sums, minimums, maximums, and the time each case took from open to close. To find how many filers owned a house, we ranked the database by the \"residence market value\" field and found where the cells dropped off to zero. Only 753 people owned a home. Its average value was $41,701.\n\nTagging along\n\nWe also used Visual FoxPro to find the top creditors, total credit card debt, total unsecured debt and total medical debt. To differentiate between credit cards and regular bank loans, identifiable credit cards were tagged with a \"-C.\" We put a \"-M\" after doctor's names, physician offices, clinics, hospitals and other medicine-related creditors. We also wanted to know how much wasn't going to get paid back - total unsecured debt, which we clearly listed on a separate schedule in the files and tagged with \"-U.\"\n\nTo find the top creditors, I cut and pasted the creditor columns and the amounts into one super-long, two-column spreadsheet. To double check that I had gotten all of the creditors and amounts in the cut and paste process, I compared the sum to the sum in the original Excel spreadsheet. I imported the two-column spreadsheet into FoxPro and then used the \"order by\" and \"group by\" commands.\n\nI used the same table to select the creditors and the amounts where the creditor had the \"-C\" for credit card. The \"order by\" and \"group by\" gave the top ten. I repeated this to find top unsecured creditors and medical debt. The hardest part was finding out the average credit card debts per case because the \"-C\" tags were included with any \"-U\" and \"-M\" tags. I didn't know how to write a loop or what a scan/endscan was. Justin Mayo of NICAR worked out a program for us using a table with the creditors in the original block form.\n\nHandle results with care\n\nWe approached the data results in the same manner as a press release. I put the results in sentence format - it was as if we were reading the traditional government and commercial studies we were used to. During weekly meetings, people threw out questions like \"How many filers live with the city limits?\" and \"What's the median debt per case?\" We made cheat sheets and used them as a starting point for reporting.\n\nFor example, Sears and CEFCU, a local credit union, were owed the most money - $2.9 million and $2.7 million respectively. Central Illinois Light Co. and Ameritech were also up there. I interviewed company executives and other spokespeople. They provided the story behind the numbers.\n\nFinding real voices\n\nWe talked to the people who filed. They were hard to find because most led transient lifestyles. We used the Journal Star's electronic library to get their personal information - we publish bankruptcies in \"Matter of Record\", along with divorces, marriages, births. Getting people to talk about their filing was our biggest problem. Some didn't want to rehash old, sour memories. Others didn't want their names published in the newspaper again. So we ended up with a mix of full names and first names only. Organizing everything for our readers also challenged us. We had so much data that we could have easily bombarded them with stat-packed stories, covering a broad range. In our numerous weekly meetings, however, we had isolated specific elements of the bankruptcy phenomenon. These elements became the stories and sidebars of our week-long series.\n\nValerie Lilley can be reached by e-mail at vlilley@pjstar.com or by phone at (309) 686-3260.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-the-week-long-bankruptcy-series-focused-on-the-following-questions","slug":"1998-july-the-week-long-bankruptcy-series-focused-on-the-following-questions","url":"https://dwillis.github.io/uplink/article/1998-july-the-week-long-bankruptcy-series-focused-on-the-following-questions.html","year":1998,"month":"July","monthNum":7,"headline":"THE WEEK-LONG BANKRUPTCY SERIES FOCUSED ON THE FOLLOWING QUESTIONS","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A week-long series examining bankruptcy in the Peoria area, exploring its causes, effects, contradictions, and potential solutions while analyzing how bankruptcy impacts various stakeholders.","keywords":["bankruptcy","credit","Peoria"],"topics":["Business & Economy","Government & Politics","Investigative Methods"],"technologies":[],"full_text":"THE WEEK-LONG BANKRUPTCY SERIES FOCUSED ON THE FOLLOWING QUESTIONS:\n\nWHAT ARE THE ELEMENTS OF THE BANKRUPTCY DILEMMA IN THE PEORIA AREA?\nWHAT IS BANKRUPTCY? IT IS FULL OF CONTRADICTIONS.\nWHAT'S THE DEBATE? CONGRESS IS GRAPPLING WITH THE TWO SIDES OF BANKRUPTCY.\nWHAT CAUSES PEORIA BANKRUPTCIES? CREDIT CARDS GET MUCH OF THE BLAME.\nWHO LOSES IN BANKRUPTCIES? WE ALL DO.\nWHO WINS? THE BUSINESS OF BANKRUPTCY IS BOOMING TOO.\nWHAT CAN BE DONE? A LOOK AT THE POSSIBILITIES.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-on-the-internet-estimating-relevancy","slug":"1998-july-on-the-internet-estimating-relevancy","url":"https://dwillis.github.io/uplink/article/1998-july-on-the-internet-estimating-relevancy.html","year":1998,"month":"July","monthNum":7,"headline":"ON THE INTERNET Estimating relevancy","kicker":"","author_name":"Robert Berkman","author_title":"Information Advisor","affiliation":"The Information Advisor","summary":"This article explains how to conduct more effective Web searches by using field-specific searches, understanding record structures, and modifying search strategies when initial results are unsatisfactory or irrelevant.","keywords":["search","keywords","relevancy"],"topics":["Technology","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"ON THE INTERNET\n\nEstimating relevancy\n\nBy Robert Berkman\nThe Information Advisor\n\nIn the information world, units of related information are called \"records.\" A record can be an article, an abstract of an article, a bibliographic citation, a Web page, and so on. A \"field\" is a specified portion of a record. For example, for Web pages, fields may include the Web page's title; the date that Web page was last modified; the Web page's URL; or the Web page's meta tags. Meta tags are the textual description — though invisible to the Web page, written by its creator to inform search engines what the page is about. Some search engines allow you to instruct the engine to look for your keywords only in a specified field. When would you want to do this kind of limited field search? If you did a keyword for, say, \"Vietnam,\" you'd be swamped with sites. By restricting your search to the title field, you'd help insure that the keyword is the primary focus of the Web pages returned.\n\nIf you want to search for recent events, some search engines, such as Alta Vista, let you search for a date field. You can specify that you only want recently updated Web pages.\n\nSometimes you may only want to see Web sites produced by a specific organization or type of institution. In these cases, you can search the URL field.\n\nSay, for instance, that you're doing a story on Sony, and you want to locate Web pages produced by the company. Zero in on Sony's own sites by searching for \"Sony\" only in the URL, which normally contains the name of the institution that created the page.\n\nAnother example is when you want to see sites only from a particular type of institution. If you were looking for statistics on aircraft deaths but only wanted data from official governmental sites, you could instruct the search engine to return only sites that end in \".gov\".\n\nKnow when to modify\n\nAs any librarian knows, getting good results from an online search often means tinkering with the initial search statement. If an initial search turns up no Web pages, too many, or irrelevant ones, it's a good idea to examine possibly alter that keyword search statement. Try the following:\n\nCheck the spelling. If you are searching unfamiliar territory, make sure the words you've used are standard terminology. For example, if you were searching for Web pages on \"static electricity,\" it would be important to know that the technical term for that phenomenon is \"electrostatics.\"\n\nDouble-check your syntax if you used Boolean operators. If you didn't get enough hits, take out any AND operators you entered. If you got too many hits, either add an AND operator to restrict your search, eliminate one or more keywords, or limit your keywords to a title field.\n\nOne-way bridges\n\nA caution: even if you've done a perfectly good search, you still can't assume the Web pages that the search engines returned are really the most relevant. Why? Some site creators are manipulating their pages and the keywords on their sites with the specific goal of getting high rankings by the search engines.\n\nAccording to Danny Sullivan, who publishes an e-mail newsletter called Search Engine Watch (http://www.searchenginewatch.com), some sites do this by developing what are called \"bridge\" pages. A bridge page is created specifically to receive a high ranking from a search engine. When you click on the decoy-like page, you are routed to the site's actual home page.\n\nIn his newsletter, Sullivan found, for example, that State Farm Insurance submitted several pages to a variety of search engines. Each page was different and specifically written to rank well in search engines' ranking algorithms. Several pages were submitted to each search engine and written to \"capture\" a particular insurance-related keyword like auto insurance or boat insurance.\n\nCyber consultants\n\nWeb site owners also are contracting the services of consulting firms that claim they can help the site get higher rankings.\n\nThese companies take their clients' Web pages, run them against the major search engines, analyze placement of the page, and then provide specific strategies on how to improve rankings. One of these firms, called Webposition (http://www.webposition.com), also provides free advice on its site on how to improve rankings on a search engine.\n\nWeb illusions\n\nThings are not, then, always what they appear to be in the brave new world of the Web. On a traditional online service, like Dialog, you could assume that if you conducted a good search, you'd pretty much get what you expect. But on the Web, it appears the data creator's intentions are another force making an impact on what you'll retrieve.\n\nAlthough popular search engines like to claim that their robots scour the entire Web, the fact of the matter is no search engine can actually index all the pages out there. The April 2 issue of Science magazine reported on an extensive study by Drs. Steve Lawrence and C. Lee Giles of the NEC Research Institute. The researchers posed over 550 scientific search questions to the five largest search engines and discovered that there are about 320 million Web pages — but that search engines index only about 40% of them.\n\nInformation overload\n\nIf you feel overwhelmed by the enormous number of hits you get back from Alta Vista or one of the other well-known search engines, try a newer search engine called Northern Light (http://www.nlsearch.com). Northern Light does an admirable job in making a dent in information overload by grouping the sites it retrieves into logical folders, which contain related information.\n\nRobert Berkman can be reached at (413) 549-9924 or by e-mail at rberkman@javanet.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-berkman-writes-flooding-engines","slug":"1998-july-berkman-writes-flooding-engines","url":"https://dwillis.github.io/uplink/article/1998-july-berkman-writes-flooding-engines.html","year":1998,"month":"July","monthNum":7,"headline":"BERKMAN WRITES: Flooding engines","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article explains two main categories of web research tools: raters and robots, and describes how hierarchical indexes like Yahoo and LookSmart allow users to search by drilling down through progressively narrower categories.","keywords":["search","hierarchy","categories"],"topics":["Technology"],"technologies":["World Wide Web","Internet"],"full_text":"ON THE INTERNET, THERE ARE TWO MAJOR CATEGORIES OF WEB RESEARCH TOOLS: RATERS AND ROBOTS. ON YAHOO, THE MOST WELL-KNOWN OF THE HIERARCHICAL INDEXES, SEARCHERS LOOK FOR SITES BY \"DRILLING DOWN\" - BY FIRST SELECTING A BROAD CATEGORY, VIEWING THE NARROWER SUBCATEGORIES, THEN CHOOSING UNTIL THE DESIRED SITES ARE LOCATED.\n\nANOTHER, NEWER SITE THAT WORKS THE SAME WAY IS CALLED LOOKSMART (HTTP://WWW.LOOKSMART.COM).","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-stats-from-the-road-sampling-not-a-dirty-word","slug":"1998-july-stats-from-the-road-sampling-not-a-dirty-word","url":"https://dwillis.github.io/uplink/article/1998-july-stats-from-the-road-sampling-not-a-dirty-word.html","year":1998,"month":"July","monthNum":7,"headline":"STATS FROM THE ROAD Sampling: Not a dirty word","kicker":"","author_name":"Sarah Cohen","author_title":"","affiliation":"NICAR","summary":"Journalists investigating health care quality can use sampling techniques to efficiently explore large databases and paper records before committing to full data collection, testing whether records contain usable information and identifying what details are worth pursuing.","keywords":["sampling","database","records"],"topics":["Investigative Methods","Data & Databases"],"technologies":[],"full_text":"In a project on the quality of health care, reporters were interested in the details of malpractice claims. There were something like 50,000 payments to victims of doctors, hospitals and other health care providers in a state database. However, the computer records included only summary information. Paper records were the source for more details, such as the nature of the injury and how it happened.\n\nThis problem often arises when working with documents, especially court records. You can find docket information on bulletin boards or other electronic means. But details ranging from previous criminal records to prosecutor recommendations simply don't exist electronically.\n\nThese are perfect candidates for sampling in the early stages of a story.\n\nSampling situations\n\nMost editors and reporters would insist that when the story runs it ought to include all instances. They argue it simply worth the effort to gain more credibility with readers and viewers. But you can sample early on to find out how many and what kinds of records will - or more importantly won't your story.\n\nYou may even find early in a story's planning that the effort probably won't result in a credible The records are too muddled to be of much help.\n\nLarge databases are another candidate for sampling in the newsroom. I think most computer-assisted reporting experts would acknowledge that they've at least once waited hours for a query to run - sometimes overnight - only to find that they left out a field or forgot to include an important case.\n\nHere are some considerations to think about when you begin a project and want to consider sampling as a way to make your work more efficient. Some are geared at figuring out what you need to know before you move further into the deep, dark hole of a long-term story. Others help you figure out whether you need scientific sampling techniques.\n\nCollection difficulty\n\nSometimes, sampling has little place simply because it doesn't save you much. Collecting all of the records on OSHA violations or fatal accidents, for instance, is probably easier than sampling. There's no reason to revert to a small set of records when you can just as easily test all of them. But if you simply can't get all of the records, such as the details of thousands of court cases, then you'll need scientific sampling methods before you can say anything about what's in them as a group.\n\nPassing acquaintance\n\nWhen you're dealing with documents you've never worked with before, you'll probably want to start out with a small group of them to test your database. I usually don't recommend bothering with scientific sampling techniques here. Instead, consider typing in the 50 most recent and 50 from the earliest period you're considering. This will give you three legs up on the project.\n\nFirst, you'll see whether you have the skill (or whether it's even possible) to create queries or other analyses that will address your questions from the database the way you've designed it. Too often, reporters call after spending months typing in records without making this simple test. It makes their work much harder.\n\nAnother advantage is that you'll see how the paper records have changed over time. If you're compiling records from many local offices, though, you'll need to take another step: gathering 10 or so from a few different far-flung districts. This will show you how they vary in quality and in content from place to place, as well as over time.\n\nFinally, you'll know what difficulties you will find. In the malpractice example, for instance, reporters who had stopped early in the project and looked at the most recent records would have noticed the fatal flaw. Malpractice lawsuits take years to settle. Even the most recent records were too old to be considered \"news.\" They could have stopped before they even started.\n\nGetting comfortable\n\nHow comfortable are you with the analytic techniques you plan to use? We've all had the experience of exploring a new data set, only to wait minutes, hours or even overnight for queries that didn't show us anything of value. Consider testing your queries and analysis on a small set of records.\n\nIn this case, you'll usually want something at least moderately scientific. The reason is that you want some hint of what you'll find and, simultaneously, a list of possible problems to check. Do different data entry clerks (or inspectors) enter the data differently? Are fields you care about usually filled out? And what is the scale of the numbers you're expecting to find: one out of a million, or one out of a hundred?\n\nWhat could compare?\n\nDo you have any basis for comparison? Make sure, early on, that you've thought through a few possible leads for your story. It doesn't matter what your suspicions are, just how you'd like to word them if they prove true. Look for words like \"more than\" or \"compared with\" in that short statement. For example, imagine you're examining trends in people who return to prison over and over again. You want to know if they're more likely to test positive for drugs, be in a gang, or be of a certain ethnic group than other felons.\n\nThink carefully about the words \"more likely\" in that sentence. The biggest mistake in building databases - whether they're sampled or not - is ignoring this crucial issue for the paper. It implies that you need to test not only the group you care about but everyone else as well. Make sure, early on, that you could write this sentence based on the records you plan to acquire or the slice of a database you plan to focus on. Sampling will help when you know you can't write the sentence based on your test.\n\nOnce you've decided to sample, the next step involves figuring out how to sample a set of data. We'll tackle that topic next month.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-upcoming-boot-camps","slug":"1998-july-upcoming-boot-camps","url":"https://dwillis.github.io/uplink/article/1998-july-upcoming-boot-camps.html","year":1998,"month":"July","monthNum":7,"headline":"UPCOMING BOOT CAMPS","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering upcoming boot camp training programs in Columbia, Missouri in fall 1998 and spring 1999, with both condensed and full-length options available for investigative journalists.","keywords":["training","journalism","boot camp"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"UPCOMING BOOT CAMPS:\n\nOCTOBER 22-25, 1998 - POWER BOOT CAMP IN COLUMBIA, Mo. (A BASIC BOOT CAMP SQUEEZING SIX DAYS OF TRAINING INTO FOUR.)\n\nJANUARY 3-8, 1999 - BASIC BOOT CAMP IN COLUMBIA, Mo.\n\nMAY 16-21, 1999 - BASIC BOOT CAMP IN COLUMBIA, Mo.\n\nFOR A COMPLETE LISTING OF UPCOMING IRE AND NICAR EVENTS, INCLUDING BOOT CAMPS, ON-THE-ROAD TRAINING, SEMINARS, AND REGIONAL CONFERENCES, VISIT OUR WEB SITE AT WWW.IRE.ORG/CALENDAR.HTML","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-first-ventures-clogged-artery","slug":"1998-july-first-ventures-clogged-artery","url":"https://dwillis.github.io/uplink/article/1998-july-first-ventures-clogged-artery.html","year":1998,"month":"July","monthNum":7,"headline":"FIRST VENTURES Clogged artery","kicker":"","author_name":"John Kelly","author_title":"","affiliation":"The Shelbyville News","summary":"A small-town newspaper in Indiana used computer-assisted reporting to analyze traffic crash data and uncover safety issues on a busy road, revealing how poor planning and excessive driveways contributed to accidents.","keywords":["database","traffic","reporting"],"topics":["Investigative Methods","Public Safety","Technology"],"technologies":["9-track tape","spreadsheet","Internet"],"full_text":"You'd panic, too, if you had never seen nine-track tapes. I didn't let on to the clerk that I had no earthly idea how to get what I wanted from them. I had hired someone to take data from the tapes if necessary. It was necessary. I paid the Indiana state police about $150 for the tapes, which held three years of police reports for every traffic crash in the state, as well as my chance to do our first major computer-assisted reporting project.\n\nThe Shelbyville News already had dipped its toes into the CAR pool. We use spreadsheets to analyze government spending. We use the Internet daily. We built small databases for a feature on local inventors and to tabulate a survey about community parks.\n\nBut we had never tangled with a 500MB relational database. It was overwhelmingly scary at first, but the computer work turned out to be much easier than I ever dreamed.\n\nHead-first plunge\n\nSo we dove into the CAR deep end and gave our readers something special. We delved into every aspect of traffic crashes in Shelby County, empowering readers with information they could use to influence public officials and take action themselves to make the streets safer.\n\nWe didn't stop with ranking roads where people crashed most. We used the database to identify where crashes happened, what caused them and how they happened. Reporters talked to countless cops, business owners, road engineers, planners, safety experts and drivers.\n\nWe didn't send politicians running. We didn't stun the public. We did what we always do. We took what we discovered and developed it into stories that showed readers what's happening, why it's happening and how problems can be fixed.\n\nAccident prone\n\nWe showed the public how the city and state governments made no improvements as East State Road 44 - once a country road lined with corn fields and a drive-in theater developed over three decades into a busy business corridor. The database helped us show readers there were more accidents on three miles of that road than on any other road in Shelby County. Our reporting helped us find reasons and solutions.\n\nCity and state officials told us heavy traffic was to blame. Maybe so, but there was something else, according to police, business owners and drivers. Bad planning and a lack of communication between city and state agencies aggravated the problem, they said. Computer analysis and further reporting showed they were right.\n\nDriveway dilemma\n\nPolice, business owners and drivers told us the biggest problem was drivers crossing several lanes of speeding traffic to get in and out of strip-center parking lots. The road is packed with driveways. The state handed out permits for driveways like bank tellers hand out lollipops. In fact, they let new businesses cut more and more driveways until there were places where six driveways sat within a few hundred feet of each other - despite state safety guidelines saying driveways should be at least 150 feet apart.\n\nThe database showed us most of the accidents were caused by cars slowing or stopped to turn into the lots or by cars turning onto the highway from them. Transportation department officials said statewide there was too little money to fix too many roads. The engineer responsible for approving driveway permits told us that Shelby County residents needed to improve their driving skills to adapt to changing traffic conditions.\n\nThe road affected everyone in our community. In this small Hoosier city, this was a huge story. To buy groceries, grab a fast-food lunch or go shopping, readers must drive that road. They knew there was a problem. They had theories about what was wrong. Still, most were calling for cliché solutions like more traffic lights, lower speed limits or a center turn lane.\n\nDriver education\n\nWe could have written those things, but the database gave us a chance to show other reasons behind the problem. After reading our report, residents and public officials alike agreed the issue was planning - or lack of it.\n\nChange of plans\n\nNow city zoning officials are talking about access roads and asking about the impact of new developments on traffic. They are redesigning one intersection before a developer builds a new shopping center and housing addition there. They are pressing the state for input in planning and driveway permits.\n\nWhether the reports will affect state planning is unclear. The state's cooperation with the city on redesigning the one intersection could be a sign of progress. One transportation department official has said the district office pinned our stories on a bulletin board. We haven't found out yet whether they're learning from it or tossing darts at it.\n\nThe stories about East State Road 44 anchored the series, but they were just two of a dozen. In other reports, we showed readers how Indiana law allowed teen-agers to get driver's licenses with little behind-the-wheel experience and how teen-agers crashed much more often than any other age group. In addition, we showed how drivers of all ages were their own worst enemies on the road. Three-fourths of all crashes were caused by the fault of a driver, including half because at least one of the drivers was not paying attention.\n\nSmall paper, big project\n\nReaders loved it. The series was comprehensive, insightful, sophisticated and fun. In newspaper terms, it was good journalism. Now, we're ready to do more of it.\n\nWe learned a great deal from this project. We learned and hopefully showed others that even the smallest newspapers can do big CAR projects. We learned how to do CAR right. Our next projects will be faster and better because of the experience.\n\nWe learned what everyone doing CAR at whatever level - should remember: don't be afraid to ask for help when it's needed. IRE members offered guidance and warnings when I wrote them. Richard Mullins and the NICAR staff taught me how to clean the database at an intermediate boot camp. A data recovery company helped me move the state police data from nine-track tapes onto a CD-ROM, then turn a flat file with seven different types of records into seven comma-delimited files so I could import the tables into Microsoft Access.\n\nBecause we asked for help, we were able to overcome what could have been project-ending (and CAR-ending) obstacles for our small newsroom. Instead, we've given the paper's leaders a taste of what CAR can mean for reporting, and now we'll get more time for CAR projects.\n\nWhile I did the database work, everyone in our newsroom helped. That is critical for small papers. Two of us did the bulk of the reporting, but every reporter conducted interviews for the stories. Our editors, graphic artist and photographer put together a terrific graphic presentation so writers could leave some numbers out of stories while still giving readers gobs of stats. Copy editors ravaged the early drafts until we got it right. And our publisher laid the groundwork by investing in CAR training for me and CAR resources for our newsroom.\n\nOur next major projects (already in the works) will be more ambitious and hard-hitting.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-handout-of-the-month-spreading-car-at-small-news-organizations","slug":"1998-july-handout-of-the-month-spreading-car-at-small-news-organizations","url":"https://dwillis.github.io/uplink/article/1998-july-handout-of-the-month-spreading-car-at-small-news-organizations.html","year":1998,"month":"July","monthNum":7,"headline":"HANDOUT OF THE MONTH Spreading CAR at small news organizations","kicker":"","author_name":"David Cuillier","author_title":"Reporter","affiliation":"IRE and NICAR","summary":"A guide to promoting computer-assisted reporting (CAR) skills at small news organizations through two strategies: making CAR tools widely available and accessible to all staff, and providing focused training to interested reporters who show potential.","keywords":["training","newsroom","technology"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","CD-ROM","9-track tape"],"full_text":"Spreading CAR at small news organizations\n\nThese handouts are excerpts from handouts provided at IndyCAR and the IRE National Conference in New Orleans.\n\nDavid Cuillier attended the May 1997 Advanced Boot Camp. So you are the only one at your small news organization who is using computer-assisted reporting skills, and you want to spread the knowledge. Most everyone seems interested, but few people actually do anything. And those who do take the initiative to learn often end up leaving for bigger ponds. Many of us have been fishing for a way to lure people into learning computer-assisted reporting at small news organizations. Here's a two-pronged approach to tackling it: Gillnetting along with precision casting.\n\nGill-netting\n\nLike fishing boats that cast their nets wide, try making basic computer research tools available and easy to use for everyone, even those who don't seem to have any interest. You may find that even the most reluctant reporters will start to use the Internet, online services or CD-ROMs. Here are some ideas:\n\nLocation, location, location: If you have a designated CAR computer, put it in a central location in the newsroom on its own desk. Make it clear the computer is available for everyone to use - that it's not the domain of one person or department.\nBook smart: Set up a book shelf at the CAR station, filling it with self-help software manuals, back issues of Uplink and useful texts geared toward computer-assisted reporting.\nHot off the press: Produce a monthly in-house CAR newsletter. Give praise for stories that benefited from CAR, especially deadline work. Also, write about what stories are in the works, and offer ideas for CAR stories that might interest reporters. Make sure to route copies to the editor, publisher, advertising director and other company managers. It will help build support for CAR work, future equipment purchases and interdepartmental computer trades.\nGet linked: Subscribe to the monthly newsletter Uplink, put out by the National Institute for Computer-Assisted Reporting. Show beat reporters how they don't have to be CAR gurus to benefit from a subscription.\nSalt that data: Set up brown-bag lunches to show folks some data you acquired, how you crunched it and what you got out of it. Or show useful sites on the Internet or how spreadsheets can help look at numbers.\nNoteworthy: Type up cheat sheets for the computer resources. Laminate them or put them in plastic covers. Give each staff member a copy and put them in a binder at the CAR station.\nGroup acquisition: When you run across an interesting database, get it, even not on your beat. Give it to the appropriate reporter with some ideas for what the data could reveal. If you're an editor, seek out data for every beat to pique the interest of all the reporters.\n\nPrecision casting\n\nSome reporters become interested in learning advanced skills, and with a little focused help they may progress fast. If it's unlikely the bosses will send them to training boot camps or classes, the job may rest on you. Seize the interest fast.\n\nYou probably don't have a lot of time, so focus your energies on one person at a time. If you give them heavy training over three months with follow-up sessions, then you could train three or four people a year. For most small news organizations, that is exceptional. Even one or two a year would be great.\n\nAt least once a week, find a good time for both of you. Stick to it.\n\nUse real data. Start small and work up. Let them do the work and go to you only for questions when they get stuck or need guidance.\n\nIf you attend CAR seminars, boot camps or conferences, take notes on how they teach the sessions. Take home the exercises and try them out on your \"students.\"\n\nMaintaining CAR on duct-tape budgets\n\nThe following are big resources for small newsrooms to overcome snags and boost skills.\n\nIn-house resources:\n\nTechies: Computer services staff at your organization can help overcome problems or handle nine-track tapes. Bring them donuts.\nBean counters: Accountants on your business department are probably proficient with spreadsheets. Also, they might have a spare copy of Excel hanging around.\nMarketing: Marketing/advertising might use mapping software for developing demographic reports for readership studies and advertisers. MapInfo is popular in the business world but can work in newsrooms. ArcView is what government tends to use.\n\nOut-house resources:\n\nPros: Local computer experts are sometimes willing to help program or crunch data, often for free. Or consider giving them credit in the story.\nUniversities: Take courses at universities or community colleges. Not only do you learn, but as a student you can get substantial discounts on software.\n\nOnline guides\n\nHere are some places online to learn about CAR, use the Internet and get story ideas:\n\nCAR guides: http://www.poynter.org/carl chome.htm. A guide tapping into online information by Nora Paul of the Poynter Institute. Includes introduction into CAR and explanations of resources.\nDeadline Online: http://www.deadlineonline.com. A great breakdown for finding facts, people and sources. By Alan Schlein, who also provides specialized Internet training for journalists.\nReporter's Desktop: http://www.seanet.com/~duff1. Compilation of search engines, people finders and other resources for finding information on deadline. By Duff Wilson, investigative reporter at The Seattle Times.\nCAR/CARR: http://www.ryerson.ca/~dtudor/carcarr.htm. A list of links to CAR sites and other useful places to check out. By Dean Tudor, journalism professor at Ryerson Polytechnic University in Toronto, Canada.\nThe Beat Page: http://reporter.org/beat/ Internet links and CAR resources grouped by beat, from agriculture to transportation. Compiled by Shawn McIntosh of The Clarion-Ledger in Jackson, Miss.\nWWW Virtual Library: http://www.cais.com/makulow/jexercises.html. Many CAR resources by John Makulowich, including Internet exercises for journalists, Internet tips and links to journalism organization. Hosted by The Writers Alliance Inc.\nCARwash: http://www.wnpa.com/carwash/carhome.html. CAR stories done in Washington state, primarily by small newspapers, posted in full text. By David Cuillier, Tri-City Herald.\n\nDavid Cuillier can be reached at (509) 582-1535 or by e-mail at dcuillier@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-maintaining-car-on-duct-tape-budgets","slug":"1998-july-maintaining-car-on-duct-tape-budgets","url":"https://dwillis.github.io/uplink/article/1998-july-maintaining-car-on-duct-tape-budgets.html","year":1998,"month":"July","monthNum":7,"headline":"Maintaining CAR on duct-tape budgets","kicker":"","author_name":"David Cuillier","author_title":"Journalist","affiliation":"Tri-City Herald","summary":"Small newsrooms can access Computer Assisted Reporting (CAR) resources through in-house staff, local professionals, universities, and online guides to improve their reporting capabilities on limited budgets.","keywords":["reporting","resources","training"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Excel","MapInfo","ArcView","9-track tape","Internet","World Wide Web"],"full_text":"Maintaining CAR on duct-tape budgets\n\nThe following are big resources for small newsrooms to overcome snags and boost skills.\n\nIn-house resources:\n\nTechies: Computer services staff at your organization can help overcome problems or handle nine-track tapes. Bring them donuts.\nBean counters: Accountants on your business department are probably proficient with spreadsheets. Also, they might have a spare copy of Excel hanging around.\nMarketing: Marketing/advertising might use mapping software for developing demographic reports for readership studies and advertisers. MapInfo is popular in the business world but can work in newsrooms. ArcView is what government tends to use.\n\nOut-house resources:\n\nPros: Local computer experts are sometimes willing to help program or crunch data, often for free. Or consider giving them credit in the story.\nUniversities: Take courses at universities or community colleges. Not only do you learn, but as a student you can get substantial discounts on software.\n\nOnline guides\n\nHere are some places online to learn about CAR, use the Internet and get story ideas:\n\nCARguides: http://www.poynter.org/carlchome.htm. A guide tapping into online information by Nora Paul of the Poynter Institute. Includes introduction into CAR and explanations of resources.\nDeadline Online: http://www.deadlineonline.com. A great breakdown for finding facts, people and sources. By Alan Schlein, who also provides specialized Internet training for journalists.\nReporter's Desktop: http://www.seanet.com/-duffl. Compilation of search engines, people finders and other resources for finding information on deadline. By Duff Wilson, investigative reporter at The Seattle Times.\nCAR/CARR: http://www.ryerson.ca/-dtudor/carcarr.htm. A list of links to CAR sites and other useful places to check out. By Dean Tudor, journalism professor at Ryerson Polytechnic University in Toronto, Canada.\nThe Beat Page: http://reporter.org/beat/ Internet links and CAR resources grouped by beat, from agriculture to transportation. Complied by Shawn McIntosh of The Clarion-Ledger in Jackson, Miss.\nWWW Virtual Library: http://www.cais.com/makulow/jexercises.html. Many CAR resources by John Makulowich, including Internet exercises for journalists, Internet tips and links to journalism organization. Hosted by The Writers Alliance Inc.\nCARwash: http://www.wnpa.com/carwash/carhome.html. CAR stories done in Washington state, primarily by small newspapers, posted in full text. By David Cuillier, Tri-City Herald. David Cuillier can be reached at (509) 582-1535 or by e-mail at dcuillier@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-chasing-the-big-story","slug":"1998-july-chasing-the-big-story","url":"https://dwillis.github.io/uplink/article/1998-july-chasing-the-big-story.html","year":1998,"month":"July","monthNum":7,"headline":"Chasing the big story","kicker":"","author_name":"Hector San Miguel","author_title":"","affiliation":"Lake Charles American Press","summary":"A curated list of useful websites for journalists to improve their reporting, including databases, archives, and reference resources covering public records, newspapers, and consumer information.","keywords":["research","databases","journalism"],"topics":["Investigative Methods","Technology","Business & Economy"],"technologies":["Internet","World Wide Web"],"full_text":"The following Web sites will make your reporting job easier.\n\nPublic Integrity Research Corporation: www.pihome.com/pirc/pirc11.html. What a site. It contains great links for general reference, fact searches, statistics, phone directories, search engines, government databases, miscellaneous references, statutes text, and business reference.\n\nThe Internet Sleuth: www.isleuth.com/arts.html. Web sites don't get any cooler than this one. It contains more than 3,000 databases you can access.\n\nNewspapers Online: www.newspapers.com. This amazing site gives you access to U.S. and foreign newspapers and other publications on the Web. Many of the newspapers online feature searchable archives.\n\nConsumerworld: www.consumerworld.org. This is a great \"public service site\" that has access to more than 1,700 consumer resources on the Internet. It has everything from product reviews to consumer rights booklets. This is a definite bookmark for the business reporter.\n\nHector San Miguel can be reached at (318) 494-4084.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-tech-tip-tell-me-y2k-you-cry","slug":"1998-july-tech-tip-tell-me-y2k-you-cry","url":"https://dwillis.github.io/uplink/article/1998-july-tech-tip-tell-me-y2k-you-cry.html","year":1998,"month":"July","monthNum":7,"headline":"TECH TIP Tell me Y2K you cry","kicker":"","author_name":"Richard Mullins","author_title":"Data specialist","affiliation":"NICAR/Missouri School of Journalism","summary":"The article explains how Access and FoxPro handle Year 2000 date conversion, detailing their different assumptions for two-digit years and recommending data integrity checks when working with datasets containing ambiguous date formats.","keywords":["dates","conversion","century"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["Access","FoxPro"],"full_text":"Tell me Y2K you cry\n\nBy Richard Mullins\nNICAR/Missouri School of Journalism\n\nAccess and FoxPro don't have a year 2000 problem - they can handle dates from 100 A.D. up to 9999 A.D. That should handle the worst jail terms and back-dated purchase orders.\n\nSo the real Year 2K problem is yours. You need to understand a little of how Access and FoxPro store dates and know the assumptions that Access and FoxPro use for implying the century when you only enter or import two digits for the calendar year.\n\nFoxPro and Access do not store any dates with only two digits for the year, leaving the century ambiguous or unknown. Both of them, however, give you various ways to control the display format of dates, including whether the year has two or four digits.\n\nLet's deal with FoxPro first. You can toggle the display of the century with the SET CENTURY command. You can test this feature in the command window to see how it works. The lines in bold are the responses to the print-to-screen command. If computer clock has the current date as July 1, 1998, then:\n\nset century ON ? date ( ) 7/1/1998 * back to the default set century OFF ? date ( ) 7/1/98\n\nWhen converting dates, FoxPro assumes all two-digit years to be 20th century, regardless of the current setting of CENTURY. This default can be overridden.\n\nIt is a data-processing standard at NICAR to convert all character dates to the FoxPro date datatype. We also preserve the original character date column. That way, there is no loss of original data and conversions can be checked and re-done. For any character representations of a date that do not represent a valid calendar date, like 19950335, FoxPro deals with this by converting it to a null or empty date. You can do a query for empty dates to see what was in the character original. Here is a query using the FEC data as an example. The column chardate is the character original; cont_date contains the date datatype.\n\nSELECT chardate, cont_date; FROM Indc079 ; WHERE empty (cont_date)\n\nThe empty ( ) function in the criteria is the same as:\n\nWHERE cont_date = }\n\nUnlike FoxPro, Access makes a conditional assumption about the implied century for a two-digit year. The cutoff year number is 30. Two-digit years of 30 and higher are assumed to be 20th century. Two-digit years from zero to 29 are assumed to be 21st century.\n\nHere is what the test of this looks like in the Access Debug window. (Open a new or existing module, then choose View/Debug Window.)\n\n? cvdate (\"07/01/98\") 7/1/98 ? cvdate \"07/01/28\") 7/1/2028\n\nHere is the issue you'll have to consider when looking at new data sets. Do they have date information with two-digit years? If so, can you do integrity and reality checks to determine that some dates should be 21st Century and not 20th. Some obvious examples: an inmate gets out in the year 2015, not 1915; a building was built in 1920, not 2020. If so, you can make the appropriate conversion.\n\nNICAR has been evaluating the data we get for the data library and for custom data jobs with this issue in mind, but so far the century issue has not come up.\n\nRichard Mullins can be reached at (573) 882-2737 or by e-mail at richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-database-library-highlights-hazardous-materials","slug":"1998-july-database-library-highlights-hazardous-materials","url":"https://dwillis.github.io/uplink/article/1998-july-database-library-highlights-hazardous-materials.html","year":1998,"month":"July","monthNum":7,"headline":"DATABASE LIBRARY HIGHLIGHTS Hazardous materials","kicker":"","author_name":"Seth Hemmelgarn","author_title":"","affiliation":"NICAR","summary":"NICAR has updated its Hazardous Materials database containing Department of Transportation incident reports from 1971 through early 1998, including details on accidental releases during transport across all modes, with information on injuries, spills, and damages.","keywords":["hazmat","database","transportation"],"topics":["Data & Databases","Public Safety","Investigative Methods"],"technologies":[],"full_text":"NICAR's Database Library has recently updated the hazardous materials (Hazmat) database. The data are from the Department of Transportation's Hazardous Materials Incident Reporting System. You can find all records dating back to 1971 - when the system was established - and going through early 1998.\n\nAny time a hazardous material is accidentally released while moving from its origin to its destination, a report is made and the data is entered into the system. All modes of transport are included, such as trucks, trains, planes and boats. An accidental release can mean a barrel falling out of the back of a poorly latched truck or a plane crashing.\n\nThe data include what the material was, the carrier company's name, where the material was coming from and where it was headed, and where the release happened. There are also fields that indicate if anyone was injured or died, if there was a spill and how much monetary damage was done. These tables can be linked together using unique identifiers. The database is broken up into three main tables. These tables consist of the details and consequences of the accident, what failed people or containers - and remarks from the reports. The six lookup tables, which simply provide more details, contain chemical codes and classifications, shipper's names and addresses, and FIPS codes.\n\nSeth Hemmelgarn can be reached at (573) 884-7711 or by e-mail at seth@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-nicars-database-library-has-recently-updated-three-federal-aviation-administration-databases","slug":"1998-july-nicars-database-library-has-recently-updated-three-federal-aviation-administration-databases","url":"https://dwillis.github.io/uplink/article/1998-july-nicars-database-library-has-recently-updated-three-federal-aviation-administration-databases.html","year":1998,"month":"July","monthNum":7,"headline":"NICAR's DATABASE LIBRARY HAS RECENTLY UPDATED THREE FEDERAL AVIATION ADMINISTRATION DATABASES","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR's Database Library has updated three FAA databases related to aviation enforcement, service difficulties, and accidents, while also adding USDA meatpacking inspection data spanning from 1990 to 1997.","keywords":["databases","FAA","inspection"],"topics":["Data & Databases","Public Safety","Agriculture & Land"],"technologies":[],"full_text":"NICAR's DATABASE LIBRARY HAS RECENTLY UPDATED THREE FEDERAL AVIATION ADMINISTRATION DATABASES: THE ENFORCEMENT INFORMATION SYSTEM DATABASE, THE SERVICE DIFFICULTY REPORTING DATABASE, AND THE ACCIDENTS AND INCIDENTS DATABASE. THE LIBRARY HAS ALSO ADDED THE USDA SUMMARY DATA OF MEATPACKING AND SLAUGHTERHOUSE INSPECTIONS FROM 1990 THROUGH THE THIRD QUARTER OF 1997.\n\nFOR MORE INFORMATION ON THE CONTENTS OF THESE DATABASES AND THEIR COSTS, CALL THE LIBRARY AT (573) 884-7332.\n\nA DOWNLOADABLE ORDER FORM IS AVAILABLE AT WWW.NICAR.ORG/DATA","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-july-bits-bytes-and-barks","slug":"1998-july-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-july-bits-bytes-and-barks.html","year":1998,"month":"July","monthNum":7,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Indiana CAR audio tapes and New Orleans IRE conference recordings are available for order, while the Campaign Finance Information Center offers freely downloadable campaign finance data from 13 states and invites contributions of additional state and local data.","keywords":["conferences","campaigns","databases"],"topics":["Journalism & Newsroom","Health & Environment","Data & Databases"],"technologies":["SPSS"],"full_text":"Indiana CAR Audio Tapes\n\nAudio tapes from the sessions at Indiana CAR are now available for ordering. For more information or a downloadable order form, point your browser to www.ire.org/resources/nicar/conferences/indianalaudio.html\n\nAudio tapes from the IRE National Conference in New Orleans are also available from Sound Images, which can be reached at (303) 649-1811. For more information, point your browser to www.ire.org/resources/conferences/neworleans1audio.html.\n\nCampaign Finance Information Center\n\nCampaign finance data from 13 states is freely downloadable from the CFIC at www.campaignfinance.org. We also have links to thirteen online search engines hosted by other non-profits and state boards of election. We will soon release our \"universal\" online search engine so you can type in a contributor from your state and see where else they are giving.\n\nThe more inclusive this database, the better for everyone. So, if you have state or local campaign data you want included, contact CFIC Coordinator Jack Dolan at jack@nicar.org or (573) 884-1802. The CFIC credits everyone who contributes data.\n\nSubscribe Online\n\nYou can now subscribe to Uplink online. Point your browser to www.ire.org/resources/nicar/uplink\n\nDatabase Library welcomes\n\nThe Database Library recently welcomed two new members to its staff: Jason Grotto and Dawn Fallik, both incoming graduate students at the University of Missouri School of Journalism. Grotto, born and raised in Chicago, started his journalism career at the small independent monthly Catalyst: Voices of Chicago School Reform. He helped launch and then directed Catalyst's Web site. Fallik, a 1992 graduate of the University of Wisconsin-Madison, has been a reporter for The Record in Troy, N.Y., and the Associated Press in Philadelphia, Omaha and Baltimore. She has won state and national awards for investigative reporting, spot news and column writing.\n\nRisky Behavior Workshops\n\nThe next Risky Behavior Workshop will take place Sept. 11-13 in Kansas City. Join IRE and NICAR for a special workshop focusing on reporting about lifestyles and health. For more information, see the Web site at www.ire.org.\n\nRelevant health Web site links and sample data sets used at the January 1998 Risky Behavior Workshop in St. Louis are now available online at www.ire.org/resources/conferences/traininglotr/healthcare-9801/links.html. The data comes in two formats: *.SAV, which is compatible with SPSS, and *.DBF. The data include demographics on AIDS, health care coverage, and seat belt usage, and survey questions used by the Centers for Disease Control & Prevention.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-analyzing-school-analysis","slug":"1998-august-analyzing-school-analysis","url":"https://dwillis.github.io/uplink/article/1998-august-analyzing-school-analysis.html","year":1998,"month":"August","monthNum":8,"headline":"ANALYZING SCHOOL ANALYSIS","kicker":"Scoring systems","author_name":"Lynn Olson","author_title":"","affiliation":"Education Week","summary":"The Detroit Free Press and other major newspapers are moving beyond simple test-score rankings to produce more nuanced school report cards that account for socioeconomic factors and other contextual challenges. This shift reflects growing public demand for detailed school information and newspapers' increasing capacity to analyze large databases, though educators remain cautious about potential new forms of misleading analysis.","keywords":["schools","accountability","journalism"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["PC"],"full_text":"This spring, the Detroit Free Press announced that it would no longer rank schools and districts based simply on scores from statewide tests. The newspaper reached its verdict after conducting a six-month computer analysis of results from the Michigan Educational Assessment Program. It found that poverty and other factors outside a school's control were so strongly linked to test scores that it made straight-up comparisons \"inevitably flawed\" and \"mostly meaningless.\"\n\nInstead, the Free Press vowed that from now on it will produce a more nuanced picture of how well Michigan's schools are doing, given the challenges they face. With that commitment, the Motor City daily joins a growing number of newspapers that are investing heavily in time and resources on special reports that go far beyond the mere reporting of test scores. Many now produce regional report cards on schools.\n\nThe newspapers' reports often surpass the documents produced by states and districts in their level of detail, sophistication, and accessibility. Most are available on the Web.\n\nApplying pressure\n\nFor educators, who are already pressured by demands for greater accountability, these reports create both new challenges and opportunities.\n\nMany teachers and administrators say they welcome the potential for a deeper, more complete picture of education. But they also worry that some of the analyses may be as misleading or incomplete as the raw rankings they replaced.\n\n\"I congratulate any newspaper that attempts to put some of this stuff into context,\" said Linda Leddick, the director of research, evaluation, and assessment for the Detroit public schools. \"For too long, they've been running test results as if they're scores in a horse race, and that does not help the public understand.\"\n\nPublic demand\n\nThe media's penchant for producing report cards stems, in part, from the public's hunger for information about schools.\n\n\"Many people today shop for school districts as intensively as they shop for homes,\" said Neill Borowski of The Philadelphia Inquirer, which published its first report card last September. The public's appetite for information comes as newspapers have greatly expanded their ability to sift through large amounts of data thanks to the advent of personal computers, Web sites, and computer-savvy reporters and editors like Borowski. The National Institute for Computer-Assisted Reporting, which trains journalists to analyze databases, regularly focuses on school reports as part of its training sessions. \"Almost every major regional paper now is making some attempt to look at school reports on their own,\" said Sarah Cohen, NICAR's training. \"I think they consider it a basic public service.\" Since 1996, The Seattle Times has published an annual report on schools in its region. This year, its 256-page book includes statistics on more than 530 public and private schools. Readers can also access the information on the Web. In a few seconds, parents can determine which high schools push their students to take algebra or which schools assign at least three hours of homework a night.\n\nCreative conditions\n\nThe Charlotte Observer's report card includes snapshots of about 500 public schools in its area, including such \"top-20 lists\" as which schools have the most 3rd graders reading at or above grade level and which have improved the most on the state's U.S. history test.\n\nLast month, the Los Angeles Times published a special series on California's 8,000 public schools that combined information from dozens of databases. Among its findings: Dropout rates are down, and students from all racial and ethnic groups are taking more college-preparatory courses than in the past. But the newspaper also found more than 1,000 schools that failed to move a single student out of bilingual education last year.\n\nSome newspapers are going beyond test scores and databases to conduct surveys and poll their readers. The Seattle Times and The Philadelphia Inquirer, for example, send questionnaires to school districts, either to collect information that the state does not or to publish it in a more timely fashion. Newspapers also combine databases to make comparisons across districts or schools, often an analysis not available elsewhere. The Los Angeles Times merged data sets from the state education department and the University of California and California State University systems. \"What we needed to do was to put together something that didn't exist,\" said Richard O'Reilly, the newspaper's director of computer analysis. The resulting combined database \"pulled together all of the data from the three sources into a single record per school per year.\"\n\nA level comparing field\n\nTo measure the effects of poverty and other non-school factors on achievement, the Free Press and other newspapers use sophisticated statistical techniques like multiple regression analysis.\n\nSuch methods can determine to what extent variations in test scores are related to differences in such factors as family income, student mobility, or limited English proficiency. The findings are used to create projections of likely test results for a school or district based on its student population. Schools or districts whose actual scores are much better than predicted are judged to be particularly effective at serving their students. Based on its study, the Free Press concluded that the Detroit public schools were beating the odds, while some wealthier suburbs could be doing more. Similarly, the Omaha World-Herald last year identified 10 elementary schools that had done far better over the preceding five years than their demographics would have predicted.\n\nEveryone's a critic\n\nMany educators praise the attempts by newspapers to put test scores in a larger context. But newspapers face many of the same problems as education researchers: Are they controlling for the right variables? Are they using the most appropriate statistical techniques? Are they reaching premature conclusions or inferring causal relationships where none exist?\n\nThe Free Press has been criticized by some researchers for what they say is an overly simplistic analysis of the non-school factors that affect test scores. Shawn M. Quilter, an assistant professor of education at Eastern Michigan University, said the newspaper's report \"has a huge impact on administrators and teachers.\" Though it is just an analysis, he added, educators \"take it as authority and fact.\"\n\nJournalists counter that too many states and districts shy away from painful comparisons of schools and districts. Or they conduct such analyses and fail to publish the results. For years, the World-Herald fought with Omaha school officials over the district's refusal to provide test scores in a way that the paper could analyze them. Once the newspaper obtained the data, \"we learned that the district does the precise type of analysis that we were doing to watch its schools but never made that public,\" said staff writer Carol Napolitano.\n\nThe Sun in Baltimore publishes a report card that profiles elementary schools in its metropolitan area. While the information is on the public record, said Mike Himowitz, the paper's electronic news editor, \"the state has always seen fit to publish this data in a way that doesn't make it easy to compare one school with another, which is why our reports are so popular.\" Other journalists praise the cooperation of state education officials. \"The state people who maintain all the data were actually thrilled that somebody was interested in using it,\" said Bill Ristow, the education editor for The Seattle Times.\n\nInvest wisely\n\nBut educators and journalists alike warn that newspapers embarking on computer analyses must invest the time and money to get it right. Many of the journalists interviewed for this story had spent six months to a year on such projects and had hired expert consultants to look over their shoulders or conduct some of their analyses.\n\nHeather Newman, the CAR specialist for the Free Press, agreed. \"I think that this is a real tricky area for newspapers to get into,\" she said. \"For people who haven't had an adequate education in the proper use of statistics, it's really easy to come up with a bunch of numbers and then to make some meaning out of them.\"\n\nNumerical starting points\n\nOne of the strengths of newspapers is their ability to supplement data with more traditional reporting. \"I don't think the numbers can be the central story,\" Napolitano said.\n\nFor its report, the World-Herald sent five reporters into a dozen schools for two weeks to visit classrooms and interview parents, students, and educators. \"The more we learned about test scores and learned their limitations, the more we felt the need to get into classrooms,\" said Mike Reilly, the newspaper's projects editor.\n\nThe Philadelphia Inquirer incorporates \"points of pride\" into its report cards, in which districts identify things they are doing that stand out. \"As much as we love the numbers and love the data analysis,\" Borowski said, \"there's a lot of things you can't capture with the numbers.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-car-cooking-raw-numbers","slug":"1998-august-car-cooking-raw-numbers","url":"https://dwillis.github.io/uplink/article/1998-august-car-cooking-raw-numbers.html","year":1998,"month":"August","monthNum":8,"headline":"CAR-COOKING RAW NUMBERS","kicker":"Poverty and performance","author_name":"Harvy Lipman","author_title":"","affiliation":"Albany Times Union","summary":"New York news organizations routinely publish standardized test report cards that show wealthy suburban schools outperforming poor urban schools, but researchers find that many inner-city schools actually achieve remarkable results when test scores are adjusted for student poverty levels.","keywords":["schools","poverty","testing"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["SPSS","MapInfo"],"full_text":"Every year, New York's Department of Education releases its \"report cards\" on the public schools, listing how their students did on standardized tests. Every year those report cards show that students in wealthy and middle class suburban districts generally score higher than those in urban-particularly inner-city-schools.\n\nIf ever there was a dog bites man story, the report cards are it. To anyone who hasn't been living in a cave for the past half-century, it's hardly news that students whose parents are well-educated and financially well off usually do better on tests.\n\nYet most news organizations across New York - and the many other states that release similar reports - dutifully publish the test results, implicitly telling our readers and viewers how much better schools in well-to-do neighborhoods are than those in poorer communities. In talking with university researchers who study what differentiates \"good\" schools from \"bad\" ones, however, we found out they believe many inner-city schools are actually accomplishing astounding results given the poverty-related difficulties they have to deal with.\n\nAgainst the odds\n\nTo demonstrate that point, we decided to take the best available poverty data for each school-the percentage its students receiving free lunches - and analyze their test scores compared to their poverty level. (One caution: using free lunch as a poverty indicator works fine at the elementary level. But once children reach junior high school, many refuse to take part in the program because of the embarrassment of being identified as \"poor.\" It's not a good substitute for actual income figures.)\n\nUsing SPSS, we ran a regression analysis that, in short, showed us how each school's test scores compared to average scores for schools with the same poverty rate. The result: In our Albany backyard, we had one of the best elementary schools in the state, although we never would have known it based on its raw test scores.\n\nSchool 20's raw scores, in fact, were slightly below average. But with 96 percent of its students participating in the free lunch program, it should have had test scores that scraped the bottom. In fact, School 20 ranked among the top 70 of nearly 1,900 public elementary schools in New York. Sitting less than a mile away, in the adjoining attendance district, was Albany's Philip Schuyler Elementary School. Schuyler, with a slightly lower free lunch rate, had even lower test scores, ranking among the bottom 20 percent of schools in our regression analysis.\n\nPoor vs. poor\n\nWe decided to take an in-depth look at the two schools, which were dealing kids from essentially the same demographic groups, in order to determine what made one so much more successful than the other. But first, we had to make sure that, in fact, the student bodies were very similar in makeup. We did that by plugging the attendance boundaries for each school into MapInfo, then adding county census and welfare data. We interviewed academics about what to look for in the schools, factors like teacher involvement in educational decision-making, good library resources, or strong leadership from the school principal. The next step getting into the schools - was surprisingly easy. The district superintendent and both principals gave a reporter and photographer free rein to roam the halls and classrooms. What we found helped us dispel the notion that inner-city schools are by definition worse than their suburban counterparts. In fact, our regression analysis found that several suburban schools scored far lower than would be expected given their low poverty rates. The regression showed a 75-percent correlation between poverty rate and test scores, which means statistically that poverty \"accounts\" for three-fourths of how kids do on tests. As a newspaper based in the state capital, we didn't take long deciding, that we ought to take a look at what the state government was doing about providing extra resources to poorer districts, since poverty is a significant determinant of performance.\n\nMisapplied aid\n\nLike most states, New York funds its schools primarily through local property taxes. But it also hands out more than $10 billion annually in school aid. In theory, more money is supposed to go to school districts with the least resources per student. In reality, powerful legislators from suburban districts have made sure their schools get sizable chunks. We analyzed state Department of Education data concerning aid compared with poverty rates among school-age children in the districts and found that the state's aid system was completely out of whack. For example, the city of Albany has the 26th-highest child poverty rate in the state but ranks 543rd in state aid. Buffalo, with the highest poverty rate and the lowest median income, is 186th in state aid. With a citizens' lawsuit already challenging the state aid formula, advocates for restructuring it were planning to use the same type of analysis as we did to support their case.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-assessing-the-e-rate","slug":"1998-august-assessing-the-e-rate","url":"https://dwillis.github.io/uplink/article/1998-august-assessing-the-e-rate.html","year":1998,"month":"August","monthNum":8,"headline":"ASSESSING THE E-RATE","kicker":"Hookup hangups","author_name":"Bob Sullivan","author_title":"","affiliation":"MSNBC","summary":"A journalist discovers that wealthy school districts are disproportionately applying for technology funding through the FCC's E-rate program, while poorer regions lag behind, suggesting the initiative meant to bridge the digital divide may be widening it instead.","keywords":["schools","technology","funding"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["World Wide Web","FTP"],"full_text":"Here's a story with two lessons. First, listen to your father. Second, even if you start your story with data, you can ultimately bail on it and still end up with a good story.\n\nAnd the good news is this story involves data on virtually every school district in the country - it would be easy to localize.\n\nMy father is a computer teacher and technology coordinator at two private schools. For several months, he consistently whined to me about a new requirement to fill out a \"technology plan\" for both schools. I figured he was just complaining about paper work. But one day he mentioned that he had been inundated with junk mail from PC makers, software makers and the like. Why? Well, those technology plans were being put up on the Web. The plans, he explained, included a wish list of all the equipment his schools hoped to purchase in the next few years to connect it to the Internet. So \"service providers\" were making their pitch, and hard.\n\nOK, maybe I'm not yet smart enough to listen to my dad, but when someone says \"those have been posted on the Web,\" my ears perk up.\n\nBefore the weekend was up, I had downloaded a database of computer equipment wish lists from 23,000 schools around the country. Then I worked backward to figure out what I was looking at.\n\nLopsided access\n\nI was looking at the real-life implementation of one of the initiatives of the Telecommunications Act of 1996. That law mandated that the FCC set up a program essentially to re-route money from telecommunications companies to schools - well, really telecommunications consumers to schools. The goal: universal access to the Internet for all schools.\n\nSo the FCC created a nonprofit called the Schools and Libraries Corporation to administer the program, now called the E-rate, or Education Rate. The SLC decided to dole out approximately $10 billion over 5 years in the form of discounts. The discounts up to 90 percent for the poorest schools - were determined by the percentage of students eligible for school lunch programs. The SLC had schools file wish lists (Form 470) and then settle on contracts with service providers (Form 471) after a bidding period. After all the 471s were gathered, the SLC sent letters confirming funding for the discounts.\n\nThe database I downloaded represented Form 470s only - i.e., the wish lists, not the contracts. No dollar amounts were attached. But I was curious to see if there were any patterns in the data, if some areas of the country were taking advantage of the program more than others.\n\nWhen I sorted requests by state, it quickly became evident that states like Washington and New York applied for much more equipment than states like Arkansas or North Carolina. In other words, it looked like there might be a \"rich get richer\" story here - already technologically hip areas were getting more equipment, while \"unconnected\" parts of the country weren't even participating.\n\nApplying for uncertain funding\n\nHere's where the real reporting came in. I wouldn't have had a story without the help of Dateline NBC's Andy Lehren, formerly of NICAR. Andy had experience working with the Department of Education's Common Core database. If my hypothesis were true, we'd be able to find poor schools that didn't apply for E-rate money.\n\nSo Andy sorted schools in Arkansas, North Carolina, West Virginia and several other states that had more than 75 percent eligibility for federal school lunch programs, then cross-referenced those schools with the SLC database. It didn't take long to find poor schools not in the SLC database.\n\nSpending several weeks calling those schools gave me a great, real-life picture of how difficult the E-rate application process was - and why some schools didn't apply. More important, found an even larger story. Every school that applied for E-rate money was stuck in limbo: E-rate funding was expected in late winter or early spring. Schools that had applied either went ahead with their purchases by spending their own money and hoping E-rate funding would come through, or, more likely, just sat and waited.\n\nPosted warnings\n\nThere's plenty of caveats to using this data in your story, though. Most important, not every application for service is included in the now 24,000 or so applications you'll see. Only requests for \"new service\" are included. Schools were allowed to apply for discounts for existing service provider contracts, but they are not included. Schools were also allowed to apply for discounts on plain old phone service, but those applications are not included, either. Finally, some schools applied as cooperatives. You should also know that schools were able to enter their own applications on the Web, making the data pretty dirty. I had to remove hundreds of duplicate records. All that said, however, you'll find downloading the data and looking at schools in your area to be a great starting point for a story on how well your schools did with their E-rate applications. As an added bonus, every application comes with a contact name and phone number (usually the district's technology coordinator), so it's an invaluable resource.\n\nAlso note that the database contains only some of the raw numbers culled from each school's application. The applications themselves can also be retrieved (one at a time) from the SLC Web site. If you're going in-depth on a few schools, you'll want to see the actual application, which contains lots of additional commentary. The SLC folks agreed with me that, because of the cooperative factor, the most accurate way to sort the data was to examine it state by state. It is interesting to see which states applied for more new services than others. Interesting, but not as conclusive as the interviews to which the data led us. In the end, after considering to omit reference to the data, we only included one chart - a ranking of states that asked for the most new video hookup services per student. We gave it no more credibility than any other source in the story. One more caveat found explaining the SLC and the E-rate process to be pretty tricky. Prepare your editor for some tortured paragraphs if you're really interested in fully explaining how the funding mechanism works.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-a-few-good-cars","slug":"1998-august-a-few-good-cars","url":"https://dwillis.github.io/uplink/article/1998-august-a-few-good-cars.html","year":1998,"month":"August","monthNum":8,"headline":"A FEW GOOD CARs","kicker":"Training gains ground","author_name":"Karie Hollerbach","author_title":"","affiliation":"Southeast Missouri State University","summary":"Journalism schools are increasingly teaching computer-assisted reporting (CAR) skills, but many programs don't require students to take these courses despite managing editors actively seeking recruits with data analysis capabilities. A 1996-97 survey found that while 85% of universities teach CAR, qualified faculty shortages remain a significant barrier to widespread adoption.","keywords":["journalism","computer","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Education"],"technologies":["Internet"],"full_text":"Newsroom managing editors expect basic skills from new recruits with freshly minted journalism school degrees. At the top of the list usually is the ability not only to write but to write well. Working a beat is important, as is generating story ideas. And let's not forget interviewing technique, basic copy editing skills, and the motivation to get it first and get it right.\n\nBut what about computer-assisted reporting skills? Are managing editors actively seeking recruits who can demonstrate that they have spent some time in front of a PC analyzing data and looking for a story? Are undergraduate journalism programs teaching students the CAR skills they need to meet the expectations of managing editors?\n\nSprouting media\n\nCAR's impact on print journalism has been growing steadily since the early 1990s. A 1992 survey of 192 daily newspaper editors found that more than half of the newspaper readers in the U.S received newspapers that were doing stories with computer-assisted record analysis, not only for special projects but for beat reporting as well.\n\nThe top two reasons given by the editors for not using CAR were a lack of equipment and a lack of skilled personnel. One editor commented that his impression was that university journalism programs were behind the curve on this new trend of computer analysis.\n\nStunted schools\n\nA national survey of journalism schools, also completed in 1992-93, indicated that the majority of universities had no formal CAR instruction nor plans to add it. Of the 258 responding schools, 24 percent taught some form of record analysis while 13 percent offered formal instruction in online database searching.\n\nBetty Medsger's nationwide study on journalism education for the Freedom Forum in 1996 found that computer skill education had become more widespread 87 percent of the journalism programs surveyed offered some type of it. Special courses had been created at 35 percent of the institutions surveyed, while 72 percent had included it in existing courses. However, only one-third of the journalism programs offering computer instruction of any kind required journalism majors to take those courses.\n\nSupply and demand\n\nIn an exploratory study I conducted in 1996-97, both university journalism/communication departments and newspaper managing editors were surveyed simultaneously to determine the current supply and demand of CAR skills.\n\nUniversities teaching CAR in their programs and newspapers practicing CAR ranked differently - though the differences were not statistically significant the most important skills for journalism students to master: Internet use, database use, data acquisition, statistical analyses, and spreadsheet use.\n\nNearly 85 percent of the universities said that they were teaching CAR. Of those not currently teaching CAR in their programs, 82 percent said they had future plans to do so. A lack of qualified faculty was the most frequent reason given for not teaching CAR. No one indicated that a lack of equipment posed a barrier to CAR's entrance into their program.\n\nNearly 42 percent of the newspapers surveyed said that they were using CAR. Special projects and investigative reporting topped the list as the most frequent uses of CAR, but CAR as a part of daily news coverage was reported at 50 percent. Internet use was a frequently expected skill not only of new journalism school recruits but also of current staff members. A lack of funds and a lack of qualified staff were the most frequently reported reason for not using CAR. Whether CAR is actively being taught or used, the skill sets valued by both university programs and newspapers are relatively the same. The real key is in teaching students how to apply those CAR skills. Progress is really made when journalism students can enter a newsroom prepared to conduct computer-assisted reporting at the touch of a button.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-report-cards-revisal","slug":"1998-august-report-cards-revisal","url":"https://dwillis.github.io/uplink/article/1998-august-report-cards-revisal.html","year":1998,"month":"August","monthNum":8,"headline":"REPORT CARDS REVISAL","kicker":"Multi-layered ratings","author_name":"Jeff Porter","author_title":"","affiliation":"Arkansas Democrat-Gazette","summary":"The Arkansas Democrat-Gazette created a comprehensive school statistics report ranking 100 schools across Pulaski County by developing a methodology that quantifies school performance using standardized metrics like test scores, dropout rates, and teacher ratios, while also capturing qualitative school characteristics through interviews.","keywords":["statistics","rankings","schools"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["SPSS","Excel","FoxPro"],"full_text":"Creating reams of school statistics is not difficult in itself. But the more challenging task was our mission to organize the reams to be able to help our readers make educated choices about schools. We also wanted to show intangible qualities of schools with interviews from educators, parents, students and others.\n\nSo how did we do it?\n\nThe Arkansas Democrat-Gazette published its second annual school statistics report on Aug. 23 in a 24-page tabloid ranking of 73 elementary, 15 junior high and 12 high schools in Pulaski County.\n\nQuantifying performance\n\nThe rankings used a broad set of categories to quantify school performance in areas identified by educators, parents and federal courts.\n\nWith most categories, the newspaper collected raw numbers then calculated rates - mostly per 100 students - to compare fairly schools of any size. The calculation was simple. For example, to calculate the high school dropout rate, we used the formula: (dropouts/enrollment) X 100. We chose SPSS but could have easily used Excel or Visual FoxPro. The categories:\n\nElementary schools: rates per 100 students for teachers, advanced-degree teachers, parent-group membership, volunteer hours and suspensions; a percentage showing either crowded or unused classroom space, calculated by enrollment/capacity; average SAT scores for the fifth grade; racial disparity in those test scores (an issue a federal court is monitoring); and spending per student.\n\nJunior high schools: all the elementary figures except the test scores and racial disparity were calculated for seventh-graders; and rates per 100 students for dropouts and expulsions.\n\nHigh schools: the elementary and junior high school categories except the scores and disparity were for 10th-graders; ACT examination averages; percentages of seniors taking the ACT; percentages of previous graduates who were forced to take remedial courses in state colleges and universities; percentages of seniors taking college preparation courses; and the total number of courses available.\n\nThe Democrat-Gazette constructed tables with rankings - that is, which school is the \"best\" given this set of categories, all of which were valued equally. In 1997, we ranked with Excel. This year, we used SPSS, a statistical software package that is much handier for this purpose.\n\nWe identified schools that beat the odds and did significantly better in the test scores than predicted. Then we wrote about how those schools are making a difference.\n\nWe had SPSS rank each category in reverse order. That, in effect, created point scores for each school. For example, the worst elementary school in the teacher-student ratio category was ranked \"1.\" The best school, then, would have a score of 73, or the total number of elementary schools.\n\nThen we had SPSS create an average score using all of the scoring columns. The top average score became our top-ranked school. We showed the average scores, but instead of using unfamiliar groupings such as quartiles or quintiles, we assigned letter grades for schools. Those letter grades helped readers understand that average scores were quite close for many schools.\n\nBut was it fair? Not if that were all we did. Like last year, and even more in-depth this year, the newspaper showed readers how to zero-in on categories and to find schools doing well in them. Graphics portrayed how schools fared in each category. Readers can track categories they deemed most important and judge how particular schools are doing.\n\nChasing intangibles\n\nWe wrote serious stories about some of the higher-ranked schools and showed how they have been successful. Those stories aren't top-heavy with numbers but rather emphasize interviews. We did stories about some schools, though not ranking highest, that ranked well in specific categories. We also looked at schools that fell in the rankings this year to learn why.\n\nAnd we went even beyond that, especially with the elementary schools. Using fifth grade average SAT scores and the percentage of children taking part in the free and reduced lunch program, we created a regression analysis showing how the two factors are related. We identified schools that beat the odds and did significantly better in the test scores than predicted. Then we wrote about how those schools are making a difference.\n\nFor our purposes, we looked at one other scatterplot. Pleasingly, the plot showed that race had no bearing on schools' average scores.\n\nMost newspapers avoid ranking and grading schools. We rank and grade them because we are confident of our statistics and want to help our readers make sense of the data instead of merely presenting a jumble of numbers that lead to no conclusions.\n\njeff Porter can be reached by e-mail at jporter@ardemgaz.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-nicars-database-library-has-recently-added-the-following-data-sets","slug":"1998-august-nicars-database-library-has-recently-added-the-following-data-sets","url":"https://dwillis.github.io/uplink/article/1998-august-nicars-database-library-has-recently-added-the-following-data-sets.html","year":1998,"month":"August","monthNum":8,"headline":"NICAR's DATABASE LIBRARY HAS RECENTLY ADDED THE FOLLOWING DATA SETS","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR's Database Library has expanded its collection to include several new datasets covering mortgage disclosures, small business loans, arts grants, and tax-exempt organization filings.","keywords":["databases","loans","grants"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"NICAR'S DATABASE LIBRARY HAS RECENTLY ADDED THE FOLLOWING DATA SETS:\n\nTHE HOME MORTGAGE DISCLOSURE ACT DATABASE FOR 1997, COLLECTED BY THE FEDERAL RESERVE BOARD TO TRACK HOME LOANS, HOME-IMPROVEMENT LOANS AND BANK-PURCHASED LOANS\n\nTHE SMALL BUSINESS ADMINISTRATION'S 7A DATABASE OF LOANS APPROVED BY THE SBA UNDER ITS MAIN LENDING PROGRAM, KNOWN AS 7A\n\nENDOWMENT FOR THE ARTS GRANTS PROGRAM DATABASE OF GRANTS TO INDIVIDUALS AND ORGANIZATIONS FROM 1987 THROUGH 1997\n\nTHE IRS' TAX EXEMPT ORGANIZATION BUSINESS FILE, WHICH CONTAINS BASIC INFORMATION FROM ALL 990 FILINGS\n\nTHE NATIONAL\n\nTO ORDER OR HAVE YOUR QUESTIONS ANSWERED ABOUT ANY OF THESE DATABASES, CALL THE LIBRARY AT (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-tech-tip","slug":"1998-august-tech-tip","url":"https://dwillis.github.io/uplink/article/1998-august-tech-tip.html","year":1998,"month":"August","monthNum":8,"headline":"TECH TIP","kicker":"On being and nothingness","author_name":"Richard Mullins","author_title":"Technology Writer","affiliation":"NICAR","summary":"This article explains null values in databases—how they represent unknown or missing data and differ from zero or empty strings—and their practical implications for queries and data management across software like Visual FoxPro and Access.","keywords":["nulls","databases","queries"],"topics":["Technology","Investigative Methods"],"technologies":["FoxPro","Access","SQL"],"full_text":"On being and nothingness\n\nIf I were clever, I would title this Tech Tip \"On being and nothingness.\" But I'm not and I did, so there's an existential dilemma for you. It's about nulls, an explicit, unambiguous and logically tricky way of talking about nothing. Knowing about nulls is significant for at least two kinds of queries: summary queries using the count ( ) aggregate function and queries filtering for empty character fields. If you use FoxPro versions 2.x, then you may never have heard of nulls. Even if you've used the last of the old FoxPro, version 2.6, you might not have noticed the idea of nulls. Nothing lost, however: the implementation of nulls in 2.6 was incomplete and not worth bothering with. If you made the switch from old FoxPro to new Visual FoxPro, or to Access, you might have bumped into nulls in the documentation - or been assaulted by one in an error message. Or you might have been using bad words in front of your 11-month-old computer because you kept doing an Access query for blank occupations, the query kept saying there weren't any, but you knew there were plenty. You were in the twilight edge of the null zone and things are different there. Here is an explanation from the Visual FoxPro documentation: Visual FoxPro provides support for null values. This support simplifies the task of representing unknown data and makes it easier to work with Microsoft Access or SQL databases that may contain null values.\n\nStandardizing the unknown\n\nThe explanation is intended to be gentle, but it borders on the euphemistic. I would say that nulls standardize the representation of unknown data. And, obviously, software is easier - and gives the user fewer problems when it conforms to standards.\n\nAlso from the Visual FoxPro documentation, these definitions:\n\nNull values are:\nEqual to the absence of any value. Different than zero, the empty string (\"6\" or blank.\nSorted ahead of other data. Propagated in calculations and most functions.\nNull values affect the behavior of commands and functions, logical expressions and parameters.\nNull value: Having no explicitly assigned value. NULL is not equivalent to zero or blank. A value of NULL is not considered to be greater than, less than, different than, or equivalent to any other value, including another value of NULL.\n\nAny datatype can contain a null. Nulls should be used to indicate things like missing data or unknown value. It does not stand for a quantity with a meaning. For example, an age column with a null means that the age is unknown, not that the person is unborn or dead. Same thing for a column indicating sex or gender: null means unknown, not a new biological condition.\n\nNullified values\n\nHere is how nulls affect summary queries using the count ( ) function. Using count (ColumnName) will only include rows where that particular column has some non-null value in it. Using count (*) will count all rows. FoxPro 2.6, for example, allows both methods because this is standard SQL. It just treats them identically and gives the same answer, which is not standard SQL. The other item to explain from the documentation is this statement: \"Nulls propagate through calculations and most functions.\" If a single column or variable containing a null is anywhere inside a nest of calculations or functions, then the answer is null. This fact can be especially tricky if you are looking at logical or true/false columns. Null is neither true nor false. Without knowing this, you could misinterpret your results.\n\nRichard Mullins can be reached by e-mail at richard@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-handout-of-the-month","slug":"1998-august-handout-of-the-month","url":"https://dwillis.github.io/uplink/article/1998-august-handout-of-the-month.html","year":1998,"month":"August","monthNum":8,"headline":"HANDOUT OF THE MONTH","kicker":"Analyzing test scores","author_name":"Carol Napolitano","author_title":"","affiliation":"The Omaha World-Herald","summary":"Journalists should plan strategically before requesting school test score data by researching competitor coverage, consulting newsroom colleagues and community members, and maintaining focus to produce quality stories rather than numerous mediocre ones. When requesting data, journalists should seek student-level information spanning multiple years including demographic, socioeconomic, and geographic details to enable meaningful longitudinal analysis and contextualization.","keywords":["data","education","journalism"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Before you ask a school district or state for test score data, think about how you may want to use it. Do your homework. Take the time to look at what other news organizations have done with similar data. Solicit ideas from reporters and editors in your own newsroom and ask them what they'd like to see written. If you have a good relationship with any teachers or members of the local PTA, ask them the same question. Keep things focused. There are so many potentially good stories in these data. Without focus, you risk getting lost, getting frustrated, and producing a lot of mediocre stories instead of a few really good ones.\n\nDemanding details\n\nTry to get student-level data. Some districts keep only minimal information; others gather demographic and socioeconomic information on students and their families. Get as many years of data as you can. Analysis of test scores for a single year is pointless if the focus of your stories is performance. Try to get three to five years at a minimum.\n\nFollowing is a sample wish list for student-level data. If you can get only school-level data, try to get aggregates like gender and race percentages.\n\nStudent ID (Student names are confidential, but sometimes the district or state will generate an ID number for each student that you can use. This will allow you to do longitudinal analysis.)\n\nGrade level, teacher, school attended\nGender, race\nGrade point average\nParent education level\n\nFamily income information. Participation in free or reduced lunch programs is the most commonly used indicator of poverty or low income. Free lunch participation tends to dwindle as students get older and it becomes more of a social stigma. So, if the district keeps a more stable indicator, like parent's income range, get that, too.\n\nStudent ZIP codes or a more specific geographic level, like nodes or census subdistricts. (Student addresses usually are confidential.) If your district has a desegregation plan, chances are it has divided the district into small geographic areas. Even if there is no desegregation plan, the district may divide its area into smaller sections. With any luck, you can match these areas to census and property information to get a feel for the demographics of a student's home environment: Do they come from an area with a lot of rental units or single-parent families? An area where high school or college graduation rates are very low or very high? An area where the average cost of a home is six figures?\n\nTest scores for all subjects tested, plus total battery scores. Ask for NCE scores, as well as percentiles. You can't do math with percentiles.\nMobility figures: How often has the student moved within the district? Is there an indicator denoting when the student entered the district from another district?\nParent participation numbers. Some schools keep data on what percentage of parents participate in the PTA, school events, report meetings, or teacher conferences.\n\nStatistical differences\n\nSchool districts will vary widely in terms of the statistical expertise they have on staff. But even the least sophisticated school researchers may assume they know more than you do and that you will mess things up. They may be right. You may mess it up if you don't know what you are doing. Bottom line - know the difference between a percentage and a percentile; know the difference between NCE, scale and percentile scores; and have a good understanding of statistics.\n\nDo you know what this means and how to use it?\n\ny = B + + E\n\nIf you don't know what that formula is or how to use it, take a course on linear regression or get yourself a consultant. While you can spend big bucks on consultants, there are less expensive alternatives: another journalist who has done this type of work, someone at your own news organization in quality control or marketing who has a statistics background, or a professor at a local college willing to work for a nominal fee or for free.\n\nPerils and pitfalls\n\nTest score data is full of pitfalls; some may be unique to your data. A thorough knowledge of the school district will help you spot any problems. Things you might encounter:\n\nUniversal lunch schools. These are schools where the percentage of low-income students is so high that all students are eligible for free/reduced lunch whether their family income qualifies them or not. This will affect your use of free/reduced lunch as an income indicator.\nGrade bias. In some districts, certain grade levels traditionally do better on standardized tests than others. In Omaha, it was the 2nd grade. Because of the city's desegregation plan, this biased the results for schools that had only 2nd grade test-takers in their school. If this bias exists, you can choose to exclude these schools or disclose this problem to the readers.\nApples-and-oranges demographic comparison. When accumulating demographics to accompany the test score data, make sure you're comparing the same population. For example, if the standardized test is only given to 2nd, 4th and 6th graders, use demographics for the test-taking population only.\nMeans versus medians. If your dataset is small or the scores are not normally distributed, consider using median scores for your analysis instead of mean scores. The more you boil down your dataset (schools, grade levels or even individual classrooms), the more you should consider using medians.\nMobility. If you are using mobility as a variable in your regression model, make sure to find out how it is defined. Some schools count only students entering a school or district after the last registration day; some count only students leaving; and some count both types of movement.\nInflating prediction. Don't overload your analytic model with predictor variables. If you dump too many variables into a regression model, you will artificially inflate the results. Watch for variables that correlate highly with each other. Putting them both into the model also can cause problems.\n\nUnavoidably incomplete\n\nRemember to acknowledge to readers the limitations of your analysis. You can't create a statistical model that explains all the differences in test scores.\n\nRegression is better at analyzing groups than individual students. You'll find this out yourself if you build a model and apply it to individual student scores and then to aggregate school scores. The model will explain significantly more difference in school level scores than individual ones.\n\nA common criticism of test score analysis is that it \"makes excuses\" or \"sets the bar lower\" for poor or minority students. This perspective often comes from people believing that your analysis is predicting how students are going to score on future tests because of their socioeconomic status. While it is true that regression analysis is used for prediction, the model based on historic performance letters of these students or schools (providing you have ample years of data in your analysis). In your stories, you may also consider avoiding words like 'expected,' which for some may connote that you are setting different levels of expectations for different students or groups. Explain how you did your analysis, but expect that you'll never satisfy all your readers. You'll hear from different people if you keep your explanation too simple or vague than if you make the explanation too detailed or complex. Consider providing a moderate amount of detail in the stories and then publishing a more detailed appendix that people can order.\n\nCarol Napolitano can be reached by e-mail Carolnap@owh.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-stats-from-the-road","slug":"1998-august-stats-from-the-road","url":"https://dwillis.github.io/uplink/article/1998-august-stats-from-the-road.html","year":1998,"month":"August","monthNum":8,"headline":"STATS FROM THE ROAD","kicker":"Sampling large groups","author_name":"Sarah Cohen","author_title":"Columnist","affiliation":"NICAR","summary":"This article explains different sampling methods for journalists analyzing large datasets, ranging from simple judgmental samples based on convenience to rigorous random samples, and discusses when each approach is most useful for investigative reporting.","keywords":["sampling","data","journalism"],"topics":["Data & Databases","Investigative Methods"],"technologies":["Access","Excel"],"full_text":"This is the second installment of a two-part Last month, this column advised you how to decide whether testing a small set of data might help focus your project, to provide a game plan for gathering documents and to hone your data analysis. That's all well and good. But how do you sample a large group?\n\nFirst decide the purpose of your sample. That will often drive you toward one of the common sampling methods, though more complex methods exist.\n\nThere's really a ladder in sampling techniques. At the bottom, researchers simply pick some cases, sometimes for a reason. Then they put a fancy word on the technique- \"judgmental sample.\" For purists, though, only a true random sample, which carries the imprimatur of science, will do.\n\nBut random samples often won't do what you want, at least not with huge samples that might be beyond your means. In the middle, samples are based on the concept of random numbers but are \"assisted\" or \"stratified\" to give a relatively large set of particular instances you know you need.\n\nExploratory tools\n\nLet's start at the bottom with the \"judgmental sample,\" or a sample of convenience. Here, you pick out documents or records you know interest you and examine them in depth. An example is the most recent 50 entries into a database compared with the original documents that generated the records. Looking at what happened lately can give you a big hint at what's to come.\n\nThese kinds of seat-of-the-pants tests are particularly useful as exploratory tools to determine whether to go further in your project. You might want to treat them the same way researchers treat \"content analysis\" the science of studying documents for trends. It's the closest analogy in social science for much of what we do in the newsroom.\n\nResearchers will sometimes check whether data entry clerks categorize items the same way. This step is particularly important when asking for help in collecting as well as entering data. Using this kind of sampling would have helped one newspaper that hired clerks to collect and type demographic information on college coaches. Simple questions about the coaches, such as years of experience, were interpreted differently. One clerk counted junior college. Another counted high school. A third counted only assistantships at major universities.\n\nWhen you care about specific instances for your story - people who drive red cars, people who live in your county, or airplanes that land at your airport - judgmental samples are also useful. But if you stop there, you'll miss your story's crucial \"compared to what\" statement. You may know what happens locally, but you won't have context of what happens elsewhere.\n\nPlanning randomness\n\nAt the other end of the spectrum is the pure random sample, usually reserved for stories in which you plan to state your findings without ever studying the \"population.\"\n\nThe major criteria for random sample: The sample must be selected from a list of every possible person, document or company you're studying. In most polls, this is virtually impossible. Researchers use samples of every known telephone number, registered voter, or company paying unemployment insurance.\n\nEvery person or case has an equal chance of selection. No one is excluded.\nPicking one case has no influence on picking any other.\n\nGetting a random sample is easy. First, make sure your data contain unique record identifiers. In Access, you'll want to create an \"AutoNumber\" field. If you already have one, make sure you haven't deleted any records so you can use it as a record number. Note the number of records in your database.\n\nThen go into Excel and create random numbers. (These aren't exactly random- they won't repeat your numbers for quite some time.) Your function will look something like this: =RANDBETWEEN(1,2000000) This picks a random number between 1 and 2 million, which you'd use if you had exactly 2 million records in your original database.\n\nCopy to as many rows as you want for your sample. Unless you want to look at sub-groups, in which you will need more records, most samples are about 350 records. Then copy and paste this list into a new Access table. You may need to change the to Integer.\" Join the two tables on your AutoNumber field, which only keeps records listed on both tables.\n\nImplicitly stratified\n\nThe dirty little secret of sampling is that hardly anyone - neither the major statistical agencies nor the pollsters - conducts pure random samples. In fact, some statistics books say that only a \"probability-based\" sample, not a random one, is needed to make statements about the larger group.\n\nStatisticians, then, often do something close to - but not quite - random sampling. It's often called \"list-assisted\" or \"implicitly stratified\" sampling. Instead of choosing a list of purely random numbers, these methods take a random starting point, then choose every subsequent 10th or 100th (or some other number) record. It guarantees the sample has the mix you care about: demographics, time, or even data entry clerks.\n\nThere are three steps to these samples: Sort by an important element, like date, keypunch operator's initials, or ZIP code. Determine the \"sampling interval,\" or the number of records between each record selected, to give you the sample size you want. Divide the number of records by the sample size. For instance, a sample of 350 records out of 2 million records would have a sampling interval of 5,714.\n\nFigure out which records to select. You want what's called a \"random start.\" Select a random decimal between 0 and 1, then multiply by the sampling interval. If your sampling interval is 5,714 and your random decimal is .35, then you'll start with the 2,000th record, followed by the 7,714th, and so on. Again, it's easiest to create the list of numbers in Excel, then paste and join it in Access.\n\nExplicitly stratified\n\nOther sampling variations include what statisticians call \"explicitly stratified\" samples, used when comparing one group to another or when you have evidence that one group would be under-represented in more random sample. The key to this method is predicting into which group an item or a person falls.\n\nSay, for instance, you want to compare recidivists with non-recidivists, who are rare. If you can identify which cases fall into the two groups in advance, you can use either random method to select an equal number of cases from each group. Or you can sub-sample from the larger group, keeping only one of four who returned to jail. But here you cannot discuss all prisoners-you're limited to discussing differences between the groups.\n\nNo matter which method you use, select enough cases for your story. A sample of 385 or so will let you predict about the larger population within five percentage points of your estimate. Often, we want much more precision. You have to raise the sample quite a bit to lower this range to one or two percentage points. Look in a statistics book if you need to find a sample size. But don't start down this road if you aren't willing to look at enough cases or talk with enough people to make your story's point. Sarah Cohen can be reached by e-mail at sarah@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-first-ventures","slug":"1998-august-first-ventures","url":"https://dwillis.github.io/uplink/article/1998-august-first-ventures.html","year":1998,"month":"August","monthNum":8,"headline":"FIRST VENTURES","kicker":"Valley of the dollars","author_name":"Hugo Martin","author_title":"Political Reporter","affiliation":"Los Angeles Times","summary":"A political reporter uses computer-assisted reporting (CAR) and database tools to analyze campaign finance data for a hotly contested state Senate race, discovering that most contributions came from outside the district rather than from local residents.","keywords":["campaign","database","analysis"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Access","Excel"],"full_text":"To be honest, math has always been a four-letter word for me. If it were up to me, the Geneva Convention's list of cruel and unusual punishments would include the task of figuring out percentages. So I was delighted to learn during my first CAR lesson that a computer can be used to sort reams of data, add rows and rows of figures, and oh joy - figure out percentages. My first thought upon taking CAR class was to use the computer to spare me the agony of calculating from those mind-numbingly long campaign finance reports that I was forced to analyze as the political reporter for the San Fernando Valley edition of the Los Angeles Times. With some tips from Sarah Cohen, NICAR's on-the-road trainer, I more than 500 campaign contributions made to the top two candidates for a hotly contested state Senate race in the Valley.\n\nSeeking sources\n\nI started the project with two main goals: Find out where, geographically speaking, the contributions came from and what occupations donated the most money. Then I could compare the two candidates: former assemblyman Richard Katz and city councilman Richard Alarcon.\n\nMy first challenge was converting the data from hard copies of campaign finance reports to a database I could manipulate. Once I downloaded the campaign data into Access, I found lots of missing data, in particular the occupations and addresses of some contributors. To fill the holes, I called the campaign offices for each candidate and entered the information manually.\n\nThe campaign finance data of the Times library included a column that contained the ZIP code for every contributor to the two candidates. I linked the ZIP code listing in the database to a table of district ZIP codes that I created. That showed me how many contributions came from within the district.\n\nI also created separate tables with the ZIP codes for the San Fernando Valley, Los Angeles and Sacramento, home of many of the state's lobbying groups and labor unions. I then sorted the contributions by those geographic areas.\n\nLimited local loot\n\nThe results were surprising. District residents and businesses contributed only eight percent of the money raised by each candidate. The vast bulk of contributions came from addresses spread throughout southern and northern California, sizeable amount from Sacramento.\n\nTo sort the contributors by occupation, I created a new column in the campaign database and plugged in generic descriptions for each contributor so that they could be sorted into multiple general categories, such as transportation firms, lawyers, real estate businesses, labor unions and so on.\n\nI then sorted to add the amounts of money generated from each occupation. This did not reveal many surprises. Much of the money came from expected sources.\n\nExcel-baked pie\n\nThe real magic came when I used the Excel program to make pie charts. With a few simple mouse clicks, a chart illustrated how much money came from within and without the district. Another chart showed how much came from lawyers, unions and other groups. This impressed my editors because the numbers were easier to visualize and present to readers.\n\nBut the best part was that Excel automatically figured out the percentages for each slice of the pie.\n\nHugo Martin can be reached by e-mail at Hugo.Martin@latimes.com or by phone at (213) 237-7086","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-on-the-internet","slug":"1998-august-on-the-internet","url":"https://dwillis.github.io/uplink/article/1998-august-on-the-internet.html","year":1998,"month":"August","monthNum":8,"headline":"ON THE INTERNET","kicker":"Unearthing experts","author_name":"Kitty Bennett","author_title":"","affiliation":"St. Petersburg Times","summary":"A 1998 guide for journalists on finding expert sources online through university directories, specialized databases, newsgroups, and advanced search engine techniques, emphasizing creative use of existing digital repositories.","keywords":["experts","databases","search"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","World Wide Web","Nexis"],"full_text":"The following is excerpted from a handout provided at the June 1998 IRE National Conference in New Orleans.\n\nCheck out experts directories on the Web. You know those \"Guide to the Faculty Experts at Wahoo University\" books your local institutions of higher learning or favorite think tanks send out annually? Chances are good that they're also available on the Web. These directories typically offer experts' credentials in excruciating detail, as well as contact information. Who needs big collections of paper directories?\n\nOnline directories are often featured prominently on an organization's Web site. Sometimes they're grouped together with other information like press releases.\n\nRepositories of experts come in many different forms. Think creatively: When you're looking for experts, Amazon.com isn't a place to buy books, it's a database of over a million experts. Same goes for the Library of Congress. The New York Times Book Review is not only a great place to check out an author's credibility but also a good place to search for experts. It's free and contains over 50,000 reviews going back to 1980.\n\nOther databases of experts masquerading as something else: AskEric (www.askeric.org), the Educational Resources Information Center, and the National Library of Medicine (http://www.nlm.nih.gov/databases/freemedl.html), a database of nine million references to articles in 4,000 medical journals.\n\nLook for experts in newsgroup and listserv postings, but be cautious. These are wild and woolly places to be searching for experts and can become black holes into which you valuable time. If you do find someone you think is an expert, it can be difficult to gauge his credibility.\n\nAn intelligent, targeted search can turn up some gems. To foster productive searching in DejaNews (www.dejanews.com), the premier newsgroup search engine that contains 100 million postings going back to March 1995, go to the Search Language Help page (www.dejanews.com/help/help_lang.shtm), print it and keep it next to you. Always use the Power Search option.\n\nThere are far fewer options for searching electronic mailing lists, but a good first step is Reference.COM (www.reference.com), an archives of three months' worth of postings to 8,000 mailing lists (of an estimated 100,000+). You can store queries and run them automatically every day if you want.\n\nTry the Directory of Scholarly and Professional E-Conferences (www.n2h2.com/KOVACS/). It's a good source for 4,000 mostly high-quality mailing lists and newsgroups on topics of interest to scholars and professionals, divided into more than 50 subjects.\n\nUse traditional search engines efficiently. AltaVista (www.altavista.digital.com) supports proximity searching, truncation and all sorts of sophisticated search capabilities that makes it ideal for finding experts. It can limit searching to URLs with certain words in them or to particular domains, but only if you click on \"Advanced\" and put your search into the \"Boolean expression\" box. You can do somewhat complicated - but effective - searches like this one: (expert or author* or research* or professor) near astron* and (url:mit or url:princeton or url:stanford).\n\nYou can't ignore HotBot (www.hotbot.com) because of the huge size of its database.\n\nBeaucoup! (www.beaucoup.com) has more than 1,000 engines, directories and indices listed, divided into 20 categories. Search.com (www.search.com) gives you 100 different ways to search the Web, divided into 14 different categories. The Mining Co. (www.miningco.com) is a search engine but also has 500 interest areas, moderated by expert guides, on everything from the aviation industry to zines.\n\nCheck your expert's credibility. If you found your expert at a reputable university or think tank, you're on pretty firm ground. Someone else checked out his credentials before he got there.\n\nFind out who else has quoted your expert because of her expertise. Run her name through a commercial online service such as Lexis/Nexis. Many reputable publications have free searchable archives. For links to free archives, see News Hunt (www.newshunt.com). Run your expert's name through several search engines. Try Northern Light (www.nlsearch.com), which has a \"Special Collection\" of 3,400 high quality publications. Don't forget to check your expert out in DejaNews for indications of her expertise and/or psychoses. For all you know, she may be a frequent poster in alt.binaries.pictures.tasteless or alt.impeach.clinton.\n\nWhen using experts found in expert witness directories, like ExpertPages.com, remember that these people are part of the testimony-for-hire industry. That doesn't mean that they aren't experts, but you really want to check out their reputations. James Derk in his excellent story on experts (\"Charlatans or Authorities?\", Editor and Publisher, June 14, 1997) quotes ProfNet's Dan Forbush as saying, \"Ultimately it's up to the journalists to decide the qualifications of the expert.\"\n\nKitty Bennett can be reached by e-mail at bennett@sptimes.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-bits-bytes-and-barks","slug":"1998-august-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-august-bits-bytes-and-barks.html","year":1998,"month":"August","monthNum":8,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This journalism newsletter covers available conference audio tapes, a campaign finance information database with data from 13 states, staff changes at news organizations, and upcoming IRE and NICAR conferences scheduled for 1999.","keywords":["conferences","campaign","journalism"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nConference Audio Tapes\n\nAudio tapes from the March 1998 sessions at Indiana CAR are now available for ordering from Sound Images, which can be reached at (303) 649-1811. For more information or a downloadable order form, point your browser to Audio tapes from the June 1998 IRE National Conference in New Orleans are also available. For more information, point your browser to wwww.ire.org/resources/conferences/neworleanslaudio.html.\n\nCampaign Finance Information Center\n\nCampaign finance data from 13 states is freely downloadable from the CFIC at www.campaignfinance.org We also have links to sixteen online search engines hosted by other non-profits and state boards of election. We will soon release our \"universal\" online search engine so you can type in a contributor from your state and see where else they are giving.\n\nThe more inclusive this database, the better for everyone. So, if you have state or local campaign data you want included, contact CFIC Coordinator Jack Dolan at jack@nicar.org or (573) 884-1802. The CFIC credits everyone who contributes data.\n\nOther services available on the CFIC Web site include a directory of campaign finance reporters, a collection of campaign finance stories and other suggested readings to generate story ideas, and information on joining the CFIC-L mailing list.\n\nMoving On\n\nAs posted to the IRE-L and NICAR-L mailing lists, investigative reporter Mike Wendland of WDIV in Detroit announced that he will be leaving the station, where he has worked since 1980. Ray Robinson, formerly with The Virginian-Pilot, has become an archive manager at the Associated Press headquarters in New York where he will be responsible for creating and managing a text and photo archive. If you have recently switched locales, let us know of your whereabouts. Send an e-mail to Brent Johnson at bjohnson@nicar.org\n\nConferences: The Next Wave\n\nInformation on upcoming IRE and NICAR national and regional conferences is now available on the IRE Web site at www.ire.org/resources/conferences/index.html.\n\nNICAR's national conference, co-sponsored by the Boston Globe, will be held March 11-14, 1999 in Boston, Mass. IRE's national conference, co-sponsored by the Kansas City Star and KCTV, will be held June 3-6, 1999 in Kansas City, Mo. Downloadable registration forms and other information are posted on the site.\n\nFor information on other events-including boot camps, on-the-road training, and an IRE regional conference to be held Sept. 9-12, 1999 in Los Angeles - point your browser to www.ire.org/calendar.html.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-conference-audio-tapes","slug":"1998-august-conference-audio-tapes","url":"https://dwillis.github.io/uplink/article/1998-august-conference-audio-tapes.html","year":1998,"month":"August","monthNum":8,"headline":"Conference Audio Tapes","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Audio recordings from the March 1998 Indiana CAR conference and June 1998 IRE National Conference in New Orleans are now available for purchase through Sound Images and IRE's website.","keywords":["audio","conference","recordings"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Audio tapes from the March 1998 sessions at Indiana CAR are now available for ordering from Sound Images, which can be reached at (303) 649-1811. For more information or a downloadable order form, point your browser to Audio tapes from the June 1998 IRE National Conference in New Orleans are also available. For more information, point your browser to wwww.ire.org/resources/conferences/neworleanslaudio.html.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-campaign-finance-information-center","slug":"1998-august-campaign-finance-information-center","url":"https://dwillis.github.io/uplink/article/1998-august-campaign-finance-information-center.html","year":1998,"month":"August","monthNum":8,"headline":"Campaign Finance Information Center","kicker":"","author_name":"","author_title":"","affiliation":"CFIC (Campaign Finance Information Center)","summary":"The Campaign Finance Information Clearinghouse offers freely downloadable campaign finance data from 13 states and links to 16 additional search engines, with plans to launch a universal search tool allowing users to track contributors across states.","keywords":["campaign","finance","database"],"topics":["Government & Politics","Data & Databases","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Campaign finance data from 13 states is freely downloadable from the CFIC at www.campaignfinance.org We also have links to sixteen online search engines hosted by other non-profits and state boards of election. We will soon release our \"universal\" online search engine so you can type in a contributor from your state and see where else they are giving.\n\nThe more inclusive this database, the better for everyone. So, if you have state or local campaign data you want included, contact CFIC Coordinator Jack Dolan at jack@nicar.org or (573) 884-1802. The CFIC credits everyone who contributes data.\n\nOther services available on the CFIC Web site include a directory of campaign finance reporters, a collection of campaign finance stories and other suggested readings to generate story ideas, and information on joining the CFIC-L mailing list.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-moving-on","slug":"1998-august-moving-on","url":"https://dwillis.github.io/uplink/article/1998-august-moving-on.html","year":1998,"month":"August","monthNum":8,"headline":"Moving On","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Investigative reporter Mike Wendland is departing from WDIV in Detroit after 24 years, while Ray Robinson has taken a position as archive manager at AP headquarters in New York.","keywords":["reporter","departure","positions"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Moving On\n\nAs posted to the IRE-L and NICAR-L mailing lists, investigative reporter Mike Wendland of WDIV in Detroit announced that he will be leaving the station, where he has worked since 1980. Ray Robinson, formerly with The Virginian-Pilot, has become an archive manager at the Associated Press headquarters in New York where he will be responsible for creating and managing a text and photo archive. If you have recently switched locales, let us know of your whereabouts. Send an e-mail to Brent Johnson at bjohnson@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-august-conferences-the-next-wave","slug":"1998-august-conferences-the-next-wave","url":"https://dwillis.github.io/uplink/article/1998-august-conferences-the-next-wave.html","year":1998,"month":"August","monthNum":8,"headline":"Conferences: The Next Wave","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR have announced upcoming national and regional conferences for 1999, including NICAR's conference in Boston in March and IRE's conference in Kansas City in June, with registration information available online.","keywords":["conferences","training","journalism"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Information on upcoming IRE and NICAR national and regional conferences is now available on the IRE Web site at www.ire.org/resources/conferences/index.html.\n\nNICAR's national conference, co-sponsored by the Boston Globe, will be held March 11-14, 1999 in Boston, Mass. IRE's national conference, co-sponsored by the Kansas City Star and KCTV, will be held June 3-6, 1999 in Kansas City, Mo. Downloadable registration forms and other information are posted on the site.\n\nFor information on other events-including boot camps, on-the-road training, and an IRE regional conference to be held Sept. 9-12, 1999 in Los Angeles - point your browser to www.ire.org/calendar.htmL.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-discovering-intersections","slug":"1998-september-discovering-intersections","url":"https://dwillis.github.io/uplink/article/1998-september-discovering-intersections.html","year":1998,"month":"September","monthNum":9,"headline":"DISCOVERING INTERSECTIONS","kicker":"","author_name":"Jennifer LaFleur","author_title":"","affiliation":"San Jose Mercury News","summary":"Geographic Information Systems (GIS) and mapping software enable journalists to visualize complex data sets and uncover patterns that would be invisible in raw data, such as crime trends or building permits in mudslide-prone areas.","keywords":["mapping","data","GIS"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["GIS","ArcInfo","ArcView"],"full_text":"Although mapping can be used to visually enhance any story, in some cases it's the key to the story. Geographic Information Systems (GIS) allows you to map one or more sets of raw data, which can show intersections between data sets or clusters in one data set that otherwise you would not be able to find.\n\nMost of you have used a database manager to look at data in many ways. With geographically based information, that's easy to do - if for example you're dealing with county names and know where those counties are. But what if your data is listed by Census tract? You probably wouldn't look at a list of tract numbers and say: \"Hey, look there at tracts 5001.02 and 5002.03. Isn't that interesting.\" Seeing the same information on a map makes the data much more powerful.\n\nWhen Carol Napolitano of the Omaha World-Herald needed to look at trends in more than 300,000 crime incidents for the last seven years, she used a map. \"The fastest way to see is visually,\" Napolitano said.\n\nFlowing foundations\n\nBefore El Nino hit California, the San Jose Mercury News wanted to look for potential mudslide problems in the Santa Cruz Mountains. By overlaying permit data and geological data on one map, our story addressed the fact that many new building permits have been issued in areas susceptible to landslides and debris flows.\n\nGetting the maps of the slides and flows was easy (well, sort of). We were lucky in that the U.S. Geological Survey uses ArcInfo mainframe software to do its mapping. ArcInfo is compatible with ArcView, the mapping software the San Jose Mercury News uses.\n\nThe San Jose Mercury News also obtained a database of building permits since 1985 from the Santa Cruz County planning department. We would have loved to have gone back to 1982, but the county started computerizing its records in 1985. The building permit data contained addresses for all the building permits.\n\nUsing a technique called \"geocoding\" (as opposed to \"Gee, coding!\" that programmers say when they get really excited about something they've done), mapping software converts addresses to latitudes and longitudes and, eventually, to points on a map.\n\nJennifer LaFleur of the San Jose Mercury News describes overlaying building permit data with geological mudslide data to predict potential trouble areas and talks to other mapping practitioners.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-the-daily-commute","slug":"1998-september-the-daily-commute","url":"https://dwillis.github.io/uplink/article/1998-september-the-daily-commute.html","year":1998,"month":"September","monthNum":9,"headline":"THE DAILY COMMUTE","kicker":"","author_name":"Neill A. Borowski","author_title":"Reporter","affiliation":"The Philadelphia Inquirer","summary":"A journalism team created a comprehensive map of average daily traffic (ADT) data across the Philadelphia metropolitan area by collecting vehicle count information from state transportation departments, demonstrating how geographic visualization can transform overlooked government datasets into useful reporting tools.","keywords":["traffic","mapping","data"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["ArcView","CD-ROM"],"full_text":"\"Ca-chunk, ca-chunk, ca-chunk.\" Cars and trucks travel over the black hose stretched across the road. Each \"ca-chunk\" registers on a roadside counter as the government tallies the vehicles using the road each day.\n\nThe data are loaded into databases at state departments of transportation or county planning departments - and adjusted for the number of axles. Fascinating information, but it is most often used only by the departments themselves for proposing future road projects or by marketers to decide where to locate the latest fast-food restaurants.\n\nEarlier this year we decided to do a project looking at the daily commute. For several years, I had collected snippets of Average Daily Traffic (ADT) information from a variety of sources. I had pieces from various marketing reports and text files from online DOT databases. What I wanted to do, however, was pull together a comprehensive look at ADT in the Philadelphia metropolitan area. A map was the logical choice to convey the commute to readers. (The special section, called \"Drive Time,\" was published on May 31, 1998.)\n\nTraffic averages\n\nThe result was a map loaded with about 600 different points. Each point represented a spot on a road or highway and showed the average daily traffic. I produced the early map with ArcView and passed the project along to our graphics department, which also uses ArcView. Graphics artist Matthew Ericson broke down the points into five different categories, ranging from less than 10,000 cars per day to more than 75,000 cars per day. We acquired the data from the Delaware Valley Regional Planning Commission in Philadelphia. The commission traffic specialists' first reaction was: \"What intersections are you interested in?\" Our answer was the usual CAR response: \"We want the whole database.\" The planners viewed the request as odd, but after some negotiating they shared the data. The result was an illuminating map. One marketing firm in the Philadelphia suburbs eagerly ordered several copies of the map because they had never been able to capture the whole region on one map before.\n\nCoordinated accidents\n\nWhile the project, we thought, was interesting and useful, the most exciting mapping aspect came in the use of latitude and longitude coordinates provided with the file. I brought the lat/lon coordinates in as an added event theme in ArcView .dbf file) and overlaid them on our regional streets file. We found the coordinates were amazingly accurate, matching perfectly with the street file.\n\nLesson learned: Always ask if the file has lat/lon coordinates built in. Shortly after the \"Drive Time\" project, we put together a large story looking at where accidents occurred in South Jersey (Pennsylvania won't give exact locations of accidents). This was far more difficult than the Drive Time/ADT project.\n\nThe database file listed accidents by intersection or milepost on highways. That prompted my call to the New Jersey Department of Transportation in which I asked, \"How am I supposed to know just where milepost 27.33 is on Route 70?\" \"Oh,\" the highway guy said, \"you need our straight-line diagrams. We'll send you a CD-ROM.\"\n\nWe found the coordinates were amazingly accurate, matching perfectly with the street file. Lesson learned: Always ask if the file has lat/lon coordinates built in.\n\nThe CD is a compilation of Acrobat .pdf files. You click on a route, and it gives you the breakdown in miles. Click on 26-28, for example, and you can see exactly where milepost 27.33 is located and its nearest cross street.\n\nThe process of finding these sites was far from automated, but the straight-line file made it easier than it might have been. Check with your state DOT to see if it has straight-line diagrams in book or computer form. I passed a copy of the CD along to our graphics department, which was delighted to get it. Seems that they are forever getting accident reports on deadline and trying to figure out how to locate mileposts on highways.\n\nAccident files provided to us by the state did not include lat/lon coordinates. However, a more recent accident file, which the state told us not to use because it had bugs that could affect our findings, did include the coordinates. I managed to look up latitude and longitude for the accident spots we wanted to map. However, when we mapped the coordinates with our streets file, we discovered that many were off a bit. Some were several miles away from where they should have been.\n\nNavigation's convert\n\nDespite some wrinkles, I've come to believe in latitude and longitude as an essential tool. I remember doing a story on some GPS (Global Positioning System) fans a few years ago. One predicted that latitude and longitude would supplant ZIP codes and addresses in the future. Your address would be your latitude and longitude - a universal system. I rolled my eyes a bit, but now I'm wondering whether they were on to something. Yes, I've joined the GPS Cult.\n\nA couple of years ago, we purchased two hand-held GPS receivers. We bought Eagle Explorers, but there are many good brands out there for less than $200. I gave one to a technically astute The Philadelphia Inquirer photographer, Michael Mally. We kept the other one in our CAR department.\n\nThe GPS receiver uses readings from three (out of a total of 24) GPS satellites to calculate the latitude and longitude of the spot where you're standing. The accuracy is said to be within 100 feet.\n\nMally became a navigational convert. Besides the GPS unit, he also uses a laptop computer with Delorme's Street Atlas USA. He's often the photographer who goes up in helicopters to shoot aerials. Mally realizes helicopter is precious - particularly when it runs as high as $600 an hour.\n\nHe routinely uses Street Atlas to find the lat/lon of the points he wants to visit and maintains that this cuts deeply into time in the air. He merely gives the headings to the pilot; then they sit back without having to locate highways or landmarks below to determine if they are on the right track.\n\nThe navigational tool can work in the other direction, too. Mally says he has been flying when they discover a column of smoke. From it's difficult to determine where the smoke is coming from. But, with a lat/lon reading, he can plug the coordinates into his map program and pinpoint the location.\n\nMally also used the handheld GPS unit when shooting the construction of an artificial reef off the Jersey coast (the National Guard was sinking old tanks). The GPS allowed him not only to determine where the reef was being built but also to estimate the reef's length by calculating the difference between the first coordinates and the last.\n\nA GPS future\n\nI believe that one day the Metro Desk will have a stock of GPS units in a drawer. Maybe they'll be common \"pool\" items like company cars and cell phones. Those company cars also might come with the GPS units built in to provide directions to reporters as well as to report coordinates.\n\nOne GPS fan predicted that latitude and longitude would supplant ZIP codes and addresses in the future. Your address would be your latitude and longitude - a universal system. I rolled my eyes a bit, but now I'm wondering whether they were on to something.\n\nA reporter running out to cover a major forest fire would grab a GPS to give accurate coordinates to the graphics department to map the fire. In fact, the report could take coordinates for the perimeter of the fires the map in the next day's paper would be an accurate portrayal of where the fire was burning.\n\nWe were prompted to purchase the GPS units after trying to figure out the site of a major bus accident - on-deadline in a wooded area in South Jersey's Pine Barrens. Even with regular maps open in front of us, we weren't clear where the accident occurred. A photographer or reporter with a GPS unit could have given us lat/lon coordinates, and we could have pinpointed the location.\n\nThe potential uses seem endless: a murder victim is dug up in the woods; an oil truck flips and spills on a back road; a new development is going up in the middle of a farm in a rural area without new roads.\n\nNeill A. Borowski can be reached by email at nborowski@phillynews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-for-a-list-of-local-arcview-users-groups","slug":"1998-september-for-a-list-of-local-arcview-users-groups","url":"https://dwillis.github.io/uplink/article/1998-september-for-a-list-of-local-arcview-users-groups.html","year":1998,"month":"September","monthNum":9,"headline":"FOR A LIST OF LOCAL ARCVIEW USERS GROUPS","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A resource guide for journalists interested in mapping and data visualization, providing references to ArcView user groups and recommended books on cartography, statistics, and visual display of information.","keywords":["mapping","cartography","visualization"],"topics":["Technology"],"technologies":["ArcView","World Wide Web"],"full_text":"FOR A LIST OF LOCAL ARCVIEW USERS GROUPS, GO TO WWW.ESRI.COM.\n\nFOR MORE INFORMATION ON MAPPING, CHECK OUT SOME OF THESE BOOKS RECOMMENDED BY STEVE DOIG AND OTHERS:\n\n\"VISUAL DISPLAY OF QUANTITATIVE INFORMATION,\" BY EDWARD TUFTE, 1992\n\n\"ELEMENTARY STATISTICS FOR GEOGRAPHERS,\" BY GERALD BARDER AND JAMES E. BURT, 1996\n\n\"HOW TO LIE WITH MAPS,\" BY MARK MONMONIER AND H. J. DE BLIJ, 1996\n\n\"COMPUTER-ASSISTED CARTOGRAPHY: PRINCIPLES AND PROSPECTS,\" BY MARK S. MONMONIER, 1982","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-infrastructure-outline","slug":"1998-september-infrastructure-outline","url":"https://dwillis.github.io/uplink/article/1998-september-infrastructure-outline.html","year":1998,"month":"September","monthNum":9,"headline":"INFRASTRUCTURE OUTLINE","kicker":"","author_name":"Jeff Porter","author_title":"Database reporter","affiliation":"Arkansas Democrat-Gazette","summary":"A newspaper used affordable Maptitude mapping software to visualize Little Rock's $400 million infrastructure wish list, converting the city's complex spreadsheet data into an accessible published map showing projects by ward.","keywords":["mapping","infrastructure","data"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["Maptitude","Lotus 1-2-3","Excel","dBase","FoxPro","ArcInfo","ArcView","MAPublisher","Adobe Illustrator"],"full_text":"I found a story in our city government's infrastructure wish list. We reported it because a wish of our own was granted: mapping software. Our choice was Maptitude - among the least expensive mapping software available. But we were pleased with our results. The Arkansas Democrat-Gazette published, on Oct. 12, 1997, a page-wide map showing city wards, major streets and the drainage, bridge and traffic-signal projects that the city planned. We skipped street reconstruction or resurfacing because a much larger map than we could publish would be required. The best part was that we could export our Maptitude file, helping our graphics department create what turned out to be a handsome map. The Public Works Department of Little Rock worked for months preparing a list of infrastructure needs. A detailed file showed exactly where and what types of projects were needed, with estimates of the costs. The total - more than $400 million-made it clear that the city was about to set its infrastructure course for the coming decades. But the thick printout listing about 6,000 projects in hundreds of pages just sat there. We changed that.\n\nWards forward\n\nBefore we could begin to map, we had to examine the data. Here's what we did. The department created its list using a Lotus 1-2-3 spreadsheet. Importing it was a snap. First, we looked at the file using Microsoft Excel, cleaned up the data, and saved the file as a dBase IV file.\n\nThen we went to work. We used Visual FoxPro to take quick looks at how the projects were divided by ward. That way, readers could quickly see how their areas of the city would be affected. We broke down spending by each ward. Then we broke down projects by ward.\n\nWe saw, for example, that in one ward drainage improvement was a vital need. In another, reconstructing streets accounted for 75 percent of the city's planned improvements.\n\nWith that information, we organized the data to show exactly where the major projects were planned.\n\nWe exported some data to Excel, where we created easy-to-understand pivot tables for the reporter to work with. Then, we used Maptitude to inspect visually the city's plans by opening a *.dbf file with the mapping software.\n\nThe city helped. The public works department has a skilled Special Programs division that uses ArcInfo mapping software. It was quite willing to share mapping files with the newspaper, generating ESRI-exported, uncompressed files that Maptitude could easily import.\n\nWe imported ward boundaries, streets and other files in order to create maps showing total costs and percentages for each ward. The map and spreadsheet printouts helped the reporter find the areas of the city he should ask about and visit.\n\nGraphics challenge\n\nBut then what? Maptitude made maps that were just fine for reporters to look at, but maps fit for publication were entirely different. We had to find a way to export the Maptitude files - from a Compaq, no less - to a file that the graphics department's Macintoshes could handle. Our answer was software called MAPublisher. We downloaded and used the limited-time demo version of the software for this project. We exported our files, layer by layer, to ArcView shape files, which MAPublisher imported as a plug-in for Adobe Illustrator. While the files still needed considerable work to make them presentable, we were not starting from scratch. The result: Our detailed stories about infrastructure needs were accompanied by a six-column map showing the pinpointed projects. The large graphic included breakdowns by ward of the total number of projects, how much money was to be spent for each type of project, and the percentage of the money to be spent in each ward. The best reaction came at a meeting of city officials. They asked employees to give them the same detailed information as the newspaper had provided its readers.\n\nJeff Porter can be reached by email at jporter@ardemgaz.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-crimes-of-the-week","slug":"1998-september-crimes-of-the-week","url":"https://dwillis.github.io/uplink/article/1998-september-crimes-of-the-week.html","year":1998,"month":"September","monthNum":9,"headline":"CRIMES OF THE WEEK","kicker":"","author_name":"James Derk","author_title":"","affiliation":"The Evansville Courier","summary":"The Evansville Courier pioneered weekly crime mapping on the web to expand coverage of neighborhood crimes beyond major incidents, partnering with local police to visualize crime data and help residents stay informed about their communities.","keywords":["mapping","crime","journalism"],"topics":["Crime & Justice","Investigative Methods","Technology"],"technologies":["MapInfo","Map Publisher","World Wide Web"],"full_text":"The Evansville Courier (65,000 daily, 125,000 Sunday) was the first newspaper in the world to publish weekly crime maps on the World Wide Web.\n\nI say that not to boast, but instead to emphasize the major benefit of working in a smaller market the ability to map out every crime in an entire city for a whole year and not have the whole town covered by thousands of colored dots.\n\nEven so, our beginnings in mapping were modest. I talked my editor into forking out the $1,295 for Mapinfo for Windows after seeing Jennifer LaFleur of the San Jose Mercury News use the software at NICAR's boot camp several years ago.\n\nExtending coverage\n\nI launched the weekly crime map in an effort to spread more information to residents of our community. When I got my first phone call from an angry realtor that my maps had \"destroyed property values\" in one neighborhood, I knew we were on to something. But we didn't do it just to tick off the realtors.\n\nAnyone who has worked the city desk knows the phone call. \"My neighbor's garage was broken into last night, and someone took their Snapper lawn mower and their chainsaw, nearly new. How come there was nothing in the paper?\"\n\nThere was nothing in the paper because newspapers and the people who run them don't care about neighborhood crime. Sure, we cover the bank robberies and murders. But we don't cover the burglaries, thefts, vandalized mailboxes and other parts of daily life in a community.\n\nThe crime map helps us do that. We can't print a story in the paper every time some kid kicks in a yard barn and takes the mower. We can at least give our readers a dot on a map and maybe help prevent some crimes in the process.\n\nQuid pro cop\n\nHow is this done? In 1995, Art Gann, Evansville's chief of police, agreed to a trial run of giving The Evansville Courier online access to police blotter information. We started to archive the logs and soon had a month's worth of information. I started to play with Mapinfo and showed the cops what we could do with their boring data.\n\nThe police had a civilian employee whose job it was to stick colored pins into a large map of Evansville that hung on the wall of the detective office. I showed them how we could do that with Mapinfo. We soon struck a deal:\n\nMapinfo is not a beast to be conquered in a few days. If you are going to spend the money for mapping software, include training in the price.\n\nThey would email me a week's worth of data, and I would produce a color map for them every week.\n\nIn case the map could inadvertently identify victim's home, the police had a moment's pause about mapping the location of sex crimes. After a prototype map was developed, their fear was allayed because one dot covered an entire city block.\n\nUncontrolled exports\n\nOnce the maps were on the Web, we turned to getting them published in the newspaper. What I thought would take a day or two ended up taking more than a year and a half. Turns out Mapinfo does not export data in a graphical form in a high-enough resolution to be used in the newspaper.\n\nI waited about a year for the next upgrade of Mapinfo because an engineer told me exporting would be improved in Version 4.1. It was, but not enough. I later bought Version 4.5, which also did not help.\n\nWe eventually heard about Map Publisher, an add-on tool from Avenza (www.avenza.com) that for $495 made exporting to our graphics and pagination system possible. Because of shortcomings in both programs, we still can print only in black and white the weekly map in the newspaper.\n\nFiltered arrests\n\nBe careful about the address given by the police. The address given by the police may not be the address where the crime occurred. A woman is assaulted but drives to her parents' home across town. They call police. The report (at least in Evansville) will list the parents' address because that's where the report was taken. Some reports will list the addresses of hospitals and police stations because that's also where the victims reported the crimes. They need to be filtered out and handled differently - or it will look like those areas have major crime sprees. Be careful about theft. Mapping theft can take up lots of time and show nothing. My weekly maps filter out shoplifting because there are hundreds of reports every week. Consider which crimes deserve weekly reporting. Just because you get all the data doesn't mean you use it all. My maps show all felonies and some Class A misdemeanors, such as sexual battery. Be careful about arrests. There are two ways to look at mapping crime. When a burglary report is filed, it's because a crime has taken place. When a prostitution report is filed, it's because an arrest has taken place. Just because a lot of arrests are made in a certain area does not mean that's where the prostitutes are. That just means it's where the police are.\n\nBe prepared to fiddle with the \"street-level maps\" you buy from vendors. Some are old; some are inaccurate. If you live in Washington, D.C., or Detroit, I would expect they are reasonably up to date. But it does take them a while to work their way to Evansville, Indiana. So if someone builds a new subdivision or renames a road for Don Mattingly, then you need to change your database, too. And be prepared for cops who still use old names for roads long changed. Old habits die hard, and the data can be filled with addresses like \"540 Old Highway 41\" or \"540 Business 41 N\" or \"540 Fares Ave.\" - all of which are the same place. Mapinfo is not a beast to be conquered in a few days. If you are going to spend the money for mapping software, include training in the price. It also helps if you know how to use a relational database, such as Access or Paradox, and a spreadsheet for basic tasks.\n\nJames Derk can be reached at (812) 464-7409 or by email at jderk@evansville.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-tech-tip","slug":"1998-september-tech-tip","url":"https://dwillis.github.io/uplink/article/1998-september-tech-tip.html","year":1998,"month":"September","monthNum":9,"headline":"TECH TIP","kicker":"","author_name":"Carl Neiburger","author_title":"","affiliation":"San Jose Mercury News","summary":"This article provides a detailed technical guide for converting GIS maps created in ArcView into publishable formats using Freehand and the shareware program epsConverter.","keywords":["GIS","maps","export"],"topics":["Technology","Investigative Methods"],"technologies":["ArcView","Freehand","epsConverter","Adobe Illustrator","EPS"],"full_text":"It's great to be able to use a GIS program to turn data into a map that you can display on your computer but it's much nicer to be able to share the results with your readers.\n\nUnfortunately, the process of turning a viewable GIS map into a publishable map is anything but easy. It is possible, though, using View 3.0 and Freehand version 7.0 or 8.0, plus a small Mac shareware program called epsConverter.\n\nBefore exporting\n\nBefore I export ArcView maps, I make sure that:\n\nOnly the themes I need are turned on. Otherwise, ArcView will export unnecessary - and possibly confusing - data. I'll have to spend a lot of time later deleting it.\nEvery feature is a unique color or color blend - either line color or fill color. It doesn't matter which color is which as long as it is unique. After I export the file, I can change the colors to whatever I want.\nObjects shown as lines, such as rivers and roads, are simple (single) lines, not filled lines, dotted lines or patterned lines.\nIf the map uses symbols, they are made with simple geographic shapes, such as circles or squares.\nAll text labels are deleted. If they are not, ArcView will convert them on export to *paths* that will have to be deleted.\n\nA suitable map projection is chosen. To do this, I pull down View Properties and select the Projection button. I can select from at least one projection for every state or from several projections of the continental United States.\n\nAfter this, I use File Export and select NEW EPS to save the view. (While ArcView has an option to export to Adobe Illustrator format, it has a bug that renders it useless.) I use a network drive to move it to a Mac, but I could also export it by copying the file to a PC diskette and then inserting the diskette into a Macintosh.\n\nAfter exporting\n\nNeither Freehand nor Illustrator can directly open the EPS file created by ArcView.\n\nBut epsConverter, a $25 Macintosh shareware program, will convert it into a form that both programs can use. epsConverter is available from http://users.aol.com/AvtAge or ftp://users.aol.com/ArtAge.\n\nTo use epsConverter, I drag the EPS file onto the epsConverter icon. If my original file was called MYMAP.EPS, epsConverter creates a new file in the same directory called MYMAP.EPS.art.\n\nFreehand requires lots of memory to handle an EPS file from ArcView, so I generally quit all other Mac applications, select the Freehand program icon and hit Control-I. I then boost the memory up as far as I can - to about 50 megs. (If you can't do this, and if Freehand gives you out-of-memory errors when opening the file, try opening it first in Adobe Illustrator and saving it in Illustrator 5.0 format. Freehand should be able to open the Illustrator file.)\n\nOnce I open the file in Freehand, I have to alternately ungroup the map and cut its contents twice before all the pieces are free. Then I go to the Xtras menu and Colors Name All Colors. Naming all the colors lets me do two things:\n\nI can use the graphic search function to select groups of items by color. If I made all the roads red before exporting them from ArcView, I can search for every red line to find all the roads.\nI can change colors en masse. If I want all the oceans and lakes to be our approved map water color, I just drag the official water color to the right place on the Colors information box.\n\nToo much information\n\nOnce I've done this, I still have a lot of work to do turn the map into a publishable product. ArcView EPS files typically have thousands of extra data points that could choke an EPS printer. I use the Extras Path operations Union and Extras Cleanup Simplify functions to join lines and polygons together and to eliminate as much garble as possible.\n\nSometimes, I simply draw new roads by retracing ArcView's lines. It's the fastest thing to do.\n\nWhen it comes to map symbols, the job is easy: I copy whatever symbols are on the map (remember, I only used simple ones, like squares and circles) and paste them into the \"From\" side of Freehand's find-replace-shape tool. I then paste a replacement shape into the \"To\" side and press the \"Change\" button.\n\nWorking with polygons is harder. Sometimes, ArcView exports what should be a single polygon as a group of splinters. I have to use the join function to reunite them. It often takes Freehand a lot of time to do this. When adjacent census tracts are the same color, I'll frequently join them.\n\nDepending on how complex the map is, the process can take anywhere from an hour to several days. I recently worked three days on a series of five U.S. maps showing health data derived from the 1998 Dartmouth Atlas of Health Care.\n\nBut the result was well worth it: Our readers were able to look at the maps, see for themselves what the story was talking about, and draw their own conclusions.\n\nCarl Neiburger can be reached at (408) 920-5874 or by email at cneiburger@sjmercury.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-on-the-internet","slug":"1998-september-on-the-internet","url":"https://dwillis.github.io/uplink/article/1998-september-on-the-internet.html","year":1998,"month":"September","monthNum":9,"headline":"ON THE INTERNET","kicker":"","author_name":"Sarah Cohen","author_title":"Senior Contributing Editor","affiliation":"NICAR","summary":"A guide to free and paid government databases for accessing employment, economic, and demographic data at local, state, and regional levels, including resources from the Bureau of Labor Statistics, Census Bureau, and Department of Agriculture.","keywords":["employment","databases","economics"],"topics":["Data & Databases","Business & Economy","Investigative Methods"],"technologies":["Internet","World Wide Web","FTP"],"full_text":"Bureau of Labor Statistics: http://stat.bls.gov\nBasic local data is available under Local Area Unemployment Statistics and Current Employment Statistics - States and Areas. The LAUS area is difficult to use. Besides, there's much more if you make some phone calls.\nFor example, the Current Employment Statistics is the best monthly guess on employment. But BLS has two other surveys, conducted by states. The \"202\" program actually counts the names of people covered by employers' unemployment insurance programs. That means it's compiling a list of almost everyone in the state who works for companies with more than five employees. The \"790\" program is a quarterly update of current employment figures. It provides greater detail on the number of companies and their payrolls.\nCensus Bureau's Ferret: http://www.census.gov/main/wwwlaccess.html\nThe Current Population Survey periodically uses \"supplements\" on special programs ranging from poverty and welfare to health insurance coverage. While you can cut up data any way you like, it often takes a day to come back.\nRegional Economic Information System, University of Virginia: http://fisher.lib.Virginia.edu/reis/county.html\nA much better way to get your local income figures than from the Commerce Department. This allows you to download the values for wage and salary earnings, welfare payments and the like going back 30 years.\nBureau of Economic Analysis: http://www.bea.gov\nThe free area of BEA's Web site contains basic regional figures in the \"Regional Indicators\" section. You can also use alternative inflation adjustments with the BEA's \"Gross Domestic Product implicit price deflators.\" These are considered more accurate for some adjustments, like government spending or company profits.\nOnly the Commerce Department could justify keeping all of its good data in a pay-only area, called Stats-USA. The BEA actually gives exclusive rights to its database to a business organization. The business cycle indicators used in preparing the Leading Economic Indicators the most comprehensive database around - are now maintained by a private consultant. To pay, reach the organization at http://www.tcb-indicators.org/subscripinfo/subscripinfo.htm\nAgriculture's State Fact Sheets: http://www.econ.ag.gov/epubs/other/usfact\nThis is one of the more comprehensive views of state economies available and is regularly updated. It also tells you everything you wanted to know about the farms in your state.\nSarah Cohen can be reached by email at sarah@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-upcoming-boot-camps","slug":"1998-september-upcoming-boot-camps","url":"https://dwillis.github.io/uplink/article/1998-september-upcoming-boot-camps.html","year":1998,"month":"September","monthNum":9,"headline":"UPCOMING BOOT CAMPS","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering upcoming journalism boot camps in Columbia, Missouri, including a power boot camp in October 1998 and basic boot camps scheduled for January and May 1999.","keywords":["training","journalism","bootcamp"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"UPCOMING BOOT CAMPS:\n\nOCTOBER 22-25, 1998 POWER BOOT CAMP IN COLUMBIA, Mo. (A BASIC BOOT CAMP SQUEEZING SIX DAYS OF TRAINING INTO FOUR.)\n\nJANUARY 3-8, 1999 - BASIC BOOT CAMP IN COLUMBIA, Mo.\n\nMAY 16-21, 1999 - BASIC BOOT CAMP IN COLUMBIA, Mo.\n\nFOR A COMPLETE LISTING OF UPCOMING IRE AND NICAR EVENTS, INCLUDING BOOT CAMPS, ON-THE-ROAD TRAINING, SEMINARS AND REGIONAL CONFERENCES, VISIT OUR WEB SITE AT WWW.IRE.ORG/CALENDAR.HTML","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-stats-from-the-road","slug":"1998-september-stats-from-the-road","url":"https://dwillis.github.io/uplink/article/1998-september-stats-from-the-road.html","year":1998,"month":"September","monthNum":9,"headline":"STATS FROM THE ROAD","kicker":"","author_name":"Jennifer LaFleur","author_title":"","affiliation":"San Jose Mercury News","summary":"Data journalists can use advanced statistical tools like the Gini coefficient to measure income inequality more effectively than basic descriptive statistics, allowing for meaningful comparisons across regions and time periods.","keywords":["inequality","coefficient","income"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["spreadsheet"],"full_text":"As you work with data more, you'll find that basic descriptives don't always serve your purpose. In this way, data analysis is like home repair. I could use a screwdriver to do most of my projects, but it would be a lot more work and the results would not be as useful. With data, you may need to go beyond the basic and turn to social science or other disciplines for analysis tools as indices.\n\nFor example: When San Jose Mercury News economics reporter Scott Thurm wanted to look at the income gap in Silicon Valley to see if it fit with national trends, simply using percentage breakdowns did not show what he needed to know. So we turned to two tools to look at the data in a different way: percentiles and the Gini coefficient of income inequality.\n\nThe Gini coefficient\n\nFirst of all, it's not every day that you get to use a big word like \"coefficient.\" But other than impressing your nerdy friends, the Gini is a useful tool. This tool was developed by Mr. Gini (Italian economist Corrado Gini) in 1912 to estimate the inequality of incomes and wealth.\n\nSpecial tools are needed to measure income inequality because standard income measurements, such as median, don't give you the whole picture.\n\nLet's consider two income distributions.\nCounty 1\tCounty 2\n100\t18000\n888\t19001\n1000\t19300\n1200\t20000\n48800\t30000\n50000\t31600\n51800\t32000\n70000\t34000\n\nThe median for both of these is $25,000. However, the range is significantly different. County 1 has incomes at extreme ends of the spectrum, while County 2 tends to be in the middle.\n\nBecause I didn't have time to come up with a good tool to measure inequality, and Mr. Gini had already done the work, I used a spreadsheet for the Gini formula, which J.J. Thompson had developed at the University of North Carolina.\n\nThe data used for the Gini is categorical income data such as that provided in Census figures. For example: 500 people in the $10,000 to $14,999 category, and so on. By multiplying the midpoints of each category by the number of people in the category, you can derive the weighted income, and therefore the other parameters.\n\nAccording to J.J., the problem arises in determining the midpoint for the top category of income - the Census doesn't limit the income, nor does it give you the value of the top income. What it does do, however, is provide aggregate income for those $150,000 and higher, which can be divided by the number of people in that category to come up with a midpoint.\n\nThe result of the Gini is one number you can use to compare your county to other counties or to your county over time. Here's the formula:\n\nGini=1-SUM(Xi-Xi)(Yi+Yj)\n\nWhere:\n\nX is the cumulative proportion of recipients\nY is the cumulative proportion of income\ni is a particular income category\nj is i-1 (the previous category)\n\nIncome Midpts. 1250 Tot. Households 526 Weighted Inc. 657500 Total Inc. 113981250 Total Pop. 7734 PropPop 0.06801138 CumPropPop (PropInc X 0.06801138 PropInc 0.00576849 CumPropInc (Y) 0.00576849 (Xi- Xj)(Yi+Yj) 0.0003923\n\nThis is repeated for all the income categories and then totaled so you end up with one number for the county or whatever area you're calculating. You can calculate that same number for other counties to compare geographically or to other years to compare over time.\n\nJennifer LaFleur can be reached by email at jlafleur@sjmercury.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-also-see-lafleurs-article-on-mapping","slug":"1998-september-also-see-lafleurs-article-on-mapping","url":"https://dwillis.github.io/uplink/article/1998-september-also-see-lafleurs-article-on-mapping.html","year":1998,"month":"September","monthNum":9,"headline":"ALSO SEE LAFLEUR'S ARTICLE ON MAPPING","kicker":"","author_name":"Jennifer LaFleur","author_title":"","affiliation":"IRE","summary":"The article references LaFleur's mapping analysis and recommends statistical tools like the Index of Diversity and Percentiles for journalists, along with suggested reading on data literacy and statistical analysis.","keywords":["statistics","analysis","journalism"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"ALSO SEE LAFLEUR'S ARTICLE ON MAPPING THAT STARTS ON PAGE ONE. OTHER ANALYSIS TOOLS DISCUSSED IN LAFLEUR'S HANDOUT (#782 IN IRE's RESOURCE CENTER) INCLUDE THE INDEX OF DIVERSITY, PUNXUTAWNEY PHIL'S WEATHER INDEX, AND PERCENTILES.\n\nSHE ALSO SUGGESTS FURTHER \"LIGHT\" READING:\n\n\"NEWS AND NUMBERS,\" BY VICTOR COHN\n\"HOW TO LIE WITH STATISTICS,\" BY DARRELL HUFF\n\"INNUMERACY: MATHEMATICAL ILLITERACY AND ITS CONSEQUENCES,\" BY JOHN ALLEN PAULOS\n\"A MATHEMATICIAN READS THE NEWSPAPER,\" BY JOHN ALLEN PAULOS\n\"THE JOURNALIST AND THE GINI COEFFIENCT: A STATISTICAL APPROACH,\" BY J.J. THOMPSON","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-handouts-of-the-month-software-hardline","slug":"1998-september-handouts-of-the-month-software-hardline","url":"https://dwillis.github.io/uplink/article/1998-september-handouts-of-the-month-software-hardline.html","year":1998,"month":"September","monthNum":9,"headline":"Handouts of the Month / Software hardline","kicker":"","author_name":"Andy Lehren, Dan Keating","author_title":"","affiliation":"NBC; IBM","summary":"ArcView and MapInfo are the leading GIS software products for journalists, each offering distinct advantages in data integration and accessibility, with additional free alternatives available through government sources and online tools.","keywords":["mapping","software","data"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["ArcView","ArcInfo","MapInfo","Atlas GIS","Excel","Access","Adobe Illustrator","LandView"],"full_text":"The two GIS software products that seem to be best suited to reporters are ArcView and MapInfo. ArcView is the little brother of ArcInfo, the mainframe computer mapping program widely used by government agencies. The biggest advantage of ArcView is the volume of government data that can be obtained for free or for minimal costs. Since it has a Macintosh version, it is also a good system if you need to get maps you've produced over to graphics artists who work on Macs. The Windows ArcView can export Adobe Illustrator (.ai) files, a format that most graphic programs can use. ArcView can also import MapInfo and Atlas* GIS formats and several other formats without an add-on module.\n\nMapInfo's strength is integration with Microsoft and other commercial vendors who sell demographic data used by the marketing industry. It reads native MS Access format and there is an enormous amount of data that can be purchased for use with this product. It does require an add-on module to read Atlas* GIS formats and ArcInfo formats.\n\nThere are a couple of other ways you can do mapping. The latest versions of MS Excel have a mapping add-on. You can also produce maps of various local demographics using the Census Bureau's Tiger online mapping program at http://www.census.gov under Data Tools.\n\nAnother free product is called LandView. You can download it for free from RTK.NET (The Right-to-know Network) at http://www.rtk.net. When you download it, it automatically comes with the Toxic Release Inventory data for your area.\n\nGetting more data\n\nI strongly recommend trying to find the GIS department of your local government. It may be affiliated with the planning division. If they don't have it, you can move up to larger regional government bodies above them. You may have to go to a regional planning agency. The state is sure to have a GIS group.\n\nSince reporters haven't been in the habit of pestering the GIS departments, I've found the officials to be extraordinarily cooperative and helpful. They're excited to be able to share the wonderful maps they've created while toiling in obscurity. This is a great opportunity for reporters.\n\nWhen getting data, make sure to get the explanations of what's there. The term used in GIS is metadata, meaning \"the data about data.\"\n\nThe data you work hard to get is geolayers. It takes great effort to create the polygons and lines for a geolayer. But once you've got a geolayer - for instance, counties in your state you can map every piece of data that comes along with county classifications. The hardest things to get are smaller subdivisions for your target area and zones that are unique, like school attendance boundaries, police grids, precincts, etc. They usually have to be created locally. Mapping geographic files (geolayers) are proprietary in format. There is no generic format, like ASCII or dbf, that everyone can share. The government is pushing a new format called SDTS intended to fill that role.\n\nKEATING WRITES ABOUT SITES FOR DOWNLOADING GEOLAYERS:\n\nCIESIN: THE WEB SITE IS AT HTTP://WWW.CIESIN.ORG, BUT IT'S EASIEST TO GET TO THE MAP GEOLAYERS BY GOING TO THE FTP SITE AT FTP://FTP.CIESIN.ORG. YOU CAN GET MAPS OF CENSUS BLOCKS, BLOCK GROUPS, TRACTS, MUNICIPAL AND \"PLACE\" BOUNDARIES, COUNTIES AND STATES IN FORMAT USABLE BY ATLAS*GIS AND ARCVIEW. MAP INFO REQUIRES AN IMPORT MODULE.\nNATIONAL GEOSPATIAL DATA CLEARINGHOUSE: HTTP://NSDI.USGS.GOV/NSDI\nU.S. FEDERAL GEODATA COMMITTEE: HTTP://MAPPING.USGS.GOV\nBEST POINTER TO FREE GEOLAYERS THAT I'VE FOUND: HTTP://WWW.CAST.UARK.EDU/LOCAL/HUNT\n\nAndy Lehren can be reached by email at alehren@nbc.com. Dan Keating can be reached by email at dtkeats@ibm.net.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-september-bits-bytes-and-barks","slug":"1998-september-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-september-bits-bytes-and-barks.html","year":1998,"month":"September","monthNum":9,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"This journalism newsletter covers available conference audio tapes, campaign finance data resources, staff moves in the industry, and upcoming IRE and NICAR conferences for investigative reporters.","keywords":["conference","campaign","reporting"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nConference Audio Tapes\n\nAudio tapes from the March 1998 sessions at Indiana CAR are now available for ordering from Sound Images, which can be reached at (303) 649-1811. For more information or a downloadable order form, point your browser to www.ire.org/resources/nicar/conferences/indianalaudio.html\n\nAudio tapes from the June 1998 IRE National Conference in New Orleans are also available. For more information, point your browser to www.ire.org/reources.conferences/neworleanslaudio.html.\n\nCampaign Finance Information Center\n\nCampaign finance data from 13 states is freely downloadable from the CFIC at www.campaignfinance.org. We also have links to 16 online search engines hosted by other non-profits and state boards of election. We will soon release an online search engine where you can track a contributors money trail across states.\n\nThe more inclusive this database, the better for everyone. So, if you have state or local campaign data you want included, contact the CFIC coordinators, Jack Dolan (jack@nicar.org) or Cindy Eberting (cindy@nicar.org). They can also be reached at (573) 884-1802. The CFIC credits everyone who contributes data.\n\nOther services available on the CFIC Web site include a directory of campaign finance reporters, a collection of campaign finance stories to generate story ideas, and information on joining the CFIC-L mailing list.\n\nMoving On\n\nAlan Levin is joining USA Today to report on aviation safety issues. Previously, he was a staff reporter at The Hartford Courant and ran the paper's CAR program. His computer-assisted work at The Hartford Courant included reports on speeders who escaped punishment in the courts, drinking water systems that repeatedly violated health standards, and felons who obtained gun permits. He also set up the paper's Intranet data search system.\n\nIf you have recently switched locales, let us know of your whereabouts. Send an email to Brent Johnson at bjohnson@nicar.org\n\nConferences: The Next Wave\n\nInformation on upcoming IRE and NICAR national and regional conferences is now available on the IRE Web site at www.ire.org/resources/conferences/index.html.\n\nThe National Computer-Assisted Reporting Conference, presented by IRE and NICAR along with The Boston Globe, will be held March 11-14, 1999 in Boston. The IRE National Conference, co-sponsored by The Kansas City Star and KCTV, will be held June 3-6, 1999 in Kansas City, Mo. Downloadable registration forms and other information are posted on the IRE Web site.\n\nFor information on other events - including boot camps, on-the-road training, and the IRE regional conference to be held Sept. 9-12, 1999 in Los Angeles - point your browser to www.ire.org/calendar.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-car-covered-planet","slug":"1998-october-car-covered-planet","url":"https://dwillis.github.io/uplink/article/1998-october-car-covered-planet.html","year":1998,"month":"October","monthNum":10,"headline":"CAR-covered planet","kicker":"ENVIRONMENTAL CAR","author_name":"Ken Ward Jr.","author_title":"","affiliation":"Charleston Gazette","summary":"Environmental journalists increasingly rely on computer-assisted reporting (CAR) to analyze large datasets of toxic emissions, hazardous waste locations, and chemical accidents, with tools like databases and mapping software becoming essential for modern environmental coverage.","keywords":["environmental","journalism","reporting"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["dBase","ToxNet","Internet","World Wide Web","spreadsheet"],"full_text":"Environmental journalism has always been a hotbed of computer-assisted reporting. A reporter who wants to analyze thousands of toxic chemical emissions better learn CAR or face spending a couple of years with a notepad and calculator. A newspaper that wants to give its readers the locations of hundreds of hazardous waste dumps might never get the story in print without a database manager and mapping software.\n\nRussell Clemings of The Fresno Bee recently conducted a quick survey on the Society of Environmental Journalists' mailing and found that environmental CAR is alive and well.\n\nCAR growth\n\nMy own experience with environmental CAR dates to 1991, when I first came to the Gazette. At the time, we had only one (old) computer with a decent-sized hard-drive. I learned to do a few crude searches on the National Library of Medicine's ToxNet service and put together the standard \"Top 10\" polluters stories. Later, I used ToxNet to compare the potential emissions from a pulp and paper mill proposed here to similar mills across the country.\n\nBack then, I used a copy of dBase that I borrowed from Gazette CAR expert Paul J. Nyden. With a little help from Paul, and a lot of time spent trying to understand the manual, I taught myself how to use it.\n\nMy CAR work got a major boost in the last year. The Gazette bought decent computers for four reporters, including me. Then, I attended the NICAR national conference in Indianapolis in March 1998. There, I got training in spreadsheets, database managers, and the Internet. I also learned a lot more about how other reporters are using computers to improve their environmental journalism.\n\nSince then, I've produced a variety of CAR stories - from daily stories with \"computer-assisted paragraphs\" backing them up to a lengthy investigation of the coal industry's strip mining in West Virginia.\n\nFor example, this summer there was a toxic tort case in federal court in Charleston filed against a Rhone-Poulenc chemical plant for a chemical leak a few years ago. The jury found for the company. But when the verdict came out, the plaintiffs' lawyer - who represented folks who live near the plant and believed they were hurt by the leak claimed his suit had forced the company to clean up its act and operate more safely.\n\nI was skeptical. So, with a few hours of research on the EPA Emergency Response Notification System Web site (http://www.epa.gouldoos/ERNSI), I downloaded local chemical accident reports and showed that the company had continued to have leaks - some even bigger than the one that prompted the lawsuit. The lawyer was just plain wrong. This story wasn't a huge investigation, but it did add punch to our coverage of the trial, and our afternoon newspaper competition didn't have it.\n\nStrip mining bared\n\nI have also found that CAR is the only way to really do long-term environmental reporting projects in any manageable or timely way. Since March, the Gazette has been publishing a series of articles called \"Mining the Mountains\" that examines mountaintop removal strip mining.\n\nUnlike old-time strip mining, mountaintop removal doesn't chip away at hillsides to uncover coal. Instead, it blasts away entire mountaintops to reach coal. Normally, strip mines must be reclaimed to their \"approximate original contour,\" or AOC. If they submit concrete plans for how the flattened land will be used for future economic development, mountaintop removal mines are allowed to ignore that standard and flatten out the land.\n\nHaving talked to environmentalists who follow mining, I suspected that most mountaintop removal mines did not have plans for future development.\n\nI asked the state Division of Environmental Protection what the stated post-mining land plans were for all mountaintop removal mines. They said they didn't know but would have field inspectors go through permits to try to find out. However, they would have to go through dozens of paper files, an exercise that could take months.\n\nMissing data, missing plans\n\nIn the meantime, I huddled with a couple of guys in the DEP computer record-keeping section. They told me they had a computer system that maintained information about surface mines with permits in the state. I got a copy of parts of their database and started doing my own research. I found a number of disturbing trends. First of all, the state didn't even know how many mountaintop removal mines it had authorized. Permit writers and inspectors never bothered to fill in the computer database field that indicated what type of mine they were granting a permit. Even when the state came back to me with the list of mines from paper files, a lot of mines that I found in the computer system were missing.\n\nSecond, I analyzed permitted area and production numbers to determine whether, as environmentalists had told me, mountaintop removal was increasing. Industry officials had argued this wasn't true. But, by simply running a few queries in FoxPro on a state database, I could write a story explaining which side was right.\n\nMost important, of the 134 mountaintop removal mines I identified, most did not submit post-mining development plans when they were given permits by the state.\n\nThe results of my work were two lengthy Sunday features published in May and in August. The articles prompted investigations by the U.S. Office of Surface Mining and by at least one congressman, Rep. Nick J. Rahall, D-W.Va. Already, the state DEP is modifying its database and completing a project to fix its record-keeping problems.\n\nIn September, Gardiner Harris of the Louisville Courier-Journal conducted a similar investigation of mountaintop removal mining in Kentucky. Harris, who used CAR to produce a landmark series earlier this year on black lung disease among coal miners, examined computerized state mining permit databases. He found similar problems, prompting an investigation by OSM officials in Kentucky.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-waste-deep","slug":"1998-october-waste-deep","url":"https://dwillis.github.io/uplink/article/1998-october-waste-deep.html","year":1998,"month":"October","monthNum":10,"headline":"Waste deep","kicker":"CLEAN WATER ACT","author_name":"Mike Lee","author_title":"","affiliation":"Tri-City Herald","summary":"A citizen activist group's threat of Clean Water Act lawsuits against Yakima Valley dairies exposed the region's rapid growth as Washington state's dairy capital and revealed widespread water pollution from manure runoff into irrigation canals. Using state water quality databases, the Tri-City Herald documented high fecal coliform levels proving agricultural pollution in the lower Yakima River.","keywords":["dairy","pollution","water"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Access"],"full_text":"When a citizen activist group threatened 10 Yakima Valley dairies with Clean Water Act suits last fall, Washington state's largest dairy region erupted. Farmers and activists lived in fear of each other. Politicians looked for solutions, and regulators tried to soothe tensions.\n\nUnderneath the bedlam was the real story: After a decade of phenomenal growth, the dairy industry dominated the communities on the lower half of the river. Any threat to the industry was a threat to a way of life and the relative prosperity of arid eastern Washington. Few even realized the Yakima Valley had quietly become the state's dairy capital. A quick check of the U.S. agricultural census showed that fact: The number of dairy cows in the area increased from 27,000 to 68,000 in eight years.\n\nPeople in the industry knew why: Over the last 10 years, population pressures in western Washington and California forced dairy farmers to the dry, less populated stretches of eastern Washington. But they weren't taking care of the cow manure. The runny muck was routinely running into irrigation canals that drained to the river. It had been going on for two decades. The story was as emotional as they come. With the threat of lawsuits, few farmers would talk.\n\nList service\n\nThe state Department of Ecology, however, had all the numbers to validate the Tri-City Herald's four-day series that ran in early March 1998. The agency had been gathering water quality data for several years. The Herald acquired two databases kept by the state to track water quality problems. The lists are mandated by the Clean Water Act and are known as 303(d) and 305(b) for their sections in the act. They show state waters that aren't up to federal standards and also tell what's wrong with the water.\n\nAs my first attempt at using Access, with lots of help from our CAR coordinator David Cuillier, I looked at the state data, zeroing in on water monitoring stations on the lower Yakima River. I found that several spots had more fecal coliform than 90 percent of the state's waters. That proved the poop problem.\n\nWe also isolated key water stations and determined the level and general cause of the pollution, which is largely agricultural. For a broad look at a state's waterways, the 305(b) list is a good start. It's an overall indicator of water quality and - at least in Washington - comes in a much easier format for database searches. Every monitored waterway is tested for a handful of pollutants, including temperature, dissolved oxygen, pH, and fecal coliform. That makes it easy to search for pollutants from a specific industry, such as dairy farms.\n\nStandard usage\n\nObviously, however, even something as specific as coliform has a variety of sources, and those sources probably will be listed generally. The 305(b) list is handy because it shows whether a water segment is clean enough for specific uses, such as fish spawning, swimming or boating. That way, it's possible to search for all the places in an area where fish can't survive. Or where people shouldn't swim.\n\nPollutants in the 305(b) list are also given a rating that indicates the percent of tests in which they exceeded quality standards. That can be used to rank state waters by pollutant. Unfortunately, the sample size is sometimes rather small, so a stream that failed 100 percent of the time may only have been tested two or three times in the last 10 years.\n\nThat's where the 303(d) list comes in. It is more specific and details tests.\n\nStates use the test data to create Total Maximum Daily Load studies that target specific pollution problems and cleanup measures. The only such study completed in Washington is on the Yakima River. It focuses on the long-lived pesticide DDT and sediment, which were both included in the Herald's package.\n\nWashington has had trouble producing enough of those studies to please Northwest environmentalists. The state is now on a 15-year schedule under threat of lawsuit. The 303(d) list is great for looking at specific water sections and finding out who did the tests.\n\nFor all the hours spent searching the databases, the amount of data-crunching results that made it into the final stories was minimal. It was too abstract and technical for the average reader. But knowing how an agency keeps records helped us ask better questions. And having the data allowed us to say the Yakima River was - at least by some parameters - the dirtiest waterway in the state of Washington.\n\nMike Lee can be reached by email at mlee@tri-cityherald.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-mire-on-the-mountain","slug":"1998-october-mire-on-the-mountain","url":"https://dwillis.github.io/uplink/article/1998-october-mire-on-the-mountain.html","year":1998,"month":"October","monthNum":10,"headline":"Mire on the mountain","kicker":"","author_name":"Penny Loeb","author_title":"Reporter","affiliation":"Charleston Gazette","summary":"A journalist describes how public data access and database analysis revealed the extent of mountaintop removal coal mining in West Virginia, uncovering violations and complaints that exposed the environmental damage of the practice.","keywords":["mining","database","violations"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":[],"full_text":"The mountaintop removal strip mining stories are available on the Gazette's Web site at http://wvgazette.com/mining. Ken Ward can be reached by email at kward@wvgazette.com\n\nI've never met an agency official as willing to disseminate data as J.R. Hodel at the West Virginia Division of Environmental Protection. During the last week of February 1997, I began a project on mountaintop removal coal mining in West Virginia, which eventually became \"Shear Madness.\" I had never seen a coal mine, let alone a mountain chopped off. But I blithely faxed a request for all complaints and violations concerning mines to a top DEP official. In less than 24 hours, Hodel called to ask how I would like my data. When I met Hodel, he gladly handed over all the data I wanted, explaining that he believes in absolute public access. (His wife Martha is the daughter of a coal miner and an Associated Press reporter, who has written stories about the mountaintop issue this year.) Two databases were vital to my story. I used the complaints to find the most problematic mines. Then I scoured the permit files on the mines to find who complained. I called them cold; only one person refused to talk to me. Most were glad someone finally cared about their problems with blasting damage, flooding and dust from the mines. In the story, I counted violations and complaints by category, which helped make the case about problems created by such mining.\n\nDownloading gives me quicker access to the primary sources than relying on regional EPA flaks to send me press releases or industry lobbyists to provide their take on the controversy. And keeping the hefty reports on my hard drive instead of in piles on and around my desk has cut down substantially on the cubicle clutter - something every environmental reporter would welcome.\n\nOne of my first questions to the DEP chief of mining and reclamation was: How many acres are being mountaintopped?\n\nAs Ken Ward points out, they didn't know. I flew over the mines with our photographer and a top DEP official. The photographer and I estimated 10 percent of the peaks in the southern coalfields had been decapitated. The DEP official made a higher estimate: percent.\n\nOne day, I was trying to download the maps of mine locations from the DEP site. Lo and behold, there was a database file underlying the map that had every permit, including acreage.\n\nThe database was also supposed to designate mountaintop mines. That field was usually blank. But I added up all acreage for surface permits, so we said about 512 square miles had been surface-mined. I also told a DEP official that the database was missing crucial information.\n\nKen Ward has done a wonderful job of keeping this issue alive, with stories nearly every day. This has been a unique journalistic effort with reporters keeping the focus on this practice.\n\nA searchable database of the complaints and violations used in the story is available at www.wvcoalfield.com designed by Loeb. She is currently writing a book about how coalfield citizens are fighting oppression by the mountaintop removal coal mines. Penny Loeb can be reached by email at ploeb@usnews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-polluters-online","slug":"1998-october-polluters-online","url":"https://dwillis.github.io/uplink/article/1998-october-polluters-online.html","year":1998,"month":"October","monthNum":10,"headline":"Polluters online","kicker":"TOXIC RELEASES","author_name":"Katie Schallert","author_title":"","affiliation":"Commercial-News (Danville, Illinois)","summary":"The Toxic Release Inventory database from the EPA provides journalists with detailed information about chemical emissions from companies, accessible through the EPA website or the more user-friendly Right-To-Know Network for reporting on local polluters.","keywords":["pollution","database","environment"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["Excel","Internet","World Wide Web","FTP"],"full_text":"Polluters online\n\nFor those who think covering the environment means donning hiking boots and bug spray, think again. Environmental stories abound on the Internet, due largely to the Environmental Protection Agency's compulsion to make companies fill out scores of reports.\n\nAlmost a decade ago, the government decided to use these finished forms to reduce pollution. Officials started releasing how much stuff a company emitted into the environment and let public pressure do the rest. Thus came about the Toxic Release Inventory.\n\nCompany profiles\n\nTRI lists how much of every regulated chemical a particular plant releases in a given year and whether the chemicals were emitted to water, air or land. The database is wonderful for reporting who the top polluters are in an area and what exactly is being released into the local environment. The most recent list is from 1996.\n\nTo reach the original database, visit the EPA Web site at www.epa.gov/envirol. Then \"Generate Reports\" from the TRI. A geography search gives you a list of every local company that has shown up even once in the TRI and provides links to their files. If you want all the specifics on a company's record, this is a gold mine, which also includes contact names and numbers for each plant. You can import the company files one by one into a Web-savvy spreadsheet like Excel 97 and manipulate them from there.\n\nAvoid information glut\n\nBefore long, you'll probably note the database is like many things involving the government and the Internet - there is so darn much information.\n\nIf you face common newsroom time constraints and just want the highlights, visit the Right-To-Know Network at www.rtk.net. This site lets you pick from summary to extremely detailed searches and lists all the companies in one file, saving you from importing company by company. You also can have the companies listed in order of emissions.\n\nAnd, best yet, the results can be compiled in comma or tab delimited format, ready to import into any semi-sophisticated spreadsheet or database for further analysis. (An EPA official told me she uses the RTK site after I called her for help with my frustrations at the EPA database page.)\n\nRTK also lets you search by industry in your state or nationally. You can search for the whole state in descending pollution order, giving you a quick feel for where local companies rank. Also useful is more detailed background information about what the TRI does and doesn't include.\n\nFor example, you should note in any story that the report only includes regulated companies and chemicals. Public works and smaller polluters like dry cleaners and diesel trucks are not included.\n\nRanking releases\n\nUnfortunately, both these databases are designed to put a spotlight on individual firms. If you want to know how much stuff was put in your local environment in general, check out the Environmental Defense Fund's Scorecard at www.scorecard.org. This site will tell you the top releases in your county and which pose the greatest threat to human health. It also ranks your county nationally. The drawback is that this database is from 1995, though the designers say they will soon have 1996 numbers.\n\nResponse time\n\nAfter you've crunched the numbers, you are ready to make phone calls. Call the plants or their parent companies and give them a chance to respond. The information is public, so while they might not like a story, most companies will talk to try to get their spin on it. Generally, companies stress the releases in the report are legal.\n\nTo be fair, you may want to note the information is old. So if Company X spent a few million upgrading their system recently, its numbers may already be lower. And always explain larger companies are generally going to have more pollution but also provide more jobs. You can use your spreadsheet to rank companies by pollution per worker.\n\nKatie Schallert can be reached by email at katiejo@soltec.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-political-science","slug":"1998-october-political-science","url":"https://dwillis.github.io/uplink/article/1998-october-political-science.html","year":1998,"month":"October","monthNum":10,"headline":"Political science","kicker":"MOUNTAIN JAM","author_name":"Mary Manning and Steve Kanigher","author_title":"reporter","affiliation":"Las Vegas Sun","summary":"Las Vegas Sun reporters used internet research to cut their investigation time from six months to three months while reporting a five-day series on the federal government's Yucca Mountain nuclear waste project, tracking both political funding and scientific concerns.","keywords":["nuclear","research","Yucca"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Internet","World Wide Web","Email"],"full_text":"As Las Vegas Sun reporters, we collaborated on a five-day series appearing from May 31 until June 4, 1998 on the federal government's Yucca Mountain Project. We cut our research time considerably thanks to computer research.\n\nInstead of more than six months of research, it took us about three months to investigate the politics and the science of the nuclear waste project at Yucca Mountain, 90 miles northwest of Las Vegas.\n\nThe series, \"Science vs. Politics,\" examined both scientific issues and political maneuvers that put Yucca Mountain on the federal government's map as the only site in the nation under study as a high-level nuclear waste repository.\n\nDeep PACground\n\nBackground information on politicians, pro-nuclear lobbyists and anti-nuclear groups were all available at various Web sites through one of five dedicated computers available in the Sun's newsroom. We started with the names of major political players such as Senators Frank Murkowski and Larry Craig, who are leading the congressional pack to put nuclear waste in Nevada.\n\nKanigher did his initial research on where and to whom political funds went by exploring the Internet. He tracked campaign contributions, nuclear PAC monies and how much money had been spent on the nuclear waste management project.\n\nBy starting with individual senators and congressmen, we found nuclear PACs that contributed to their campaigns, then tracked the nuclear funds through Federal Election Commission records, all available on the Web.\n\nNuclear waste deposits\n\nManning tracked scientific leads. She researched articles and corresponded with scientists working in New Mexico, Texas and Siberia on deep, hot water possibly rising inside Yucca Mountain and disrupting the engineered repository.\n\nFormer DOE scientist Jerry Szymanski had proposed the theory that Yucca Mountain's seismic and volcanic activity could force water deep within the earth to invade the repository. Although Szymanski is no longer a DOE geologist, he is still presenting his case to oversight committees. Other scientists around the world had weighed in on the Yucca Mountain debate, so Manning contacted them through email.\n\nThere are also Web pages for the Nuclear Regulatory Commission, the agency in charge of licensing a nuclear repository, as well as the independent scientific panel called the Nuclear Waste Technical Review Board.\n\nWith email interviews, the questions can be precise, and follow-up questions and clarifications can be completed in one day, even in Siberia, rather than playing telephone tag or using snailmail over days.\n\nResearch tools\n\nWhile the Las Vegas Sun encourages all its reporters to use the Internet as much as possible, computer research is a tool, not the end-all for getting the story. Besides computer-assisted searches, you have to do a lot of interviews and use other sources to verify information.\n\nFor example, after following the trail of nuclear industry funds into congressional pockets from initial searches conducted on the Web, Kanigher sent the computer-generated information and his questions to the Center for Responsive Politics, a congressional watchdog group. For a $75 fee, the center offered a detailed road map of how the nuclear PACs spent their money.\n\nWe think that computer research is great for checking out Web sites, gathering background information and focusing the scope of research for large projects such as the nuclear waste series.\n\nThe other side of Internet research - after the excitement of finding mountains of information - brought us back to earth. Legwork, plenty of perspiration and old-fashioned, follow-up digging were necessary to do a thorough job.\n\nMary Manning can be reached by email at manning@lasvegassun.com\nSteve Kanigher can be reached at steve@lasvegassun.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-manning-and-kanigher-offer-these-tips-for-starting-internet-searches","slug":"1998-october-manning-and-kanigher-offer-these-tips-for-starting-internet-searches","url":"https://dwillis.github.io/uplink/article/1998-october-manning-and-kanigher-offer-these-tips-for-starting-internet-searches.html","year":1998,"month":"October","monthNum":10,"headline":"Manning and Kanigher offer these tips for starting Internet searches","kicker":"","author_name":"Mary Manning and Steve Kanigher","author_title":"Reporters","affiliation":"Las Vegas Sun","summary":"Las Vegas Sun reporters Manning and Kanigher share internet research strategies used to investigate the Yucca Mountain nuclear waste project, demonstrating how online resources including government databases, scientist contacts, and political funding records reduced their research time from six months to three months.","keywords":["research","nuclear","internet"],"topics":["Investigative Methods","Technology","Government & Politics"],"technologies":["Internet","World Wide Web","Email"],"full_text":"MANNING AND KANIGHER OFFER THESE TIPS FOR STARTING INTERNET SEARCHES:\n\nSTART WITH SOMETHING YOU KNOW: A PERSON'S OR COMPANY'S NAME, PLACE OR A KEY WORD FROM YOUR PROJECT. THIS WILL LEAD TO MAJOR WEB SITES LINKED TO YOUR ORIGINAL REQUEST AND A WEALTH OF INFORMATION, MUCH MORE THAN YOU NEED. BUT A PATTERN OF INFORMATION WILL EMERGE.\n\nLOOK CAREFULLY AT HOW OFTEN THE WEB SITE IS UPDATED. SOME SITES ARE SO OUT-OF-DATE THAT THEY FUNCTION AS HISTORICAL SOURCES, NOT NEWS SOURCES.\n\nONCE YOU HAVE COMPUTER-GENERATED INFORMATION, DOUBLE-CHECK IT WITH OTHER SOURCES. TIE UP LOOSE ENDS THROUGH OTHER SOURCES.\n\nAs Las Vegas Sun reporters, we collaborated on a five-day series appearing from May 31 until June 4, 1998 on the federal government's Yucca Mountain Project. We cut our research time considerably thanks to computer research.\n\nInstead of more than six months of research, it took us about three months to investigate the politics and the science of the nuclear waste project at Yucca Mountain, 90 miles northwest of Las Vegas.\n\nThe series, \"Science vs. Politics,\" examined both scientific issues and political maneuvers that put Yucca Mountain on the federal government's map as the only site in the nation under study as a high-level nuclear waste repository.\n\nDeep PACground\n\nBackground information on politicians, pro-nuclear lobbyists and anti-nuclear groups were all available at various Web sites through one of five dedicated computers available in the Sun's newsroom. We started with the names of major political players such as Senators Frank Murkowski and Larry Craig, who are leading the congressional pack to put nuclear waste in Nevada.\n\nKanigher did his initial research on where and to whom political funds went by exploring the Internet. He tracked campaign contributions, nuclear PAC monies and how much money had been spent on the nuclear waste management project.\n\nBy starting with individual senators and congressmen, we found nuclear PACs that contributed to their campaigns, then tracked the nuclear funds through Federal Election Commission records, all available on the Web.\n\nNuclear waste deposits\n\nManning tracked scientific leads. She researched articles and corresponded with scientists working in New Mexico, Texas and Siberia on deep, hot water possibly rising inside Yucca Mountain and disrupting the engineered repository.\n\nFormer DOE scientist Jerry Szymanski had proposed the theory that Yucca Mountain's seismic and volcanic activity could force water deep within the earth to invade the repository. Although Szymanski is no longer a DOE geologist, he is still presenting his case to oversight committees. Other scientists around the world had weighed in on the Yucca Mountain debate, so Manning contacted them through email.\n\nThere are also Web pages for the Nuclear Regulatory Commission, the agency in charge of licensing a nuclear repository, as well as the independent scientific panel called the Nuclear Waste Technical Review Board.\n\nWith email interviews, the questions can be precise, and follow-up questions and clarifications can be completed in one day, even in Siberia, rather than playing telephone tag or using snailmail over days.\n\nResearch tools\n\nWhile the Las Vegas Sun encourages all its reporters to use the Internet as much as possible, computer research is a tool, not the end-all for getting the story. Besides computer-assisted searches, you have to do a lot of interviews and use other sources to verify information.\n\nFor example, after following the trail of nuclear industry funds into congressional pockets from initial searches conducted on the Web, Kanigher sent the computer-generated information and his questions to the Center for Responsive Politics, a congressional watchdog group. For a $75 fee, the center offered a detailed road map of how the nuclear PACs spent their money.\n\nWe think that computer research is great for checking out Web sites, gathering background information and focusing the scope of research for large projects such as the nuclear waste series.\n\nThe other side of Internet research - after the excitement of finding mountains of information - brought us back to earth. Legwork, plenty of perspiration and old-fashioned, follow-up digging were necessary to do a thorough job.\n\nMary Manning can be reached by email at manning@lasvegassun.com Steve Kanigher can be reached at steve@lasvegassun.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-daily-nuclear-feed","slug":"1998-october-daily-nuclear-feed","url":"https://dwillis.github.io/uplink/article/1998-october-daily-nuclear-feed.html","year":1998,"month":"October","monthNum":10,"headline":"Daily nuclear feed","kicker":"PLANT STATUS REPORTS","author_name":"Tom Henry","author_title":"","affiliation":"The (Toledo) Blade","summary":"The NRC's public website provides daily nuclear reactor status reports that journalists can use to identify story opportunities when plants operate at reduced power or shut down unexpectedly, as demonstrated by a Fermi II plant shutdown in Michigan.","keywords":["nuclear","reporting","monitoring"],"topics":["Investigative Methods","Technology","Public Safety"],"technologies":["Internet","World Wide Web"],"full_text":"One of the more useful Web sites I've come across when looking for a quick daily story or an idea for a long-range project is the one maintained by the U.S. Nuclear Regulatory Commission's office of public affairs (http://www.nrc.gov/OPA/). In addition to glancing through the latest press releases, each day you can spot-check the level at which each of America's 110 nuclear reactors are operating. Information about the most recent 24-hour period is usually posted by 7 a.m. EST, which means you can do a routine check as you're getting settled with your morning coffee. Simply click on \"Plant Status\" and scroll to the nuclear power stations in your region.\n\nRoutine checks\n\nThis may seem like a trivial pursuit. You're likely to find most of the plants operating at or near capacity. But, if the plant is operating at, say, 26 percent of its power, or if it is shut down when you know it isn't off-line for an ordinary refueling, a red flag should go up.\n\nYou likely find a brief narrative on log, probably only a few words in NRC jargon. Either way, go back to the home page and click on \"Event Reports\" to see if there is more.\n\nAt the minimum, finding your plant operating at reduced power or not at all is enough to tip you off that something isn't right and should be checked out. At least call the NRC regional office and the utility.\n\nThis scenario occurred when I went online the morning of July 20 and found one of the nuclear power cover-Detroit Edison Co.'s Fermi II plant in Michigan's Monroe County - was shut down after operating temporarily at reduced power the night before. I wrote a 10-inch story by finding out the plant had been taken off-line manually by control room operators who noticed an unexplained fluctuation in power. The facility stayed shut down for a couple of days until the apparent problem was fixed.\n\nA blockbuster? No, but it was one of those nifty little stories your competition might not pick up on. Or, it was the kind of thing you might take note of if you're keeping track of how often your plant is being shut down unexpectedly.\n\nNRC Updates\n\nJan Strasma, senior public affairs officer at the NRC's Midwest regional office in Lisle, Ill., said the daily plant status reports get the most hits of anything on the agency's Web site. Journalists are only a fraction of the users; many are tracking patterns for the financial community.\n\nStrasma, who has been involved with the NRC's Web site since its inception, said the agency gathers information about each plant in the wee hours of the morning while most of the country is asleep.\n\nPlant status reports were initially circulated only within the agency. They eventually were posted on an electronic bulletin board, then an electronic mailing list and, finally on the NRC's Web site.\n\nPrice of free information\n\nUtilities aren't always as thrilled about having the status of their plants made instantly available to the general public.\n\nRichard Wilkins, spokesman for FirstEnergy Corp., explained that it can put a utility at a disadvantage when it is forced to buy bulk electricity on the open market, especially in the summer when there is a shortage because of the amount being used to run air conditioners.\n\nIf they see that a utility's nuclear plant is or operating at reduced power, potential suppliers get bargaining leverage in a flash. \"It affects your negotiating position,\" Wilkins said.\n\nFirstEnergy is based in Akron, Ohio, and has two subsidiaries-Toledo Edison Co. and Cleveland Electric Illuminating-licensed to operate northern Ohio's Davis-Besse and Perry 1 nuclear plants, respectively.\n\nThe parent company experienced a temporary shortage last summer after a June 24 tornado ripped apart Davis-Besse's electrical switchyard, knocking down 11 distribution towers and causing the plant to lose its off-site power.\n\nBy clicking on the NRC's \"Selected Reports\" category, I read a 34-page report that the agency's inspection team had filed on the matter before the hard copy had made it to the utility's headquarters. I know that because Wilkins told me at the time that they were still waiting for it to come in via overnight mail. By finding it on the NRC's Web site, albeit with Strasma telling me where to look, I showed the utility spokesman where he could find it so that he could make a company statement that day.\n\nMade to order\n\nStrasma defends the agency's decision to post some of the information it collects almost as soon as it gets it. Obviously, some ultra-sensitive information may still be guarded closely.\n\nBut if you had scanned across the NRC Web site recently, you could have found out about a draft report the agency is circulating concerning the pros and cons of stockpiling potassium iodide pills for the general public in the event of a nuclear accident. You could have found out if a nuclear plant in your area is one of 21 plants the agency has cited for failing to move quickly enough to replace faulty fire barriers called Thermo-Lag 330-1. (Years ago, the nuclear industry spent $58 million on that product, years before the government charged the manufacturer with falsifying lab tests.) Or you could have found out the agency's progress on auditing computers at nuclear plants as part of the government's Year 2000 program.\n\n\"Why go to a Web site?\" Strasma asked. \"Because information is fresh and different. If it's not, you're not going to go there.\"\n\nTom Henry can be reached at (419) 724-6079 or by email at parkerpapa@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-jobs-and-online-car-projects-notices","slug":"1998-october-jobs-and-online-car-projects-notices","url":"https://dwillis.github.io/uplink/article/1998-october-jobs-and-online-car-projects-notices.html","year":1998,"month":"October","monthNum":10,"headline":"Jobs and Online CAR Projects notices","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The Oregonian and WNBC-TV are hiring a reporter and investigative producer respectively, while IRE maintains a resource page showcasing computer-assisted reporting projects and award-winning journalism stories.","keywords":["jobs","reporting","resources"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"JOBS:\n\nFOR RECENT POSTINGS OF JOBS - INCLUDING A REPORTER TO JOIN THE OREGONIAN'S ENVIRONMENT & NATURAL RESOURCES TEAM AND AN INVESTIGATIVE PRODUCER TO JOIN WNBC-TV IN NEW YORK CITY - POINT YOUR BROWSER TO WWW.IRE.ORG/JOBS\n\nONLINE CAR PROJECTS: TO VIEW A LISTING OF LINKS OF RECENT COMPUTER-ASSISTED REPORTING STORIES POSTED ON THE WEB, POINT YOUR INTERNET BROWSER TO WWW.IRE.ORG/RESOURCES/CONFERENCES/TRAINING/CARPROJECTS.HTML THE SITE INCLUDES A DESCRIPTION OF THE STORIES AS WELL AS LINKS TO IRE AWARD WINNERS. IF YOU WOULD LIKE TO SEE A STORY ADDED TO THE LIST OF LINKS, SEND AN EMAIL TO BRENT JOHNSON AT BJOHNSON@NICAR.ORG","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-environmental-data","slug":"1998-october-environmental-data","url":"https://dwillis.github.io/uplink/article/1998-october-environmental-data.html","year":1998,"month":"October","monthNum":10,"headline":"Environmental data","kicker":"ON THE INTERNET","author_name":"Ken Ward Jr.","author_title":"reporter","affiliation":"Charleston (W.Va.) Gazette","summary":"An overview of how computer-assisted reporting (CAR) is transforming environmental journalism, providing analyses of toxic emissions and hazardous waste locations that would otherwise be unmanageable.","keywords":["environmental","CAR","journalism"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["dBase","ToxNet","Internet","World Wide Web"],"full_text":"Environmental journalism has always been a hotbed of computer-assisted reporting. A reporter who wants to analyze thousands of toxic chemical emissions better learn CAR or face spending a couple of years with a notepad and calculator. A newspaper that wants to give its readers the locations of hundreds of hazardous waste dumps might never the story in print without a database manager and mapping software.\n\nRussell Clemings of The Fresno Bee recently conducted a quick survey on the Society of Environmental Journalists' mailing and found that environmental CAR is alive and well.\n\nCAR growth\n\nMy own experience with environmental CAR dates to 1991, when I first came to the Gazette. At the time, we had only one (old) computer with a decent-sized hard-drive. I learned to do a few crude searches on the National Library of Medicine's ToxNet service and put together the standard \"Top 10\" polluters stories. Later, I used ToxNet to compare the potential emissions from a pulp and paper mill proposed here to similar mills across the country.\n\nBack then, I used a copy of dBase that I borrowed from Gazette CAR expert Paul J. Nyden. With a little help from Paul, and a lot of time spent trying to understand the manual, I taught myself how to use it.\n\nMy CAR work got a major boost in the last year. The Gazette bought decent computers for four reporters, including me. Then, I attended the NICAR national conference in Indianapolis in March 1998. There, I got training in spreadsheets, database managers, and the Internet. I also learned a lot more about how other reporters are using computers to improve their environmental journalism.\n\nSince then, I've produced a variety of CAR stories - from daily stories with \"computer-assisted paragraphs\" backing them up to a lengthy investigation of the coal industry's strip mining in West Virginia.\n\nFor example, this summer there was a toxic tort case in federal court in Charleston filed against a Rhone-Poulenc chemical plant for a chemical leak a few years ago. The jury found for the company. But when the verdict came out, the plaintiffs' lawyer - who represented folks who live near the plant and believed they were hurt by the leak - claimed his suit had forced the company to clean up its act and operate more safely.\n\nI was skeptical. So, with a few hours of research on the EPA Emergency Response Notification System Web site, I downloaded local chemical accident reports and showed that the company had continued to have leaks - some even bigger than the one that prompted the lawsuit. The lawyer was just plain wrong. This story wasn't a huge investigation, but it did add punch to our coverage of the trial, and our afternoon newspaper competition didn't have it.\n\nStrip mining bared\n\nI have also found that CAR is the only way to really do long-term environmental reporting projects in any manageable or timely way. Since March, the Gazette has been publishing a series of articles called \"Mining the Mountains\" that examines mountaintop removal strip mining.\n\nUnlike old-time strip mining, mountaintop removal doesn't chip away at hillsides to uncover coal. Instead, it blasts away entire mountaintops to reach coal. Normally, strip mines must be reclaimed to their \"approximate original contour,\" or AOC. If they submit concrete plans for how the flattened land will be used for future economic development, mountaintop removal mines are allowed to ignore that standard and flatten out the land.\n\nHaving talked to environmentalists who follow mining, I suspected that most mountaintop removal mines did not have plans for future development.\n\nI asked the state Division of Environmental Protection what the stated post-mining land plans were for all mountaintop removal mines. They said they didn't know but would have field inspectors go through permits to try to find out. However, they would have to go through dozens of paper files, an exercise that could take months.\n\nMissing data, missing plans\n\nIn the meantime, I huddled with a couple of guys in the DEP computer record-keeping section. They told me they had a computer system that maintained information about surface mines with permits in the state. I got a copy of parts of their database and started doing my own research. I found a number of disturbing trends. First of all, the state didn't even know how many mountaintop removal mines it had authorized. Permit writers and inspectors never bothered to fill in the computer database field that indicated what type of mine they were granting a permit. Even when the state came back to me with the list of mines from paper files, a lot of mines that I found in the computer system were missing.\n\nSecond, I analyzed permitted area and production numbers to determine whether, as environmentalists had told me, mountaintop removal was increasing. Industry officials had argued this wasn't true. But, by simply running a few queries in FoxPro on a state database, I could write a story explaining which side was right.\n\nMost important, of the 134 mountaintop removal mines I identified, most did not submit post-mining development plans when they were given permits by the state.\n\nThe results of my work were two lengthy Sunday features published in May and in August. The articles prompted investigations by the U.S. Office of Surface Mining and by at least one congressman, Rep. Nick J. Rahall, D-W.Va. Already, the state DEP is modifying its database and completing a project to fix its record-keeping problems.\n\nIn September, Gardiner Harris of the Louisville Courier-Journal conducted a similar investigation of mountaintop removal mining in Kentucky. Harris, who used CAR to produce a landmark series earlier this year on black lung disease among coal miners, examined computerized state mining permit databases. He found similar problems, prompting an investigation by OSM officials in Kentucky.\n\nSupplementing stories\n\nEven if you're not up given time a big project, you can use CAR to help with environmental reporting. For example, in writing about the ongoing debate over EPA's ozone transport regulations, I have used the agency Web site to download lengthy documentation about the issue.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1998-october-recalled-emissions","slug":"1998-october-recalled-emissions","url":"https://dwillis.github.io/uplink/article/1998-october-recalled-emissions.html","year":1998,"month":"October","monthNum":10,"headline":"Recalled emissions","kicker":"FIRST VENTURES","author_name":"Peter Kendall","author_title":"Reporter","affiliation":"Chicago Tribune","summary":"A journalist discovers through spreadsheet analysis that Illinois had quietly weakened its emissions testing program, reducing failure rates to one-third before launching an expensive new program that would theoretically return to previous levels, revealing state deception.","keywords":["emissions","investigation","spreadsheet"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["spreadsheet"],"full_text":"After spending a couple of good weeks poking into Illinois' auto-emissions testing program, a colleague and I had a little problem. We didn't have a story. Our desks were stacked with tables the agency used to show the program's effectiveness and piles of spreadsheets that had been spit out of the state's database. Our notebooks were overflowing with dozens of interviews. But there was nothing to grab onto, no handhold to pull ourselves up to answer our fundamental questions.\n\nAnd the questions had seemed so simple. Illinois had a decade of experience with automobile emissions testing and was about to embark on a new program, the most expensive in the nation. Was the program working and would the changes make it better?\n\nSudden siege\n\nThe state's take was that the old program was great and the new one would be even better, allowing them to find three times as many dirty cars.\n\nI recall exactly when the Illinois Environmental Protection Agency decided to take a siege mentality to our reporting, but it got to that sorry stage where all questions-no matter how incidental-had to be submitted in writing, to be followed days later by a written response.\n\nProgress was tedious to say the least. Making matters worse, my partner on the story, Laurie Cohen, took off for a few days, and she was far better than I at extracting information from the state, anyway.\n\nJump-charting CAR\n\nHaving recently finished a CAR course with Sarah Cohen, I decided to create spreadsheets of my own. I can't say I had any great plan. By the end of the day, however, I had a very neat statistical chart of the state's progress, showing how many people had been going through the testing process, how many had been failing at various stages, and how many had been given waivers.\n\nTaped together, the printouts stretched across Laurie's desk, making it look like I did something while she was gone. Scanning my crisp columns, something jumped out at me, like an image you stare at and all of a sudden something pops out in three dimensions.\n\nOver six years, the state had cut its failure rate to one-third of what it had been, and now it was starting up a new program to catch three times as many polluting cars. Cut something to a third, multiply it by three, and you get back to where you started.\n\nFalling in\n\nEverything started to fall into place that random comment from a service repair manager that he doesn't have as much business fixing cars, that anecdotal evidence that people weren't complaining about failing the test anymore. It started to make sense in light of the fact that the state had been dumbing down its testing program. Over the next week or two, this little nugget that seemed to glow so bright on the spreadsheet drove our reporting, leading us to uncover how the state had been quietly dismantling its emissions program while the U.S. Environmental Protection Agency looked the other way. Fact is, the failure rate number could have been derived from tables the state provided. \"Any moron looking at these numbers would have seen this, and we had two morons looking at them,\" my colleague later reflected. My point exactly. Which is why that afternoon putting together the spreadsheet was such a good idea.\n\nPete Kendall can be reached by email at pkendall@tribune.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-upcoming-boot-camps","slug":"1998-october-upcoming-boot-camps","url":"https://dwillis.github.io/uplink/article/1998-october-upcoming-boot-camps.html","year":1998,"month":"October","monthNum":10,"headline":"Upcoming Boot Camps","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering basic and advanced boot camp training programs in Columbia, Missouri throughout 1999, with additional events listed on their website calendar.","keywords":["training","bootcamp","journalism"],"topics":["Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"UPCOMING BOOT CAMPS:\n\nJANUARY 3-8, 1999 - BASIC BOOT CAMP IN COLUMBIA, Mo.\n\nMAY 10-13, 1999 - ADVANCED BOOT CAMP IN COLUMBIA, Mo.\n\nMAY 16-21, 1999 - BASIC BOOT CAMP IN COLUMBIA, Mo.\n\nFOR A COMPLETE LISTING OF UPCOMING IRE AND NICAR EVENTS, INCLUDING BOOT CAMPS, ON-THE-ROAD TRAINING, SEMINARS AND REGIONAL CONFERENCES, VISIT OUR WEB SITE AT WWW.IRE.ORG/CALENDAR.HTML","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-gleaning-cleanliness","slug":"1998-october-gleaning-cleanliness","url":"https://dwillis.github.io/uplink/article/1998-october-gleaning-cleanliness.html","year":1998,"month":"October","monthNum":10,"headline":"Gleaning cleanliness","kicker":"HANDOUT OF THE MONTH","author_name":"Penny Loeb","author_title":"","affiliation":"U.S. News and World Report","summary":"Water quality in U.S. streams, rivers, and lakes has declined over the past decade, with agriculture being the leading source of pollution including sediment, nutrients, bacteria, and pesticides. The article provides resources for finding water quality information through state 305(b) reports, EPA websites, and NPDES discharge violation databases.","keywords":["pollution","water","agriculture"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["FTP","Internet","World Wide Web"],"full_text":"Chances are the streams, rivers, lakes and estuaries in your area are more polluted than you thought. In fact, the quality of the nation's water bodies has declined in the past decade. The trouble with water is that it mostly looks okay to the naked eye. The only visible signs of trouble are dead fish, raw sewage, and a horrible smelling toxic spill.\n\nThose were the kinds of problems that won passage of the Clean Water Act 26 years ago. Today's troubles are less obvious, but equally important. According to the Environmental Protection Agency, the leading pollutants are sediment, nutrients (phosphorous and nitrogen), bacteria, pesticides, habitat alterations and metals. The leading source, by far, is agriculture. Sewage plants, urban runoff, industry and physically altering the stream are other, relatively equal, causes.\n\n305(b) Report: The best place to start is with your state's 305(b) report. These are issued every other year and are required under the Clean Water Act. Whatever agency that oversees water pollution will have it. These are chock-full of statistics. Most helpful are each state's assessment of its water problems and the list of impaired rivers and streams.\n\nEvery two years, the EPA compiles the state reports into the national Water Quality Inventory as a report to Congress. These show how your state stacks up against others. They are always two years behind. In May 1998 the EPA released the 1996 report. To obtain copies, call Barry Bergen at (202) 260-7060.\n\nOne caveat: 305(b) reports are self-reported by the states, and each state measures water quality a bit differently.\n\nEPA Information: Two places for basic water information are the EPA Water site: http://www.epa.gov/OWOW/ and the EPA Watershed Indicators http://www.epa.gov/surf/wil. You can search for the watershed covering your area. Each one has a laundry list of water quality measures, including toxic discharges, fish consumption advisories, and agricultural runoff.\n\nNPDES Discharge Violations: The National Pollutant Discharge Elimination System requires permits and reporting of discharges from \"point sources,\" which can be factories and municipal waste treatment systems. Each discharger is given a limit for pollutants released to rivers and other water bodies. They are supposed to report when the discharge exceeds the limit. Studies have shown that one-fifth of the dischargers routinely violate their limits. See \"Dirty Water Scoundrels\" from the U.S. Public Interest Research Group at http://www/pirg.org.\n\nStates usually put violations into a database and send that to the EPA. For the past decade, the EPA has maintained its data in a cumbersome, hierarchical format - though the database is finally being updated. States have had to use the same format and often go to the EPA to retrieve their own data. You can try to get your state's data in spreadsheet format, which lets you easily find violations. You can also request the data directly from the EPA. The agency has a lot of data available through its Web site, but NPDES data was not in an obvious place. The best Web site is the Right-to-Know Network's at www.rtk.net/www/data/pcs_comp.html, though the data seem to be out of date.\n\nWater Tests: The state environmental agencies, the U.S. Geological Survey and other environmental agencies continually monitor certain streams, rivers, lakes and estuaries. The test results for nutrients, bacteria and sedimentation will help you see how much non-point sources affect your rivers. Non-point sources include fertilizer and manure washed off farm fields and dirt and chemicals washed out of construction areas.\n\nState data is available either from the states or from the EPA. The STORET system is EPA's test repository (http://www.epa.gov/OWOW/STORET/). Again, it's a bit arcane. You can submit a request for specific data. Or you can call STORET at (800) 424-9067 and request a FTP transfer of data.\n\nThe USGS has collected data for many years from 679 sites around the country. You can buy a CD-Rom of the testing results from the USGS in Reston, Va., for $42. (703) 648-5921. Code is DDS-37.\n\nThe USGS has recently finished studies of the first 20 of 59 areas of the country. These encompass about two-thirds of the population. Known as the National Water-Quality Assessment, this program provides the first uniform measures for all areas. See the NAWQA Web site: http://www.ruares.er.usgs.gov/nawqa/studyu.html. The USGS wants to share these studies and will make the scientists available for explanations. They are very knowledgeable.\n\nDrinking Water: About half of public drinking water comes from rivers. Municipal systems treat the water with varying chemicals. However, a number of pesticides make it through because there are no limits for them in drinking water. The Environmental Working Group (http://www.ewg.org), has done several excellent studies of pesticides and nitrates in drinking water. (Nitrate can cause blue-baby syndrome). The Working Group can also provide custom analyses of EPA data and violations by public water systems.\n\nToxic Release Inventory: Though all substances don't have to be reported, this data can still give a picture of what's being released into your rivers. Some of the chemicals discharged are perfectly legal, though potential health threats. The Environmental Working Group analyzed TRI data and found the water bodies receiving the most cancer-causing chemicals. See \"Dishonorable Discharge.\"\n\nFish Consumption Advisories: In 1996, 2,193 advisories against eating fish caught in certain rivers were in effect. For more information see http://www.epa.gov/OST/fishadvice.\n\nReports you can't do without\n\nPresident Clinton's \"Clean Water Action Plan:\" The $2.3 billion, five-year cleanup plan now being considered by Congress and hammered out by the EPA. Call (202) 260-7060.\n\n\"A Prescription for Clean Water: How to Meet the Goals of the Clean Water Act.\" Clean Water Network. (202) 289-2395 http://www.cwn.org.\n\nPenny Loeb can be readed at (202) 955-2640 or by email at ploeb@usnews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-means-to-appends","slug":"1998-october-means-to-appends","url":"https://dwillis.github.io/uplink/article/1998-october-means-to-appends.html","year":1998,"month":"October","monthNum":10,"headline":"Means to appends","kicker":"TECH TIP","author_name":"Justin Mayo","author_title":"Data Manager","affiliation":"NICAR","summary":"NICAR journalists developed a FoxPro solution using the ADIR() function to efficiently append hundreds of identically structured database tables into a single dataset, avoiding repetitive manual work and demonstrated this approach with USDA meat-packing inspection data split across nearly 800 regional files.","keywords":["FoxPro","database","automation"],"topics":["Data & Databases","Technology","Investigative Methods"],"technologies":["FoxPro","dBase","CD-ROM"],"full_text":"Have you ever had to append hundreds of tables together? At NICAR we were faced with such a repetitive task often enough that we came up with a programming solution. Rather than permanently disfiguring ourselves with carpal tunnel syndrome, we turned to one favorite data-processing FoxPro functions-ADIR().\n\nThe ADIR() function makes use of arrays. An array can be thought of as a temporary table. In the case of ADIR(), a temporary array is made up of the contents of a specified directory on your computer. Five pieces of file information are included in an ADIR() array: names, sizes, dates, times and attributes. This information makes up the columns in the array, but the most important for our purposes is the file name. Here's an example of how ADIR() can save your sanity.\n\nWhen we processed the USDA meat-packing inspections database, ADIR() was indispensable. The meat-packing data arrived on a CD-ROM in dBase format but were separated into nearly 800 identically structured tables, broken down by the region. What we wanted, however, was one table for the entire country. Here's what the basic code that appended the USDA tables looked like:\n\ngcOutputDir = 'c:\\usda\\outgoing\\'\ngcSourceDir = 'c:\\usda\\incoming\\'\nUSE (gcOutputDir) + 'all_usda.dbf' in 1\n=ADIR(temparray, (gcSourceDir) + '*.dbf')\nSELECT 1\nFOR i = 1 to ALEN(temparray, 1)\nAPPEND FROM (gcSourceDir) + (temparray(i,1))\nENDFOR\n\nExplaining the code\n\nThe first two lines of the above code simply set our working directories. The source tables are in the incoming directory. The outgoing directory contains an empty table structure - all_usda.dbf - that will become the USDA database for the entire country. (NOTE: This code assumes that the structures of your individual source tables are the same. If not, you could lose data when you append them.)\n\nThe third line opens the table structure (all_usda.dbf) into which we will be appending.\n\nThe fourth line is where ADIR() comes into play. The function can handle up to three arguments, though in this case we only use two: temparray and gcSourceDir + '*.dbf'\n\nThe first argument (temparray) is simply the name of the array that will contain the file names, sizes, dates, times and attributes. In this example, the array did not exist, so FoxPro created one that we named temparray. If the array did exist, FoxPro would have modified the size of the array so that the information would fit.\n\nThe second argument (gcSourceDir + '*.dbf') tells FoxPro to look in the source directory and put all the files with a .dbf extension into our array. With all of the file names and attributes stored in this temporary array, we are ready to append the source tables into the outgoing table.\n\nIn the fifth line, we select the outgoing table (all_usda.dbf). Next, we need to tell the program to go through the array one row (or one file name) at a time. This is why the FOR ENDFOR loop is used. In the loop, the 'i' is the counter variable that keeps the loop going. So that it begins with the first file name in the array, we set it to 1. (ALEN(temparray, 1)) tells FoxPro to continue the loop until it hits the end of the array. The '1' in the ALEN function makes sure that we are counting the number of rows in the array (which will be the same as the number of file names).\n\nFinally, in the seventh line, we append the source tables into the outgoing table (all_usda.dbf). The last part of that line - (temparray(i,1)) - is important. The (i,1) singles out the first column in temparray, which, in the case of an ADIR() array, will always be the name of the file. As we loop through the array, we are telling FoxPro to keep appending the source tables until it reaches the end of the array.\n\nJustin Mayo can be reached by email at justin@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-database-library-additions","slug":"1998-october-database-library-additions","url":"https://dwillis.github.io/uplink/article/1998-october-database-library-additions.html","year":1998,"month":"October","monthNum":10,"headline":"Database Library additions","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR's Database Library has expanded its collection with several new government datasets including mortgage disclosure records, Small Business Administration loans, arts grants, tax-exempt organization filings, transportation accident data, and Social Security death records.","keywords":["databases","government","loans"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"NICAR's Database Library has recently added the following data sets: the Home Mortgage Disclosure Act database for 1997, collected by the Federal Reserve Board to track home loans, home-improvement loans and bank-purchased loans\n\ntwo databases from the Small Business Administration: the loans approved under the SBA's main lending program, known as the 7A program; and disaster loans, granted to businesses and homeowners in the wake of hurricanes, earthquakes, floods, and other disasters.\n\nthe National Endowment for the Arts Grants Program database of grants to individuals and organizations from 1987 through 1997\n\nthe IRS' Tax Exempt Organization Business File, which contains basic information from all 990 filings\n\nthe Department of Transportation's database of fatal road accidents from 1975-1997\n\nthe Social Security Administration Master Death Records, kept to track beneficiaries and compiled from sources including families of the recipients, funeral homes, churches, and state health organizations.\n\nTo order or have your questions answered about any of these databases, call the library at (573) 882-0684. Also view the library's collection online at www.nicar.org/data/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-society-of-environmental-journalists-notice","slug":"1998-october-society-of-environmental-journalists-notice","url":"https://dwillis.github.io/uplink/article/1998-october-society-of-environmental-journalists-notice.html","year":1998,"month":"October","monthNum":10,"headline":"Society of Environmental Journalists notice","kicker":"","author_name":"","author_title":"","affiliation":"Society of Environmental Journalists","summary":"The Society of Environmental Journalists offers a comprehensive web resource hosted on IRE's website, providing journalists with topic-organized environmental links, story idea tip sheets, archived content, event calendars, and membership information.","keywords":["journalists","environmental","resources"],"topics":["Health & Environment","Journalism & Newsroom","Technology"],"technologies":["World Wide Web","Internet"],"full_text":"CONNECT TO THE SOCIETY OF ENVIRONMENTAL JOURNALISTS VIA IRE's WWW.REPORTER.ORG WEB SITE, WHICH HOSTS A VARIETY OF JOURNALISM SITES.\n\nSERVICES OF THE SEJ WEB PAGES INCLUDE:\n\nEXTENSIVE LINKS TO OTHER ENVIRONMENTAL WEB SITES, ORGANIZED BY TOPICS (SUCH AS BIOTECHNOLOGY, ENERGY, OR STRATOSPHERIC OZONE LAYER), PLUS A LISTING OF ENVIRONMENTAL DATABASES AND MAILING LISTS\n\nBIWEEKLY TIP SHEETS TO GENERATE STORY IDEAS AND AN ARCHIVE OF PRIOR TIP SHEETS\n\nBACK ISSUES OF THE SEJOURNAL\n\nCALENDAR OF UPCOMING ENVIRONMENTAL EVENTS\n\nSEJ MEMBERSHIP INFORMATION\n\nLINK TO THE NEW WEB SITE OF THE INTERNATIONAL FEDERATION OF ENVIRONMENTAL JOURNALISTS","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-october-bits-bytes-and-barks","slug":"1998-october-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-october-bits-bytes-and-barks.html","year":1998,"month":"October","monthNum":10,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"This newsletter updates on available conference audio tapes, campaign finance data resources, database processing capabilities, and upcoming IRE and NICAR conferences in 1999.","keywords":["conference","data","campaign"],"topics":["Journalism & Newsroom","Technology"],"technologies":["9-track tape"],"full_text":"Bits, Bytes and Barks\n\nConference audio tapes\n\nAudio tapes from the March 1998 sessions at Indiana CAR are now available for ordering from Sound Images, which can be reached at (303) 649-1811. For more information or a downloadable order form, point your browser to www.ire.org/resources/nicar/conferences/indiana/audio.html\n\nAudio tapes from the June 1998 IRE National Conference in New Orleans are also available. For more information, point your browser to www.ire.org/resources/conferences/neworleans/audio.html.\n\nCampaign Finance Information Center\n\nDownloadable campaign finance data from 14 states is free from the CFIC at www.campaignfinance.org. We also have links to 16 online search engines hosted by other nonprofits and state boards of election. We will soon release an online search engine where you can track a contributors money trail across states.\n\nThe more inclusive this database, the better for everyone. So, if you have state or local campaign data you want included, contact the CFIC coordinators, Jack Dolan (jack@nicar.org) or Cindy Eberring (cindy@nicar.org). They can also be reached at (573) 884-1802. The CFIC credits everyone who contributes data.\n\nOther services available on the CFIC Web site include a directory of campaign finance reporters, a collection of campaign finance stories to generate story ideas, and information on joining the CFIC-L mailing list.\n\nMoving on\n\nIf you have recently switched affiliations, let us know of your whereabouts. Send an email to Brent Johnson at bjohnson@nicar.org\n\nPower boost\n\nThe Database Library recently purchased a 3480/3490/3490E tape drive. In the library's experience processing local and federal data sets, it appears that many agencies are moving toward the 3490E format, which is written at 36-track. If you receive data in this format, NICAR can now process it. Call (573) 882-0684 for more information.\n\nConferences abound\n\nInformation on upcoming IRE and NICAR national and regional conferences is now available on the IRE Web site.\n\nThe National Computer-Assisted Reporting Conference, presented by IRE and NICAR along with The Boston Globe, will be held March 11-14, 1999 in Boston. You may register online or download an early registration form at www.ire.org/resources/conferences/boston\n\nThe Investigative Reporters and Editors National Conference, co-sponsored by The Kansas City Star and KCTV, will be held June 3-6, 1999 in Kansas City. You may also register online or download an early registration form at www.ire.org/resources/conferences/kansascity\n\nInformation on upcoming IRE and NICAR national and regional conferences is now available.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-november-census-preparation","slug":"1998-november-census-preparation","url":"https://dwillis.github.io/uplink/article/1998-november-census-preparation.html","year":1998,"month":"November","monthNum":11,"headline":"Census preparation","kicker":"UNITED STATS OF AMERICA","author_name":"Paul Overberg","author_title":"Reporter","affiliation":"USA Today","summary":"This article discusses the uncertainty and political battles surrounding the preparation for Census 2000, focusing on the controversy over using sampling in the census count. It highlights the potential impact on reapportionment and federal fund distribution.","keywords":["Census","sampling","reapportionment"],"topics":["Demographics & Census","Government & Politics","Data & Databases"],"technologies":[],"full_text":"Almost six months ago, I spoke on a panel at the IRE national conference about the wisdom of starting Census 2000 preparations immediately. Remarkably, the shape of Census 2000 is even less clear now. That's why it's even more important to start planning now. Much of the uncertainty stems from a battle that's erupted over the Census Bureau's plan to use sampling to help count the nation. Republicans oppose it, and they control Congress. Democrats favor it, and they control the White House...\n\nIn Washington shorthand, sampling has been reduced to 'like polling.' It is, but what's planned would be much more sophisticated and complicated. There are actually two forms of sampling planned in the Census 2000 count, further complicating the task of explaining it to our editors and readers. At this writing, two appellate courts have ruled that sampling can't be used in lieu of a count-to-the-last-person approach to produce the state totals that will reapportion the House of Representatives. Neither court, however, has forbidden the use of sampling to produce totals for other purposes, such as distributing federal funds or redistricting state legislatures. The Supreme Court's final word is due by spring. Congress and the White House have agreed to make peace by mid-June or shut down a big chunk of the government. The Census Bureau hasn't decided whether or how it would use sampling if it can't use it for everything. The bureau's overseers in the Commerce Department and the White House won't even discuss this prospect because they're still fighting to conduct just one census with built-in sampling...\n\nJust like in 1990, it's quite possible the bureau will produce two sets of numbers: one from the count, the other from the sampling-adjusted count. If this happens, President Clinton could order federal agencies to use adjusted numbers to dole out $100 billion a year...\n\nMuch of the publicity over sampling focuses on how it corrects for the undercount, those missed by the mail survey and door-knockers - most frequently racial minorities, the young, the poor and illegal immigrants...","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1998-november-2000-tuneup","slug":"1998-november-2000-tuneup","url":"https://dwillis.github.io/uplink/article/1998-november-2000-tuneup.html","year":1998,"month":"November","monthNum":11,"headline":"2000 tuneup","kicker":"CENSUS ACHIEVEMENT TEST","author_name":"Steve Doig","author_title":"","affiliation":"Arizona State University","summary":"A preparatory guide for journalists to get ready for analyzing data from the 2000 Decennial Census, emphasizing the need to develop spreadsheet and mathematical skills before the data's release.","keywords":["Census","spreadsheet","preparation"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"The first release of data from the 2000 Decennial Census is more than two years away, but it's none too soon to start preparing. Here's a checklist from someone who wrestled a lot with the 1990 Census, covering what you need to know and what you should do to get ready.\n\nThe basics\n\nMaster your spreadsheet skills: The majority of census analysis can be done using a spreadsheet program such as Excel. Make sure you know how to import a text file, build a spreadsheet, write and copy formulas, sort and filter, export your results into a table that can be published, and other such tasks.\n\nMaster your math skills: The census is numbers, lots of 'em. Memorize my formula for percentage change: NEW divided by OLD, minus one. (You don't believe it's that easy? Try it.) Make sure can be made to work with census data, too.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-november-surfing-data-waves","slug":"1998-november-surfing-data-waves","url":"https://dwillis.github.io/uplink/article/1998-november-surfing-data-waves.html","year":1998,"month":"November","monthNum":11,"headline":"Surfing data waves","kicker":"ON THE INTERNET","author_name":"Jennifer Small","author_title":"","affiliation":"San Antonio Express-News","summary":"This guide helps journalists use the Census Bureau's website to find statistical data for reporting, explaining key resources like news releases, expert contacts, and major data files (STF1 and STF3) while providing tips for accessing information quickly on deadline.","keywords":["census","journalists","data"],"topics":["Demographics & Census","Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","CD-ROM"],"full_text":"Why would a reporter want to access the Census on the Web? Easy. You can find data to add depth and context to your daily reporting or download large data sets for projects. You can keep yourself updated on developments at the Census Bureau and generate story ideas. It's a great resource, a site with statistical information ranging from population and housing to minority-owned businesses and international fertility rates.\n\nConsisting of more than 40,000 pages, the Census Web site can be hard to navigate. Familiarize yourself with some of the tools mentioned below and you will be well on your way to utilizing its full potential.\n\nPress releases, alerts and experts\n\nFollow the News link off the main page to access tools the Bureau offers reporters. The Census Bureau posts all news releases in both chronological and subject order. News releases can also be sent directly to your e-mail. See www.census.gov/pubinfo/www/subscriptions.html. Each news release links to appropriate data sets, tables and reports. The press can also view embargoed news releases. Contact the Public Information Office.\n\nIf you need the names and phone numbers of Census Bureau employees who specialize in various areas of the Census, link to Ask the Experts at http://www.census.gov/contacts/www/contacts.html. Use State Data Centers to find more specifics for your state.\n\nUnderstanding the data\n\nFinding the information you need on deadline can be a challenge. If you are looking for demographic data, be aware of the two major summary data files: STF1 and STF3. The STF1 data are generated form responses to the more general Census short form. This form is sent to everyone and represents the 100 percent count of the population. The STF3 data are generated from the Census long form, which is sent to a sample of the population. The numbers provided are estimates, but the information tends to be more detailed.\n\nThe 1990 Census Lookup Tool http://venus.census.gov/cdrom/lookup provides quick access to every variable in the 1990 Census. From the Lookup, choose the appropriate data set – STF3A provides the greatest level of geographic detail.\n\nSome of the most up-to-date yet overlooked data can be found in Census surveys. (See a complete listing at http://www.census.gov/main/www/surveys.html. Survey topics range from American Housing, Minority-Owned Business and Wildlife Recreation to Business Owners and Nationwide Truck Activity. The Current Population Survey (CPS) at http://www.bls.census.gov/cps/cpsmain.htm is one of the site's most useful surveys. However, because surveys occur more frequently, they tend to lack the geographic specificity of the census. Numbers are typically generated for the nation, states and, sometimes, metropolitan areas.\n\nQuick hits\n\nIf you are on deadline, take advantage of the many lookup tools. In addition to the 1990 Census Lookup, tools are available for the Economic Census at http://www.census.gov/epcd/econ/www/lookups.html. Search County Business Patterns, the Census of Agriculture and U.S. Imports and Exports History.\n\nAs a member of the media, you have access to the great lookup tools in CenStats, located at http://www.census.gov/apsd/www/censtats.html. Call the PIO office for a user name and password. My personal favorite is the Census Tract Street Locator. Type in a street address and determine its census tract. Take that tract number back to the 1990 Census Lookup and get more information on a smaller corner of your city.\n\nUse MapStats at http://www.census.gov/datamap/www/index.html to access county population data, county business pattern data, and Congressional district information.\n\nThe Census tries to provide quick access to the most popular Census variables (population, health insurance, marital status) at Hot Sites for the Media at http://www.census.gov/pubinfo/www/hot_frm.htm. Although you have to navigate through an unwieldy frames interface, this may be the fastest way to view comparative data broken down by race and ethnicity.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-november-census-resources","slug":"1998-november-census-resources","url":"https://dwillis.github.io/uplink/article/1998-november-census-resources.html","year":1998,"month":"November","monthNum":11,"headline":"Census resources","kicker":"HANDOUTS OF THE MONTH","author_name":"Jeff South","author_title":"","affiliation":"Virginia Commonwealth University","summary":"Census data can be leveraged for both quick news stories and long-term investigative projects by combining population estimates with other datasets to reveal trends and disparities across communities. Beyond the Census Bureau, journalists should explore alternative demographic sources and develop relationships with human sources at regional offices and state data centers.","keywords":["census","demographics","trends"],"topics":["Demographics & Census","Investigative Methods","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"Quick hits and the long term\nBy Jeff South\nVirginia Commonwealth University\n\nFrom quick-hit dailies to long-term projects, census data can be used in various ways:\n\n• Every year, the bureau updates population estimates for states, counties and metropolitan areas. These updates are fodder for breaking news and trend pieces.\n\n• The 1990 census and updated estimates are especially valuable when combined with other data, such as the number of crimes, gun permits, pawn shops, supermarkets, public swimming pools, billboards – you name it. Use census data to compute rates, like zoning violations per 1,000 people in different parts of your city. Then use the census to see if the rates are different between low-income and high-income areas or between predominantly minority and predominantly white areas.\n\n• The bureau produces such reference materials as the Statistical Abstract of the United States, the County and City Data Book and USA Counties. These materials include data ranging from health care to the cost of living.\n\n• The bureau continually cranks out studies and statistical briefs that spotlight trends.\n\n• Beyond demographics, the Census Bureau tracks rental housing vacancies, government spending, business development and other trends.\n\nOther demographic data\n\nThe Census Bureau is by no means the only source of demographic data on the Internet. The bureau's page has links to other sites, like the University of Missouri http://www.oseda.missouri.edu, census data at the Lawrence Berkeley National Laboratory http://www.lib.berkeley.edu.GSSI/uc_cdrom.html and the Consortium for International Earth Science Information Network http://www.ciesin.org. Also, check you state government's Web page, as well as sites operated by publications like American Demographics http://www.demographics.com, which often posts the data used in its articles.\n\nDevelop human sources, too – not just at the Census Bureau but at its regional office nearest you, and at your state data center. The centers, usually affiliated with a state government agency or university, help distribute and analyze census information.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-november-quick-hits-and-the-long-term","slug":"1998-november-quick-hits-and-the-long-term","url":"https://dwillis.github.io/uplink/article/1998-november-quick-hits-and-the-long-term.html","year":1998,"month":"November","monthNum":11,"headline":"Quick hits and the long term","kicker":"","author_name":"Jeff South","author_title":"reporter","affiliation":"Austin American-Statesman","summary":"Census data offers journalists multiple storytelling angles, from annual population updates to trend analysis comparing demographic patterns across geographic and socioeconomic lines. Beyond the Census Bureau, numerous public sources and human contacts can provide demographic data for investigations.","keywords":["census","demographics","data"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":["Internet","World Wide Web","CD-ROM"],"full_text":"From quick-hit dailies to long-term projects, census data can be used in various ways:\n\n• Every year, the bureau updates population estimates for states, counties and metropolitan areas. These updates are fodder for breaking news and trend pieces.\n\n• The 1990 census and updated estimates are especially valuable when combined with other data, such as the number of crimes, gun permits, pawn shops, supermarkets, public swimming pools, billboards – you name it. Use census data to compute rates, like zoning violations per 1,000 people in different parts of your city. Then use the census to see if the rates are different between low-income and high-income areas or between predominantly minority and predominantly white areas.\n\n• The bureau produces such reference materials as the Statistical Abstract of the United States, the County and City Data Book and USA Counties. These materials include data ranging from health care to the cost of living.\n\n• The bureau continually cranks out studies and statistical briefs that spotlight trends.\n\n• Beyond demographics, the Census Bureau tracks rental housing vacancies, government spending, business development and other trends.\n\nOther demographic data\n\nThe Census Bureau is by no means the only source of demographic data on the Internet. The bureau's page has links to other sites, like the University of Missouri http://www.oseda.missouri.edu, census data at the Lawrence Berkeley National Laboratory http://www.lib.berkeley.edu.GSSI/uc_cdrom.html and the Consortium for International Earth Science Information Network http://www.ciesin.org. Also, check you state government's Web page, as well as sites operated by publications like American Demographics http://www.demographics.com, which often posts the data used in its articles.\n\nDevelop human sources, too – not just at the Census Bureau but at its regional office nearest you, and at your state data center. The centers, usually affiliated with a state government agency or university, help distribute and analyze census information. You can find a list of the centers at http://www.census.gov/ftp/pub/sdc/www/\n\nIf you need inspiration for census stories, read Who We Are: A portrait of America based on the latest U.S. Census. This 1993 book, by Sam Roberts of The New York Times, shows how to put a face on census data.\n\nSouth's handout, #495 in the Resource Center, was provided when he was with the Austin American-Statesman.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-november-county-population-estimates","slug":"1998-november-county-population-estimates","url":"https://dwillis.github.io/uplink/article/1998-november-county-population-estimates.html","year":1998,"month":"November","monthNum":11,"headline":"County population estimates","kicker":"","author_name":"D'Vera Cohn","author_title":"","affiliation":"The Washington Post","summary":"County population estimates provide valuable demographic breakdowns including age groups, births, deaths, and migration patterns that can fuel reporting on aging populations, immigration, and regional migration trends. The Census Bureau's recently published 1996 estimates by county offer journalists a way to track demographic changes since 1990, though they are less accurate than full census data and should be used cautiously for small counties.","keywords":["demographics","migration","estimates"],"topics":["Data & Databases","Demographics & Census","Investigative Methods"],"technologies":["spreadsheet"],"full_text":"What makes county population estimates useful is not just the population totals, but the underpinnings of those estimates. Included in the \"components of change\" are the bureau's year-by-year estimates of the number of people over 65, people under 65, births, deaths, immigrants and domestic migrants (which means people who moved to any county from any other county in the U.S.). This is good fodder for stories about the growth of the elderly and immigration populations, ups or downs in births (best to double-check those with actual birth figures, usually compiled by health departments), and the appeal of your region to people from other parts of the country.\n\nCensus has recently published 1996 estimates by county of the age, race and Hispanic origin population. Slap them into a spreadsheet and track change since 1990.\n\nThe numbers are not as accurate as the Census, because they are estimates from administrative records rather than results of a door-to-door survey. You should be cautious about using them for very small counties. But they sure beat figures that are nearly a decade old.\n\nAnother useful data set – and about to get somewhat more useful – is the International Revenue Service County Migration files. These records count the number of taxpayers and dependents who moved from one county to another in any given year and include their total and median income.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-november-out-of-date-data","slug":"1998-november-out-of-date-data","url":"https://dwillis.github.io/uplink/article/1998-november-out-of-date-data.html","year":1998,"month":"November","monthNum":11,"headline":"Out-of-date data","kicker":"COMMERCIAL CENSUS UPDATES","author_name":"Paul D'Ambrosio","author_title":"","affiliation":"Asbury Park (NJ) Press","summary":"The author argues for annual census counts instead of the current decade-long gap, highlighting how outdated census data makes local demographic reporting inaccurate and forces journalists to rely on commercial vendor estimates that use varying methodologies.","keywords":["census","demographics","estimates"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":["Claritas"],"full_text":"Note to the next Constitutional Convention: Please make the census an annual event. The Founding Fathers sure didn't have journalists in mind when they came up with the count-everyone-once-a-decade concept, confounding our obsessive need for total accuracy.\n\nTry answering these questions on a municipal (or lower) level: How many immigrants live in town? How many toddlers are there (whose parents may not even have met before the last census)? What is the richest neighborhood in the county?\n\nThe 1990 census is almost a decade out of date, making such microdata as age woefully inadequate – if not downright paradoxical – to use: If there were 200 toddlers in town in 1989, does that mean there are none now, but there are lots of 10- to 15-year-olds? There are only two solutions. Ignore the 1990 census data by taking a sabbatical until 2000 or buy census updates from commercial vendors.\n\nEducated guesses\n\nCommercial updates have been the lifeblood of census information at the Press for the last five years. But keep in mind that the methodology for calculating estimates differs from one company to another. Most use algorithms based on death and birth lists, change-of-address data from the post office, and even mini-surveys.\n\nIn short, what you get with an estimate is a big ol' guess. Maybe a pretty good educated guess, but a guess nonetheless. Given the choice between using the 1990 census or an update, I would choose the update every time, with a note to readers that the information is based on estimates.\n\nI use Claritas, which packages data mainly for businesses and marketers. Claritas comes with a mapping program and is","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-undermining-defining","slug":"1998-december-undermining-defining","url":"https://dwillis.github.io/uplink/article/1998-december-undermining-defining.html","year":1998,"month":"December","monthNum":12,"headline":"UNDERMINING DEFINING","kicker":"","author_name":"Craig R. McCoy and Tom Torok","author_title":"","affiliation":"The Philadelphia Inquirer","summary":"The Philadelphia Police Department has systematically misclassified crimes to artificially lower reported crime statistics, which The Philadelphia Inquirer exposed through data analysis and investigative reporting, revealing significant impacts on victims and public safety.","keywords":["crime","statistics","fraud"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Access","Excel"],"full_text":"Philadelphia sure is a law-abiding place. It's a city where people aren't held up at gunpoint. Instead, they have \"disturbances,\" or, at worst, are subject to \"threats.\" Victims aren't beaten or stabbed. They are deemed mere \"hospital cases,\" coding for the police taking someone to the hospital. Pocketbooks aren't ripped off; they're booked as \"missing property.\" Thieves don't steal cars. Instead, the car's forgetful owners lose them, and the helpful cops write up the missing vehicles as a \"try and locate.\"\n\nWith artful classifications such as these, Philadelphia's police department for years has been fudging statistics for the FBI's influential Uniform Crime Reporting (UCR) program. This, in turn, has permitted mayor after mayor to boast of the city's low crime rate.\n\nUntil the whole statistical deck of cards came undone. In a series of articles over the last year, The Philadelphia Inquirer has challenged the city's crime statistics, using computerized reporting and a lot of old-fashioned, shoe-leather interviewing to document flaws in the numbers.\n\nThe finagling has had a big impact, both for the citizenry as a whole and, on a very personal level, for victims:\n\nAt this point, no one really knows what the true crime picture is for the city. In stages, police over the last year have disowned the citywide crime stats for 1996, 1997 (twice withdrawn) and the first half of 1998.\nThe scandal has damaged Philadelphia's attempt to follow New York's lead and switch over to police deployment based on computerized trend-spotting and mapping.\nSome \"downgradings\" - the term for converting a major crime to a minor one - have cut off cases from detectives, leaving suspects free to strike again.\nVictims have lost their claim for compensation under a state program that helps pay for uninsured medical bills. No crime, no victim, no compensation.\n\nThe stage for our reporting was set when Philadelphia Mayor Edward Rendell, finally responding to requests from the Inquirer and the Philadelphia Daily News, released seven years of underlying crime-by-crime data. This massive computer file provided details on more than 700,000 crimes deemed serious by police from 1991 to 1997. Philadelphia is one of few major American cities that release such detailed information, besides totals made public twice a year by the UCR program.\n\nUsing Access and Excel, an Inquirer team began to slice and dice the data. In some ways, our focus was on the dogs that weren't barking - crimes that weren't being booked.\n\nBoom and bust thefts\n\nAmong our early articles was a straightforward account, pegged to the FBI's national release, of the city's overall crime rate for 1997. Even as we wrote it, we noticed a weird trend in the data. When crimes were analyzed by month, Philadelphia had a puzzling drop in car thefts in the first half of the year, followed by a boom in the second. The change was without precedent for Philadelphia; we could not explain it as seasonal.\n\nWe then took our analysis to the city's police chief, John F. Timoney, for comment.\n\nThe numbers at issue had already been withdrawn from the FBI count once. In late 1997 the Daily News first reported that the city had been counting crimes based on when they entered data into their computer system rather than when crimes occurred. This method made comparisons between years and among cities haphazard at best.\n\nTimoney, who inherited the mess when he took command of the force in March 1998, had released new 1997 figures after tallying them again for a regular Jan. 1-Dec. 31 calendar year.\n\nOnce he had digested the Inquirer's new analysis, Timoney announced in May 1998 yet another inquiry into the 1997 figures, less than a week after their re-release. In September, he provided the results: for the first half of the year, police had by mistake dropped 3,000 property crimes - mostly stolen cars - from the city's count.\n\nNovelties and oddities\n\nAs we continued to explore the database, other puzzling patterns emerged. A key finding was Philadelphia's remarkably low count of aggravated assaults compared to murders. While almost every major city routinely books 50 such serious assaults for every murder, Philadelphia was logging just 15 per murder. Criminologists pointed us to data showing that murder and serious assaults rose and fell together in that ratio in most cities - a not surprising tidal effect if you think of a murder as an assault that goes too far. Another oddity: attempted burglaries had just about vanished in Philadelphia. For example, the data let us see that two police districts, with a total population of 138,000 people, reported a total of only six attempted break-ins in 1997. Shoplifting figures had been similarly low-balled. So had thefts of auto licenses, a particularly troublesome Philadelphia problem.\n\nThough Philadelphia's pervasive culture of numbers chicanery may seem unusual, there is no doubt that the problem prevails in municipalities large and small. Since the system's founding nearly 70 years ago, major UCR scandals have flared repeatedly. Experts have been warning that the problem is growing more virulent as departments put more stress on the numbers, both to fight crime and to rank commanders. In the last year statistical controversies have been sparked in New York, New Orleans, Baltimore, Atlanta and Boca Raton.\n\nShoe-leather series\n\nAs we worked over the data, we also interviewed serving and retired police from beat cops to commanders; victims and victims' advocates; town watch and civic group leaders; and academic and policing experts nationwide. We also consulted court records and ordered scores of individual incident reports.\n\nThe department's long-time culture of \"going down with crime\" - as the cops called it internally - began to surface. Detectives and street cops told us how crimes were blown off to lighten their workload. Others spoke of constant pressure from the top, of district commanders and even more senior officials seeking to raise performance ratings by putting a thumb on the scale of justice. Our work culminated Nov. 1-2 in a series that was actually sparing in its use of numbers and charts. Instead, we told story after story of actual victims and how the crime against them had been downgraded. One article explored violent crime; another tackled crimes against property. Among the more dramatic evidence were graphic reproductions of complete police reports that had been rewritten to downgrade the crimes. In the rewrites, key information vanished, including descriptions of suspects and weapons.\n\nWrongdoing and response\n\nThe issue has engendered considerable outrage in Philadelphia. For the first time, the city controller, the top financial watchdog for the city, is planning an audit of crime figures. U.S. Attorney General Janet Reno has asked her aides to examine the Philadelphia figures. Timoney has sworn the 1998 figures will be accurate, \"if it kills me.\" He has transferred two captains whose numbers were questioned and created an aggressive auditing unit to check figures. So far their investigation suggests that as many as 5,500 crimes were buried in the first half of 1998 - one out of every 10. After our series, Timoney ratcheted the pressure higher. He announced that undercover police would pose as crime victims to \"sting\" officers mishandling paperwork.\n\nCharacter overload\n\nWhile examining the Philadelphia figures, we also looked at national data. We drew, in part, upon unpublished FBI statistics that the agency provides on magnetic tape (available from NICAR) that includes in its entirety crime data collected for the UCR program - much more detailed information than what is available in print in the well-known Crime in the United States publications. One problem is the FBI sends the data in text that runs 8,000 characters across, four times the length that most databases can handle. Using a Visual Basic script, Torok helped break down the data into usable sections. (Note from the Database Library: NICAR processes the raw FBI data so software using the .dbf format can handle them.)\n\nAt the paper's request, a sociologist at Temple University had built an ArcView map of Philadelphia, which tagged each of the city's 400 police sectors by neighborhood. With this in hand, for one article graphics artist Matthew Ericson allocated crime by sector to each of Philadelphia's 56 neighborhoods. We ranked them by violent crime and property crime rates.\n\nOnline database\n\nWith leads flowing in, we are still at work on the issue. In one recent development, we posted on The Inquirer's Web site, Philadelphia Online, the 700,000-crime database that underpinned the entire effort. It took considerable grooming to get the database into usable form. Using Visual Basic, Torok \"parsed\" the chaotic address field into discrete block and street fields. His program correctly separated all but 15,000 of the crimes. Ericson then wrote a program in Perl (CQ) that solved another particularly bad 6,000 addresses or so. We also deployed scripts that fixed more than 100,000 misspellings of street names. Using Microsoft SQL Server, Torok then created active server pages on the Web site that permit readers to search the database rapidly for crimes on their block over seven years, on any one day, and by other methods. His program ingeniously separates and indexes the data by search method to produce quick results from an immense dataset. It received more than 24,000 page views its first day online. For comparison, a well-browsed Inquirer sports story receives around 3,000 \"hits.\" One first responses came in an e-mail, edited slightly for confidentiality: \"I looked for an assault,\" a browser wrote us. \"It must have been downgraded since it is not listed at all. It infuriates me that an assault is not reported. The two men who did this meant to cause bodily damage, but their act of cowardice won't be known because the police department is playing games with our lives.\" We're looking into it.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-correctional-care","slug":"1998-december-correctional-care","url":"https://dwillis.github.io/uplink/article/1998-december-correctional-care.html","year":1998,"month":"December","monthNum":12,"headline":"CORRECTIONAL CARE","kicker":"","author_name":"David Heath","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"The St. Louis Post-Dispatch investigated deaths of inmates under privatized correctional health care, finding that obtaining information about doctors with disciplinary histories required overcoming significant legal and organizational obstacles through creative use of public databases.","keywords":["prison","healthcare","investigation"],"topics":["Crime & Justice","Health & Environment","Investigative Methods"],"technologies":["Visual Basic","Internet","World Wide Web"],"full_text":"While in an Alabama prison, 18-year-old Calvin Moore melted before the eyes of correctional doctors and nurses. Moore, a burglar who suffered from severe mental illness, lost 56 pounds in less than a month - more than one-third of his body weight. On Feb. 21, 1996, he died from dehydration and starvation in the care of St. Louis-based Correctional Medical Services, the industry giant in the rapidly growing field of privatized correctional health care.\n\nCMS said it provided Moore with appropriate and compassionate care. A state report attributed the death to natural causes. However, a renowned expert on forensic medicine and pathology from the University of Colorado described Moore's death as \"a homicide resulting from criminal negligence.\"\n\nWord of Moore's death prompted the St. Louis Post-Dispatch to spend five months looking at the privatized correctional health-care industry. Reporters found more than 20 cases in which inmates died as a result of alleged negligence, indifference, under-staffing, inadequate training or cost-cutting by for-profit HMOs.\n\nInformation obstacles\n\nThe search for information was not easy. State correctional agencies were generally uncooperative, concerning the backgrounds of CMS physicians. We knew from anecdotal evidence that some had checkered pasts. The problem was obtaining a list of all CMS doctors and matching them against a database of disciplinary actions by state medical boards.\n\nAlthough the data we needed existed, it required sweat to obtain and make usable. A database of unprofessional and incompetent behavior by medical doctors, called the National Practitioner Data Bank (www.npdb.com), is maintained by the U.S. Department of Health and Human Services. However, federal law prohibits the names of the doctors in the database from being divulged to the public. That information is only disclosed to hospitals, health care entities, state licensing boards, and, in some cases, attorneys.\n\nThe Missouri State Board of Registration for the Healing Art is more open with disciplinary data. The board even posts the physician license database, which includes a field of disciplinary actions (www.ecodev.state.mo.us/prl).\n\nIndex end-run\n\nThe only comprehensive database available to the public is provided by Public Citizen, a consumer group founded by Ralph Nader. The data come in a four-volume set of books priced at $307.50. It can be ordered from the Web site (www.publiccitizen.org). Regional listings are available for $23.50.\n\nThe books provide an abstract of actions taken against doctors (including arrests), actions taken by state licensing boards, and actions taken by the Drug Enforcement Administration. While it is a gold mine of information, it is organized by state rather than by doctors' names - a problem since many doctors with disciplinary histories move around. Even the index was difficult to navigate, because it was grouped by type of action.\n\nWe solved these problems by scanning the 75-page index and writing a short program in Visual Basic to parse the fields into a database. We posted a search form on a password-protected Web site that we shared, at first, only with members of the project. When the series ran, we made that database available to everyone in the newsroom on our Intranet. The Web site allows reporters to search doctors by name.\n\nPrying with contracts\n\nOur bigger problem was obtaining the names of doctors working for CMS in order to match them against our data. Our computer-assisted-reporting specialist, Julie Lucas, contacted 26 states in which CMS does business to ask for a list of doctors working there and the contract the state signed with CMS.\n\nIn virtually every case, the states at first refused to provide us with the list of doctors, claiming that they did not have that information or directing us to contact CMS directly. (CMS declined to share the data as well.) We did routinely obtain the CMS contracts, however. In some cases, those contracts provided the leverage we needed to pry the list of CMS doctors from the states. In nearly every contract, CMS was bound to provide the state correctional department with a current list of their doctors. We used that contract language to persuade several states that the doctors' names were a public record.\n\nEven so, two states, Virginia and Arkansas, claimed that we weren't entitled to the information because their open record acts apply only to state residents. In actuality, Virginia's law specifically states that newspapers circulating in the state are entitled to public records. We proved that we sell some papers there. In Arkansas, we enlisted the help of Jeff Porter, the computer-assisted reporting editor at the Arkansas Democrat-Gazette, who managed to get the list of doctors within a day.\n\nOur efforts paid off. We had numerous hits on our database from the list of doctors we received from the states. In fact, doctors with questionable backgrounds became a major theme in the series.\n\nDavid Heath can be reached by e-mail at dheath@gw.pulitzer.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-investigating-the-fbi","slug":"1998-december-investigating-the-fbi","url":"https://dwillis.github.io/uplink/article/1998-december-investigating-the-fbi.html","year":1998,"month":"December","monthNum":12,"headline":"Investigating the FBI","kicker":"TRAC & DATA FIELDS","author_name":"David Burnham and Susan Long","author_title":"","affiliation":"TRAC","summary":"Challenges the FBI's secrecy and lack of accountability, using data analysis and the Freedom of Information Act to access and evaluate FBI enforcement actions.","keywords":["FBI","accountability","investigation"],"topics":["Investigative Methods","Government & Politics","Crime & Justice"],"technologies":[],"full_text":"The FBI is an extraordinarily secretive organization that routinely denies Congress, reporters and the public comprehensive information about the matters it investigates, the number of individuals who are indicted and convicted as a result, or the distribution of its staff in different parts of the country. For many decades, this absence of concrete information has made it nearly impossible to judge the bureau's actual performance. Given America's faith in the theory of accountability, the public acceptance of the FBI position is surprising. It's as if General Motors could sell stock without publishing an annual report certified by an independent accounting team. Fortunately, however, the FBI's steadfast resistance to public scrutiny can now be challenged. This is possible because of the Freedom of Information Act, the availability of computerized data analysis programs and the recognition that very detailed information about the enforcement activities of the bureau are recorded by the Justice Department's Executive Office of United States Attorneys (EOUSA).","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1998-december-we-dont-file-out-tapes","slug":"1998-december-we-dont-file-out-tapes","url":"https://dwillis.github.io/uplink/article/1998-december-we-dont-file-out-tapes.html","year":1998,"month":"December","monthNum":12,"headline":"\"WE DON'T FILE OUT TAPES\"","kicker":"","author_name":"Bob Warner","author_title":"Reporter","affiliation":"Philadelphia Daily News","summary":"Police departments in major U.S. cities have resisted releasing electronic crime data to news organizations despite publicizing crime reductions, with Philadelphia's Daily News eventually obtaining records after a two-year battle involving the mayor's intervention.","keywords":["transparency","police","data"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Police departments around the country have been happy to take public bows for declining crime rates. But in many cases, they're still refusing to provide news organizations with electronic data that would allow closer scrutiny of their records.\n\nAs of early 1998, police departments in the nation's three biggest cities — New York, Los Angeles and Chicago — were unwilling to release any data to reporters describing individual crimes. Houston's police department said it would be willing to release electronic data, but as of last January nobody had asked. Of the five biggest cities, only the Philadelphia police were providing the media with electronic data on local crime — and only after a two-year fight requiring intervention by a progressive mayor.\n\nThe Philadelphia Daily News began asking the police department for various pieces of electronic data - drug arrests, stolen cars, etc. - in 1994. With the exception of a small database describing city homicides, our requests were turned down, flatly and repeatedly, by the department's chief counsel and its day-to-day spokeswoman. In early 1996, we began asking for more comprehensive data - the past five years of criminal offenses reported to police, known to a small group inside the department as the \"offense tapes.\" Once again, we were turned down. \"We don't give out tapes,\" the department's spokeswoman told us.\n\nPennsylvania's public records law offered us little support. Written in 1957, it does not mention computers or electronic data. It contains a troublesome exemption for any records disclosing the institution or progress of a criminal investigation. Since the police claimed to investigate all Part 1 crimes reported to it, its lawyers would occasionally argue that this exemption covered any records on crimes reported to police.\n\nMedia allies\n\nBut we had an ally in the city's mayor, Ed Rendell, whose top aides had the common sense to manage the news media by providing information, not withholding it. Unofficially, they had long since agreed to the principle that documents considered public records on paper should also be considered public records when they're kept in an electronic format. We asked the mayor's office to review our dispute with the police.\n\nIt was a ticklish matter for the mayor's office, which had repeatedly been accused by the police officers' union of trying to run the department from the second floor of City Hall. But some sort of message was actually delivered to Police Commissioner Richard Neal, telling him he'd have to give us some data or provide reasonable explanations for withholding them.\n\nCalled and cornered\n\nWe still had trouble getting Neal to return our calls. Finally, we cornered him as he was kicking off a pedestrian safety campaign. That led to a series of three meetings, going through the department's record layouts and negotiating precisely what data would be turned over. The newspaper agreed to live without victim's names and addresses, but the department agreed to identify the blocks or intersections where crimes had occurred.\n\nThe agreement nearly fell apart with Neal's next phone call. Our data was ready, he said, but the Daily News would have to pay $2,089 - in advance - to cover the department's programming costs. Reluctantly, we agreed.\n\nIt didn't take long to find evidence of the department's bogus crime-counting practices: thousands of crimes from 1995 had been included in the data used to come up with 1996 crime totals for the FBI. It soon became clear that the department's annual crime counts were based on when they fed data into their computer system rather than when the crimes occurred.\n\nOur stories forced the department to withdraw crime figures previously submitted to the FBI for 1996 and 1997. More importantly, they became Exhibit A in a hot public debate over the effectiveness of the police department and the competence of Commissioner Neal. Five months later, Neal submitted his resignation. His successor, John Timoney, made clear on taking office that reform of the department's criminal data system would be one of his top priorities. Timoney has followed through.\n\nBob Warner can be reached at (215) 854-5885.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-crime-without-punishment","slug":"1998-december-crime-without-punishment","url":"https://dwillis.github.io/uplink/article/1998-december-crime-without-punishment.html","year":1998,"month":"December","monthNum":12,"headline":"CRIME WITHOUT PUNISHMENT","kicker":"","author_name":"Jim Hopkins","author_title":"Reporter","affiliation":"The Courier-Journal","summary":"A Louisville jail death investigation revealed that nearly 90 percent of the jail's guards had been disciplined for workplace violations, with serious punishment rarely enforced due to successful appeals to grievance boards. The Courier-Journal used open records requests and computer database analysis to uncover systemic discipline problems that eventually led to contract negotiations aimed at making officer accountability more effective.","keywords":["jail","discipline","records"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Paradox","Internet"],"full_text":"Shortly after midnight on Jan. 7, 1998, a 34-year-old inmate died after a struggle with guards in the Jefferson County jail in Louisville, Ky. Within days, the county coroner ruled his death a homicide, setting off multiple investigations by the jail, the police, prosecuting attorney and The Courier-Journal.\n\nEarly on, we reported that three of the five guards involved had prior on-the-job disciplinary records, including one who had been accused of inmate abuse. That piqued my curiosity and a question that formed the basis of our inquiry: How many other officers in the department had been disciplined, and for what violations?\n\nThe answer came the following month when we reported that nearly 90 percent of the jail's 350 guards had been disciplined for violating workplace rules. One-fifth had been found guilty of major offenses such as inmate abuse, insubordination and sleeping on the job. Serious punishment, such as termination, was rare. One officer had been disciplined 48 times - more than any other for violations that included pummeling an inmate in the face and then trying to cover up the incident. Management had tried to fire him twice but failed after he appealed to an employee grievance board.\n\nOpen-records accommodations\n\nTo reach our conclusion, we relied heavily on computer analysis and Kentucky's open-records laws to review all the disciplinary records of current and former jail-house staff. With our findings in hand, we turned to the Internet to find jail-management experts to place the situation in a national context.\n\nIn the wake of our report and a follow-up piece that looked exclusively at inmate abuse, jail managers negotiated a new contract with the union representing guards that is supposed to make it easier to punish officers who don't follow the rules while keeping an eye on the jail's approximately 1,600 inmates.\n\nWhen we first asked the jail for all employee disciplinary records, we were presented with a loose-leaf log that listed every employee offense, the date, a brief description of the offense, the recommended punishment, and the outcome. The log became our road map.\n\nUnfortunately, the log wasn't computerized. Using Paradox, I entered each incident and eventually created a 2,236-record database. I included every scrap of information in the log because I wasn't sure what would be important until I was finished.\n\nI detected patterns almost immediately: Some employees had been punished dozens of times; management seldom recommended termination; and employees often got their punishments overturned on appeal. For example, our analysis showed that nearly 34 percent of all recommended suspensions had been reduced by the employee grievance board. About 70 percent of the recommended firings taken to the board were reduced to lesser punishments.\n\nIn the case of the inmate who had died, three of the five officers involved had prior infractions that could have led to termination. But in only one instance was firing recommended, and that recommendation was tossed out during the grievance process. The employee grievance panel, it turned out, gave management and the Teamsters union an equal number of votes when deciding cases appealed to the board. Our analysis showed that virtually all discipline that involved a suspension or termination were appealed to the board. Anxious to avoid a prolonged appeal if they stuck to their guns, managers repeatedly struck bargains with the union.\n\nData entry demands\n\nThe lack of a pre-existing computer database was the biggest problem; virtually all the jail records were paper. I don't enjoy data-entry work, but there didn't seem to be any other way. It took about three solid days, but in cases like this I do the work myself because I can see trends early on, which helps me draft the story as I go. Inputting data myself also makes me feel more confident about accuracy.\n\nThe employee log was only a starting point; it included virtually no details from individual disciplinary cases. To learn more about inmate abuse and excessive force, for example, I had to isolate the names of all 34 officers accused of such violations in the past. Then I requested their complete personnel files. That led to a second story, which concluded that although inmate deaths at the hands of guards were rare, incidents of excessive force and abuse were not.\n\nLessons learned\n\nWhat's true in traditional reporting also is true in computer-assisted reporting: A newspaper's greatest ally is the open-records law. In this case, however, the records told only part of the story. Although jail managers knew they were having a hard time enforcing the rules, they didn't know how serious the problem was because they had never taken a close look at their own records. That's where computer analysis proved invaluable.\n\nFinally, database stories can be pretty dull stuff without concrete examples. Part of what made our two stories compelling was our focus on case studies, including the officer who had been disciplined 48 times. Using the database as a guide to inmate abuse cases, for example, I described in detail situations where guards had beaten inmates held down by other officers, falsified reports to cover up their actions, and gotten off scot-free.\n\nJim Hopkins can be reached by e-mail at hopkinsj@courier-journal.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-first-ventures","slug":"1998-december-first-ventures","url":"https://dwillis.github.io/uplink/article/1998-december-first-ventures.html","year":1998,"month":"December","monthNum":12,"headline":"FIRST VENTURES","kicker":"","author_name":"Bill Muller","author_title":"Reporter","affiliation":"The Arizona Republic","summary":"A journalist investigating workplace deaths in Arizona using OSHA data discovered critical gaps in the database, finding 26 undocumented deaths after verification with state officials. The analysis revealed that workplace fatality fines are often reduced by 40% through appeals and that Arizona provides minimal survivor benefits to families of deceased workers.","keywords":["workplace","deaths","fines"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["Access","Excel"],"full_text":"Death on the job\n\nIt was one of those moments that every reporter dreads. I woke up in the middle of the night staring at the ceiling. After analyzing an OSHA database, I had just finished writing a story about workplace deaths in Arizona. In cases where there was a serious safety violation, I had identified 108 deaths since 1992 and established the median fine per fatality: about $3,000. But now I tossed and turned. What if there were more than 108 deaths? What the data was flawed? analysis would be worthless. Luckily, though I already had written the story, its run date was still weeks away. The next day, I called the director of the state industrial commission, who had come to know well during the project, and told him of my misgivings. Without hesitation, he agreed to have a member of his staff go through all workplace deaths since 1992 and make sure we had them all. We didn't.\n\nUndocumented deaths\n\nDue to some reporting problems on OSHA's part, 26 deaths were not in our analysis. Using data provided by the commission, we added the deaths into the Excel spreadsheet that we had used to calculate the median. A funny thing happened: The median was still about $3,000.\n\nIn picking median for analysis, we had done ourselves a favor. When we added the missing cases, some came in above and some came in below, and the median stayed about the same.\n\nStill a good lesson. Some other findings:\n\nFrom 1992 until mid-1997, 79 of the 134 worker deaths investigated by OSHA in Arizona led to citations for serious safety violations. Just 12 fines were more than $10,000.\nThough OSHA fines related to workplace deaths might start high, the appeal system is designed to reduce them. In fact, an analysis of initial fines and current fines showed that they dropped by about 40 percent, saving the companies involved about half a million dollars.\n\nFor widows and widowers of workers who die on the job, Arizona ranks low for survivor benefits. State law protects companies from wrongful death lawsuits, saying essentially that employees covered by workers' compensation cannot recover damages in the event of injury or death.\n\nThough OSHA fines related to workplace deaths might start high, the appeal system is designed to reduce them.\n\nEarly experiments\n\nNaturally, getting the answers wasn't as easy as we first thought. It was my first time dealing with a fairly large database—about 3 MB, with four tables and 43,000 records in the largest table. The data weren't laid out quite the way I thought they would be.\n\nFor example, when I first calculated the fines per death (sorted in Access, then crunched in Excel), I noticed that each record included only one company name. This puzzled me because I knew several cases in which more than one company had been fined in connection with a death. To get the death cases, you had to link the accident table with the OSHA table by case ID number. Each time I asked for cases with '1' in the degree field (the death code), I got the same list and only one company name for each deceased worker.\n\nI decided to experiment. I sorted the data by date and then by address, looking for investigations in the OSHA table that occurred on the same date and at the same address as deaths in the accident table. Essentially this worked. To make sure the cases were related (and not just OSHA inspectors spotting unrelated violations on the job site), we checked the related activity numbers that appear on the case records available on the OSHA Web site. We also had the industrial commission check our work. They found one related case that didn't turn up, mostly because it was added to the case file a month after the accident.\n\nScanning for the striking\n\nAfter generating our initial list of 108 deaths, we searched the data for cases that would be interesting. We looked for small fines, large fines, and large fines that hadn't been paid (by checking the \"penalty remit\" field in the data). I also went through the newspaper library, scanning for details on various cases. We found striking cases, including a small company that had dodged a $47,000 fine by declaring bankruptcy and another that was fined just $7,500 after a worker had been decapitated. To find the relatives of these victims, many of whom had scattered, we used Track, locating one relative in Rochester, N.Y., and another by sending letters to her last two known addresses. Also, after an editor asked the question, I went back to the data to find out if we could document the total amount that OSHA fines had been lowered during the appeal process. Fortunately, the data allowed such a calculation by including \"penalty\" and \"initial penalty\" fields. Because of ongoing appeals, the fines associated with more recent cases tended to change, so we updated them using the OSHA Web site establishment search. (See http://www.osha.gov/cgi-bin/est/est1)\n\nBut 26 fatalities were still missing from the records. The reason, as far as we can tell, was that state officials had not filled out the accident summary when logging the case. So while the accident appeared in the main OSHA table, it was not in the accident table, which contained the degree field and the name of the victim.\n\nI used the records provided by state OSHA to update my chart, and the rest is history. By the way, you might want to replace that frayed electric cord.\n\nBill Muller can be reached by e-mail at mullyb@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-tech-tip","slug":"1998-december-tech-tip","url":"https://dwillis.github.io/uplink/article/1998-december-tech-tip.html","year":1998,"month":"December","monthNum":12,"headline":"TECH TIP","kicker":"","author_name":"Steve Doig","author_title":"","affiliation":"Arizona State University","summary":"Computer-assisted reporting professionals tackle messy, unstructured datasets using various tools and techniques, with the NICAR-L community providing valuable collaborative problem-solving resources and demonstrating multiple effective approaches to data challenges.","keywords":["data","journalism","programming"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["SAS","Access","FoxPro","Visual Basic","Excel","Monarch","Perl","CD-ROM","magnetic tape"],"full_text":"A big part of the fun for most CAR folks is the challenge of solving puzzles. projects tend to be one-of-a-kind things, so we're constantly facing new data formats with strange layouts (at least to us), undocumented codes and other idiosyncrasies. Even though no one else in the newsroom understands how difficult the struggle was, we get a rush of satisfaction from finally beating an unruly dataset into submission.\n\nWhen tackling such data puzzles, a great resource is the NICAR-L list. Even if you've never faced variable-length records or packed hexadecimals or YYYYDDMM dates or bitmaps or other weirdness, chances are good that others have. And perhaps the most instructive thing about the flurry of postings often generated by an interesting question is the wide range of clever solutions offered. Over and over, we discover that in CAR there are a lot of ways to get from here to there.\n\nA case in point was the \"ugly data\" contest I tossed onto NICAR-L this summer. It not only provided a stimulating puzzle for some of our best propeller-heads, but the results also proved the versatility of the CAR toolbox.\n\nI found this puzzle in a big tape dataset sent to me by a reporter who needed it in a more useful format and on CD-ROM. The data, which came as a courts database involving civil suits, offered several kinds of ugliness:\n\nThe structure was hierarchical: For each case there was one case record and then as many individual party records as there were parties to the case. But the case record didn't say how many party records would follow. Nor was there a common field, like case number, to link case and parties.\n\nEach \"line\" of the data was composed of exactly 100 of the above case and party records, with no delimiters between records. Thus, a \"line\" might be anywhere from 4,500 to about 6,000 characters long. If the last case still had more parties when the 100-record limit was reached, the rest would spill across to a new \"line.\"\n\nThe layout we were given was wrong. It didn't mention that each case record began with \"0026\" and each party record with \"0055.\" Later, I figured out that this undocumented feature was supposed to be the length of each record type, though why the file creators felt the need to repeat this for every record is beyond me.\n\nNor were the supposed record lengths of 26 or 55 characters reliable. Some, but not all, records ended with one, two or three carets, for no apparent reason.\n\nIt took me an afternoon of noodling with SAS, my database program of choice, to figure out a way to read the data and transform it into something useful. I've been on a long campaign to convince CAR power users that SAS is better than more commonly used programs such as Access or FoxPro, particularly for ugly data like this. So I created the \"Ugly Data Contest,\" urging anyone interested to tackle it with whatever tools they wanted. I was sure their struggles, and the relative simplicity of my SAS program, would win some converts.\n\nThe roads to solution\n\nI won't go into all the details of the results here (you can check the NICAR-L archives of August 4-10, 1998, by going online at http://www.nicar.org/nicarl.html). But I was surprised and delighted - despite my pro-SAS agenda - at the wide variety and inventiveness of the solutions that came in.\n\nAlong with my SAS program, there were FoxPro programs from Greg Reeves of the Kansas City and Griff Palmer of the Daily Oklahoman; a Visual Basic solution from David Heath of the St. Louis Post-Dispatch; a stand-alone Visual Basic GUI from Dan Keating of The Miami Herald; a Word-and-Excel method-in-concept (that I called the \"ugly solution\" winner!) from Russ Clemings of the Fresno Bee; a Visual FoxPro/SQL approach from Jeff Porter of the Arkansas Demo-Gat-Gazette, and a text editor/Monarch solution from Dick Eijk of NRC Handelsblad in Rotterdam, The Netherlands.\n\nBut my favorite solution used an entirely unexpected tool: PERL, the \"print extraction and report language\" loved by Unix wizards and cgi-scripters. Matt Ericson of the Philadelphia Inquirer's web operation did it with these five lines:\n\nPut a new line after any party record\nperl -i -pe 's/ (\\d|d\\d\\d\\d\\d) (\\d\\d\\d\\d) ([CP]) / \\1\\n\\3/g' ug.txt\n\nPut a new line after any case record\nperl -i -pe 's/(\\d\\d\\d\\d\\d\\d\\w\\w\\w\\w\\w\\w) (\\d\\d\\d\\d) ([CP])/\\1\\n\\3/g' ug.txt\n\nFix any line that still starts with a 4 digit number, which seems to appear after the ^s\nperl -i -pe 's/^\\d\\d\\d\\d//;' ug.txt\n\nKill any ^s\nperl -i -pe 's/\\^//g;' ug.txt\n\nMerge the cases and parties\nperl -i -pe 'if (/^C/) { $c=$_=\"=\"; chop ($c) } else $_=\"$c ug.txt\n\nI still say SAS is best for handling the widest range of ugly data and other CAR problems. (I am, however, starting to learn some PERL!) But the contest showed it's possible to make almost any CAR tool do almost any CAR task. As the old saying goes: \"When all you have is a hammer, every problem looks like a nail.\"\n\nSteve Doig can be reached by e-mail at steve.doig@asu.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-on-the-internet","slug":"1998-december-on-the-internet","url":"https://dwillis.github.io/uplink/article/1998-december-on-the-internet.html","year":1998,"month":"December","monthNum":12,"headline":"ON THE INTERNET","kicker":"","author_name":"Heather Browne","author_title":"","affiliation":"NICAR","summary":"This journalism article provides a guide to the best online resources for accessing crime statistics and data, including major sources like the FBI, Bureau of Justice Statistics, and specialized databases that help journalists navigate the complex landscape of criminal justice information.","keywords":["crime","statistics","resources"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["Internet","World Wide Web","spreadsheet"],"full_text":"Covering crime is not for the faint of heart. Crime statistics, for their part, can be daunting or difficult to locate. The vast array of criminal statistics out there can be hard to navigate, but the Web helps narrow it into categories. Some sites offer answers to frequently asked questions for data that needs explanation or provide tutorials for analyzing the data. The following are just a few of the many sites that offer everything from interesting crime tidbits to vast databases. They are good places to start and can lead to more specific sites.\n\nThe FBI Home Page www.fbi.gov\n\nThis site is the premier source for the latest in crime stats. Not only can you check out who's who on the \"Most Wanted\" list, but you can also get the most recent version of \"Crime in the U.S.,\" which summarizes Uniform Crime Reports (UCR) from agencies throughout the country. (Click on \"Crime Statistics\") It includes trends and thorough explanations, which save time. Also available is the National Incident-Based Reporting System, the latest wave in reporting crime.\n\nThe Bureau of Justice Statistics www.ojp.usdoj.gov/bjs\n\nThe U.S. Department of Justice much in the way of statistics on both victims and offenders. Findings from the National Crime Victimization Survey can flesh out FBI data the survey may account for unreported crimes. This site also offers spreadsheets in trend data, information on courts and sentencing, and much more. It incorporates different data sources, so it is very useful in tracking crimes through the various stages of criminal justice. A lot of the data is available to order.\n\nThe Justice Information Center www.ncjrs.org\n\nThis site offers in-depth information that supplements FBI and other crime statistics. It includes statistics in areas such as homicide reports, arrests, and juvenile court, or victim information. For topics, such as \"Crime Prevention\" or \"Juvenile Justice,\" it lists links to relevant Web sites, mailing lists, and articles. Definitely check out its extensive list of sites for criminal justice statistics http://www.ncjrs.org/statuww.htm.\n\nThe National Consortium for Justice Information and Statistics www.search.org\n\nIf the data you're looking at appears to be a scrambled mess, visit this site for background. It offers information on courts, NIBRS, and can lend technical support. The consortium site gives dates for conferences and symposia in crime statistics and criminal justice. It can also link you to the Clearinghouse, another wealthy source of information for reading and analyzing data on law and order in the U.S.\n\nFedStats www.fedstats.gov\n\nThis site is simply awesome. It links you to any government agency and is simple to follow. It calls itself \"one-stop shopping\" for statistics, programs, and policy. Maintained by the Federal Interagency Council on Statistical Policy, federal statistics from the FBI, DEA, and many more agencies are just a click away. FedStats also lists agency contacts.\n\nOrganized Crime www.crime.org\n\nNot the Mafia, though you could probably link to them. This site, which is easy to move around in, provides tutorials on reading crime statistics and links to all the major sites. It helps answer questions on crime data and points out possible inaccuracies. Check it out for nothing else but the always enlightening \"criminal quote.\" The designer, Regina Schekall, plans to develop the site further.\n\nNumber Crunchers Precision Journalism www.crunch.com\n\nThis site is devoted to journalists. It \"crunches\" the numbers, based on the adage that \"journalists don't do math.\" It's got some \"top secret\" FBI files, now available under the Freedom of Information Act. You can look up the rap sheet of Al Capone, for example. A glitzy, colorful site, it offers data and spreadsheets on crime trends.\n\nNational Archive of Criminal Justice Data www.icpsr.umich.edu/NACJD\n\nThis site offers a wide array of background information, technical support, and data. If you need to wade through all that, go ahead, but the site offers a link straight to the downloadable data, found in outline. Just click on the subject you'd like to see: corrections, police, victimization surveys, and more.\n\nOr search by letter of the alphabet. There are also forums to which you can post questions.\n\nSecurity on Campus www.soconline.org/STATS/index.ht\n\nStatistics concerning crime on college campuses is hard to come by, but this site makes it easy. It gives background information and offers links to UCR various university crime tables.\n\nFrom the mailbox\n\nOn the NICAR-L mailing list, David Milliron of The Atlanta Journal-Constitution recently suggested these Web sites for the crime beat:\n\nNational Clearinghouse for Criminal Justice Information Systems http://www.ch.search.org/\n\nCrime Data Sources in Criminal Justice http://www.ncwc.edu/-toconnor/data.htm\n\nUniform Crime Reports: County Level Data http://fisher.lib.virginia.edu/crime/ 1995 appears to be the latest year.\n\nSourcebook of Criminal Justice Statistics http://www.albany.edu/sourcebook/ Data about all aspects of criminal justice in the U.S. Includes more than 600 tables in Adobe Acrobat .pdf format.\n\nJustice Research and Statistics Association Statistical Analysis Centers with Web Servers http://www.jrsainfo.org/sac/sacsites.html\n\nFinding forensic information on Internet http://www.bart.nl/-geradts/lit.html\n\nCrime and Justice Electronic Data Abstracts http://www.ojp.usdoj.gov/bjs/dtdata.htm The Department of Justice Bureau of Justice Statistics recently updated this massive database to include selected data through 1996. Lotus spreadsheets (.wk1) are available in 55 subjects covering crimes and arrests, criminal justice, and general demography.\n\nUN Crime & Justice Information Network Statistics and Research Sources http://www.ifs.univie.ac.at/-uncjin/stats.html\n\nCrime Mapping Research Center http://www.ojp.usdoj.gov/cmrc/\n\nHeather Browne can be reached by phone at (573) 884-7711","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-from-the-mailbox","slug":"1998-december-from-the-mailbox","url":"https://dwillis.github.io/uplink/article/1998-december-from-the-mailbox.html","year":1998,"month":"December","monthNum":12,"headline":"From the mailbox","kicker":"","author_name":"David Milliron","author_title":"","affiliation":"The Atlanta Journal-Constitution","summary":"A collection of web resources compiled by a journalist for crime beat reporting, including links to criminal justice databases, statistics sources, crime mapping tools, and law enforcement information.","keywords":["crime","databases","statistics"],"topics":["Crime & Justice","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web","Lotus 1-2-3","Adobe Acrobat"],"full_text":"From the mailbox\n\nOn the NICAR-L mailing list, David Milliron of The Atlanta Journal-Constitution recently suggested these Web sites for the crime beat:\n\nNational Clearinghouse for Criminal Justice Information Systems\nhttp://www.ch.search.org/\n\nCrime Data Sources in Criminal Justice\nhttp://www.ncwc.edu/-toconnor/data.htm\n\nUniform Crime Reports: County Level Data\nhttp://fisher.lib.virginia.edu/crime/\n1995 appears to be the latest year.\n\nSourcebook of Criminal Justice Statistics\nhttp://www.albany.edu/sourcebook/\nData about all aspects of criminal justice in the U.S. Includes more than 600 tables in Adobe Acrobat .pdf format.\n\nJustice Research and Statistics Association Statistical Analysis Centers with Web Servers\nhttp://www.jrsainfo.org/sac/sacsites.html\n\nFinding forensic information on Internet\nhttp://www.bart.nl/-geradts/lit.html\n\nCrime and Justice Electronic Data Abstracts\nhttp://www.ojp.usdoj.gov/bjs/dtdata.htm\nThe Department of Justice Bureau of Justice Statistics recently updated this massive database to include selected data through 1996. Lotus spreadsheets (.wk1) are available in 55 subjects covering crimes and arrests, criminal justice, and general demography.\n\nUN Crime & Justice Information Network Statistics and Research Sources\nhttp://www.ifs.univie.ac.at/-uncjin/stats.html\n\nCrime Mapping Research Center\nhttp://www.ojp.usdoj.gov/cmrc/\nWealth of data on crime mapping and agencies that have put their crime maps on the Internet.\n\nStudents' Report of School Crime: 1989 and 1995\nhttp://www.ojp.usdoj.gov/bjs/abstract/srsc.htm\n\"It compares findings from the 1989 and 1995 SCS on student reports of victimization, drug availability, street gang presence, and gun presence at school.\"\n\nLaw Enforcement Sites on the Web\nhttp://www.ih2000.net/ira/ira2.htm\nPossibly the largest collection of law enforcement information on the Web.\n\nFederal Bureau of Prisons\nhttp://www.bop.gov/\nIncludes directory of facilities, Fact Sheets with summary statistical data on federal prison populations by gender, race, ethnicity, age and other demographic characteristics.\n\nDavid Milliron can be reached by e-mail at david-milliron@david-milliron.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-stats-from-the-road","slug":"1998-december-stats-from-the-road","url":"https://dwillis.github.io/uplink/article/1998-december-stats-from-the-road.html","year":1998,"month":"December","monthNum":12,"headline":"STATS FROM THE ROAD","kicker":"","author_name":"Sarah Cohen","author_title":"Data Journalist","affiliation":"NICAR","summary":"Journalists should use inflation-adjusted figures when comparing dollar amounts across years, and the author recommends George Landau's NewsEngin tool as the most accurate and user-friendly calculator available, while cautioning against outdated tools that lack transparent methodology.","keywords":["inflation","calculators","journalism"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Almost every comparison using dollars over a number of years should be adjusted for inflation. It's simply not fair to talk about salary or budget increases since the 1970s without building in inflation.\n\nNot enough reporters do this, and that's a shame. It's also now inexcusable. The data is sitting on the Web, as are tools that make the simple calculation for you.\n\nA recent question from an IRE member prompted me to test a variety of these inflation-adjustment (or cost-of-living) calculators on the Web. I was looking for results consistent with my own calculations, documentation that let me verify how they'd come to an answer, and a simple interface that clearly guided me to enter data for the answer I expected.\n\nNot many of the tools met my challenge.\n\nAnd the winner is\n\nOnly George Landau's NewsEngin tool passed in all categories: accuracy, documentation and ease of use. Landau, a longtime IRE member, also confirms the date of the last update, letting you know his data is up-to-the-minute accurate. The instructions of some calculators were so indecipherable that you could easily make a mistake filling them out. Others introduced rounding errors or slightly erroneous data into their calculations.\n\nAny site that doesn't keep its data up to date and doesn't provide details on how it performs its calculations ought to be eliminated from your list of bookmarks.\n\nThe rounding error isn't very worrisome. In fact, if rounding makes such a big difference in an answer, there's a good reason to report the answers at a general level. There's no reason to imply a false sense of precision when doing work based on indexes with bigger flaws than a calculation would introduce.\n\nBut any site that doesn't keep its data up to date and doesn't provide details on how it performs its calculations ought to be eliminated from your list of bookmarks. It's just not worth the effort to worry that you've filled it out incorrectly, that it's using the correct formula, or that the data is out of date.\n\nHow the calculators calculate\n\nGenerally, you type in a value like a salary. You tell the inflation adjuster what year the salary applies to and in what year you want it expressed. Then you receive an inflation-adjusted figure.\n\nAlmost all of the calculators assume you want to inflate previous year dollars. So if you type in $25,000, 1975 and 1997, you'll get what a salary of $25,000 in 1975 would be today, after inflation.\n\nThat's usually how we do it, even though the instructions on official Web sites imply you ought to use some other base year. The reason is that readers, viewers and listeners can picture what a dollar is worth today. Few accurately remember what it was worth a long time ago.\n\nOn your own\n\nInstead of depending on someone else, most reporters who deal with dollar values like budgets, salaries or home values ought to learn how to compute inflation-adjusted figures themselves.\n\nThen you can leave the Web tools where they belong - as a check on your own work. It's not hard. I promise. Here's the formula:\n\n(CPI Then) Value then = Old dollars inflated to compare in new dollars.\n\nTo get the CPI, just visit the BLS home page at http://www.bls.gov. Then choose \"Data,\" and \"Most requested series.\" Almost everyone uses the CPI-U, or the national CPI for all urban consumers, as their deflator. It doesn't matter which base period you use; they all use the same underlying data.\n\nSarah Cohen can be reached by e-mail at sarah@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1998-december-bits-bytes-and-barks","slug":"1998-december-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1998-december-bits-bytes-and-barks.html","year":1998,"month":"December","monthNum":12,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"Sarah Cohen","author_title":"training director","affiliation":"IRE and NICAR","summary":"IRE and NICAR are launching initiatives to improve crime reporting through data-driven journalism and partnerships with public health experts, while also announcing upcoming national conferences in Boston and Kansas City.","keywords":["crime","reporting","training"],"topics":["Investigative Methods","Crime & Justice","Journalism & Newsroom"],"technologies":["SAS","Perl","FoxPro","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nIRE and NICAR on the crime beat\n\nAs you can see from this issue, reporting on crime in the United States is becoming more sophisticated each year as reporters try to give it context and depth. IRE and NICAR are working with several groups to improve reporting on crime.\n\nOne project involves the Berkeley Media Studies Group and the Missouri School of Journalism The project is trying to show journalists how to strengthen daily reporting on crime and violence by making use of datasets and historical information. It also tries to link the reporters to public health experts who study the topic intensely.\n\nLori Dorfman from Berkeley and veteran reporter Jane Stevens are leading the effort. Both have some radical ideas about daily reporting on crime and violence and are taking their message into newsrooms.\n\nThe role of IRE and NICAR is to help provide meaningful hands-on training and examples of stories that use combinations of datasets and shoe-leather reporting. IRE and NICAR also hope to work with a newly created association on crime reporting to provide manuals, tipsheets and booklets about covering different aspects of the beat: the cop shop, the court, the prison.\n\nIf you have ideas or want to help on either of these projects, please contact Brant Houston by e-mail at brant@ire.org\n\nConferences abound\n\nInformation on upcoming IRE and NICAR national and regional conferences is now available on the IRE Web site. Visit the IRE Web site at www.ire.org/reourred.comferences/\n\nThe National Computer-Assisted Reporting Conference, presented by IRE and NICAR along with The Boston Globe, will be held March 11-14, 1999 in Boston You may register online or download an early registration form at www.ire.org/resources/conferences/boston\n\nThe schedule for the conference is being finalized. We hope to offer new classes in SAS basics and data management and PERL scripting. We plan to reinstate basic Fox training using Visual Fox and continue popular basic classes, a special session for broadcasters, and Intranet training.\n\nContact Sarah Cohen, IRE and NICAR training director, with suggestions for panel topics and hands-on classes, plus ideas for who might speak or teach. Cohen can be reached by e-mail at sarah@nicar.org\n\nThe Investigative Reporters and Editors National Conference, co-sponsored by The Kansas City Star and KCTV, will be held June 3-6, 1999 in Kansas City. You may also register online or download an early registration form at www.ire.org/resources/conferences/kansascity","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-charting-campaign-cash","slug":"1999-januaryfebruary-charting-campaign-cash","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-charting-campaign-cash.html","year":1999,"month":"January/February","monthNum":1,"headline":"Charting campaign cash","kicker":"CREATING CONTRIBUTION DATABASES","author_name":"Cindy Eberting","author_title":"","affiliation":"CFIC","summary":"Newsrooms face a choice between hiring campaign finance consultants or building their own databases to track political contributions, with both approaches offering advantages and leading to improved election coverage.","keywords":["database","campaign","contributions"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":[],"full_text":"Building a database to track campaign contributions is nothing new to computer-assisted reporting. But the best way to do it is in question.\n\nCampaign finance consultants are popping up and pitching their service to newsrooms to design and maintain such a database. They say, an in-house project wastes money and ties up reporter time with tedious work, such as coding contributors for occupation. Many newspapers, however, still opt to create their own database.\n\nThere are advantages to both methods. And both tactics have resulted in more in-depth election coverage. Here are tips from consultants and reporters about how to approach tracking your local political cash flow.\n\nThe biggest hurdle, says consultant Tony Raymond of Public Disclosure, Inc., can be convincing editors that the time, effort and money needed to build a good campaign finance database are worthwhile. The key is to point to stories other newspapers have produced with such databases and the state laws that changed once the stories ran.\n\n\"The cost of it can be a deterrent,\" Raymond says. \"But the benefit of a campaign finance database is that it can be used for more than campaign finance stories. Say someone in the metro area gets stopped for DUI. It's a good place to throw his name at the database and see if he is a player in the community.\"\n\nDavid Poole, a reporter-turned-consultant who helped build campaign finance databases for statewide newspaper consortiums in New York and Virginia, says both his projects have changed state campaign finance practices.\n\n\"In Virginia, the newspapers have completely helped change the equation,\" Poole says. \"There are more and more candidates that are beginning to file electronically. These projects are a way for the newspapers to have a positive effect on some positive change.\"\n\nOnce editors are convinced, the careful work of planning the database begins. It's important to think ahead about every angle from which the databases will be analyzed.\n\n\"We try to suggest a database format,\" Raymond says. \"We know that once the database happens, and people see it inside the newsroom, then the next thing they say is: 'I'd like to see this or that.' If you don't set it up right the first time, you have to do the whole thing again.\"\n\nLast fall, The Washington Post hired Public Disclosure, the company owned by Raymond and Kent Cooper, who both worked for the Federal Elections Commission and the Center for Responsive Politics. They built a database of contributions in Maryland races from 1994 to 1998. Last summer, Tom Brown at The Seattle Times combined federal and state campaign contributions into one database to determine Washington's Top 50 political contributors. The project, which they did in-house, took about two months to complete.\n\nIn both cases, some of the data were already in electronic format, eliminating the time-consuming and often costly effort of data entry. Both databases also identified the occupations and employees of as many major donors as possible.\n\nThe most tedious tasks were standardizing contributor names and coding contributors for occupation and industry. (There is no effort by federal or state election commissions to address the problem of contributor names being listed several different ways, even though the money is coming from the same person.) In both projects, to complete the coding, both the reporter and the consultants were forced to call individual contributors to verify or determine their occupation.\n\nProject proposals\n\nWhen Raymond and Cooper first start talking with a newspaper about a project, they set up a division of labor from the beginning to define who will do what. Basic questions they initially want answered include:\n\nWhat level of donations do you want? Do you want donations to federal candidates as well as state legislators and county commissioners?\n\nIs the information available electronically? What's the average number of transactions for each level: federal, state or local?\n\nIf the data are only available on paper, do you want to contract out with a data-entry firm to have in key-punched? Or does a state or local entity already key-punch the data into an in-house database, which you could request through the Freedom of Information Act?\n\nDo you want to code the contributions for occupation and employer by giving them an industry or sector classification?\n\nBoth of Poole's projects required data entry. Poole used Quality Data Systems, a nationwide data entry firm. Reporters can also find numerous data entry services near state capitols where state agencies require their work.\n\nWhen hiring a data entry firm, Poole suggests asking the firm's customers about the firm's accuracy rating. The contracts are usually based on a per-record basis. It's best to receive bids for the contract from two or three vendors, Poole says. Then ask a price for a specific number of records and an expected turnaround time.\n\nWhen doing data entry, key in twice the contributor's name, donation amount and ZIP code (to help later with verification). Also add fields for the page number on which the contribution is listed and the schedule of the contribution form. These extra fields make it easier to spot poor data entry and to cross-verify summary contribution information. \"I think it's overkill to fully verify (key in twice) every single field for cost and for time reasons,\" Poole says.\n\nThe newspaper needs to come up with clear rules for the data entry workers to follow, particularly when entering addresses. The database coordinator also needs to work closely with reporters who have covered campaign finance issues using paper records. In Poole's Virginia project, the summary contribution pages didn't have standardized date ranges. A query of the itemized data told the actual date range.\n\n\"On the summaries, it helps to know what they say and what they actually mean, which can often be two different things,\" Poole says.\n\nOccupation coding\n\nOccupation coding can be the most complicated, tedious and subjective process of building a database, but also the most enlightening.\n\n\"Some of the best stories that have been done are those where classification has been done,\" Raymond says. \"You can see trends and patterns, such as how much gambling has influenced a particular area.\"\n\nThe Post's project tried to identify employer and occupation for every donor who gave $1,000 or more. To make the matches, they ran names against other databases, looked up names in newspaper archives, and phoned several people just to ask who they were and what they did for a living. Public Disclosure did most of the work, identifying 78 percent of major donors to the governor's race.\n\nBut it's important to point out the caveats of occupation coding to editors and readers. Raymond uses the example of General Electric, which also owns a television network, makes commercial and defense jet engines, and constructs refrigerator parts.\n\n\"So what do you code GE as?\" Raymond asks. \"Is it a defense contractor, a home appliance company or jet crafter?\"\n\nWhen dealing with large companies, reporters need to be aware of the large players involved. When Raymond's company works with newspapers to code contributions, many coding decisions are left to the newspaper and reporters themselves, who are more familiar with area industries.\n\nIn the Seattle series, the reporters considered a couple as one entity when both were significant donors. They also included corporate contributions of companies known to be controlled by one individual. Brown warns that reporters looking at contributions over a long period must beware of contributors who have died. Four of their original Top 50 had died during the seven years of contributions the newspaper considered.\n\nStructuring the database\n\nRaymond suggests having a master table that lists details on all contributors. This serves as a look-up table for all the names. Raymond builds databases using Microsoft Access. Each contributor name is given a unique identifier. Then each office or race—such as the gubernatorial or state legislative races—becomes its own separate table.\n\n\"You're going to get information in the gubernatorial race that will be different than the legislature,\" he says. \"But you want to keep all that information. The most flexible thing you have going is that master table. It's really the key to it.\"\n\nWith this structure, every time a media outlet wants to add another level of campaign contributions, they don't have to change the database—just add a related table for that office and add the contributor names to the master table. The database structures for the New York and Virginia projects differ. In the Virginia database, every contributor and candidate has a unique identifying number in a relational database with 10 tables that relate to the main contributions table. They also code contributors for occupations. In New York, for competitive reasons within the consortium, the database only assigns a unique number to candidates. Each newspaper then decides whether they want to standardize names or code for occupation. \"The papers there have a much higher level of distrust,\" Poole says. \"They were all happy to have the base data, and each of them just wanted to take it from there.\"\n\nVerifying the data\n\nRaymond reminds reporters that the database is only as good as the data you obtain. The reporter has to be able to verify the database information. \"The data you're going to get needs to be scrubbed up so it's garbage in and garbage out,\" Raymond says. In Seattle, reporters ran an explanatory note with the tables pointing out potential problems with the data. The note mentioned data-entry error by public agencies and that paper records were examined selectively. The note also stated that the contribution totals reported were mostly like the minimum anyone actually gave.\n\nMaking the data available\n\nAfter the database is built, it's important to get the information to key people in the newsroom and make sure they understand how to use it—otherwise you render the system worthless. Raymond's company sets up the database on an Intranet in which reporters use their Web browser to query the database and sort the data in various ways. An Intranet also gives data access to a reporter working from home or on the road.\n\nMaintaining the database\n\nThe campaign finance database increases in value when it's maintained and contribution data is continuously collected as candidates file their reports.\n\n\"Campaign finance databases are living and breathing things,\" Raymond says. \"They don't end at the end of an election cycle. Right after the election you have a whole other set. What happens in a newsroom is that they look at these things like a project with a beginning and end date.\"\n\nRaymond's company offers to maintain databases for newspapers and suggests, though from an obviously biased point of view, contracting out that duty to an outside party. \"It's almost inviting a waste of money if you try to do it in-house,\" Raymond says. \"That's when they set it up as a project as a one-time shot. Then the next election comes and it's going to be: 'We'll pitch it to somebody again. Dig it out and start from ground zero.'\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-defied-laws-diseased-labor","slug":"1999-januaryfebruary-defied-laws-diseased-labor","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-defied-laws-diseased-labor.html","year":1999,"month":"January/February","monthNum":1,"headline":"Defied laws, diseased labor","kicker":"BLACK-LUNG PERSISTENCE","author_name":"Gardiner Harris","author_title":"Reporter","affiliation":"The Courier Journal","summary":"An investigation into the coal mining industry's negligence of black lung disease laws, revealing routine defiance of mine dust level regulations.","keywords":["black lung","coal mining","negligence"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Nearly three decades after Congress passed laws intended to end the disease, black lung still kills nearly 500 coal miners each year. The Courier-Journal (Louisville, Ky.) decided to determine why. After more than a year of work, we printed in April a series concluding that the deaths result because coal operators, aided by miners themselves, routinely defy federal laws designed to control mine dust levels. Industry representatives have fiercely denied our charges but have not explained why so many tests in the dustiest areas of mines have impossibly small amounts of dust in them - proof that the tests aren't being conducted properly. Nor have they explained why one-third of all underground mines annually flunk at least one dust test or why two-thirds of all mines are cited each year for improper dust controls. All of these conclusions came from the computer work. The computer and on-the-ground reporting set up a dialogue. As we learned something in the field, we'd figure out new questions to ask the data. The data would in turn focus our field research. I suppose we could have written something without all of the computer analysis, but I don't know what it would have amounted to. Eastern Kentucky's mines are almost entirely non-union and small operations. Figuring out how to get in touch with hundreds of miners was my first challenge. The state has at least two lists of miners; in May 1996 I asked for both. We started calling miners. I visited anyone who agreed to talk, though not for long. After a couple of weeks, I had spoken to less than a dozen miners. Since I had planned to talk to hundreds but had initially only been given five months to finish, something had to give. The next set of data that I requested was dust-test records from the federal government. We only asked for a few years' worth. This was a mistake. After playing around with the most recent data, I realized that the graph looked very odd. I asked for more years. Then I asked for databases that would tell me which mines were sending in which records. After a lengthy process, we realized a systemic problem with how the mining industry reported data, which became the centerpiece of our investigation.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-januaryfebruary-simplifying-incidents","slug":"1999-januaryfebruary-simplifying-incidents","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-simplifying-incidents.html","year":1999,"month":"January/February","monthNum":1,"headline":"Simplifying incidents","kicker":"OMAHA CRIME","author_name":"Carol Napolitano","author_title":"","affiliation":"Omaha World-Herald","summary":"A newspaper's 17-month computer-assisted reporting project to analyze Omaha crime data from 13 million mainframe records, overcoming government resistance and data cleaning challenges to produce an accessible 12-page section.","keywords":["data","journalism","crime"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["mainframe","Excel","Access","FoxPro","SPSS","ArcView"],"full_text":"When \"Omaha Crime\" rolled off the presses in November, I thought to myself, \"Is this all there is?\"\n\nThe 12-page section about reported crime from 1990 to 1997 looked pretty simple. Good stories and photos, decent maps (despite our old presses), lots of information.\n\nI realized that the simplicity itself was the great achievement of the project. We had produced a readable, balanced, fair and informative section about crime from nearly 13 million convoluted mainframe records.\n\nTo me, there was nothing extraordinary. We could not boast about doing complex statistical analyses or unearthing some government secret. There was no place where it could be seen plainly why this project took 17 months of my life. Then, I realized that the simplicity itself was the great achievement of the project. We had produced a readable, balanced, fair and informative section about crime from nearly 13 million convoluted mainframe records.\n\nMany journalists can identify with the story of how \"Omaha Crime\" came into being. Most likely it isn't 13 million records you are after, but the number doesn't really matter. It's the process-the often confounding process of trying to obtain data from a reluctant government agency and, if you get it, the incredible amount of time it can take to clean that data and determine if it is reliable.\n\n\"Omaha Crime\" is a poster child for computer-assisted reporting. During the life of the project, we used Excel, Access, Visual FoxPro, SPSS and ArcView. We also designed a poll to assess how safe residents felt in Omaha.\n\nFirst steps\n\nThe journey began in January 1991 with written requests for electronic crime data from the police and sheriff. The newspaper, having limited CAR knowledge at that time, found itself unable to respond effectively to the government technobabble it got in response to those requests. So, the issue landed on the sidelines, arising only sporadically during the next five years.\n\nIn early 1996, with a more extensive understanding of CAR, we tried again. At our first negotiating session, we faced a roomful of lawyers and law enforcement officials who did not want the data in our hands.\n\nNeither the police nor the sheriff had ever used the data they collected. They had an antiquated mainframe system that allowed for little more than counting incidents for the FBI's Uniform Crime Report. As one police crime analyst put it: \"We come armed every month with a handful of pencils.\"\n\nThe last thing government officials wanted was a newspaper analyzing what they should have been analyzing themselves. So, at our final negotiating session in August 1996, the police department demanded a \"handling fee\" of $1.00 per record in addition to other data costs.\n\nThey figured we'd go away. Instead, we sued.\n\nTwo months later, they agreed to give us the data - reported incidents, bookings, arrest warrants, criminal histories and full personal information on perpetrators and victims.\n\nReliability check\n\nThe first installment of data arrived in June 1997. That's when crime numbers first consumed my life. More arrived throughout the year.\n\nHere are some of the things we did to help make sure the data was as reliable as possible:\n\nWe excluded from our study places that had their own security or police forces, such as universities.\n\nIncidents with \"unfounded\" status codes were removed.\n\nViolent incidents reported as having happened at police or sheriff offices, hospitals and extended-hour medical care centers, and jails and juvenile detention centers were removed. Non-violent incidents at police or sheriff facilities and jails and juvenile centers also were removed.\n\nIncidents reported at shopping malls were excluded from per capita calculations so they would not skew rates in surrounding neighborhoods.\n\nWe mapped where the incident happened, not where it was reported. If you are going to group your locations into larger areas like census tracts or block groups, be careful when mapping intersections. Determine on which corner the incident happened. If that is not feasible, randomly assign intersection incidents to the converging geographic areas.\n\nIf looking at incidents over a number of years, get population figures or estimates for as many years during that period as possible, then average them for your per capita rate. Using just one year may not be accurate if population is growing or declining substantially.\n\nBecause our data arrived in installments, the incident reports did not all end at the same time. We had homicides through July 9, 1997, drug offenses and robberies through July 22, arson cases through August 25, etc. So we weighted our crime rates for each offense.\n\nCarol Napolitano can be reached by e-mail at carolnap@owh.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-mapping-crimes","slug":"1999-januaryfebruary-mapping-crimes","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-mapping-crimes.html","year":1999,"month":"January/February","monthNum":1,"headline":"Mapping crimes","kicker":"FROM THE INCIDENT TO THE RATE","author_name":"Naomi Aoki and Toni Whitt","author_title":"","affiliation":"The Virginian-Pilot","summary":"Crime mapping helps readers assess neighborhood safety by visualizing either crime rates across areas or plotting specific incidents; The Virginian-Pilot successfully uses crime-rate maps that attract thousands of monthly visitors seeking information about local safety.","keywords":["crime","mapping","safety"],"topics":["Crime & Justice","Investigative Methods","Technology"],"technologies":["Access","FoxPro"],"full_text":"Like moths to a flame, people are drawn to news about crime. They want to know where, how and why it happened. But, most of all, they want to know if it could happen to them in their neighborhoods. There is no way to answer that question, but mapping crimes can help readers determine their risk.\n\nCrime maps fall into two categories: thematic maps showing crime rates or maps plotting specific incidents. Crime-rate maps are great for comparing the relative safety of different states, cities or neighborhoods. Maps plotting specific incidents are valuable for tracking crime in small towns, a burglary ring or a serial rapist operating in a certain area.\n\nRegardless of how you decide to map crimes, the first step is to get the incident report data from police. You need to know where the crime happened. For a crime-rate map, the 'where' needs to be a census tract, planning district, neighborhood or any other area for which you have population data. To plot individual crimes, you need an exact address or, failing that, a block number and street name.\n\nDepicting crime rates\n\nAt The Virginian-Pilot, we publish crime-rate maps showing the previous year's property and violent crime rates for four cities in our region. Depending on the city, the maps are broken down into census tracts, police zones or city planning districts.\n\nMapping the crime rates, rather than the individual incidents or number of crimes, allows readers to evaluate the safety of neighborhoods throughout the city. Readers like prospective home owners often call asking for the maps; close to 15,000 people visit the online version each month.\n\nStep-by-step\n\n1. Get police incident data in an electronic format, including the incident number (or other unique identifier), offense, date, time, address or block number, census tract or other area for which you have population data, and anything else you can get.\n2. Calculate crime rates for the census tract or designated areas, using a database manager like Access or FoxPro. (For a crime rate per 1,000 residents, divide the number of crimes in an area by the population and multiply that number by 1,000.)\n3. Create the necessary census tract or area files for your mapping program. Most programs come with census tract files, and most city planning departments can provide files for other district designations the cities use.\n4. Then, using the mapping program, map the crime rates you've calculated.\n\nPlotting crimes\n\nPlotting individual crimes can be a great tool for spotting patterns in crimes, but it is not a fair way of comparing crimes in different areas of the city. Areas that are densely populated and heavily trafficked are going to have more crimes. But if you notice a rash of rapes occurring in one geographical area, mapping will help you see any patterns. For example, in Virginia Beach, the state's largest city and popular resort area, several women living near the ocean reported being raped by a man who had broken into their homes. They gave similar descriptions of the suspect. Plotting those incidents on a map would have shown that the rapes all took place at the north end of the beach. That information alerted readers to be more cautious, to keep their windows and doors locked. Maps plotting incidents can also be useful for readers in small towns with fewer crimes. In larger cities, the sheer number of crimes makes plotting each incident impractical. But in areas with few crimes where it is possible to plot each incident, it gives readers a greater degree of specificity.\n\nMore step-by-step\n\n1. As with the crime-rate map, get all the incident data in an electronic format. The only difference is that you do not need census tract information for this map.\n2. Again, using a database manager, create a table of crimes you want to analyze. The table must include the exact address or block number and street name.\n3. Make sure the mapping program includes street level information for your city. Street level information is available (at a cost) with most mapping programs. You may also be able to find a friendly person in your city planning department who can help provide that information.\n4. Then, again, using your mapping program, plot the table of crimes.\n\nHappy mapping!\n\nNaomi Aoki can be reached by e-mail at aoki@pilotonline.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-build-your-own-tables","slug":"1999-januaryfebruary-build-your-own-tables","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-build-your-own-tables.html","year":1999,"month":"January/February","monthNum":1,"headline":"Build-your-own tables","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Journalists can build databases of campaign contributions by combining federal and state data, with key challenges including standardizing contributor names and coding occupations, requiring careful planning and often outsourcing data entry work.","keywords":["contributions","database","data"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"Disclosure, the company owned by Raymond and Kent Cooper, who both worked for the Federal Elections Commission and the Center for Responsive Politics. They built a database of contributions in Maryland races from 1994 to 1998. Last summer, Tom Brown at The Seattle Times combined federal and state campaign contributions into one database to determine Washington's Top 50 political contributors. The project, which they did in-house, took about two months to complete.\n\nIn both cases, some of the data were already in electronic format, eliminating the time-consuming and often costly effort of data entry. Both databases also identified the occupations and employees of as many major donors as possible.\n\nThe most tedious tasks were standardizing contributor names and coding contributors for occupation and industry. (There is no effort by federal or state election commissions to address the problem of contributor names being listed several different ways, even though the money is coming from the same person.) In both projects, to complete the coding, both the reporter and the consultants were forced to call individual contributors to verify or determine their occupation.\n\nProject proposals\n\nWhen Raymond and Cooper first start talking with a newspaper about a project, they set up a division of labor from the beginning to define who will do what. Basic questions they initially want answered include:\n\nWhat level of donations do you want? Do you want donations to federal candidates as well as state legislators and county commissioners?\n\nIs the information available electronically? What's the average number of transactions for each level: federal, state or local?\n\nIf the data are only available on paper, do you want to contract out with a data-entry firm to have in key-punched? Or does a state or local entity already key-punch the data into an in-house database, which you could request through the Freedom of Information Act?\n\nDo you want to code the contributions for occupation and employer by giving them an industry or sector classification?\n\nBoth of Poole's projects required data entry. Poole used Quality Data Systems, a nationwide data entry firm. Reporters can also find numerous data entry services near state capitols where state agencies require their work.\n\nWhen hiring a data entry firm, Poole suggests asking the firm's customers about the firm's accuracy rating. The contracts are usually based on a per-record basis. It's best to receive bids for the contract from two or three vendors, Poole says. Then ask a price for a specific number of records and an expected turnaround time.\n\nWhen doing data entry, key in twice the contributor's name, donation amount and ZIP code (to help later with verification). Also add fields for the page number on which the contribution is listed and the schedule of the contribution form. These extra fields make it easier to spot poor data entry and to cross-verify summary contribution information. \"I think it's overkill to fully verify (key in twice) every single field for cost and for time reasons,\" Poole says.\n\nThe newspaper needs to come up with clear rules for the data entry workers to follow, particularly when entering addresses. The database coordinator also needs to work closely with reporters who have covered campaign finance issues using paper records. In Poole's Virginia project, the summary contribution pages didn't have standardized date ranges. A query of the itemized data told the actual date range.\n\n\"On the summaries, it helps to know what they say and what they actually mean, which can often be two different things,\" Poole says.\n\nOccupation coding\n\nOccupation coding can be the most complicated, tedious and subjective process of building a database, but also the most enlightening.\n\n\"Some of the best stories that have been done are those where classification has been done,\" Raymond says. \"You can see trends and patterns, such as how much gambling has influenced a particular area.\"\n\nThe Post's project tried to identify employer and occupation for every donor who gave $1,000 or more. To make the matches, they ran names against other databases, looked up names in newspaper archives, and phoned several people just to ask who they were and what they did for a living. Public Disclosure did most of the work, identifying 78 percent of major donors to the governor's race.\n\nBut it's important to point out the caveats of occupation coding to editors and readers. Raymond uses the example of General Electric, which also owns a television network, makes commercial and defense jet engines, and constructs refrigerator parts.\n\n\"So what do you code GE as?\" Raymond asks. \"Is it a defense contractor, a home appliance company or jet crafter?\"\n\nWhen dealing with large companies, reporters need to be aware of the large players involved. When Raymond's company works with newspapers to code contributions, many coding decisions are left to the newspaper and reporters themselves, who are more familiar with area industries.\n\nIn the Seattle series, the reporters considered a couple as one entity when both were significant donors. They also included corporate contributions of companies known to be controlled by one individual. Brown warns that reporters looking at contributions over a long period must beware of contributors who have died. Four of their original Top 50 had died during the seven years of contributions the newspaper considered.\n\nStructuring the database\n\nRaymond suggests having a master table that lists details on all contributors. This serves as a look-up table for all the names. Raymond builds databases using Microsoft Access. Each contributor name is given a unique identifier. Then each office or race—such as the gubernatorial or state legislative races—becomes its own separate table.\n\n\"You're going to get information in the gubernatorial race that will be different than the legislature,\" he says. \"But you want to keep all that information. The most flexible thing you have going is that master table. It's really the key to it.\"\n\nWith this structure, every time a media outlet wants to add another level of campaign contributions, they don't have to change the database just add a related table for that office and add the contributor names to the master table.\n\nThe database structures for the New York and Virginia projects differ. In the Virginia database, every contributor and candidate has a unique identifying number in a relational database with 10 tables that relate to the main contributions table. They also code contributors for occupations. In New York, for competitive reasons within the consortium, the database only assigns a unique number to candidates. Each newspaper then decides whether they want to standardize names or code for occupation.\n\n\"The papers there have a much higher level of distrust,\" Poole says. \"They were all happy to have the base data, and each of them just wanted to take it from there.\"\n\nVerifying the data\n\nRaymond reminds reporters that the database is only as good as the data you obtain. The reporter has to be able to verify the database information. \"The data you're going to get needs to be scrubbed up so it's garbage in and garbage out,\" Raymond says.\n\nIn Seattle, reporters ran an explanatory note with the tables pointing out potential problems with the data. The note mentioned data-entry error by public agencies and that paper records were examined selectively. The note also stated that the contribution totals reported were mostly like the minimum anyone actually gave.\n\nMaking the data available\n\nAfter the database is built, it's important to get the information to key people in the newsroom and make sure they understand how to use it, otherwise you render the system worthless. Raymond's company sets up the database on an Intranet in which reporters use their Web browser to query the database and sort the data","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-coding-questions","slug":"1999-januaryfebruary-coding-questions","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-coding-questions.html","year":1999,"month":"January/February","monthNum":1,"headline":"Coding questions","kicker":"","author_name":"Cindy Eberting","author_title":"","affiliation":"CFIC","summary":"Journalists can use database classification and occupation coding to identify trends in campaign contributions, but must carefully consider how to categorize donors, especially large corporations with diverse business interests. Proper database structure using a master table with unique identifiers allows news organizations to flexibly track and analyze contributions across multiple races and offices.","keywords":["database","contributions","occupation"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Access"],"full_text":"ing a database, but also the most enlightening.\n\n\"Some of the best stories that have been done are those where classification has been done,\" Raymond says. \"You can see trends and patterns, such as how much gambling has influenced a particular area.\"\n\nThe Post's project tried to identify employer and occupation for every donor who gave $1,000 or more. To make the matches, they ran names against other databases, looked up names in newspaper archives, and phoned several people just to ask who they were and what they did for a living. Public Disclosure did most of the work, identifying 78 percent of major donors to the governor's race.\n\nBut it's important to point out the caveats of occupation coding to editors and readers. Raymond uses the example of General Electric, which also owns a television network, makes commercial and defense jet engines, and constructs refrigerator parts.\n\n\"So what do you code GE as?\" Raymond asks. \"Is it a defense contractor, a home appliance company or jet crafter?\"\n\nWhen dealing with large companies, reporters need to be aware of the large players involved. When Raymond's company works with newspapers to code contributions, many coding decisions are left to the newspaper and reporters themselves, who are more familiar with area industries.\n\nIn the Seattle series, the reporters considered a couple as one entity when both were significant donors. They also included corporate contributions of companies known to be controlled by one individual. Brown warns that reporters looking at contributions over a long period must beware of contributors who have died. Four of their original Top 50 had died during the seven years of contributions the newspaper considered.\n\nStructuring the database\n\nRaymond suggests having a master table that lists details on all contributors. This serves as a look-up table for all the names. Raymond builds databases using Microsoft Access. Each contributor name is given a unique identifier. Then each office or race—such as the gubernatorial or state legislative races—becomes its own separate table.\n\n\"You're going to get information in the gubernatorial race that will be different than the legislature,\" he says. \"But you want to keep all that information. The most flexible thing you have going is that master table. It's really the key to it.\" With this structure, every time a media outlet wants to add another level of campaign contributions, they don't have to change the database—just add a related table for that office and add the contributor names to the master table. The database structures for the New York and Virginia projects differ. In the Virginia database, every contributor and candidate has a unique identifying number in a relational database with 10 tables that relate to the main contributions table. They also code contributors for occupations. In New York, for competitive reasons within the consortium, the database only assigns a unique number to candidates. Each newspaper then decides whether they want to standardize names or code for occupation. \"The papers there have a much higher level of distrust,\" Poole says. \"They were all happy to have the base data, and each of them just wanted to take it from there.\"\n\nVerifying the data\n\nRaymond reminds reporters that the database is only as good as the data you obtain. The reporter has to be able to verify the database information. \"The data you're going to get needs to be scrubbed up so it's garbage in and garbage out,\" Raymond says. In Seattle, reporters ran an explanatory note with the tables pointing out potential problems with the data. The note mentioned data-entry error by public agencies and that paper records were examined selectively. The note also stated that the contribution totals reported were mostly like the minimum anyone actually gave.\n\nMaking the data available\n\nAfter the database is built, it's important to get the information to key people in the newsroom and make sure they understand how to use it—otherwise you render the system worthless. Raymond's company sets up the database on an Intranet in which reporters use their Web browser to query the database and sort the data in various ways. An Intranet also gives data access to a reporter working from home or on the road.\n\nMaintaining the database\n\nThe campaign finance database increases in value when it's maintained and contribution data is continuously collected as candidates file their reports.\n\n\"Campaign finance databases are living and breathing things,\" Raymond says. \"They don't end at the end of an election cycle. Right after the election you have a whole other set. What happens in a newsroom is that they look at these things like a project with a beginning and end date.\" Raymond's company offers to maintain databases for newspapers and suggests, though from an obviously biased point of view, contracting out that duty to an outside party. \"It's almost inviting a waste of money if you try to do it in-house,\" Raymond says. \"That's when they set it up as a project as a one-time shot. Then the next election comes and it's going to be: 'We'll pitch it to somebody again. Dig it out and start from ground zero.\"\n\nCindy Eberting can be reached by e-mail at cindy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-let-the-buyer-know-fares","slug":"1999-januaryfebruary-let-the-buyer-know-fares","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-let-the-buyer-know-fares.html","year":1999,"month":"January/February","monthNum":1,"headline":"Let the buyer know fares","kicker":"PRICING DOGFIGHT","author_name":"Bob Warner","author_title":"Reporter","affiliation":"Philadelphia Daily News","summary":"A Philadelphia newspaper analyzes airline fare data from the U.S. Transportation Department to show that Philadelphia has the third-highest average fares among major U.S. cities, revealing pricing disparities that affect travelers.","keywords":["fares","airlines","Philadelphia"],"topics":["Business & Economy","Investigative Methods","Technology"],"technologies":["Access","Excel","Internet","World Wide Web"],"full_text":"Let the buyer know fares\n\nBy Bob Warner\nPhiladelphia Daily News\n\nHow do airline fares from your city stack up against airfares from other cities? In Philadelphia, a hub for US Airways, we've always had the sense that our fares were on the high side. Planning a trip to the West Coast or overseas, you can usually save serious money by traveling up to Newark, N.J., or down to Baltimore. We've long been irked that it costs more to fly from Philadelphia to Pittsburgh, another US Airways hub, than to catch a flight all the way to Florida.\n\nBut how do you document the cost of air travel from Philadelphia or compare it to other cities? We never figured out a way until we noticed a brief item in The New York Times travel section, mentioning a Web site on airline fares, set up by the U.S. Transportation Department at http://ostpoweb.dot.gov/aviation/domfares/domfares.htm\n\nWith just a few hours' work after downloading several files and working them into Access and Excel, we calculated average fares per mile on the busiest air routes from Philadelphia and compared them to fare levels (known in the airline industry as \"yields\") on the busiest routes from other cities.\n\nIn our opinion, the results were newsworthy: average fares from Philadelphia in the period we looked at (the first quarter of 1998) were 24.2 cents a mile - third highest among the nation's 30 busiest air destinations, just behind Minneapolis (a Northwest Airlines hub, 25.5 cents a mile) and Dallas (an American Airlines hub, 24.8 cents a mile).\n\nPreserving competition\n\nThe Transportation Department has assembled this data as part of a federal effort to deal with anti-competitive practices in the airline industry. The original source of the data is the airlines themselves, which are required to submit voluminous data every quarter that list the origin, destination, ticket price and more on 10 percent of all tickets they sell. The data is known inside the department as the \"10 percent O & D\" (for origin and destination).\n\nWhat's posted on the Web is not the original data but tables derived from it, covering more than 1,100 of the busiest domestic air routes. For us, the most valuable table was the first one, which described each of 1,179 routes with the following information: origin and destination cities; the mileage between those cities; the average number of passengers each day; the average fare; the carrier with the largest market share and its average fare and market share; and the carrier offering the lowest average fare and its market share.\n\nOther tables describe average fares from each city, changes in fare levels between 1997 and 1998, and more detailed information on specific airlines and market shares on specific routes.\n\nWobbly tables\n\nThe data goes back to the beginning of 1997 and is updated quarterly. There are several pitfalls.\n\nFirst, the tables available on the Web do not differentiate between multiple airports in a single metropolitan area. For flights beginning or ending in Chicago, for example, there's no way to separate fares from O'Hare and Midway. For New York City, the problem is more difficult: flights using Newark Airport, where several low-cost carriers operate, are lumped together with flights from JFK and LaGuardia. Reporters in New York City would have to go to the original O & D data to deal with this problem.\n\nSecond, the federal data relies on calculations of an \"average\" fare for each city-to-city route. As airlines are quick to point out, in the real-life airline industry there's no such thing as an \"average\" fare. On most major routes, the airlines start with a three-tiered rate structure: first-class, short-notice \"business\" fares and lower-cost \"leisure\" fares requiring ticket purchases weeks in advance. But on top of that structure, each major airline has developed ways to modify its fare offerings around the clock - to deal with competitive pressures from other airlines and to confront the real-earnings specter of large blocks of empty, unsold seats.\n\nUS Airways told us that the federal study is seriously flawed because it fails to account for variances between business travel and leisure travel from different airports. We quoted the airline to this effect, but they didn't convince us to drop the story. In Philadelphia, we do have a relatively high proportion of business travel. But there is sufficient anecdotal evidence to suggest that this proportion is an effect - not a cause - of high local fares. Many leisure travelers, those paying for their own tickets, are traveling to other airports to save money, raising the business-to-leisure travel ratio further.\n\nThird, working with the data, you have to be careful to avoid \"averaging averages.\" If you're dealing with 40 different routes involving your local airport, you can't just add up the 40 \"average\" fares, add up the 40 distances, and divide to get an average fare per mile. You have to make adjustments based on how many passengers used each route. Fortunately, the posted federal data include the number of passengers traveling each route, enabling reporters to simply multiply to find figures for total passenger miles and total fares. (We made this mistake, though it's easy to avoid, in our first pass at the data.)\n\nBob Warner can be reached by phone at (215) 854-5885 or by e-mail at bwarner@philly.infi.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-dealing-wheels","slug":"1999-januaryfebruary-dealing-wheels","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-dealing-wheels.html","year":1999,"month":"January/February","monthNum":1,"headline":"Dealing wheels","kicker":"SCHOOL VAN SAFETY QUESTIONS","author_name":"Andy Lehren","author_title":"Producer","affiliation":"Dateline","summary":"Dateline investigated how federal agencies fail to enforce laws prohibiting the sale of unsafe vans to schools, using computerized vehicle databases to show that enforcement could be easily improved and that hundreds of schools illegally operate vans despite their inferior safety features.","keywords":["safety","enforcement","databases"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["FARS"],"full_text":"Producer Shachar Bar-On came into my office wanting to know whether we could use computerized records to show a federal agency was not enforcing school bus safety laws. We could, and it's a story that could be done by many local and regional news organizations around the country.\n\nA little-known federal law forbids car dealers from selling vans to schools. Schools save money by buying vans, which cost less to operate than traditional, small-sized yellow school buses. But experts say vans are far less safe for transporting students - mainly because of inferior structural protection and interior padding. Manufacturers do include warning letters in these vans that clearly state \"not a school bus.\"\n\nThe safety question is not hypothetical. In July 1994, a truck rammed into a school van in South Carolina, killing 6-year-old Jacob Strebler. Yet 18 months earlier, in an almost identical accident in the same part of town, a truck had slammed into a small-sized school bus - and none of the children was seriously hurt. Dateline found Strebler was one of at least a dozen children who died in vans they were never supposed to be in. According to experts, they most likely would have survived if they had been in school buses.\n\nThe National Highway Traffic Safety Administration is supposed to check whether vans are being sold to schools. The law is a little unusual: it is illegal for dealers to sell the vans to schools but not illegal for schools to own them. Since the 1974 law took effect, NHTSA has taken 10 enforcement actions against dealers for selling vans to schools. NHTSA said it only has so much staff to handle all its duties.\n\nBut by using vehicle databases, Dateline showed how easily the agency could be checking on vans sold to schools for transporting students. We picked a state, Texas - and obtained its database for motor vehicles. We looked at vehicles owned by schools and then analyzed the data to find which schools owned vans for transporting students. We found hundreds. The point was not only to show that vans were being sold to schools but also to prove that it was not hard to enforce the law. It helped us question NHTSA about how it was not taking simple steps to prevent the deals from happening.\n\nRecord roadblocks\n\nThe school van safety story is one that many news organizations across the country could do for their own communities. Bar-On and Dateline correspondent Bob McKeown did terrific reporting. We encountered a few difficulties. For example, we examined hundreds of thousands of computerized accident reports in the U.S. Department of Transportation's Fatal Accident Reporting System. These records were not perfect. In a few instances, vans were incorrectly marked as being used by schools. More often, vans were not properly identified as being used by schools.\n\nTo help determine which vans are used to transport children, we paid particular attention to vehicle identification numbers - an automobile's equivalent of a Social Security number. It is not a random set of numbers and letters. It describes the vehicle, including any seating arrangement, and can be decoded with easily obtainable reference books or databases. By doing this, we separated vans for hauling maintenance equipment from those for transporting students.\n\nThis is also an example of a story that may not be possible in certain states, depending on privacy laws and rules on electronic access to data. While many states provide access to vehicle ownership records, this is not true everywhere - another reminder that journalists need strong open-records laws to do their job.\n\nAndy Lehren can be reached by e-mail at alehren@nbc.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-neighborhood-rap-sheet","slug":"1999-januaryfebruary-neighborhood-rap-sheet","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-neighborhood-rap-sheet.html","year":1999,"month":"January/February","monthNum":1,"headline":"Neighborhood rap sheet","kicker":"FIRST VENTURES","author_name":"Pete Basofin","author_title":"Reporter","affiliation":"Sacramento Bee","summary":"Sacramento Bee reporters analyzed police crime data by subdistrict to identify neighborhood crime patterns and trends, then conducted community interviews to provide context and depth to their findings.","keywords":["crime","data","neighborhoods"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["World Wide Web"],"full_text":"The idea to examine crime patterns in Sacramento came from the city police Web site, where citizens can examine current neighborhood crime statistics. Initially, Bee reporter Nancy Weaver Teichert and I thought we would gather older data to support a story about neighborhood crime trends. We also wanted to rank neighborhoods in terms of crime patterns.\n\nBut it quickly became apparent that Sacramento neighborhoods could not be compared with any accuracy. Besides the great variation in size and population, the boundaries of the geographic areas employed in police crime reporting did not fit exactly with accepted neighborhood boundaries. While per capita comparisons might have worked, we did not have easy access to population figures for such segments of the city.\n\nFortunately, the reported crime data could be analyzed for a smaller, more uniform geographic parcel. The police subdistrict is an administrative unit less than a square mile in size. Although the population density does vary, we felt comparisons and sorting were valid for these 520 subdivisions. By ranking subdistricts, we could identify specific trouble spots in the city.\n\nTerminology translation\n\nPolice provided the Bee with annual data (1991-1997) in seven files. To simplify the process of creating crime categories, we followed police practice in translating California penal code numbers into common terminology - homicide, kidnapping, auto theft, etc. For summary purposes, we then grouped these crime categories into broader crime types - property crime, violent crime and weapons/narcotics. We used the latter to rank police subdistricts and display historical data for each area.\n\nPolice estimated a 10 percent error in the reported crime statistics. Since the material originated from busy cops on the beat, errors did creep into the complex reporting form. Officers must make quick, complicated judgements about the exact nature of the crime and its assignment to a penal code number.\n\nClose examination of the data revealed occasional inaccuracy in the identification of police subdistrict numbers. In general, it was important to discuss the accuracy of field information with the police. For example, we were told that the tagging of gang-related crimes should not be used because it was done so erratically.\n\nSubdistrict statistics\n\nThe project published in the newspaper was inspired - but not dominated - by statistics. Reporters used the data to identify parts of the city that suffered from high crime or enjoyed low crime. Then they went into the community to speak with many sources, including police, crime victims, shopkeepers, homeowners, apartment residents, commuters, and neighborhood activists. These interviews breathed life into the series.\n\nThe series uncovered particular conditions fostering various types of crime. Neighborhood profiles identified specific problem spots, such as malls, bus stations, and convenience stories, which attract criminals. A commonly held belief about the relative high crime in one inner-city neighborhood was dispelled by the data presented.\n\nBecause of heavy emphasis on police subdistricts, the project featured large maps locating the regions of high property and violent crime. It was clear at the onset that we couldn't display locator maps and statistics for all 520 subdistricts. So we turned to the Bee's Web site to create a permanent database with which readers can call up historical data on the area immediately surrounding their home or business.\n\nThe biggest challenge in displaying data on the Web was helping users find their police subdistricts. News artist Mitchell Brooks gave the Bee's New Media department a highly detailed map showing the streets bordering each parcel. These were divided into twelve smaller maps, which the user navigates through a clever \"zoom-in, zoom-out\" feature. Once the subdistrict of interest is found, the user simply enters the four-digit code into a box to get total numbers of property, violent and weapons/narcotics crimes for the years 1991-97.\n\nPete Basofin can be reached by e-mail at pbasofin@sacbee.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-upcoming-boot-camps","slug":"1999-januaryfebruary-upcoming-boot-camps","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-upcoming-boot-camps.html","year":1999,"month":"January/February","monthNum":1,"headline":"UPCOMING BOOT CAMPS","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering an advanced statistics workshop in May 1999 at UNC's journalism school, instructed by Philip Meyer and Sarah Cohen, with an optional SPSS basics session. Additional boot camp training programs are scheduled throughout May, July, and August in Columbia, Missouri.","keywords":["statistics","training","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["SPSS"],"full_text":"ADVANCED STATISTICS WORKSHOP MAY 10-13, 1999 CHAPEL HILL, NC IRE AND NICAR AND THE UNIVERSITY OF NORTH CAROLINA SCHOOL OF JOURNALISM WILL HOST AN ADVANCED SEMINAR ON USING STATISTICS IN COMPUTER-ASSISTED REPORTING INSTRUCTORS WILL INCLUDE PHILIP MEYER AND SARAH COHEN. AN OPTIONAL SESSION ON THE BASICS OF THE STATISTICAL SOFTWARE SPSS WILL TAKE PLACE ON SUNDAY MAY 9. (MOTHER'S DAY)\n\nEDITOR BOOT CAMP MAY 13-15, 1999\nCOLUMBIA, MO\n\nBASIC BOOT CAMP MAY 16-21, 1999\nCOLUMBIA, MO\n\nBASIC BOOT CAMP JULY 18-23, 1999\nCOLUMBIA, MO\n\nBASIC BOOT CAMP AUG. 8-13, 1999\nCOLUMBIA, MO\n\nCALL (573) 882-2042 FOR MORE INFORMATION OR VISIT THE IRE WEB SITE AT www.IRE.ORG/TRAINING","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-planes-trains","slug":"1999-januaryfebruary-planes-trains","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-planes-trains.html","year":1999,"month":"January/February","monthNum":1,"headline":"Planes, trains","kicker":"HANDOUTS OF THE MONTH","author_name":"","author_title":"","affiliation":"","summary":"A guide for CAR reporters on using accident and transportation safety databases, emphasizing the importance of normalizing data, understanding local versus federal sources, and verifying data completeness before publication.","keywords":["data","accidents","databases"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["World Wide Web"],"full_text":"The following excerpts from handouts concerning CAR on the transportation beat are all available in their entirety from the Resource Center. Request the number listed in parentheses. Call (573) 882-3364 for more information.\n\nCrash course (#816)\nBy Justin Mayo\nThe Seattle Times\n\nWhen things collide, one of the results is data. Generally, the destruction of property or human injury generates huge amounts of paperwork, which eventually ends up in a database. This grim reality is good news for CAR reporters because of the abundance of accidents - and therefore accident data. Additionally, transportation safety is an issue that will always be important to the audience and one that can clearly be enhanced through computer-assisted reporting.\n\nThree things to keep in mind:\nFirst, although many databases are constructed or disseminated at the federal level, remember that you can also tap local agencies - especially for automobile accidents.\n\nMost of the time the state will be more current and complete. However, local agencies might be reluctant to give out the entire database without redacting some information. So be prepared to know what you really need to get out of the database and be ready to fight for it. A good strategy is to first request paper records and the record layout of the database. That way you can see what's collected at the scene of the accident and what information is available in the database.\n\nIf you're going to write about individual accidents, always ask for the paper records.\n\nSecond, when trying to make comparisons, you'll need to normalize the data. Usually, raw numbers will not be enough. Normalizing data will depend on the nature of your story. For example, if you're looking at fatality rates for the elderly, you'll have to get statistics based on the number of miles driven by age of the driver. On the other hand, if you want to look at the type of vehicles involved in rollover accidents, the number of cars on the road by type would be necessary.\n\nThird, beware of online data sources. Many of the federal accident databases are available online, particularly aviation data. These sites can be a great resource for deadline searches, but be warned that some are incomplete. It's best to get the entire dataset directly from the agency (or NICAR) if you plan to use a particular database on a regular basis.\n\nNOTE: In the remainder of his handout, Mayo discusses available databases organized under the topic headings of air, roads, rail and agencies, the Web and/or NICAR.\n\nJustin Mayo can be reached by e-mail at jmayo@seattletimes.com\n\nMust-have resource (#677)\nBy Heather Newman\nDetroit Free Press\n\nThe Directory of Transportation Data Sources, published by the Bureau of Transportation Statistics (the Mecca of free transportation data), is the best collection of database descriptions anywhere. (And it's also free.) It includes a comprehensive listing of data collected by dozens of federal agencies and who to call for information. It also gives you that crucial what-is-this-darn-database-called edge when filing your FOIAs. Call BTS at (202) 366-DATA or order from the Web site at http://www.bts.gov.\n\nKeep in mind that nearly everything collected federally comes from a local agency first - so updated and complete information might be just around the corner instead of in D.C. Cultivate your state police department for freeway accident data, your city police department for pedestrian hit-and-runs, your county transportation department for road condition and traffic information, your state Department of Natural Resources for boating or snowmobile accidents.\n\nHeather Newman can be reached by e-mail at newman@freepress.com\n\nColliding databases (#816)\nBy Elizabeth Marchak\nThe Plain Dealer\n\nAviation reporting is complicated and time-consuming. The computer makes it even more so. A few notes of caution: Don't wait until deadline. Learn what offices the FAA, the NTSB and the IG have in your area. If they hang up, don't give up. The aviation community has a low opinion of reporters. You have to convince them you are serious.\n\nDevelop sources you never quote. People you can double-check your findings with are more valuable than anybody who can give you a pithy quote on deadline. Secret sources are your reality check.\n\nHave a plane crash plan in place in your newsroom. Know how your community plans to respond. Develop contacts at the NTSB, which investigates plane crashes (not the FAA). Learn the schedule for the release of reports and opening of dockets.\n\nKnow how to search for accidents and incident data. Know how to search for a whole fleet using the four-digit OP code, or operator code. (Computer data for one plane is rarely sufficient and is unfair to the operator.) Know ATA or airplane system codes. Know how to search for N-numbers and serial numbers. (A tail number or an N-number may change, but a serial number shouldn't.) And learn the lingo, but keep it out of your stories. Learn to put repairs and problems in context. A rash of reports of lavatory and exit lights problems are far less serious than air returns for lack of radar. Remember, it's what isn't reported that is often the real problem. Don't use data on deadline you've never seen before. There's a really good reason Web pages have disclaimers about completeness - they really don't have everything. Many news organizations have ignored these warnings and shoved data into the paper or on the air, making huge mistakes, missing important angles or ruining their credibility with potential sources in the process.\n\nFile FOIAs on every single aviation entity in your area. This is the best way to find out who has been inspected and when, who has been \"violated\" (FAA slang for who has been investigated for violating regulations), and what action the FAA took. File for every single significant event, too. There are FOIA officers out there who love a well-written request on something nobody else is paying attention to, so let er rip.\n\nElizabeth Marchak can be reached by e-mail at marchak@digex.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-crash-course-816","slug":"1999-januaryfebruary-crash-course-816","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-crash-course-816.html","year":1999,"month":"January/February","monthNum":1,"headline":"Crash course (#816)","kicker":"","author_name":"Justin Mayo","author_title":"Reporter","affiliation":"The Seattle Times","summary":"A guide for computer-assisted reporters on using accident and transportation safety data from federal and local sources, emphasizing the importance of data normalization and caution with online databases.","keywords":["accidents","databases","transportation"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":[],"full_text":"Crash course (#816)\nBy Justin Mayo\nThe Seattle Times\n\nWhen things collide, one of the results is data. Generally, the destruction of property or human injury generates huge amounts of paperwork, which eventually ends up in a database. This grim reality is good news for CAR reporters because of the abundance of accidents - and therefore accident data. Additionally, transportation safety is an issue that will always be important to the audience and one that can clearly be enhanced through computer-assisted reporting.\n\nThree things to keep in mind:\n\nFirst, although many databases are constructed or disseminated at the federal level, remember that you can also tap local agencies - especially for automobile accidents.\n\nMost of the time the state will be more current and complete. However, local agencies might be reluctant to give out the entire database without redacting some information. So be prepared to know what you really need to get out of the database and be ready to fight for it. A good strategy is to first request paper records and the record layout of the database. That way you can see what's collected at the scene of the accident and what information is available in the database.\n\nIf you're going to write about individual accidents, always ask for the paper records.\n\nSecond, when trying to make comparisons, you'll need to normalize the data. Usually, raw numbers will not be enough. Normalizing data will depend on the nature of your story. For example, if you're looking at fatality rates for the elderly, you'll have to get statistics based on the number of miles driven by age of the driver. On the other hand, if you want to look at the type of vehicles involved in rollover accidents, the number of cars on the road by type would be necessary.\n\nThird, beware of online data sources. Many of the federal accident databases are available online, particularly aviation data. These sites can be a great resource for deadline searches, but be warned that some are incomplete. It's best to get the entire dataset directly from the agency (or NICAR) if you plan to use a particular database on a regular basis.\n\nNOTE: In the remainder of his handout, Mayo discusses available databases organized under the topic headings of air, roads, rail and agencies, the Web and/or NICAR.\n\njustin Mayo can be reached by e-mail at jmayo@seattletimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-must-have-resource-677","slug":"1999-januaryfebruary-must-have-resource-677","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-must-have-resource-677.html","year":1999,"month":"January/February","monthNum":1,"headline":"Must-have resource (#677)","kicker":"","author_name":"Heather Newman","author_title":"","affiliation":"Detroit Free Press","summary":"The Bureau of Transportation Statistics publishes a free Directory of Transportation Data Sources that catalogs databases from dozens of federal agencies; journalists are also advised to contact local and state agencies directly for the most current transportation data.","keywords":["transportation","data","government"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"The Directory of Transportation Data Sources, published by the Bureau of Transportation Statistics (the Mecca of free transportation data), is the best collection of database descriptions anywhere. (And it's also free.) It includes a comprehensive listing of data collected by dozens of federal agencies and who to call for information. It also gives you that crucial what-is-this-darn-database-called edge when filing your FOIAs. Call BTS at (202) 366-DATA or order from the Web site at http://www.bts.gov.\n\nKeep in mind that nearly everything collected federally comes from a local agency first - so updated and complete information might be just around the corner instead of in D.C. Cultivate your state police department for freeway accident data, your city police department for pedestrian hit-and-runs, your county transportation department for road condition and traffic information, your state Department of Natural Resources for boating or snowmobile accidents.\n\nHeather Newman can be reached by e-mail at newman@freepress.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-colliding-databases-816","slug":"1999-januaryfebruary-colliding-databases-816","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-colliding-databases-816.html","year":1999,"month":"January/February","monthNum":1,"headline":"Colliding databases (#816)","kicker":"","author_name":"Elizabeth Marchak","author_title":"","affiliation":"The Plain Dealer","summary":"Aviation reporting requires meticulous research, developing reliable sources, understanding FAA and NTSB databases, and filing FOIA requests to uncover safety issues and regulatory violations. Journalists must verify data thoroughly, avoid deadline mistakes, and build credibility with the aviation community.","keywords":["aviation","reporting","databases"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":[],"full_text":"Colliding databases (#816)\nBy Elizabeth Marchak\nThe Plain Dealer\n\nAviation reporting is complicated and time-consuming. The computer makes it even more so. A few notes of caution: Don't wait until deadline. Learn what offices the FAA, the NTSB and the IG have in your area. If they hang up, don't give up. The aviation community has a low opinion of reporters. You have to convince them you are serious.\n\nDevelop sources you never quote. People you can double-check your findings with are more valuable than anybody who can give you a pithy quote on deadline. Secret sources are your reality check.\n\nHave a plane crash plan in place in your newsroom. Know how your community plans to respond. Develop contacts at the NTSB, which investigates plane crashes (not the FAA). Learn the schedule for the release of reports and opening of dockets.\n\nKnow how to search for accidents and incident data. Know how to search for a whole fleet using the four-digit OP code, or operator code. (Computer data for one plane is rarely sufficient and is unfair to the operator.) Know ATA or airplane system codes. Know how to search for N-numbers and serial numbers. (A tail number or an N-number may change, but a serial number shouldn't.) And learn the lingo, but keep it out of your stories. Learn to put repairs and problems in context. A rash of reports of lavatory and exit lights problems are far less serious than air returns for lack of radar. Remember, it's what isn't reported that is often the real problem. Don't use data on deadline you never seen before. There's a really good reason Web pages have disclaimers about completeness they really don't have everything. Many news organizations have ignored these warnings and shoved data into the paper or on the air, making huge mistakes, missing important angles or ruining their credibility with potential sources in the process.\n\nFile FOIAs on every single aviation entity in your area. This is the best way to find out who has been inspected and when, who has been \"violated\" (FAA slang for who has been investigated for violating regulations), and what action the FAA took. File for every single significant event, too. There are FOIA officers out there who love a well-written request on something nobody else is paying attention to, so let er rip.\n\nElizabeth Marchak can be reached by e-mail at marchak@digex.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-using-aviation-safety-documents-677","slug":"1999-januaryfebruary-using-aviation-safety-documents-677","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-using-aviation-safety-documents-677.html","year":1999,"month":"January/February","monthNum":1,"headline":"Using aviation safety documents (#677)","kicker":"","author_name":"Byron Acohido","author_title":"Reporter","affiliation":"Seattle Times","summary":"The NTSB's aircraft accident investigation process relies heavily on input from manufacturers and airlines who have financial incentives to minimize culpability, potentially delaying preventative safety measures until crashes occur. As aviation traffic increases and aircraft age, experts warn that this approach is inadequate and the public remains largely uninformed about safety concerns.","keywords":["investigation","manufacturers","safety"],"topics":["Public Safety","Investigative Methods"],"technologies":[],"full_text":"Know the landscape. Determining the probable cause of an aviation disaster is the responsibility of the NTSB, a small, quirky federal agency whose investigators pride themselves on objectivity and scientific certainty. Yet the NTSB openly solicits technical expertise from aircraft manufacturers, airlines, pilots and mechanics. You can bet each group possesses a strong self-protection bias. The NTSB's crash-site \"go-team\" usually includes engineers from the aircraft and engine manufacturers, who invariably take the lead in identifying and retrieving wreckage material. The manufacturers then use their own labs and technicians to analyze and test suspect parts. Pilots, mechanics and airline representatives are assigned to check maintenance records and research what role, if any, ground and flight crew missteps may have played in the accident. Commercial jets crash, on average, less than two times for every 1 million flights. Assuming the overall accident rate remains low for the foreseeable future, air travel should continue to expand about 5 percent annually. That means the total number of crashes will continue to climb year to year. Factor in aging aircraft issues and increasing competition among airlines, and a disturbing trend isn't hard to predict. In fact, many experts think the trend toward messy air disasters occurring weekly is already well under way.\n\nThe present approach of dealing with accidents by permitting the potentially culpable parties to sift through wreckage, negotiate a probable cause and then methodically take steps to make sure it doesn't happen again is under strain. It gives manufacturers and airlines, who are driven by economic concerns, the benefit of the doubt; and it skews against preventative measures taken in a timely fashion. Too often a long trail of problem flights, dead-end investigations and incremental upgrades - precursors of a disaster in the making - is ignored until a crash or crashes occur. Meanwhile the public is by and large left out of the loop.\n\nByron Acohido can be reached by e-mail at bacchido@seattletimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-transportation-scams-and-bungling-some-tips-847","slug":"1999-januaryfebruary-transportation-scams-and-bungling-some-tips-847","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-transportation-scams-and-bungling-some-tips-847.html","year":1999,"month":"January/February","monthNum":1,"headline":"Transportation scams and bungling - some tips (#847)","kicker":"","author_name":"Bill Krueger","author_title":"","affiliation":"The News & Observer","summary":"A guide for journalists investigating potential corruption and mismanagement in state transportation departments, including curb cuts, right-of-way settlements, discretionary spending, road maintenance standards, accident prevention priorities, and industrial access road funding.","keywords":["transportation","corruption","infrastructure"],"topics":["Investigative Methods","Public Safety","Government & Politics"],"technologies":[],"full_text":"Figure out who's getting access to the roads. Find the traffic engineers who are responsible for considering requests for curb cuts on major roads-requests from developers and business owners - ask them if their recommendations have ever been overturned or ignored.\n\nLook for right-of-way scams. Most deals aren't worth spending your time on, but check with the Federal Highway Administration on any right-of-way settlements they refused to sign. Chances are, there's something funny about the deal if the Feds won't sign off. Ask to see the appraisals prepared for the department and compare those amounts to the amount actually paid. In fact, ask to see the entire project file.\n\nFind the slush funds. Talk to veteran transportation officials, politicians, and advocacy groups to find the \"discretionary\" funds, who controls them, and what they're used for. Get the records on how the money has been spent (ask if they're in a computer database) and then compare them to the department's priorities. It's a sure bet the two lists won't match up.\n\nTake the bite out of the watchdog. When you find the slush funds, see if a lot of money is going to members of legislative oversight committees or other legislators with power over the transportation department. Ask for any correspondence or records of phone calls from those legislators.\n\nAre your roads being maintained? Most states have goals, or standards, for resurfacing their roads. Find out what those goals are; then find out how many miles the state is resurfacing each year; then do the math. Is your state meeting its own goals? If not, why not? Talk to the maintenance engineers out in the field - they can tell you where the problems are.\n\nCould that accident have been prevented? State transportation agencies spend a lot of money analyzing intersections and other potential danger spots on roads. If there is a bad accident on a road near you, check with the state's traffic safety engineers to see if they've studied that intersection. If they already knew it was dangerous, why didn't they fix it? Had they spent the money on a project that was a lower priority but had more political muscle behind it? Again, ask to see the project file.\n\n\"If it's good for bizness\" Find out if your state has a fund for industrial access roads. Such roads often amount to little more than private driveways for companies - paid for by taxpayers. Check the files to find out who weighed in on the company's behalf and whether the department followed its own rules for paying for such roads.\n\nBill Krueger can be reached by e-mail at bkrueger@nando.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-hidden-hazards-618","slug":"1999-januaryfebruary-hidden-hazards-618","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-hidden-hazards-618.html","year":1999,"month":"January/February","monthNum":1,"headline":"Hidden hazards (#618)","kicker":"","author_name":"Penny Loeb","author_title":"","affiliation":"U.S. News & World Report","summary":"This guide helps journalists investigate train and subway accidents using federal databases to identify safety patterns, problem railroads, and dangerous crossings. It provides reporting tips on accessing FRA data, understanding accident records, and identifying root causes like poor track maintenance and defective equipment.","keywords":["trains","accidents","safety"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["Lotus 1-2-3","World Wide Web","FTP"],"full_text":"Nearly once every hour, somewhere in the country, a train falls of the track, collides with another train or has some other sort of mishap. Often the accident is miles from nowhere. Only occasionally does a fiery and fatal crash in a metropolitan area draw national attention.\n\nFortunately, there are lots of data to help reporters find out where accidents are occurring and which railroads have the worst records. The Federal Railroad Administration has downloadable data files for the past six years on its Web site. These cover accidents by every railroad, employee injuries, and accidents at railroad crossings. The site also has text and tables from the annual report on safety.\n\nCaution: One accident can have multiple records - one for each railroad involved and one for the track owner. You must group by accident ID, date, time and the track owner when counting accidents. Double-check your results against the annual report.\n\nA rail-crossing accident is the most frequent cause of deaths. You can easily find how many accidents occurred in your area in the federal data. Then check out the safety features of the crossings. Many crossings are without gates. Watch drivers when they cross; sometimes they will ignore warning signals just to save a couple of seconds.\n\nOld and rickety track is the leading cause of train accidents. Find out which railroad has the most accidents caused by bad track and then figure out why. As railroads cut back workers, there just aren't enough to repair the broken track. Railroads, too, haul ore and other loads too heavy for some kinds of track.\n\nTo increase profits, railroads are sending out cars with serious defects. These include broken wheels inoperative brakes, and bad shock-absorbing mechanisms. Ask for FRA inspections of the railroad you are covering. Talk to the workers who fix the trains.\n\nSUBWAYS:\n\nWant to know how many people were injured in your subway or metro? How many were robbed, and how that compares to big cities? Or how much it costs to run the system?\n\nThe answers can all be found in great detail in data on the Web site of the Federal Transit Administration. There are data on accidents, crime, and costs. The data are in Lotus spreadsheets and cover 1993-96. The data also contain the miles traveled for each system. You can establish a rate of accidents per mile traveled to equalize results between systems of different sizes.\n\nPenny Loeb can be reached by e-mail at pioeb@usnews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-transportation-sites","slug":"1999-januaryfebruary-transportation-sites","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-transportation-sites.html","year":1999,"month":"January/February","monthNum":1,"headline":"Transportation sites","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A guide to transportation-related websites and databases organized by mode (air, rail, and road) that provide journalists with access to safety data, accident information, and regulatory filings for covering the transportation beat.","keywords":["transportation","databases","safety"],"topics":["Public Safety","Technology","Investigative Methods"],"technologies":["Internet","World Wide Web","CD-ROM"],"full_text":"The following Web sites were mentioned in various handouts discussing CAR and the transportation beat.\n\nAir\n\nAirsafe Financial - http://airsafe.com\nThis site lists airline safety information for the airline passenger and contains various aircraft accident data.\n\nDaily Airline Filings - http://airline info.com\nThis site offers the full text of regulatory filings by airlines.\n\nFederal Aviation Administration Office of System Safety - http://nasdac.faa.gov\nThe site will allow you to access aircraft accident data and a myriad of other databases, including the NTSB Aviation Accident/Incident Database, the Aviation Safety Reporting System, the Near Midair Collisions System (NMACS) Database, NTSB Safety Recommendations to the FAA with FAA Responses, the International Safety Recommendations Database, and the Aviation Safety Statistical Handbook.\n\nLandings.com - www.landings.com\nThis privately maintained site has the blessing of the FAA and the private blessings of dozens of reporters. Among the databases you'll find sophisticated searches of FAA regulations, Canadian regulations, AIM, the Pilot Controller Glossary, Service Difficulty Reports, Airworthiness Alerts, NTSB Briefs, N-Numbers and the FAA Airmen knowledge test information.\n\nRail\n\nFederal Railroad Administration - www.fra.dot.gov\nThe FRA site is your one-stop shop for railroad accident data. Who hit what and when and how, and sometimes even why. Not on its Web site, however, are the \"PC Accident Prediction System\" and rail crossing inventory databases. The \"Accident Prediction System\" is a searchable database that assigns an accident prediction to all public rail crossings nationwide. (The prediction system is available on CD-ROM.) The rail-crossing inventory is a database with specific information on all public rail grade crossings nationwide.\n\nRoad\n\nFatality Analysis Reporting System\nThe FARS is a database that theoretically contains detailed information about every fatal accident on the nation's roads. Because it relies upon local police departments, FARS is actually not that complete, but it's the best national picture of what's killing us on the highways. Some fields are the result of interpretation by traffic analysts, so be careful making judgments.\n\nFederal Highway Administration - www.fhwa.dot.gov\n\nInsurance Institute for Highway Safety - www.huysafety.org\nA good source of studies on urban crashes, red-light cameras and insurance-related issues. Offers other links to road-related sites.\n\nNational Highway Traffic Safety Administration - www.nhtsa.dot.gov\nLook for the Crashworthiness Data System for in-depth crash investigations. See the Crash Outcome Data Evaluation System, a relatively obscure NHTSA database that tracks the outcome of accidents, medical bills, etc. This information is usually hard to get because it's only collected by insurance companies, which are not public entities. Under the \"CARS\" link, see the downloadable collection of databases on consumer complaints, recalls, and Technical Service Bulletins (minor fix-it-for-free notices by car companies). The records detail whether injuries, deaths or property damage resulted from the problem that prompted the complaint.\n\nOther\n\nBureau of Transportation Statistics - www.bts.gov\nThe granddaddy of all free transportation database collections. Get information on waterways, rail, roads, accidents, travel, you name it - and everything is free. See the National Personal Transportation Survey at www.bts.gov/ntdaln.pts for measures of travel by American households. BTS is developing less esoteric measures of transportation and exploring new ways to visualize data.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-searching-for-influence","slug":"1999-januaryfebruary-searching-for-influence","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-searching-for-influence.html","year":1999,"month":"January/February","monthNum":1,"headline":"Searching for influence","kicker":"ON THE INTERNET","author_name":"Matt Jacob","author_title":"","affiliation":"Campaign Finance Information Center","summary":"The Campaign Finance Information Center provides journalists with tools, resources, and training to track and report on candidate fundraising and spending patterns during elections. The organization offers a searchable database of state campaign finance data, story examples, and networking opportunities to help reporters investigate the influence of money in politics.","keywords":["campaign","finance","journalism"],"topics":["Government & Politics","Data & Databases","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"With the 2000 elections quickly approaching, we at the Campaign Finance Information Center are giving our all to help journalists make the most of election stories. Wondering how much money the local insurance salesman running for state representative received? Interested in how much money a candidate received from out-of-state contributors?\n\nFollowing the campaign money trail in its entirety is one way journalists can provide more in-depth coverage of candidates. CFIC helps by collecting state campaign finance data from across the nation, making the data available to journalists, teaching reporters how to work with the data and offering story ideas.\n\nThe CFIC Web site - www.campaignfinance.org - is our central resource for journalists. A recent overhaul of the site provides what we think is a more user-friendly environment. Listed here are the site's features, about which we welcome your comments via e-mail at cfic-comments@ire.org.\n\nIndividual state pages\n\nHaving collected information on every state, we have built a page for each. State pages offer the source and range of our information, and when available, a breakdown of top contributors, state contribution limits, contribution reporting forms and links to state-specific search engines and to state election Web sites.\n\nPower search\n\nOut of our compiled data we've built a comprehensive search engine that allows journalists to track contribution cash flow across state lines for both federal and state races. The search provides who gives what to which candidates (contributions) as well as how the candidates spend their donations (expenditures). In some cases, it includes money flow to and from political action committees (PACs), candidate committees and party committees.\n\nStories and tips: Covering campaign finance\n\nThe CFIC collects stories that use campaign finance data to show how private money influences public policy. Journalists can search stories by publication, state, date and headline keyword. We also offer a list of sources specializing in campaign finance and tip sheets written by reporters that cover campaign finance issues.\n\nTraining\n\nUsing the resources and excellent training programs of Investigative Reporters and Editors and the National Institute of Computer-Assisted Reporting, the CFIC plans to offer seminars specific to covering campaign finance. Journalists will learn the skills they need to dig through data and come out with a story. The CFIC Web site will offer details on these seminars as they become available. There will also be sessions on covering campaign finance at both the IRE and NICAR national conferences.\n\nNetworking\n\nThe CFIC has compiled the names, phone numbers and e-mail addresses of reporters covering campaign finance into a searchable database. Journalists can search by name, affiliation, city or state. The center also maintains a mailing list for journalists to exchange ideas and tips. To subscribe to CFIC-L, send an e-mail to majordomo@campaignfinance.org In the body of the message, type: subscribe cfic-1 your e-mail address. The archives of the listserv can be searched online at http://notes.reporter.org:8080/cfic-l.nsf\n\nTracker\n\nTracker is the CFIC's newsletter dedicated to campaign finance coverage. Published quarterly, it is also available for viewing on our Web site. Each issue contains stories written by reporters working with campaign finance records. They share tips and tactics for tackling these often complex investigations. Past issues are also posted.\n\nHelpful links\n\nWe've pulled together a detailed list of other Web sites especially useful to journalists who are following the campaign money trail. These sites can help journalists put their data in some context with candidate profiles and candidate voting records. We also give other sources for campaign finance data.\n\nState sites\n\nNew York State Campaign Finance Consortium (http://www.timesunion.com/capitol/contributions)\n\nNewsday, The New York Times, Associated Press and the Albany Times Union joined together to build a statewide database. The Albany Times Union has put the data in a searchable database on this site.\n\nVirginia Public Access Project (http://www.crp.org/vpap)\n\nThe VPAP is a non-partisan, non-profit group that demonstrates how information technology can improve public awareness of money's role in Virginia politics. The Center for Responsive Politics helped make the data searchable.\n\nWisconsin Democracy Campaign (http://wisdc.org)\n\nFrom this site journalists can search gubernatorial contributions and expenditures and view legislative candidate contributions and expenditures to senate and house candidates.\n\nFlorida Division of Elections Campaign Finance Database (http://election.dos.state.fl.us/campfin/contrib.htm)\nCampaign finance contribution data is searchable online and downloadable in a file of tab-separated columns suitable for importing into most spreadsheets or databases.\n\nFederal sites\n\nCenter for Responsive Politics (http://www.crp.org)\nReports describing candidates' sources of funds broken down by industry, metro area and ZIP code. This site also provides in-state and out-of-state comparisons. The CRP site also hosts a number of online engines for searching individual state campaign finance data.\n\nFEC Info (http://www.tray.com/fecinfo)\nUp-to-date FEC filings are searchable from this site, an excellent alternative for most users to downloading FEC files.\n\nMatt Jacob can be reached by e-mail at matt@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-cfic-revamped","slug":"1999-januaryfebruary-cfic-revamped","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-cfic-revamped.html","year":1999,"month":"January/February","monthNum":1,"headline":"CFIC revamped","kicker":"","author_name":"Matt Jacob","author_title":"","affiliation":"NICAR","summary":"A guide to state and federal campaign finance databases that allow journalists and the public to search and analyze political contributions and expenditures across multiple jurisdictions including New York, Virginia, Wisconsin, Florida, and federal elections.","keywords":["campaign","finance","database"],"topics":["Government & Politics","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"New York State Campaign Finance Consortium (http://www.timesunion.com/capitol/contributions)\n\nNewsday, The New York Times, Associated Press and the Albany Times Union joined together to build a statewide database. The Albany Times Union has put the data in a searchable database on this site.\n\nVirginia Public Access Project (http://www.crp.org/vpap)\n\nThe VPAP is a non-partisan, non-profit group that demonstrates how information technology can improve public awareness of money's role in Virginia politics. The Center for Responsive Politics helped make the data searchable.\n\nWisconsin Democracy Campaign (http://wisdc.org)\n\nFrom this site journalists can search gubernatorial contributions and expenditures and view legislative candidate contributions and expenditures to senate and house candidates.\n\nFlorida Division of Elections Campaign Finance Database (http://election.dos.state.fl.us/campfin/contrib.htm)\nCampaign finance contribution data is searchable online and downloadable in a file of tab-separated columns suitable for importing into most spreadsheets or databases.\n\nFederal sites\n\nCenter for Responsive Politics (http://www.crp.org)\nReports describing candidates' sources of funds broken down by industry, metro area and ZIP code. This site also provides in-state and out-of-state comparisons. The CRP site also hosts a number of online engines for searching individual state campaign finance data.\n\nFEC Info (http://www.tray.com/fecinfo)\nUp-to-date FEC filings are searchable from this site, an excellent alternative for most users to downloading FEC files.\n\nMatt Jacob can be reached by e-mail at matt@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-cant-get-a-date","slug":"1999-januaryfebruary-cant-get-a-date","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-cant-get-a-date.html","year":1999,"month":"January/February","monthNum":1,"headline":"Can't get a date?","kicker":"TECH TIP","author_name":"Richard Mullins","author_title":"","affiliation":"NICAR","summary":"The article explains how data types work in database queries, specifically addressing a common mistake where journalists query date fields with just a year number instead of a complete date, resulting in unexpected or empty results.","keywords":["databases","queries","dates"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Access","FoxPro","SQL"],"full_text":"Before we started using databases and spreadsheets, we probably never thought about data types - the difference between text and numbers and dates and all that. A few bewildering sort results and a couple of type-mismatch errors later (plus the one about the ZIP codes with the missing zero in the front), and we're all the wiser.\n\nWe learned the rules for queries: Use quotes for text but not for numbers. Use a special sign to quote dates: {4/1/98} in FoxPro, or #4/1/98# in Access.\n\nIn Access, you get full-time data-type help along these lines in the query design window. Use a wildcard for a field criteria, and the program will automatically quote the criteria and add the keyword LIKE. Use a date for a criteria, and it will automatically put the pound signs around it. Of course, you are on your own if you type the query text yourself in the SQL window.\n\nNow to the point. The column this month comes from a question we got at NICAR recently. There is a short answer to make the query work. The long answer involves understanding how Access does things so you'll understand why things work the way they do. Knowing your software is not just technical trivia or the mastering of minutiae to impress people at work, although you can get by for quite some time without the ordeal of reading the fine print in the manual - or a long-winded tech tip. Eventually, doing your journalistic duty to know exactly how and why you got a certain answer will require you to look behind the curtain.\n\nIt is possible to type a query in Access, using a date criteria, which Access decides is just peachy, syntax-wise, and you'll get an answer back that will make you think: \"My query didn't work\" or \"This can't be right.\"\n\nLet's say you are working with a dataset in Access that has a date field called Insp Date. The table could be dams or rental property. You decide you want to see all the dams that were inspected in 1995. You could, for any number of reasons, type a query like this:\n\nSELECT * FROM Dams WHERE InspDate = 1995\n\nOr, if you are using Design view, you typed 1995 in the criteria grid under the InspDate column.\n\nBecause it has no syntax errors, the query would run. Most likely, however, your result would have zero rows, which you would think couldn't be right because you know you saw some 1995 dates when browsing the data. If you happened to be seeking inspections after 1995, so that the criteria was > 1995, you most likely would get a result that included every row in the table. Again, you'd think, this can't be right.\n\nYou get what you ask for\n\nHere's the real problem. Access gave you just what you asked for; you just didn't know why or how.\n\nAt this point, a FoxPro user might point out that the problem is a data-type mismatch. You didn't use a date in the criteria, they'd say. You used a number that represents only the year portion of a complete calendar date. It's a good answer, but it still doesn't help. How come you didn't get a type-mismatch error when you compared a number to a date? How come, if you used the design view, Access didn't put the pound signs around the criteria like it usually does? How come = 1995 got no rows and > 1995 got all the rows?\n\nThe answer to all these questions is that Access stores date/time information as a number. For dates in 1999, for example, the number is 36161 or higher. Specific times of day are stored internally as a fraction of the whole number for that day. So for the date we know on the calendar as January 11, 1999, Access stores this internally as 36171. For noon on that date, the number is 36171.5. For 11:33 that night, the number is 36171.98125. This is all internal. You can display the calendar equivalent of the date in your table and in queries in many different ways, from short (1/11/99) to long (January 11, 1999).\n\nThe number is based on the number of days elapsed since a certain base date. In Access 97, the base date, or Day Number One, is December 31, 1899. Dates before that base date are stored as negative numbers. Part of the explanation here is that Access is being flexible, allowing you to refer to date values by the conventional calendar representations or the underlying number.\n\nThis explains the answers to queries like InspDate = 1995 or InspDate > 1995. Like all software, Access doesn't know what you meant to ask, only what you actually asked. And it didn't give an error message because you tried to use a number as a criteria for a date column. It interpreted your criteria of 1995 not as the year 1995 but as the 1995th day since December 31, 1899, or June 17, 1905. If your data are apartment inspections, then probably none of them have a recorded inspection date that old, so your query would return zero rows. But most of them probably have a date later than 1905, so your query would return all the rows that have some date recorded. Both results were not what you expected, hence this long attempt to explain how to reconcile what you got with what you expected to get.\n\nUnleashing built-in functions\n\nAccording to a calendar I checked, the vernal equinox this year occurs March 20, 1999 at 8:46 PM, Eastern Standard Time. That's 36239.8652777778 in the Access date-storage scheme.\n\nIf you want to see more of how this works, try a query like this on a table that has a date column in it.\nSELECT InspDate, CDbl(InspDate) FROM Dams\n\nThe Access function, CDbl converts whatever is inside the parentheses to a double-precision number.\n\nHere are two solutions for what to do when you meant to ask for all the dams inspected some time in 1995. The first one involves remembering to use full dates - not just the year - in the criteria.\n\nSELECT * FROM Dams WHERE InspDate between #1/1/1995# and #12/31/1995#\n\nThe other solution involves taking advantage of built-in functions in Access (and FoxPro) that let you pluck out certain parts of a calendar date.\n\nSELECT * FROM Dams WHERE year(InspDate) = 1995\n\nTo find all the dams inspected in March:\n\nSELECT * FROM Dams WHERE month(InspDate) = 3\n\nTo find all the dams inspected on the 5th of the month:\n\nSELECT * FROM Dams WHERE day(InspDate) = 5\n\nTo find all the dams inspected on Monday:\nSELECT * FROM Dams WHERE DayOfWeek(InspDate) = 2\n\nAll of these functions look at the date and return a number that describes the component of the date you asked for: the number of the year; the number of the month (between 1 and 12); the number of the day (between 1 and 31); or the number of the day of the week (between 1 and 7). For day of the week, Sunday is 1 by default. This can be changed.\n\nKnowing your software is not just technical trivia or the mastering of minutiae to impress people at work. Eventually, doing your journalistic duty to know exactly how and why you got a certain answer will require you to look behind the curtain.\n\nRichard Mullins can be reached by e-mail at richard@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-januaryfebruary-bits-bytes-and-barks","slug":"1999-januaryfebruary-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1999-januaryfebruary-bits-bytes-and-barks.html","year":1999,"month":"January/February","monthNum":1,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Investigative Reporters and Editors launched a redesigned website with improved navigation, new member features, and an online store, while also announcing staff changes including a new trainer focused on computer-assisted reporting instruction.","keywords":["website","training","journalism"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["World Wide Web"],"full_text":"Investigative Reporters and Editors has launched a revamped Web site aimed at better serving current members and drawing future members. The site, while containing many of the features of the former site, is easier to navigate, offers more options for doing business with IRE staff and debuts some new features. There are more online registration forms and order forms. The IRE Store offers books, tapes and other IRE goods for easy purchase. Along with the popular job posting board, a fellowship/grant posting board is now available. The site will soon make use of members only areas allowing members to access the latest issue of The IRE Journal, reporter tipsheets, archived listservs, a personal information update area and more. Members will receive passwords in the mail before any members only gates go up. Work on the site will continue. Educators especially will be sought out to contribute to the IRE Journalism Education Center. Suggestions are welcome. Contributions are urged. E-mail should be sent directly to Deputy Director Jen Bruzzese at jen@ire.org.\n\nSarah Cohen, formerly the trainer for IRE and NICAR, has joined The Washington Post. If you have recently switched locales, let us hear from you. Send an e-mail to Brent Johnson at bjohnson@nicar.org.\n\nTom McGinty, a reporter with The Times of Trenton, has joined Investigative Reporters and Editors as its new on the road trainer.\n\nMcGinty will primarily teach computer-assisted reporting techniques at regional and newsroom workshops for NICAR.\n\nMcGinty, who coordinated the CAR efforts of The Times, served in beats ranging from cops and courts to local and state government and general assignment.\n\nReporters and editors will identify with this front line experience, said Brant Houston, IRE's executive director. Tom knows how to translate the training into terms news people will understand. McGinty earned a journalism degree from Jaca College of Syracuse University.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-elderly-evictions","slug":"1999-march-elderly-evictions","url":"https://dwillis.github.io/uplink/article/1999-march-elderly-evictions.html","year":1999,"month":"March","monthNum":3,"headline":"Elderly evictions","kicker":"NURSING HOME ENTRANCE","author_name":"Doug Stanley","author_title":"","affiliation":"The Tampa Tribune","summary":"The Tampa Tribune uncovered that a large nursing-home chain planned to evict more than 50 poor residents in Tampa. This initiated a broader investigation revealing systemic issues with elderly care in Florida's nursing homes operated by big corporations.","keywords":["nursing homes","evictions","investigation"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"When The Tampa Tribune broke the news that one of the nation's largest nursing-home chains intended to evict more than 50 of its poorest residents in Tampa, reaction was swift. A circuit court judge issued an injunction, state health officials launched an investigation, a powerful state senator called for fines, and the state attorney general's office threatened to broaden its ongoing probe into fraud within the Medicaid system. But what initially appeared to be a local story about an isolated incident grew into a six-month Tribune investigation of the quality of care in Florida nursing homes run by large corporations whose stock is traded on Wall Street. Vencor Inc., the same company evicting residents in Tampa under the guise of a remodeling project, was evicting or planning to evict Medicaid residents in at least 13 other states. Peterson began calling stock analysts who told her Vencor's stock had been dropping. Investors were worried cuts in Medicare and Medicaid would cut into profits. The Tribune computer analysis found that residents of the homes operated by the Big Six companies are about 20 percent more likely to be harmed or put at risk than are residents of all other Florida nursing homes. It also showed that quality of care varied greatly among the big chains - when compared both to each other and to all other homes. With that key finding in hand - and reams of computer data on the care records of individual homes - Peterson and Tribune medical reporter Vickie Chachere descended on public records repositories around the state gathering the details that would put the flesh on the computer-constructed skeleton. They combed through thousands of pages of paper records, which held the human detail. Published as a special 8-page section on Nov. 15, the project struck a chord with readers and with lawmakers. Moderated by executive editor Gil Thelen, a community forum on a weeknight just before the holidays brought 150 people to the newspaper's downtown offices. State and federal lawmakers filed reform bills. Published during the midst of a change in administrations at the state level, the real impact of the project won't be felt until after the legislative session.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-march-unsafe-taps","slug":"1999-march-unsafe-taps","url":"https://dwillis.github.io/uplink/article/1999-march-unsafe-taps.html","year":1999,"month":"March","monthNum":3,"headline":"Unsafe taps","kicker":"DRINKING WATER ANALYSIS","author_name":"Peter Eisler","author_title":"Reporter","affiliation":"USA Today","summary":"USA Today's investigation reveals that thousands of American public water systems violate safe drinking water laws with impunity because enforcement is virtually nonexistent, based on a comprehensive analysis of 170,000 water systems across all 50 states.","keywords":["enforcement","contamination","investigation"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"It's no secret that millions of Americans get bad drinking water. Year after year, private and governmental studies find that thousands of public water systems violate safe drinking water laws by failing to monitor and control contamination. News organizations (USA Today included) give those studies plenty of coverage.\n\nLast year, we decided to figure out how these bad water systems — from big-city utilities to mom-and-pop operations serving tiny rural communities — keep getting away with it again and again.\n\nThe answer turned out to be simple: No one enforces safe drinking water laws.\n\nBut proving it — and detailing the problem's breadth and impact — required one of the most challenging computer-assisted investigations we've ever done. We reviewed records on each of the 170,000 public water systems subject to state and federal regulation. We made FOIA requests to all 50 states and the federal government. We spent hundreds of hours compiling, standardizing and scrutinizing data. Our analysis was so thorough and unassailable that the U.S. Environmental Protection Agency inquired about hiring our database editor, Barbara Hansen.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-money-or-mercy","slug":"1999-march-money-or-mercy","url":"https://dwillis.github.io/uplink/article/1999-march-money-or-mercy.html","year":1999,"month":"March","monthNum":3,"headline":"Money or mercy?","kicker":"","author_name":"Doug Stanley","author_title":"database editor","affiliation":"Tampa Tribune","summary":"A journalism investigation used nursing home inspection data and SEC financial records to compare quality of care at six major publicly traded nursing home companies against other Florida facilities, finding that residents at the large chains were 20 percent more likely to be harmed or put at risk.","keywords":["nursing","deficiencies","profit"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["OSCAR database","ASCII","Access","EDGAR"],"full_text":"shareholders also must be served.\n\nThe project began to take shape when it appeared the answers might lie in public documents — particularly nursing home inspection data from the Health Care Financing Administration and financial information from the Securities and Exchange Commission.\n\nComparison standards\n\nWe acquired the OSCAR database for Florida from HCFA in ASCII on diskettes and imported the data into Access. Then came the requisite cleanup and false starts associated with unfamiliar data. Because our objective was to measure the homes of the six largest public companies against each other and against all other homes, the first step was to ensure accuracy in the field containing ownership information. Often that information was out of date. We used state and company documents to make revisions where necessary. This step is crucial.\n\nOwnership changes among some homes owned by the Big Six companies dictated that we look only at one year's inspection data. The universe became 642 nursing homes whose most recent inspections occurred between January 1997 and May 1998.\n\nWe chose to look at total deficiencies as an overall indication of quality. But we were most interested in severe deficiencies that endangered or harmed patients. A field in OSCAR that assigned a letter-code ranking to each deficiency made that easy.\n\nHow to compare the homes? Should comparisons be based upon numbers of beds or numbers of homes? After careful consideration, we settled on comparing deficiency rates based on numbers of homes because of varying bed-occupancy rates and an inspection system that uses the same size sample regardless of home size. Also, comparisons based on numbers of homes would be easier for readers to grasp.\n\nA \"deficiency\" is any violation of a nursing home rule or regulation. A total deficiency rate and a severe deficiency rate (both per 10 homes) were calculated for each of the Big Six companies; for all other Florida nursing homes; for all other for-profit homes; and for all nonprofit homes.\n\nWhat we found\n\nThe Tribune's computer analysis found that residents of the homes operated by the Big Six companies are about 20 percent more likely to be harmed or put at risk than are residents of all other Florida nursing homes. It also showed that quality of care varied greatly among the big chains — when compared both to each other and to all other homes.\n\nKey questions became obvious: Do large, publicly traded nursing-home corporations take better or worse care of residents than other home operators? What role, if any, does the pursuit of profit play in the equation?\n\nWith that key finding in hand — and reams of computer data on the care records of individual homes — Peterson and Tribune medical reporter Vickie Chachere descended on public records repositories around the state gathering the details that would put the flesh on the computer-constructed skeleton. They combed through thousands of pages of paper records, which held the human detail. Hundreds of interviews followed. Countless hours were spent in nursing homes. The SEC's EDGAR database and the companies' own Web sites provided financial context.\n\nPublished as a special 8-page section on Nov. 15, the project struck a chord with readers and with lawmakers. Moderated by executive editor Gil Thelen, a community forum on a weeknight just before the holidays brought 150 people to the newspaper's downtown offices. State and federal lawmakers filed reform bills. Published during the midst of a change in administrations at the state level, the real impact of the project won't be felt until after the legislative session.\n\nDoug Stanley can be reached at (813) 259-7364 or by e-mail at dstanley@tampatrib.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-upcoding-downfall","slug":"1999-march-upcoding-downfall","url":"https://dwillis.github.io/uplink/article/1999-march-upcoding-downfall.html","year":1999,"month":"March","monthNum":3,"headline":"Upcoding downfall","kicker":"ILLNESS MANIPULATION","author_name":"Dave Davis and Joan Mazzolini","author_title":"","affiliation":"The Plain Dealer","summary":"Rural for-profit hospitals in Ohio are allegedly inflating the complexity of elderly patients' illnesses on Medicare claims to receive higher reimbursements, a practice known as upcoding that contradicts the reality that large teaching hospitals treat sicker patients.","keywords":["upcoding","Medicare","fraud"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Excel"],"full_text":"Out in the farmland of Ohio, far from sophisticated 1,000-bed teaching hospitals, hospital administrators say they are caring for the sickest elderly patients. It defied logic, experts told us. They were right. The rural hospitals aren't treating the most serious cases, but they are trying to make it look that way on paper to get higher Medicare reimbursements.\n\nUsing four years of Medicare reimbursement data compiled by a private company, we found that in our state for-profit hospitals — without exception small in size and located in suburban or rural areas — received larger Medicare reimbursements than other hospitals. They claimed more often that their elderly patients suffered from an illness with complications.\n\nIn Ohio, this defies logic. It means that small hospitals such as 52-bed Knox Community Hospital in the rich farmland of southwestern Ohio are treating sicker patients than the large, teaching hospitals in Cincinnati, Columbus and Cleveland. Health care experts told us that the opposite was true: Big-city hospitals treated the sickest, usually poorest, patients.\n\n\"No one who is sober would dispute that,\" Frank Sloan, a researcher at Duke University told us. \"It's like disputing the Earth is round.\"\n\nComplication claims\n\nBut we did come across a few administrators — mostly from hospitals owned or managed by Quorum Health Group, Ohio's most powerful for-profit player — who argued the illogical. That's not surprising — our analysis of Medicare reimbursements for 1994 through 1997 found that Quorum's hospitals, on average, claimed that 89 percent of their elderly patients suffered from an illness with complications. For large teaching hospitals, the average was 81 percent.\n\nIn Ohio, Quorum's rate of claiming illnesses with complications was considerably higher than even the rate of Columbia HCA/Healthcare Corp., whose deftness in the art of \"upcoding,\" the practice of making patients sicker on paper than they were in the hospital, has caught the attention of federal investigators and was the subject of a revealing and exhaustive investigative project by The New York Times. (See Josh Barbanel's cover story in the May 1998 issue of Uplink.)\n\nTo understand how this works, you need to understand that Medicare pays hospitals a fixed dollar amount for treating the elderly based on the illness, known as a Diagnostic Related Group (DRG) in Medicare-speak. Some DRGs are related to one another, one being a more complex (and more expensive) case of the other, such as \"simple pneumonia\" and \"respiratory infection with complications.\" Respiratory infection is the higher reimbursed DRG here. Not surprisingly, we found the number of reported respiratory infections on the rise in Ohio.\n\nLet's now make this more confusing. Each DRG, in addition to being part of a hierarchy of related illnesses, is part of a pair. DRG pairs are the exact same illnesses, but one has complications and the other doesn't, such as \"simple pneumonia\" and \"simple pneumonia with complications.\" It should come as no surprise that Medicare pays more for treating the latter.\n\nDRG analysis\n\nThe New York Times investigation centered on true upcoding, where Columbia claimed reimbursement for treating a more complex DRG. Our stories focused on an offshoot of upcoding, where hospitals claimed high rates of treating an illness with complications.\n\nOur stories, published in October, took about three weeks to complete and could be easily duplicated in any state in the nation. The analysis included about one million admissions in 170 Ohio hospitals.\n\nIt was pretty simple, really. We found a Louisville, Ky.-based company that combines information from two U.S. Department of Health and Human Services data files — the Medicare Cost Reports and Medicare Provider and Analysis and Review File (MedPar). The company, Data Advantage, (502) 587-9500, sold us the data for about $200.\n\nThe data file we received contained hospital-level records by DRG. For each DRG, the records showed the number of elderly patients treated, the average amount the hospital was reimbursed and its bed size, among other things.\n\nWe then examined each of the several hundred DRG pairs and calculated how often hospitals claimed their patients suffered an illness with complications rather than without.\n\nNext, we grouped hospitals together based on their size (small, medium and large) and who operates them (for-profits or non-profits).\n\nWe found that size mattered little in how often hospitals claimed complications. Ownership, however, was a big deal. Because the records included the average amount hospitals received for each DRG, we also estimated how much tax money would have been saved if the small, for-profit hospitals had received reimbursement for patients no sicker than patients in their urban, non-profit counterparts. The figure was a whopping $22 million for the four-year period.\n\nDave Davis can be reached by e-mail at ddavis@plaind.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-treatment-disparity","slug":"1999-march-treatment-disparity","url":"https://dwillis.github.io/uplink/article/1999-march-treatment-disparity.html","year":1999,"month":"March","monthNum":3,"headline":"Treatment disparity","kicker":"CARE AND COLOR","author_name":"Adam Pitluk","author_title":"","affiliation":"Newsday","summary":"A Newsday investigation reveals significant racial disparities in healthcare outcomes between Black and white patients in the New York region, with Black patients receiving less effective treatments for conditions like diabetes, heart disease, and kidney failure.","keywords":["healthcare","disparities","amputation"],"topics":["Health & Environment","Investigative Methods","Demographics & Census"],"technologies":[],"full_text":"School safety officer Dedri Brown, an African-American diabetic, lost a leg to amputation. Yet it might have been saved if she had been aware of available treatments. Amputation is just one of several fates suffered more frequently by black diabetics than white diabetics, a Newsday investigation found.\n\nGerald Adams is another example of insufficient health care for Blacks in New York: He has been waiting for a kidney transplant for five years — almost twice as long as the average white patient.\n\nFor the past year, reporters from Newsday examined the health care provided to people in their region. The newspaper conducted a behavioral-risk study of more than 2,000 residents of Long Island and Queens to parallel surveys done at the federal level. At the helm of the investigation was Newsday data specialist and reporter Ford Fessenden, now with The New York Times.\n\nFessenden's story started as an offshoot of a 1996 report card on all of the hospitals in the area. Newsday Associate Managing Editor Rich Galant wanted to know what the numbers had to say about the health issues facing readers. He and projects editor Bob Tiernan sent Fessenden to find out.\n\n\"We found that the health of people in our region was good virtually on everything except heart disease,\" Fessenden said, \"and nobody had ever really told people they had some of the highest heart attack rates in the country.\"\n\nTransformed investigation\n\nIt also quickly became clear that health care varied widely among different populations in Queens and Long Island. The study manifested into an issue of who receives better and more effective health care, whites or blacks. Here is what Newsday found:\n\n• Whites get more intensive treatment. They die of coronary heart disease at a lower rate than blacks but are more likely to undergo advanced heart procedures. Whites also have hip replacements at a rate twice as high as blacks, knee replacements at a rate 23 percent higher, and gall-bladder operations at a rate 53 percent higher. (All figures are adjusted for differences in age between the races.)\n\n• Blacks receive more radical surgery when less severe alternatives are an option. Newsday cited diabetes as an example: Blacks are more likely to lose feet or legs to amputation; whites are more likely to have surgery to restore blood flow and save their legs. Among 1,907 whites and 385 blacks from Nassau, Suffolk and Queens who had surgery for arteriosclerosis or diabetes-related circulatory problems in 1995, 61 percent of whites had surgery to restore blood flow, but only 43 percent of blacks had leg-saving surgery. Dedri Brown was not among them.\n\n• Black women are more likely to have the more invasive kind of hysterectomy treatment. Blacks with serious kidney problems wait longer for transplants and are less likely to receive one than whites. For every 1,000 blacks in the hospital with kidney failure, there were 8.2 transplants; for whites, 13.3 transplants.\n\nDelayed visits\n\nFessenden and his team analyzed some 3 million records of hospital admissions in Long Island and Queens covering a period of five years.\n\nAs part of the inquiry, Newsday, joined by The New York Times, sued the state — which had refused to release fields in health care databases identifying doctors and insurance companies — and won. Newsday then showed that the best hospitals and busiest doctors treated white patients almost exclusively. Newsday also commissioned a pollster and concluded that blacks are more likely to seek alternatives to traditional medicine and delay visiting a doctor.\n\n\"Even blacks with good insurance are sicker when they arrive at the doors of medical practitioners,\" he said. \"Black cancer sufferers are more likely to die of their disease — a strong indicator they aren't getting care early enough.\"\n\nTreatment disparities have been an ongoing problem in medicine, Fessenden said. The purpose of his coverage was to let the numbers tell the story and make patients and doctors alike aware of this dirty little secret. On Feb. 14, 1999, a local conference of health officials addressed the problem, a direct result of Fessenden and Newsday's findings.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-wanted-mentally-ill","slug":"1999-march-wanted-mentally-ill","url":"https://dwillis.github.io/uplink/article/1999-march-wanted-mentally-ill.html","year":1999,"month":"March","monthNum":3,"headline":"Wanted: mentally ill","kicker":"NURSING HOME ENTRANCE","author_name":"Dawn Fallik","author_title":"","affiliation":"NICAR","summary":"Chicago Tribune reporter Mike Berens' investigation revealed that Illinois was placing mentally ill patients in nursing homes instead of institutions to save costs, leading to federal and state investigations and resulting in regulatory changes requiring specialized staff training.","keywords":["nursing","mentally","Medicaid"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["OSCAR"],"full_text":"Mike Berens' three-part series, \"Dangerous Bedfellows,\" began with the wrong question.\n\n\"I had seen that the cost of health care in jails was going way up, and a tip told me it was mostly psychotropic drugs,\" said the Chicago Tribune reporter. \"So I wanted to find out how many mentally ill patients were in the jails.\"\n\nInstead, he found the state taking mentally ill patients off the streets and out of the institutions and placing them into nursing homes. The reasoning: The state split the costs of care for nursing home patients with Medicaid. But, if the patient were placed in an institution, the state paid the full cost.\n\nUsing about a dozen federal and state databases, including the OSCAR nursing home database, Berens pieced together the skeleton of a dangerous and occasionally deadly practice. He found that nursing home owners openly searched for mentally ill patients and that the Department of Public Aid often turned a blind eye to nursing homes filled with non-elderly residents.\n\nThe series, published Sept. 27-29, 1998, resulted in a federal investigation of Medicaid fraud. The state responded by launching its own investigation. Illinois recently admitted that 17 of its nursing homes were actually being operated as mental institutions; all were then withdrawn from the Medicaid program.\n\nBecause of the series and the investigation that followed, many nursing homes will be relicensed to allow such patients, but staffs will be required to receive specialized service and rehabilitation training.\n\nViolent histories\n\nBerens found that many of the incoming mentally ill patients had histories of violence and needed intensive psychiatric care and drug therapies, neither of which nursing home employees were trained to handle.\n\n\"In 99 percent of the cases, the attacks — and the resulting injuries — were never reported to police. This lack of reporting is part of a major problem,\" Berens said. \"I documented hundreds of incidents of attacks and injury from the state nursing home inspection reports.\"\n\nCreating his own database of incidents, Berens found evidence of sexual and physical abuse of elderly patients and neglect of those who needed psychiatric care. In one case, he found a man whose behavior goals included keeping sexual fondling of other patients down to three times a week.\n\nDepartment of Public Aid officials denied putting mental patients into nursing homes. In order to subvert a rule that Medicaid would pay only for patients in homes with more than 50 percent actual nursing home clients, psychiatric patients were sometimes reported as physically ill.\n\nIn one case, a woman whom state doctors had verified as mentally ill was re-classified in a nursing home as suffering from \"cranial dermatitis,\" or bad dandruff, Berens said.\n\nUnderlying numbers\n\nMuch of the evidence Berens used in his series came from a Department of Public Health database, which included a patient-by-patient listing of about 1,200 nursing homes. DPH's state twin, the Department of Public Aid, was not so helpful.\n\n\"I would go to them with the numbers I had, showing that thousands of patients in nursing homes were mentally ill. And they would tell me I didn't understand the numbers and that I wasn't calculating the figures right,\" Berens said. \"I've never had public officials just out-and-out lie to my face before.\"\n\nAlthough the federal Medicaid cost report shows the owners of nursing home facilities, it only includes those who own more than five percent.\n\nWhen Berens requested the same information from state sources, he was denied. After several months of wrangling with attorneys, the state relented. Berens then used the data to track down those owning less than 5 percent of the homes. \"The whole thing was like an octopus network,\" he said.\n\nIt wasn't until Berens completed his investigative work that he finally stepped foot into the nursing homes themselves.\n\n\"The thing that helped me most was finding real advocates inside who helped guide me through the system. The series was a real melding of computer database technology and old-fashioned reporting.\"\n\nDawn Fallik can be reached by e-mail at dawn@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-revisiting-hours","slug":"1999-march-revisiting-hours","url":"https://dwillis.github.io/uplink/article/1999-march-revisiting-hours.html","year":1999,"month":"March","monthNum":3,"headline":"Revisiting hours","kicker":"NURSING HOME SECOND ENTRANCE","author_name":"Mark J. Rochester","author_title":"","affiliation":"The Indianapolis Star and The Indianapolis News","summary":"The Indianapolis Star and Indianapolis News expanded their nursing home investigation beyond the initial exposé, uncovering alarming new findings including a 50% increase in violations, racial disparities in care quality, and preventable deaths caused by inadequate feeding and patient care.","keywords":["investigation","nursing","reform"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["Paradox","SPSS"],"full_text":"Too often, daily newspapers are reluctant to pursue major investigative efforts after the so-called \"big story\" has been published and the problem exposed. But do we really go back and take a second look?\n\nSure, we all do follow-up stories: What is the reaction, if any, of the public or government officials to the findings of the investigation? Will anything likely change as a result of the investigation?\n\nThose are important stories, but they may signal the premature end of an investigation. Often, there is much more work to be done. For various reasons — lack of resources, other investigative projects in the pipeline, fear of repetition — we fail to keep digging following an investigation.\n\nWe miss a lot of good stories when we do that.\n\nWhen state government and health officials promised significant reforms following publication of a year-long investigation of nursing home care printed in June, it was tempting for The Indianapolis Star and The Indianapolis News to simply bundle up our documents, source lists and computer databases and move on to the next project.\n\nPast and present\n\nThe newspapers had documented the pervasiveness of poor care in nursing homes and the shortcomings of numerous state and federal regulations that allowed the problems to linger. State and federal authorities reacted swiftly: State officials announced a series of reform initiatives, including tougher penalties against the worst providers.\n\nWe could have assigned a reporter to follow those reforms and considered our work done on the issue, but we knew there was more to the story. Stick with an investigation until you've exhausted all crucial subject areas.\n\nThe reporters had uncovered numerous leads and tips during the initial reporting effort. Others came flooding in after publication of the series, \"The Tarnished Years.\" After several meetings, we decided to take the extraordinary step of undertaking another six-month investigation. Instead of reducing the staffers dedicated to the effort, the newspapers expanded the reporting team to three reporters, who, along with the CAR coordinator and a dedicated photographer, tackled an entirely new set of questions about the quality of care in nursing homes.\n\nAmong the key findings of the second investigation:\n\n• As the need for reform was being debated by state health and government officials, problems in nursing homes rose dramatically from the previous year — there were 50 percent more health and safety violations, a computer analysis of state inspection records showed.\n\n• In Indianapolis, residents in predominantly black nursing homes received far worse care than those in white facilities. Among the disparities: Homes serving low-income blacks had nearly five times as many harm-level violations as homes serving whites.\n\n• Needless deaths and illnesses occur in nursing homes every year because of problems associated with the routine feeding and care of patients.\n\nSeparate and unequal\n\nAside from the significant increase in health and safety violations, the most striking element of the latest investigation were the racial disparities in quality of care. Tackling that issue was difficult.\n\nOur first attempt to obtain data containing demographic information about nursing homes from was blocked by government officials, who denied our FOIA requests. Rather than continue that fight, however, we decided to pursue the matter at the state level, a strategy that had worked well while putting together our original series.\n\nWe knew the state of Indiana collected the same nursing home census information as the federal government, so we hoped it would be a simple matter of obtaining that database. It wasn't. State officials denied the existence of any such electronic records, claiming only hard copy forms tucked into thousands of files warehoused in their offices.\n\nEven though we were running into roadblocks set up by officialdom, we had other successes that inspired us to keep going. Using Paradox and SPSS software to analyze several different databases we had already obtained, we determined other patterns in the data that let us know we were on the right track. For example, we documented the impact poverty has on quality of care by examining state inspection records, U.S. census data and Medicaid reimbursement rates from the government. Analysis showed that most poorly performing facilities were in low-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-laxity-allowed","slug":"1999-march-laxity-allowed","url":"https://dwillis.github.io/uplink/article/1999-march-laxity-allowed.html","year":1999,"month":"March","monthNum":3,"headline":"Laxity allowed","kicker":"","author_name":"Peter Eisler","author_title":"","affiliation":"USA Today","summary":"An investigative journalism project analyzed drinking water safety violations across all 50 states and federal agencies, finding that approximately 40,000 water systems violate safe drinking water laws annually, yet most violators face minimal enforcement despite serious public health threats.","keywords":["water","violations","enforcement"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Paradox","Monarch"],"full_text":"tems subject to state and federal regulation. We made FOIA requests to all 50 states and the federal government. We spent hundreds of hours compiling, standardizing and scrutinizing data. Our analysis was so thorough and unassailable that the U.S. Environmental Protection Agency inquired about hiring our database editor, Barbara Hansen.\n\nMatters of \"significance\"\n\nSome of our findings:\n\n• About 40,000 water systems violate safe drinking water laws each year; 9,500 of them have \"significant\" problems that pose \"serious threats to public health,\" according to the EPA. Yet just one in 10 \"significant\" violators face binding enforcement action such as an administrative order, fine or lawsuit. One-quarter of all \"significant\" violators have been in that category for at least three years.\n\n• Eleven states have not implemented all the Safe Drinking Water Act's contamination limits, which should have been in place by 1995. At least 13 states don't meet federal guidelines by inspecting water systems every three to five years. A half-dozen have not given their drinking water regulators the power to levy fines.\n\n• The EPA is more than a year behind in completing required assessments of drinking water programs in at least 11 states. What's more, the agency's Safe Drinking Water Information System (SDWIS), the main national database used to monitor water systems' performance, is terribly inaccurate.\n\nThe stories got a strong response. We met requests for more than 30,000 reprints. In Congress, the project prompted an inquiry to the EPA by Reps. Henry Waxman, D-Calif., and Sherrod Brown, D-Ohio. Waxman's office says he intends to raise the issue with EPA officials at hearings this year.\n\nA path full of hurdles\n\nThe project began simply, but it didn't stay that way. First, we went about learning how drinking water rules work. Then we set measures for evaluating their success: Are states implementing required regulations? Are states conducting regular sanitary surveys (inspections) of water systems? Are state and federal officials taking action against systems that violate health and safety rules?\n\nWe sent FOIA requests to the EPA and the states for the records we needed. The states' data were wildly inconsistent — each state has different rules on what constitutes a violation of drinking water laws and what warrants an enforcement action.\n\nMore troublesome: the states' data didn't jibe with much of what the EPA had in SDWIS. The federal database was riddled with errors. Some we were able to resolve, but we had to adjust our expectations. We used SDWIS to draw conclusions at the state and national levels, but information on specific water systems had to be verified independently.\n\nWe finished with four databases: SDWIS, which we imported into Paradox using Monarch parsing software, and three we built ourselves. The first, based on sanitary survey records, allowed us to calculate average inspection rates for water systems in each state. The second, based on census data and information from EPA's infrastructure needs survey, allowed us to calculate per-capita costs for bringing water systems into compliance with federal requirements. The third, based on state budget data, allowed us to determine each state's per-capita spending on drinking water oversight.\n\nLessons learned\n\n• Know the rules: You can't interpret data without a keen understanding of how it was compiled and categorized. Plus, you'll win over sources with your technical expertise.\n\n• Check data: To confirm findings' validity, develop alternate queries and different approaches that should yield the same result.\n\n• Be honest: Explain your data's limits. We wrote a separate story on the problems we found — especially with SDWIS — to help readers understand what conclusions could and could not be reached.\n\n• Allow time: When you're getting data from varied sources, budget time for cleanup and verification. Also, many states aren't used to handling requests for computerized records, so there's a lot of haggling.\n\nPeter Eisler can be reached by e-mail at peisler@usatoday.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-medicare-milked","slug":"1999-march-medicare-milked","url":"https://dwillis.github.io/uplink/article/1999-march-medicare-milked.html","year":1999,"month":"March","monthNum":3,"headline":"Medicare milked","kicker":"","author_name":"","author_title":"","affiliation":"Plain Dealer","summary":"A study reveals that hospital ownership type significantly impacts how often complications are reported, with for-profit hospitals claiming complications more frequently than non-profits, potentially costing taxpayers $22 million in unnecessary Medicare reimbursements over four years.","keywords":["hospitals","ownership","reimbursement"],"topics":["Data & Databases","Health & Environment","Business & Economy"],"technologies":["DRG"],"full_text":"Next, we grouped hospitals together based on their size (small, medium and large) and who operates them (for-profits or non-profits).\n\nWe found that size mattered little in how often hospitals claimed complications. Ownership, however, was a big deal. Because the records included the average amount hospitals received for each DRG, we also estimated how much tax money would have been saved if the small, for-profit hospitals had received reimbursement for patients no sicker than patients in their urban, non-profit counterparts. The figure was a whopping $22 million for the four-year period.\n\nDave Davis can be reached by e-mail at ddavis@plaind.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-finding-the-unknown","slug":"1999-march-finding-the-unknown","url":"https://dwillis.github.io/uplink/article/1999-march-finding-the-unknown.html","year":1999,"month":"March","monthNum":3,"headline":"Finding the unknown","kicker":"","author_name":"Adam Pitluk","author_title":"","affiliation":"Newsday","summary":"A Newsday investigation revealed significant racial disparities in healthcare outcomes across the New York region, with Black patients receiving less effective treatments and more radical surgeries compared to white patients for conditions like heart disease, diabetes, and kidney failure.","keywords":["disparities","healthcare","diabetes"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Finding the unknown\n\nSchool safety officer Dedri Brown, an African-American diabetic, lost a leg to amputation. Yet it might have been saved if she had been aware of available treatments. Amputation is just one of several fates suffered more frequently by black diabetics than white diabetics, a Newsday investigation found.\n\nGerald Adams is another example of insufficient health care for Blacks in New York: He has been waiting for a kidney transplant for five years — almost twice as long as the average white patient.\n\nFor the past year, reporters from Newsday examined the health care provided to people in their region. The newspaper conducted a behavioral-risk study of more than 2,000 residents of Long Island and Queens to parallel surveys done at the federal level. At the helm of the investigation was Newsday data specialist and reporter Ford Fessenden, now with The New York Times.\n\nFessenden's story started as an offshoot of a 1996 report card on all of the hospitals in the area. Newsday Associate Managing Editor Rich Galant wanted to know what the numbers had to say about the health issues facing readers. He and projects editor Bob Tiernan sent Fessenden to find out.\n\n\"We found that the health of people in our region was good virtually on everything except heart disease,\" Fessenden said, \"and nobody had ever really told people they had some of the highest heart attack rates in the country.\"\n\nTransformed investigation\n\nIt also quickly became clear that health care varied widely among different populations in Queens and Long Island. The study manifested into an issue of who receives better and more effective health care, whites or blacks. Here is what Newsday found:\n\n• Whites get more intensive treatment. They die of coronary heart disease at a lower rate than blacks but are more likely to undergo advanced heart procedures. Whites also have hip replacements at a rate twice as high as blacks, knee replacements at a rate 23 percent higher, and gall-bladder operations at a rate 53 percent higher. (All figures are adjusted for differences in age between the races.)\n\n• Blacks receive more radical surgery when less severe alternatives are an option. Newsday cited diabetes as an example: Blacks are more likely to lose feet or legs to amputation; whites are more likely to have surgery to restore blood flow and save their legs. Among 1,907 whites and 385 blacks from Nassau, Suffolk and Queens who had surgery for arteriosclerosis or diabetes-related circulatory problems in 1995, 61 percent of whites had surgery to restore blood flow, but only 43 percent of blacks had leg-saving surgery. Dedri Brown was not among them.\n\n• Black women are more likely to have the more invasive kind of hysterectomy treatment. Blacks with serious kidney problems wait longer for transplants and are less likely to receive one than whites. For every 1,000 blacks in the hospital with kidney failure, there were 8.2 transplants; for whites, 13.3 transplants.\n\nDelayed visits\n\nFessenden and his team analyzed some 3 million records of hospital admissions in Long Island and Queens covering a period of five years.\n\nAs part of the inquiry, Newsday, joined by The New York Times, sued the state — which had refused to release fields in health care databases identifying doctors and insurance companies — and won. Newsday then showed that the best hospitals and busiest doctors treated white patients almost exclusively. Newsday also commissioned a pollster and concluded that blacks are more likely to seek alternatives to traditional medicine and delay visiting a doctor.\n\n\"Even blacks with good insurance are sicker when they arrive at the doors of medical practitioners,\" he said. \"Black cancer sufferers are more likely to die of their disease — a strong indicator they aren't getting care early enough.\"\n\nTreatment disparities have been an ongoing problem in medicine, Fessenden said. The purpose of his coverage was to let the numbers tell the story and make patients and doctors alike aware of this dirty little secret. On Feb. 14, 1999, a local conference of health officials addressed the problem, a direct result of Fessenden and Newsday's findings.\n\nIncredulous response\n\nWhen the article ran, there was a lot of initial resistance by doctors.\n\n\"At first the doctors said, 'No way.' But some of them came back to me and said, 'Hmm. Maybe my practice is like the study.'\"\n\nFessenden said his findings are important as both public service and investigative journalism. \"The nice thing about this stuff is we found something other people didn't know. We took our findings to the health department and they said, 'What?' It's expert journalism in that it gives you information not previously known, as well as believability.\"\n\nAdam Pitluk can be reached by e-mail at c664490@showme.missouri.edu. Ford Fessenden can be reached by e-mail at fordfess@nytimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-march-measured-neglect","slug":"1999-march-measured-neglect","url":"https://dwillis.github.io/uplink/article/1999-march-measured-neglect.html","year":1999,"month":"March","monthNum":3,"headline":"Measured neglect","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Journalists investigating nursing home quality of care encountered resistance from Indiana state officials who claimed electronic census records didn't exist, but persisted using data analysis software to uncover patterns linking poverty to poor facility performance.","keywords":["nursing","database","poverty"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Paradox","SPSS"],"full_text":"that had worked well while putting together our original series.\n\nWe knew the state of Indiana collected the same nursing home census information as the federal government, so we hoped it would be a simple matter of obtaining that database. It wasn't. State officials denied the existence of any such electronic records, claiming only hard copy forms tucked into thousands of files warehoused in their offices.\n\nEven though we were running into roadblocks set up by officialdom, we had other successes that inspired us to keep going. Using Paradox and SPSS software to analyze several different databases we had already obtained, we determined other patterns in the data that let us know we were on the right track. For example, we documented the impact poverty has on quality of care by examining state inspection records, U.S. census data and Medicaid reimbursement rates from the government. Analysis showed that most poorly performing facilities were in low-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-april-officer-downings","slug":"1999-april-officer-downings","url":"https://dwillis.github.io/uplink/article/1999-april-officer-downings.html","year":1999,"month":"April","monthNum":4,"headline":"Officer downings","kicker":"D. C. 'S \"DEADLY FORCE\"","author_name":"Jo Craven","author_title":"","affiliation":"The Washington Post","summary":"The Washington Post investigated police killings in Washington D.C. during the 1990s by examining FBI crime data, discovering that justifiable homicide records were initially missing from standard data releases and had to be specially requested. The investigation revealed D.C. police shot and killed more people per resident than any other major U.S. metro police department, though the Post ultimately decided against relying solely on FBI justifiable homicide reports due to accuracy and jurisdictional reporting concerns.","keywords":["police","homicide","data"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Supplementary Homicide Report","Uniform Crime Reports","9-track tape"],"full_text":"In the 1990s, District of Columbia police shot and killed more people per resident than any metro police department in the country – including New York, Los Angeles, Chicago, Detroit and New Orleans.\n\nThe Washington Post story, \"Deadly Force,\" resulted from a close examination of the FBI's Supplementary Homicide Report, one of the computer databases included in the annual release of the Uniform Crime Reports.\n\nThat examination revealed that the 1995 SHR data for the country (released in 1996) did not have a single \"81\" – the code in the circumstance field for justifiable homicide by a police officer. Neither did any of the other years of data that I checked.\n\nThis suggested that whole records were missing from the dataset. The FBI's own documentation revealed another corroborating clue: The data is provided on nine-track tapes, and an \"analysis of records written\" printout accompanies each tape. This printout describes how many records were \"read\" in the government's computer versus how many were \"written\" on the tape provided to the purchaser. These should match; in 1995, however, the number of records written was short by 287.\n\nThey shall be released\n\nSeveral conversations with the FBI revealed that the bureau did collect justifiable homicides but that the numbers were not part of the \"standard data release.\" We ordered a nonstandard release, specifically asking for justifiable homicides.\n\nSix weeks passed before the nine-track tapes arrived. When they did, we eagerly downloaded the data, opened the records, scrolled to the circumstance field and looked for 81s. There were none.\n\nDespite a very specific written request, which had been reinforced with several telephone conversations, the FBI had sent the \"standard release,\" minus justifiable homicides. We reordered the data and reiterated the request.\n\nSeveral more weeks passed, new tapes arrived, we processed the data, and there they were: hundreds of 81s.\n\nA hot tip\n\nThe raw numbers of justifiable homicides by police officers alone were remarkable. Only officers in a handful of cities – all much larger than Washington, D.C. – had shot and killed more people. Ultimately, these records served only as a starting point.\n\nAs the project progressed, The Post decided against relying on the FBI's justifiable homicide reports for several reasons. Some criminologists have questioned the data's accuracy. By definition, FBI reports are jurisdictional, not departmental: If an officer kills someone in another jurisdiction, according to Uniform Crime Report guidelines, the death should be reported by the agency in the jurisdiction where it occurred rather than by the officer's own department.\n\nAdditionally, reports to the FBI may include justifiable homicides by armed security guards, transit police, housing police or other armed police. There is no flag in the data to indicate whether an officer is a member of the reporting agency's force.\n\nBecause of these and other concerns, The Post chose to survey directly police departments to collect fatal shootings and woundings by each department's officers for 1990 to 1997. We focused on the 27 cities that had at least 500,000 residents at any point from 1990 to 1996, the latest year that U.S. Census population estimates were available.\n\nFor verification, we compared the numbers we collected to those in the SHR. We searched news clippings for previously published reports. Often, there were small differences; sometimes, bigger ones. To address these variations, we contacted the departments for further clarification or sought other sources, such as state UCR reporting programs or state attorneys general charged with reviewing fatal shootings by police officers.\n\nFive-gauge analysis\n\nAfter consulting criminologists and reading literature on the subject, we determined that no single measure was adequate to gauge a department's number of officer-involved shootings. We decided instead to measure fatal shootings against five other categories: population, violent crime, arrests for violent crime, homicide and the number of uniformed police officers.\n\nOur data sources, all electronic, included:\n\n• Return A for violent crime and homicide totals. (This is the base table in the FBI's UCR.) Because it is the FBI standard, we used the Return A for homicides instead of the SHR. We also found its numbers to be more accurate than the SHR's. For example, in 1996, the SHR recorded four homicides for Austin, while the Return A recorded 40.\n\n• Age, sex and race of persons arrested for the number of violent crime arrests\n\n• Police employees for each department's number of uniformed officers\n\n• Instead of the FBI's population estimates for cities, which are extrapolated from state estimates, we used the more sophisticated Census Bureau city estimates for per capita calculations. (These figures are not ready early enough for inclusion in the FBI's annual reports.)\n\nArriving at violent crime, homicide and arrest totals required adding up numerous fields of coded data. To verify our data work, we checked each number that we planned to use against the appropriate volume in \"Crime in the United States,\" the FBI's annual publication. To further vet the FBI data, we checked the District's crime numbers with the police department and called a sample of other departments as well.\n\nMore shootings by all measures\n\nTo help offset possible discrepancies and to compensate for years with accurate but unusual spikes in fatal shootings, we averaged the number of fatal police shootings over the period for each department. Likewise, we averaged the numbers of violent crimes, arrests for violent crimes, homicides, uniformed officers and population.\n\nWe used these averages to calculate how many fatal shootings there were per 100 homicides, per 1,000 arrests for violent crimes, per 10,000 violent crimes and per 100,000 residents in the District.\n\nWe aggregated the other 26 cities' numbers and compared the District to the group. We also aggregated the data for nine other particularly violent cities and compared the District to that subset as well. Because of variations in how departments define and report violent crime, we used homicides to identify violent cities, choosing those with the highest homicide rates for the period.\n\nBy each of these measures, the number of fatal shootings by District officers was higher than average – often significantly higher. Only when measured against homicide did D.C. approach the norm. Some criminologists say a high rate of homicide helps explain officer-involved shootings.\n\nTo confirm the statistical significance of our findings, The Post consulted with Sarah Cohen, then NICAR's training director (The Post has since hired her), and later submitted the findings to James Fyfe, a nationally recognized criminologist, for review.\n\nLocal digs\n\nWhile we mined these federal databases, we also dug in locally, obtaining at least one electronic database, building others and scouring the courts for civil and criminal cases.\n\nWith a FOIA request, we obtained an electronic database from the District of Columbia Use of Service Weapon Review Board that purported to detail every firearm discharge by a District police officer. The records outlined the circumstances leading to each shooting, but the department redacted officer names.\n\nWe collected and built a database of nearly seven years of press releases from the D.C. police department on officer-involved shootings. We also found seven press releases on fatal shootings that had never made it to the department's Use of Service Weapon Review Board, the body charged with determining whether a shooting was justified or not.\n\nAnother FOIA request yielded a list of lawsuits against the District, the Metropolitan Police Department or its officers that involved a shooting. We examined each of these lawsuits, gleaning more details about the shootings, the officers and sometimes the department's own investigations.\n\nEventually, we expanded our search to include police brutality and built a database of more than 300 lawsuits against the District, the police department or police officers that had been settled or adjudicated, recording the settlement amount and the nature of the complaint.\n\nFinally, we mapped the local data, which provided a stark and startling picture of the volume and location of fatal shootings and woundings in the District.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-april-investor-burden","slug":"1999-april-investor-burden","url":"https://dwillis.github.io/uplink/article/1999-april-investor-burden.html","year":1999,"month":"April","monthNum":4,"headline":"Investor burden","kicker":"THE PRICE OF CONCENTRATED COMPETITION","author_name":"Christopher H. Schmitt","author_title":"","affiliation":"The San Jose Mercury News","summary":"The San Jose Mercury News used computer-assisted reporting to analyze Nasdaq trading data, revealing that concentrated dealer competition forces investors to pay significantly higher trading costs, potentially costing the public hundreds of millions of dollars annually.","keywords":["Nasdaq","trading","competition"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":["Quattro Pro","Paradox","SPSS","CD-ROM"],"full_text":"One area in which computer-assisted reporting has been less frequent is business coverage, in large part because there aren't gobs of data readily available, such as that prepared by government agencies.\n\nBut, if you scout around, it turns out great information actually is available. With some imagination, you can use it to produce good stories. One example, humbly submitted: The San Jose Mercury News recently enjoyed success in mining trading records of the Nasdaq Stock Market to show how investors are forced to pay sharply higher trading costs.\n\nBefore you yawn, understand that Nasdaq affects virtually everyone. In Silicon Valley, the Mercury News' backyard, Nasdaq has been the coin of the realm in recent years. Hundreds of high-tech companies have gone public on Nasdaq, raising the capital that has funded the valley's world-changing innovations.\n\nMore broadly, tens of millions of Americans invest in Nasdaq stocks – on an individual stock basis, through 401(k) plans or through pension plan holdings.\n\nUnlike its better-known cousin, the New York Stock Exchange, Nasdaq has no trading floor. Instead, Nasdaq exists in the form of a super-fast computer network connecting securities dealers across the country.\n\nInvestors' losses are stock dealers' gain\n\nTherein lies our story. The fundamental premise of this system is that spirited competition among the dealers is supposed to keep investors' trading costs low. But, using a CAR analysis, the Mercury News found that isn't true.\n\nAs we said in our \"Shortchanged on Nasdaq?\" report, more often than not, trading of a company's stock is concentrated in the hands of only a few dealers. When that happens, the transaction costs for investors are sharply higher.\n\nHow much higher? Add it all up and the extra cost to the investing public could reach hundreds of millions of dollars annually.\n\nWho profits? Mostly the stock dealers, which include such household names as Morgan Stanley, Merrill Lynch and Goldman Sachs, plus regional firms across the country.\n\nTo study Nasdaq, we analyzed about 11.2 million Nasdaq trades and market price postings. This data came from the Trade and Quote database of the New York Stock Exchange. Available on CD-ROM at nominal cost, TAQ contains trade-by-trade data on stocks listed on the NYSE, the American Stock Exchange and Nasdaq.\n\nWidely used by academics, TAQ is a good example of the kind of lesser-known business data that's available.\n\nOur analysis found that trading of a company's stock was concentrated in the hands of only a small number of dealers nearly two-thirds of the time. When there was the least amount of competition among dealers, the cost to investors to buy or sell those stocks was nearly twice what it was to trade when competition was vigorous.\n\nMost important, the level of dealer competition turned out to be a bigger factor in explaining investor costs than other factors customarily looked at by Nasdaq and federal regulators.\n\nDeveloping this last point was key, and it accounted for most of the heavy lifting in our CAR analysis. It turns out that assessing dealer competition by itself isn't terribly difficult. Besides dealer competition, things like trading volume also affect investor costs. It was crucial to \"filter out\" those other things to arrive at an apples-to-apples comparison.\n\nKnow statistical limits\n\nWe did this using a multiple regression model we developed. If you're interested in the gory details, check out the Web sites listed in the sidebar. In the meantime, suffice it to say this model accounted for various factors affecting transaction cost and allowed us to explore among all of them how significant the influence was of dealer competition. Our primary software tools were QuattroPro, Paradox and SPSS.\n\nAs we said in our story, we didn't find evidence of collusion or other deliberate anti-competitive behavior among dealers. But, some economists who reviewed our findings believe Nasdaq should make changes to improve market performance.\n\nWith any analysis like this, it's important to keep in mind that you can never prove a relationship – in statistical terms, you've got to be on guard against confusing correlation with causality. While we didn't prove that lagging dealer competition directly causes higher investor costs, the analysis clearly suggested that the level of competition plays a significant role, even when accounting for other factors.\n\nNasdaq executives hated what we did, calling it misguided and wildly inaccurate. In fact, they went to considerable lengths to try and prevent publication of the story.\n\nBut, the bottom line, as business people are wont to say, was this: We put CAR to work in a coverage area where data isn't as readily available as others. In doing so, we think we told a compelling pocketbook tale that affects nearly everyone. You can, too.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-april-flirting-with-disaster","slug":"1999-april-flirting-with-disaster","url":"https://dwillis.github.io/uplink/article/1999-april-flirting-with-disaster.html","year":1999,"month":"April","monthNum":4,"headline":"Flirting with disaster","kicker":"IGNORED RESTRICTIONS","author_name":"T. Christian Miller","author_title":"Reporter","affiliation":"Los Angeles Times","summary":"A Los Angeles Times investigation using mapping and data analysis revealed that Los Angeles County politicians allowed developers and campaign contributors to violate growth restrictions in the Santa Monica Mountains, resulting in homes built in excess of limits that were twice as likely to suffer damage from natural disasters while taxpayers footed the repair bills.","keywords":["development","violations","mapping"],"topics":["Investigative Methods","Government & Politics","Health & Environment"],"technologies":["Maptitude","Excel","SPSS","mainframe"],"full_text":"The Santa Monica Mountains are one of Los Angeles' most spectacular natural features. The range stretches through the heart of the region, rising high above the city sprawl. It's where the famous – from Johnny Carson to Axl Rose – build their ego mansions.\n\nIt's also where county politicians have routinely allowed developers and campaign contributors to run roughshod on environmental and growth restrictions, according to a recent Los Angeles Times analysis.\n\nFlouting those restrictions not only hurt the mountains but also stole from the public's pocketbook. Homes built in excess of the area's growth limits stood better than twice the chance of being hit by one of the natural disasters that plague the region – earthquake, flood and fire. Usually, state and federal taxpayers picked up the repair bill.\n\nMapping reality over design\n\nThe story began with a simple question: Who built all the homes in the mountains, and how did it happen?\n\nFrom the beginning, it was clear this would be a mapping project, albeit one with a depressingly large scope: 20 years of development approvals covering more than 35,000 parcels of property.\n\nTo start, we paid a consultant to scan in the area's growth plan, an outdated, 20-year-old document created when the bulk of the undeveloped part of the mountains was declared a national recreation area. The complicated map had more than 5,000 polygons, each representing different use and density classifications.\n\nAt the time of the original plan, the idea was to hold down building density. That map, we figured, showed how the mountains were supposed to look. The next step was creating a map that showed the reality: all subdivision approvals since 1978.\n\nThat turned out to be a trickier task. The Los Angeles County Assessor's Office wanted $80,000 to sell the GIS information outright. Instead, we rented time on the assessor's mainframe, then watched as one of his assistants created a special file showing the outlines of the subdivisions in our area of study.\n\nOnce we had the two maps, we imported them into Maptitude, a low-budget, though effective, GIS program. Then we overlaid the two. The growth-plan map showed proposed density. Our new map showed actual density. The overlay, the heart of the project, showed the subdivisions that violated the growth restrictions – 40 percent of the nearly 100 subdivisions.\n\nTimely contributions\n\nFrom there, we began reviewing the individual subdivision approvals, combing through musty files in numerous city halls and planning departments for the names of the developers, engineers, consultants and attorneys involved in the high-density projects. Using laptops loaded with Excel, we created our own database to track them, eventually inputting about 2,500 names.\n\nWe ran those names against the Times' in-house database of political contributions, which has been continually updated since 1984. The results were not surprising: Nearly all of the developers and businessmen involved contributed to the campaigns of various supervisors in charge of the area.\n\nSince the link between developers and county supervisors had long been known, we decided to take the story one step further. To do that, we obtained the case-tracking database from our planning agency, which included the dates of public hearings and planning commission votes going back 15 years.\n\nWe then compared the dates in that database with the dates in our political contribution database to determine that, in many cases, contributions came within the day and weeks of key votes on the projects. After approval, when the developers had gotten what they wanted, the money tap shut down.\n\nHarmed habitats\n\nNext, we decided to check the damage from the developments. From the county's Flood Control District, we obtained water-level data for various streams in the area, data they had kept continuously for 30 years.\n\nUsing SPSS, we then charted the increase in high-density area homes against an in-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-april-safety-through-gis","slug":"1999-april-safety-through-gis","url":"https://dwillis.github.io/uplink/article/1999-april-safety-through-gis.html","year":1999,"month":"April","monthNum":4,"headline":"SAFETY THROUGH GIS","kicker":"","author_name":"Jennifer LaFleur and John Woolfolk","author_title":"","affiliation":"San Jose Mercury News","summary":"A short report on using GIS to overlay landslide maps with building permits to identify risky construction areas before the 1998 El Nino storms.","keywords":["GIS","landslides","public safety"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["GIS"],"full_text":"BEFORE THE 1998 EL NINO STORMS, WE OBTAINED LANDSLIDE AND DEBRIS FLOW MAPS FROM THE U.S. GEOLOGICAL SURVEY FOR THE NEARBY SANTA CRUZ MOUNTAINS. USING GIS WE WERE ABLE TO OVERLAY THOSE MAPS WITH BUILDING PERMITS ISSUED DURING THE PAST TEN YEARS TO SHOW WHERE BUILDING PERMITS HAD BEEN ISSUED ON LANDSLIDE AREAS. THE STORY RAN JANUARY 11. JENNIFER LAFLEUR SAN JOSE MERCURY NEWS JLAFLEUR@SJMERCURY.COM FOR MORE INFORMATION.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-april-write-the-flipping-story","slug":"1999-april-write-the-flipping-story","url":"https://dwillis.github.io/uplink/article/1999-april-write-the-flipping-story.html","year":1999,"month":"April","monthNum":4,"headline":"Write the flipping story!","kicker":"HANDOUT OF THE MONTH","author_name":"Dan Browning","author_title":"Reporter","affiliation":"Saint Paul Pioneer Press","summary":"Real estate flipping—buying property and quickly reselling it for inflated profits—is a widespread scam involving appraisal fraud, straw buyers, and equity skimming that devastates poor neighborhoods and defrauds lenders and homebuyers. The practice, refined during the S&L crisis and now operating in cities across America, causes foreclosures, neighborhood decay, and financial losses that ripple through entire communities.","keywords":["flipping","fraud","foreclosure"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Write the flipping story!\n\nYou'll probably never hear an editor yell that across a newsroom. Too bad. Across the United States, speculators are engaging in real estate scams called \"flipping\" as fast as they can. It's an old gambit. You've probably seen motivational programs on the practice on late-night TV: \"Buy property with no money down! Get rich using other people's money!\"\n\nIn short, flipping is the practice of buying property and turning it around quickly for a large increase in value. It's not inherently illegal. But it often is.\n\nIn an illegal flip, a speculator must find an appraiser to inflate the value of a house so that a lender will advance a mortgage that is not supported by the property's true market value. The speculator then buys the property near or below its true market value and re-sells it, either to a dupe or to a straw buyer, who finances the transaction with the loan. The purchase and resale often take place the same day.\n\nSophisticated power-brokers refined the scam during the S & L crisis. They flipped commercial properties and entire planned communities. Now, schemers are at work in poor neighborhoods in the nation's cities. A handful of reporters have documented flipping schemes in New Jersey, Maryland, Florida, Wisconsin and Minnesota. It's undoubtedly more widespread.\n\nEquity skimming and surfing\n\nThe latest flipping schemes are intriguing. Investigators have discovered fake financial records, forgeries, misrepresentation of ownership and many other ploys. Authorities suspect some flippers of laundering drug money through the purchase and resale of these homes.\n\nWhen straw buyers are involved – that is, when the buyer is working with the seller, usually for a fee – equity skimming is often found. That's where the buyer rents out the property but fails to pay the mortgage. The buyer keeps the rents as the property winds through the foreclosure process, which can take up to 18 months in a state like Minnesota. By the time the authorities uncover the scam, the flippers may be long gone.\n\nWho suffers? Entire neighborhoods may be overassessed because of the artificial inflation of property values. Unsophisticated buyers end up in foreclosure, their credit wrecked and their eligibility for first-time home-buying programs forever lost. Buildings get boarded up and condemned. Municipal utility bills go unpaid. Lenders get stuck with properties they can't sell. And they must reimburse investors who bought securities backed by these fraudulently obtained mortgages. Shareholders in the mortgage companies therefore get less return on their investments. Lenders, feeling burned, then shy away from poorer neighborhoods.\n\nDealers deduced\n\nHow do you uncover property flips? It's easier than you might think.\n\nIn 1996, when I worked at the Saint Paul Pioneer Press, I wrote about property flips involving one well-connected speculator. A source told me that the speculator had bragged that he was putting people into business around the country, and no one would ever catch him. Two years later, I discovered that several of his associates or acquaintances had indeed gone into business.\n\nI was following one of his associates closely. Based on a tip, I pulled his property transactions. But I found far fewer deals than I expected. The reason? Dirty data. I'd type in the name of his company, but the data entry at the county was so poor that I missed many of the deals. Later, I discovered that he used several corporations to do the deals. I talked with local assessors and found that I could buy data documenting property sales, called \"certificates of real estate value\" (CRV).\n\nIn Minnesota, as in most places nowadays, a buyer must file these documents attesting to the date of the sale, the buyer, the seller, the property identification number, the purchase price and the nature of the financing, among other things. I pulled a bunch of hard copies and found that most of the time, the buyer, seller, sales date, PIN and purchase price were accurate. Most everything else on the forms were unreliable.\n\nThe assessor gave me (at no charge) the database of CRVs from 1995 through March 1998. I used FoxPro to assign a transaction number on each sequential sale for a property and to clean up the names of the buyers and sellers. I had 6,575 transactions on 3,079 Minneapolis properties. Of those, 2,148 sold more than once, for a total of 4,170 transactions. Knowing that flips occur rapidly and for significant increases in value, I calculated the number of days between each repeat sale, and the percent change in value. Because many of the transactions took place on the same day, I couldn't say whether the percent change was positive or negative. So I decided to ignore the sign (plus/minus) of the percent change.\n\nI decided to examine deals that took place in less than 45 days for greater then a 50 percent change in value. I brought everything into Access to count the number of transactions each buyer and seller made. Then I took the list of major traders and ran the corporate names through the Secretary of State's Office to find out who was behind the corporations. I called this my AKA list. Then I pulled every single deal these heavy traders made, regardless of whether it appeared to be a flip or not.\n\nIf a high percentage of the speculators' deals resembled flips, I copied their names to a short list. By contrast, if a small percentage fit the pattern of a flip, I decided to ignore them. Once my short list was built, I recounted the number of transactions these speculators had completed, both in terms of purchases and sales. Then I went to the courthouse and pulled the paper records: mortgage information, title transfers, etc. That was the hard part. It took about two months and cost me about $500 to copy these property records.\n\nWidespread speculators, scarce licenses\n\nMinnesota requires anyone who makes five or more trades in property in a 12-month period to get a \"limited broker's license.\" I took my data to the Minnesota Commerce Department and asked which of my speculators were unlicensed. The department was investigating flip sales on its own. But my database revealed that the practice was far more widespread than they had believed. They expanded their investigation. The department revealed that the FBI also was investigating.\n\nThe commerce department, concerned that the media would ignore the problem, planned a press conference. The Pioneer Press ran my story on flips to head that off. The Star Tribune followed with several stories of its own. One showed that flips can take place even when the speculator holds the property and rehabilitates it.\n\nI later moved to the Star Tribune. In February, I wrote a story about the first criminal indictment here, which charged five people with conspiracy, mail fraud, bank fraud and various money laundering charges. I also noted three major civil suits filed by WMC Mortgage Corp. against 23 people and 30 business entities. My colleagues published follow-ups showing the human pain these practices can wreak. A sidebar showed how Norwest Mortgage, the nation's largest residential lender, lobbied the U.S. Attorney's Office to bring the criminal indictment. That case involved one of the lender's former mortgage brokers. So far, the state has disciplined six license holders and has targeted a total of 50 for closer scrutiny. Two FBI investigations are moving forward. The criminal division of the IRS is reportedly investigating, as is the state attorney general's office.\n\nI'd be remiss if I failed to mention the great work done on flipping stories by the Milwaukee Journal and the Asbury Park Press in the past two years.\n\nBut I'd also be neglecting this opportunity if I failed to point out that this practice is going on around the country, yet few reporters have chosen to investigate it. I urge you to reconsider.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-april-booming-claim-busted","slug":"1999-april-booming-claim-busted","url":"https://dwillis.github.io/uplink/article/1999-april-booming-claim-busted.html","year":1999,"month":"April","monthNum":4,"headline":"\"Booming\" claim busted","kicker":"LOCAL ECONOMIC PORTRAIT","author_name":"Paige St. John","author_title":"","affiliation":"Tallahassee Democrat","summary":"A Tallahassee newsroom investigated local car dealers' claims of booming business by examining vehicle registration data, discovering sales were actually down significantly, which led them to analyze the broader regional economy using employment, wage, housing, and building permit data.","keywords":["investigation","economy","data"],"topics":["Investigative Methods","Business & Economy","Data & Databases"],"technologies":["Excel","spreadsheet"],"full_text":"\"Booming\" claim busted\n\nThe tip came from the side of the news building that most reporters suspect and shun: Advertising.\n\nTallahassee car dealerships were in a foul funk, selling fewer cars and therefore buying fewer ads. But, when we called dealers to ask what was going on, each and every one claimed business was booming.\n\nBooming? We returned from visiting the computer clerk in the county recorder's office with a stack of monthly new car registration reports, which turned out to be the first key to dissecting the local economy. (HINT: This dataset also produced shorts on the hottest selling vehicles in town and statistics on out-of-town buyers. Later we learned the reports already were being collected each month by our own advertising department.)\n\nThe car registration reports were broken down not only by dealership but also vehicle model. Month-to-month comparisons showed sales were way down. Plotting the numbers on six-month and 12-month rolling averages showed we were looking at an entrenched slump, not a glitch in the market.\n\nArmed with something solid, we called the dealers back. \"Business is great!\" they said again, but when we started citing their own slipping sales numbers, the interviews took a dramatic turn. Suddenly, the dealers were trying to explain a mysterious slump that was dragging down Tallahassee's economy.\n\nBusiness reporter Lisa Newkirk and I decided to try to take the pulse of the region, dividing the economy into sectors and, like the car dealership story, accumulating data on the performance of each one.\n\nLagging employment and wages\n\nThe Florida Division of Labor and Employment Security (www.state.fl.us/dles/main/statdata.htm) keeps an electronic warehouse of reports, downloadable in spreadsheet format. Most useful were the ES-202s, quarterly surveys of employment levels and wages broken down by county and by job sector. The standardized reports allowed easy statewide and national comparisons. Importing 10 years of reports into Excel, we could tell where Tallahassee was lagging in job and wage growth.\n\nSlim pickings, housing glut\n\nA detailed database of home listings and sales, kept by the local realtors association, reflected activity in the housing market. It showed what was selling and where. More importantly, it showed what was not selling. The \"market time,\" the period a home was on the market before being sold, had stretched to three years in what once had been Tallahassee's hottest housing developments – hallmarks of a building glut.\n\nBecause realtor records don't reflect homes sold by developers, builders or individual homeowners (the \"For-Sale-By-Owner\" crowd), we turned to a commercial service for more detailed activity reports. The Florida-based Metro Market Trends collects monthly data tapes from county clerk and building permit offices. Our advertising department already subscribed, so we had free access. In addition to home sales, the reports showed a sharp drop in building permit activity and a rise in mortgage foreclosure filings. (Some of the same data also is available directly from local building permit offices.)\n\nFrom spurt to sputter\n\nWe mapped population growth – important to fuel the home construction industry – using the Census Bureau","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-may-charting-a-flood","slug":"1999-may-charting-a-flood","url":"https://dwillis.github.io/uplink/article/1999-may-charting-a-flood.html","year":1999,"month":"May","monthNum":5,"headline":"Charting a flood","kicker":"MAPPING ANALYSIS","author_name":"Chris Williams","author_title":"Reporter","affiliation":"San Antonio Express-News","summary":"Journalists at the San Antonio Express-News used GIS mapping and database software to show that 75 percent of buildings destroyed by a major flood were located within recognized flood plains.","keywords":["mapping","flood plains","MapInfo"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["Access","MapInfo","GIS"],"full_text":"Two weeks after a freak storm dropped more than 15 inches of rain on San Antonio, more than 100 residents from a flood-ravaged neighborhood gathered in an elementary school gym. They wanted to know why the government, at any level, had not told them they were at risk of flooding. Across town, journalists at the San Antonio Express-News already were working on the same deceptively simple questions. Before the swollen creeks had receded, we were asking: \"Where did the flood water go? Was there any pattern to the damage? Did people who needed flood insurance have it? Why not?\" Some moderately sophisticated CAR techniques were needed to answer those simple questions. On Nov. 29, five weeks after the flood, we printed the results of the work of a computer-savvy reporter and news researcher. We found: • The path of the destruction closely followed the 100-year flood plains published in 1996. In fact, 75 percent of the 175 residences destroyed by the flood were in those flood plains. • Citywide, only about 10 percent of the damaged buildings were insured by the National Flood Insurance Program, which meant 90 percent of the people in 470 damaged buildings were trying to rebuild on slim federal grants or disaster loans they could not afford to repay. • Many more houses along creeks on the poor side of town were damaged than buildings along creeks in wealthier areas. The story demonstrated that you don't need a full-time data analyst or CAR guru to get high-impact stories into the newspaper. From the very beginning, we knew that we wanted to print a map showing the locations of hundreds of properties damaged by the flooding. We quickly decided the way to do that was building, then geocoding, a database of damaged addresses. Geocoding is the geek word for assigning latitude and longitude coordinates to (in our case) street addresses. We used the software at hand – Microsoft Access for building the database and MapInfo for generating the maps and analyzing the relationship between the flood plains and the locations of the actual damage. The American Red Cross gave us hard copies of their damage assessments, which included addresses, an inspector's opinion of the extent of damage and the estimated depth of the water. We built the database using a form in Access. Although most of the fields were standard, we devised a clever trick that we recommend for anyone who needs to build and then geocode a database. We copied the streets file out of our MapInfo software and imported it as a table into Access. We cut out all the columns except the street names. Then we deleted duplicate names, which whittled the table down from 87,000 to 20,000 rows. Within our data-entry form, we linked the street address field via a combo box to our new table of street names. The computer would recommend the proper spelling of a street name after we typed its first two or three letters, which saved time and ensured that the street names in the database would be exactly as they appeared in the MapInfo file. Having street names in a database that don't match those on the streets file of the GIS software is frustrating. The computer can't tell the difference between \"Villa Main\" and \"Villamain.\" When we were done, we could generate a map of red dots for the 470 buildings damaged in the flood. With MapInfo's query ability, we could sort out homes, businesses, apartments and mobile homes. The missing piece was the flood-plain data. That was easy. We bought it. The MapInfo Web page listed a company, On Target Mapping, that sold digital maps from the Federal Emergency Management Agency for about $200. From there, we printed a large map of the city and several smaller, blow-up maps of the damaged areas for the art department. We had difficulty outputting maps into a format our art department could import on their Macs. So, the artists scanned the images into their graphics software, then retouched them. Once the data wrangling was finished, the story became a matter of traditional reporting. We interviewed dozens of people, such as the mother who lost everything in the flood except five dresses and two school uniforms for her son and the city public works director who claimed he told several unheeding neighborhood groups about the dangers of flooding. After the story appeared, several members of the San Antonio City Council pledged to force the city to notify by mail everyone who lived in a flood plain about their risks of flooding. The council also embarked on a multi-million-dollar program of buying out properties in the flood plain that were damaged by the flood.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-may-mexico-and-beyond","slug":"1999-may-mexico-and-beyond","url":"https://dwillis.github.io/uplink/article/1999-may-mexico-and-beyond.html","year":1999,"month":"May","monthNum":5,"headline":"Mexico and beyond","kicker":"CAR REPORT FROM LATIN AMERICA","author_name":"Pedro Enrique Armendares","author_title":"Co-Founder","affiliation":"Periodistas de Investigación","summary":"Periodistas de Investigación is working to bring computer-assisted reporting and investigative journalism techniques to Mexico and other Latin American countries to improve public accountability.","keywords":["Latin America","training","investigative journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"Training Mexican journalists in the methods of investigative reporting has been the purpose of Periodistas de Investigación for almost three years now. PI was the brainchild of IRE members from both sides of the border and it extends IRE's programs, such as workshops on computer-assisted reporting, to Mexico thanks to a grant from the McCormick Tribune Foundation.\n\nIn September 1996 Lise Olsen and I opened PI's office in Mexico City. Lise came out of the United States with many years of experience in investigative journalism and CAR. She had also taught CAR workshops for IRE. I began my work in radio and television, later moving to the Mexico City daily La Jornada, where I covered special assignments for seven years.\n\nThe last decade has been an exciting time for investigative journalism in","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-may-in-memorium","slug":"1999-may-in-memorium","url":"https://dwillis.github.io/uplink/article/1999-may-in-memorium.html","year":1999,"month":"May","monthNum":5,"headline":"In Memorium","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Uplink pays tribute to its late managing editor, Brent Johnson, and announces the establishment of a memorial scholarship fund in his name.","keywords":["Brent Johnson","obituary","scholarship"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Many of you knew Brent Johnson, a graduate student here at Missouri, who has been managing editor of our newsletter Uplink and has contributed to the IRE Journal. Brent died April 13 of cardiac arrest. Brent had Marfans Syndrome, a connective-tissue disorder that affects the cardiovascular and skeletal systems, and had gone into the hospital to have a heart valve replaced. Although this was a serious operation, he (and we) fully expected a quick recovery and for him to go on to his internship at the Miami Herald this summer. However, complications ensued during and after the operation. Brent, who was with us for the past two years, was a gifted writer and reporter and a wonderful colleague and friend with a great sense of humor and perspective. We will miss his voice. In his memory, we are setting up at IRE a scholarship fund in his name to be given each year to the graduate student who is managing editor of Uplink. If you want to make a contribution, please make out your check to IRE and mark it for the \"Brent Johnson Scholarship Fund.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-may-cars-advantages","slug":"1999-may-cars-advantages","url":"https://dwillis.github.io/uplink/article/1999-may-cars-advantages.html","year":1999,"month":"May","monthNum":5,"headline":"CAR's advantages","kicker":"TIME, SWEAT AND SHOE LEATHER","author_name":"Paul Overberg","author_title":"","affiliation":"USA Today","summary":"Paul Overberg explores how computer-assisted reporting provides efficiency by directing journalists toward specific data-driven leads for more effective shoe-leather reporting.","keywords":["efficiency","journalism","data analysis"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Nexis","MapInfo"],"full_text":"One of computer-assisted reporting's less-celebrated virtues is its ability to save time, sweat and shoe leather.\n\nWhen we sell CAR to skeptical editors and reporters, we often focus on how it can root out corruption or do the previously impossible, such as analyzing and publishing to the Web a school test-score package.\n\nBut it's good to remember CAR's humbler virtues. Time and money are always short. Saving you two days of interviewing and saving the editor's plan to run your story the following weekend win just as many converts as a big hit.\n\nThis idea struck me again last year when discussion arose on the NICAR mailing list about how one might track and analyze the tragedy of fatally abused children who have been written off as victims of sudden-infant death syndrome. I watched colleagues at Gannett News Service do this in 1991. Most cases turned up from interviews and Lexis-Nexis searches. But they also had a statistician analyze 49,569 death certificates from 1987 of children under nine to determine autopsy rates for various causes of death, especially suspicious ones, in every state. That steered them to good places for shoe-leather work, which turned up great material for the case database. Reporters Marjie Lundstrom and Rochelle Sharpe won the Pulitzer.\n\nMore recent examples:\n\n• In analyzing county-level personal bankruptcy rates in 1997, USA Today's Barbara Pearson checked several correlations before finding that one of the best was whether a state allowed wage garnishment. She mapped the data and found the Augusta, Ga., metropolitan area offered a stark contrast. Flanking the Savannah River, Richmond County, Ga., and Aiken County, S.C., were pretty similar, but the former had twice the bankruptcy rate. Reporter Tanya Albert went to Richmond County and found a wealth of local lore – the perfect prism for this CAR story.\n\n• Water well tests around Oakland County, Mich., last year showed fairly high levels of arsenic. The Detroit Free-Press mapped test results with geological information. The worst results clustered where soil was shallow over sandstone from which the arsenic came, said Heather Newman, a Free-Press projects reporter and CAR specialist. That told reporters exactly where to find people who were most affected.\n\n• In Utica, N.Y., gasoline prices well above the national and regional averages prompted an editor at The Observer-Dispatch to wonder if they were due to concentrated gas station","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-may-calculating-aids","slug":"1999-may-calculating-aids","url":"https://dwillis.github.io/uplink/article/1999-may-calculating-aids.html","year":1999,"month":"May","monthNum":5,"headline":"Calculating Aids","kicker":"ON THE INTERNET","author_name":"Stephanie Reitz","author_title":"","affiliation":"The Hartford Courant","summary":"This collection of online resources provides journalists with essential tools for learning statistics, polling ethics, spreadsheet management, and general data calculations.","keywords":["Internet","statistics","calculators"],"topics":["Data & Databases","Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","spreadsheet"],"full_text":"1. Statistics Every Writer Should Know: nilesonline.com/stats/ Compiled by Rocky Mountain News webmaster Robert Niles, this site offers a great primer on averages, mean, median, per capita and other concepts. 2. The 10 Commandments of Polling: metalab.sunc.edu/journalism/pollcomd.txt Ken Blake of University of North Carolina (chapel Hill) explains the basic dos and don'ts of polling. Thou shalt not miss this site. 3. The Spreadsheet Page: j-walk.com/ss/ Compiled by a computer consultant in Sand Diego, the best offerings on this site can be found along the toolbar on the left-hand side of the page. Look for easy tips (and jokes) on various kinds of spreadsheet programs. 4. Martindale's Calculators Online: www-sci.lib.uci.edu/HSG/RefCalculators.html It takes a bit of navigating to get used to the format of this page, but the results are worth the effort. It claims to have more than 8,200 calculators on the site, ranging from basic percent-change calculators to translation of names from English into various languages (not helpful during budget time but fun after deadline).","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-may-title-ix-undermined","slug":"1999-may-title-ix-undermined","url":"https://dwillis.github.io/uplink/article/1999-may-title-ix-undermined.html","year":1999,"month":"May","monthNum":5,"headline":"Title IX undermined","kicker":"HIGH SCHOOL GENDER DISCRIMINATION","author_name":"Jim Hopkins","author_title":"Reporter","affiliation":"The Courier-Journal","summary":"A six-month data analysis by The Courier-Journal exposed widespread gender discrimination in Kentucky and Indiana high school sports, forcing new state compliance measures.","keywords":["Title IX","gender discrimination","high school sports"],"topics":["Education","Investigative Methods","Demographics & Census"],"technologies":["Paradox","Excel"],"full_text":"In Fourmile, on the western edge of Appalachian Kentucky, softball coach Rick Nelson bristled at the way some high school girls were treated by the Bell County School District. At the girls' softball field, about half the lights didn't work. Dugouts were little more than tin roofs propped up by two-by-fours. The restrooms were crowded, dirty and didn't always work. Seven miles down the road the boys' baseball team had manicured fields that were amply lit and featured cinder-block dugouts painted with the Bobcats' logo. Nelson told The Courier-Journal that he believed the county school district was in violation of Title IX, part of a 1972 federal law that requires schools to provide comparable athletic programs for girls and boys. That interview prompted a team of Courier-Journal reporters to conduct a six-month computer-assisted examination of athletic programs at hundreds of public high schools in Kentucky and Southern Indiana. Our conclusion in February 1998: There was rampant gender discrimination in equipment, practice times, facilities, budgets and the number of teams offered. The Courier-Journal's disclosures brought immediate results. The Kentucky Board of Education now requires all high schools to certify annually that they're complying with Title IX. For the first time, the state sponsored two Title IX training sessions for school officials. And in February of this year, after the newspaper published a follow-up report showing continued inequities, education board members accused the Kentucky High School Athletics Association of not working hard enough to deal with the problem. The newspaper's findings came after lots of traditional shoe-leather journalism mixed with newer computer-assisted reporting and analysis techniques. Under open-records laws, we mailed requests to each of the 255 Kentucky and Southern Indiana high schools that have interscholastic sports. We asked for sports participation numbers by gender, for spending on boys' and girls' sports, for a gender breakdown on athletic directors and coaches and other information. We also obtained electronic copies of schedules for more than 5,000 boys' and girls' basketball games for the 1997-98 season. Using Paradox and Excel, we created dozens of databases to track the following: spending, growth in boys' teams versus girls' teams and trends in the hiring of women. We showed, for example, that: • Although boys comprised about 50 percent of school enrollment in Kentucky's public high schools, they held 62 percent of all the team positions. • Men held virtually all the coaching slots in major boys' sports and the majority of slots in girls' softball, basketball and soccer. • Nearly 40 percent of boys' basketball games were played during \"prime time\" – Friday nights, when audiences are especially large. Less than 13 percent of girls' games were scheduled during that time slot. The biggest obstacles we faced were in gathering data. Because there was no central repository for the records we needed, the lead reporter had to write individual records requests to each school. When school officials didn't respond, we placed follow-up phone calls and made additional written requests. We were also hampered by the fact that school officials generally provided raw data, which meant sorting through copies of budgets and hiring schedules to learn, for example, how much was spent on travel for a football team or how many women were in coaching positions. Finally, to keep the story focused on the people most affected – the students – we crisscrossed the state to interview parents, team members and school officials, then supplemented that reporting by talking with experts on a national level. Going beyond the data led us to people like Jerry Owens, principal at Owen County High. \"There is probably not a high school in the state that isn't violating the law,\" he said. \"The good ol' boy network is very much in place.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-may-discovering-data","slug":"1999-may-discovering-data","url":"https://dwillis.github.io/uplink/article/1999-may-discovering-data.html","year":1999,"month":"May","monthNum":5,"headline":"Discovering data","kicker":"CAR REPORT FROM DENMARK","author_name":"Flemming Svith and Nils Mulvad","author_title":"","affiliation":"Morgenavisen Jyllands-Posten","summary":"Reporters in Denmark founded the Danish Institute for Computer-Assisted Reporting (DICAR) to promote data analysis skills and fight for better access to electronic public records.","keywords":["Denmark","DICAR","freedom of information"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web","PC"],"full_text":"During the last three years, computer-assisted reporting has developed in Denmark from something unknown to an established set of research and analytic skills. At the outset, there was a defeatist attitude toward CAR in Denmark, but that has changed fundamentally. Active reporters with different areas of expertise have cooperated with each other; a very competent freedom-of-information lawyer has joined the effort; and the management of the Danish School of Journalism has been supportive. All main Danish media today have heard about CAR and have journalists trained in it. But still only a few newspapers run CAR stories. In 1997 The Association of Computer-Assisted Reporting in Denmark (FCJ) was founded. Membership has increased steadily. In 1998 FCJ launched the first Danish CAR day. The association prepared an action program for the establishment of an institute for CAR. The program was to be based on three important principles: • It would be a nonprofit organization. • Active journalists would be in control. • Quick action in the beginning would secure a stable and democratic system for the future. In 1998 DICAR, the Danish Institute for Computer-Assisted Reporting under the Danish School of Journalism (DJH), was established. Based on experience from Sweden and the United States, the institute filled the need for a more formal and ambitious structure of CAR training and communication. The statutes and a cooperation agreement with DJH are nearly finished. DICAR has established the first courses and completed a Web page (www.dicar.dk). The institute has also organized conferences, and it is beginning to receive money. There is so much work to do that it needs to hire a part-time employee. Things are moving ahead. Two main things are helping the process. One is the experience of American journalists in this field. Other is technological development in general, which puts PCs on the desks of journalists and the Internet in a central position for research. CAR is not an obscure realm of nerds but is broadly recognized in mainstream journalism because of the developing computer knowledge in society. One of the main obstacles is access to data. Denmark has 5.2 million inhabitants. Since 1970 the population has had some freedom of information; slowly, a practice of fighting for greater freedom has taken root. Like the authorities of other Scandinavian countries, the Danish government has become largely computerized. But its attitude toward granting access to electronic information has been very restrictive. When information is electronic, there is generally no legal basis for gaining access to it. Although no right of access to electronic data exists, many authorities are prepared to pass on information, anyway. Moreover, it is possible to invoke other acts and rules in order to generate interpretations that can be used later to formulate precise requests for an alteration of the existing rules. There have been some very good CAR stories proving that the skills do work in journalism. But still there haven't been sensational CAR stories. No CAR stories until now have been nominated for the Danish version of the Pulitzer Prize, Cavling-prisen. In the coming year, an important task is making the association and the institute both run stable – on a minor scale much like IRE and NICAR in the United States. Getting the Danish authorities to place an overview of their data on the Internet will be a future task. Other considerable tasks are continuing the dialogue with authorities – both concerning specific experiments with mailing lists and access to electronic documents – and regarding the broader and more fundamental issue of formulating proposals for law reform.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-may-car-on-the-map","slug":"1999-may-car-on-the-map","url":"https://dwillis.github.io/uplink/article/1999-may-car-on-the-map.html","year":1999,"month":"May","monthNum":5,"headline":"CAR on the map","kicker":"CAR REPORT FROM THE NETHERLANDS","author_name":"Dick van Eijk","author_title":"Reporter","affiliation":"NRC Handelsblad","summary":"Dick van Eijk details how early database and mapping projects in the Netherlands, particularly concerning election results, helped legitimize computer-assisted reporting as a vital journalistic tool.","keywords":["Netherlands","mapping","elections"],"topics":["Investigative Methods","Technology","Government & Politics"],"technologies":["dBase","Lotus Approach","MapInfo"],"full_text":"In the past five years, computer-assisted reporting in the Netherlands has developed from an obscure activity of an individual reporter to a formally established practice at two national newspapers. Mapping played an important role in breaking ground for the new professional methods.\n\nTo American journalists, it may seem obvious that in their jobs they can use a computer for many tasks other than writing a story. Computer-assisted reporting is something that has existed for thirty years and can be learned at seminars, boot camps and conferences. The situation in the Netherlands, and in the rest of Europe, is entirely different. Only very recently has computer-assisted reporting been gaining momentum in a few countries, notably Sweden and the Netherlands.\n\nI did not know of the American tradition in 1993 when I used a database program (dBase III Plus for MS DOS) to enter the names and positions of more than 2,000 top officials of social-security institutions in the Netherlands. The Dutch parliament was doing a formal investigation on the execution of social-security laws, and we were curious about who actually governed these executing bodies.\n\nFrom the database analysis, it turned out that there was a relatively small network of unknown officials that met several times a week to determine policy. These officials actually ran the social-security programs. I interviewed most of them and attended, with their permission, a few of their meetings. I wrote two background stories about their hidden world. The stories did not mention that a computer database was used preparing them. Only a few of my colleagues knew.\n\nThe database program had proved useful, but at that time I did not realize that the technique had the potential of being an integral part of everyday journalistic practice. That changed during the next year.\n\nMapping wish list\n\nTo everyone's surprise, local parties gained a strong position in the March 1994 municipal elections. The polls had not predicted the rise of local parties because they concentrated on national trends and national parties. What we needed, but did not have, was a map showing the percentage of votes collected by local parties in each municipality. Speaking of maps, wouldn't it be nice to show other aspects of election results as well, such as where particular parties were strong or where they had gained or lost the most votes?\n\nThat should be possible with a computer, I thought, though I had no idea how to do it. Our national editor encouraged me to find out. I worked largely on evenings and weekends to be able to present such maps on the day after the parliamentary elections, which were only two months away. I made myself familiar with the world of Windows, with Lotus Approach and with the newly discovered mapping program, MapInfo. The application worked, and the day after the elections the newspaper ran a page containing eight maps showing the strength as well as gains and losses of the major parties. The maps were considered a big hit, an experience that would recur in later projects. The explanation, I think, is that the maps showed information not available in such a form before. Their value could be judged without knowing anything about the database analysis behind them.\n\nThe power of maps would prove to be an important argument for CAR in the Netherlands. The election maps were discussed on national television, and the professional magazine De Journalist wrote about them. We have repeated the map","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-june-deadly-practice-cosmetic-surgery","slug":"1999-june-deadly-practice-cosmetic-surgery","url":"https://dwillis.github.io/uplink/article/1999-june-deadly-practice-cosmetic-surgery.html","year":1999,"month":"June","monthNum":6,"headline":"DEADLY PRACTICE? Cosmetic Surgery","kicker":"","author_name":"Fred Schulte","author_title":"Senior writer","affiliation":"Sun-Sentinel, Ft. Lauderdale","summary":"An investigation into malpractice in cosmetic surgery offices in Florida, with a focus on poorly trained doctors performing lengthy operations and the resulting fatalities.","keywords":["cosmetic","surgery","malpractice"],"topics":["Health & Environment","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"A few hours before dawn on April 1, 1998, police found a disoriented woman lying on the sidewalk outside a Fort Lauderdale medical office. Bandaged, bleeding and still attached to a catheter and heart monitor, she was suffering from seizures brought on by an overdose of anesthetic given during liposuction. The woman had stayed overnight in a doctor's office to recover from surgery. She awoke in the middle of the night gasping for breath and panicked when nobody answered her screams for help. Thinking she would die if she didn't get help, she crawled out to the street and collapsed. She spent five days in a hospital intensive care unit recovering. I got a tip about the woman's ordeal from a source who said many poorly trained doctors did lengthy cosmetic operations — sometimes eight hours or more — in their offices and then let patients stay overnight to recuperate. This wasn't the first disaster, the source said. Identifying Victims The source was right. Courthouse records showed the surgeon whose patient had ended up on the sidewalk had nearly a dozen malpractice suits filed against him in just two years of practice in Fort Lauderdale. There were women who alleged they had been scarred horribly by botched breast-implant surgery and several people who couldn't close an eye or move their faces fully after a face lift. I decided to sample other plastic surgeons who advertised in our newspaper or in the Yellow Pages; many of them also had high numbers of malpractice suits. (Florida health officials consider three or more claims in a five-year period to be a red flag.) Next stop: Florida's Board of Medicine, which disciplines doctors and can be searched online. Some of the doctors in my sample also had been disciplined, in a few cases more than once, by the state medical board. But it was the early interviews with several plastic surgeons that persuaded me to pursue this as a major project. I had never heard so many doctors so critical of their colleagues. They spoke openly about the need for tighter state regulation of office surgery and criticized the state medical board for being too lenient in punishing incompetent surgeons. A couple of surgeons even said they knew of patients dying during what should have been routine cosmetic surgery. Unfortunately, none of these doctors could, or would, provide any details such as patient names or dates of death. Local medical examiners weren't much help either; they said they had no way to check with the name and date of death. Senior writer Jenni Bergal and I began visiting morgues in South Florida and rummaging through thousands of cases by hand. One by one, we found deaths and began contacting families. We also tried to track down police reports on patients who had died in their homes a day or two after surgery. Some of these provided additional details about the operations. By the time we published 'Cosmetic Surgery: The Hidden Dangers' in November 1998, we had confirmed 34 deaths in roughly the last 12 years. Thirteen of them occurred in 1997 and 1998. We have since documented five more deaths since 1997, bringing the total to 18. Fatal Pattern The number of deaths stunned state regulators, medical examiners and even the gossip-prone plastic surgeons of South Florida. A troubling pattern emerged in fatal cases. Many patients had more than one cosmetic procedure that took eight or 10 hours to perform in a medical office — longer than a liver transplant takes in a hospital. The victims tended to be in late middle age, were overweight or had chronic medical conditions that made them questionable candidates for such operations. About half of those who had died since 1997 were men older than 50 even though women account for the vast majority of cosmetic operations in the state. Finding Fault Still, we needed to keep the deaths in perspective. Nobody knew exactly how many cosmetic operations were performed in Florida every year, though it seemed from industry statistics to be at least 77,000. Lacking any means to compute a death rate, we searched for data to bolster the case that some cosmetic surgeons were the problem. We found the data in the Florida Department of Insurance closed-claims database, which listed all malpractice settlements paid by insurance companies. These claims include cases settled by insurance companies before any lawsuit is filed. The state insurance department charged us $150 for about 10 floppy disks, which we loaded into Access with the help of John Maines, our research center's database specialist. The data included the doctor's name, location, specialty, training, severity of the injury and the money paid to the patient. Sorting by 'plastic surgeons' gave us more than 1,100 cosmetic-surgery claims settled since 1990, including some deaths. Unfortunately, the database didn't explain the type of injury or how it occurred. We had to pay 15 cents per page for paper copies of a detailed narrative that had this information. None of the documents included the patient names, which are confidential under state law. Still, we documented scores of injuries, ranging from permanent facial scars to grossly disfigured breasts after botched implant surgery. Many doctors with multiple claims against them (several had a dozen or more) we already knew from our search of Department of Medicine disciplinary files. I also used the insurance-claims data to create graphics. For example, we published charts showing the severity of injuries, the average size of claims and the board-certification status of doctors with large numbers of malpractice claims. Board certification became important because plastic surgeons held that uncertified surgeons were injuring patients. In fact, the data showed that 80 percent of the injuries came at the hands of board-certified plastic surgeons. Since our series, state officials have tightened oversight of office surgery. Florida's Board of Medicine banned overnight stays in medical offices and limited office surgery to six hours. The Florida Legislature also passed a bill that requires office-based surgeons to report deaths and serious injuries to the state.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-june-legal-drug-hazards-inside-the-fda","slug":"1999-june-legal-drug-hazards-inside-the-fda","url":"https://dwillis.github.io/uplink/article/1999-june-legal-drug-hazards-inside-the-fda.html","year":1999,"month":"June","monthNum":6,"headline":"LEGAL DRUG HAZARDS Inside the FDA","kicker":"","author_name":"Stephen Fried","author_title":"","affiliation":"Philadelphia Magazine","summary":"The article is based on a personal investigation into the pharmaceutical industry, uncovering the inner workings of the FDA and exposing the dangers of legal drugs.","keywords":["FDA","pharmaceutical","drugs"],"topics":["Health & Environment","Investigative Methods","Government & Politics"],"technologies":["Internet","World Wide Web"],"full_text":"'Bitter Pills: Inside the Hazardous World of Legal Drugs' began as a personal quest for answers after my wife had a severe adverse reaction to one sample pill of a new, heavily-marketed antibiotic. It grew into a magazine article, which triggered an FDA investigation and won the 1993 National Magazine Award for Public Service Journalism. It eventually led to a five-year investigation of the entire pharmaceutical-industrial complex. The book focuses on issues of international drug safety while exposing the inner workings of the Food & Drug Administration, the pharmaceutical industry, physicians, nurses, pharmacists, consumer advocates and the medical press. Unlike earlier books on these subjects, 'Bitter Pills' weaves a personal narrative through the hardcore investigative reporting, putting a human face on the 'legal drug problem.' Even though most investigative journalism pieces focus on illegal drug use, the truth is that legal drugs kill many more people each year than illicit drugs. Adverse reactions to prescription and over-the-counter medications are the fourth-leading cause of death in America. 'Bitter Pills' is based on thousands of documents, many obtained from Freedom of Information requests from the FDA and other federal agencies. Others were rescued from discovery materials in lawsuits that were later settled and sealed. It is also based on an exhaustive survey of over 100 books and thousands of papers on various subjects pertaining to drug safety and the legal drug culture. But 'Bitter Pills' also incorporates hundreds of interviews with government officials, drug company executives, lawyers, physicians, academics, consumer advocates, pharmacists, nurses and patients worldwide - as well as coverage of dozens of conferences in the United States and Europe. I knew nothing about the drug business when I started this project. And I was faced with several problems: how to develop enough expertise on a highly technical field so I could report and write credibly, how to journalistically handle the personal circumstances that got me interested in the subject so I might capture my passion for drug safety without being biased against drug makers, and how to humanize very technical issues. The drug industry is rife with potential conflicts of interest, but so is the world of investigative journalism. I tried to face those conflicts head-on in my reporting, and I'd advise other investigative reporters to do the same. Because my use of Internet resources was so extensive, I published a guide to the sources in the book, which was updated and expanded for the recently published trade paperback. There is also a large Web site for the book at www.bitter-pills.com, which allows for pretty extensive research by professionals and lay people. During the research for the book, my assistants and I obtained several electronic databases from IMS, the company that does the 'Neilsen ratings' of drug use. We also obtained adverse reaction reporting databases for different drugs from the FDA, and deposition databases from most investigative journalism various lawsuits. Much of the material came for free or nominal charges (I'm good at begging, and the FDA will waive its FOI charges if you make the requests properly) but I did have to pay stenographers for some of the depositions. In fact, the FDA is so strapped for cash in its investigative branches that I was asked to donate my copy of a deposition which cost me $500 - to the regulators because they couldn't afford it and couldn't go ahead with their investigation without it.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-june-naming-the-unknown","slug":"1999-june-naming-the-unknown","url":"https://dwillis.github.io/uplink/article/1999-june-naming-the-unknown.html","year":1999,"month":"June","monthNum":6,"headline":"Naming the unknown","kicker":"USING MILITARY DATABASES, WEB SITES AND LISTSERVES","author_name":"Dawn Fallik","author_title":"NICAR Staff","affiliation":"NICAR","summary":"Sampley's investigation led to the exhumation and identification of the Vietnam-era Unknown Soldier using military databases, revealing him to be Lt. Michael Joseph Blassie.","keywords":["military","database","investigation"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Internet","Email"],"full_text":"Every day hundreds, if not thousands, of tourists gather at the Tomb of the Unknown Soldier in Washington D.C. to watch the changing of the guard and honor the memory of unknown men who died in battle. So when CBS reporter Vince Gonzales began investigating whether the tomb's Vietnam era Unknown Soldier had been identified by military officials before being buried, he treaded softly around both memories and rituals. One officer told us 'He's not a human being anymore, he's a symbol,' said Gonzales. The initial seven-month investigation involved multiple military databases as well as Internet archives and personal interviews. Four months after the first story aired, the Pentagon ordered the soldier's body exhumed from the tomb. When DNA tests proved the unknown soldier was indeed decorated Air Force Lt. Michael Joseph Blassie, his St. Louis family reclaimed their son's body and buried him with his own tombstone. Military listservs Gonzales said he began investigating the story after seeing an Internet posting by Ted Sampley, a reporter for U.S. Veteran Dispatch. Sampley's story described how documents pointed to Blassie and suggested that DNA testing could be used to confirm the suspicion. Gonzales went further, using interviews, documents and databases as confirmation until the government was forced to unearth the tomb's soldier. 'At first I thought it was another one of those Internet conspiracies,' he said. But as he started looking into the details of the Unknown Soldier selection process and talking to the individuals involved, Gonzales was convinced Blassie's body was beneath the tomb. Without mentioning the words 'tomb' or 'unknown,' Gonzales quietly began requesting documents regarding Blassie, a 24-year-old pilot killed on May 11, 1972 during a mission. He requested all documents, including forensic reports, squadron logs and e-mail records that mentioned Blassie's name. During the investigation, Gonzales sent more than 20 FOIA requests to every branch of the Department of Defense as well as the U.S. Department of State and many others. He received documents from almost all of the requests. Searching National Archives In addition, the reporter turned to the National Archives in Washington, D.C. for electronic database help. 'We asked the Center for Electronic Records for the National Archives to run several searches for us on databases we didn't have. They were really intrigued by the project and so they helped us out a lot,' Gonzales said. The center does charge for searches, and the station paid $30 for documentation and eventually purchased the entire Combat Area Casualty File. The two main databases Gonzales used were the Combat Air Summary and Detail Files (OPREA) and the Combat Area Casualty database, both available at the National Archives. OPREA includes information about all combat and non-combat air missions during the Vietnam War, including information about Cambodia and Laos. The casualty database included information on where and how military personnel died, their home of record, date of birth and other death details. Gonzales also used The Situation Report Army File, a database of ground combat operations in Southeast Asia, looking for army officers who were supposed to be on the ground and recovered the remains. 'Not all of it was vastly useful; mostly we used it for some tidbits,' he said. Data confirmed tips Most of the data work was used to confirm information received from sources: who had gone down where on what day and whose remains were recovered. All records related to the unknown selection process were supposed to be destroyed after the soldier was buried, but Gonzales found government officials who had refused to comply. 'Some people would hide the information in other files. One guy hid stuff in his garage. I now have a 5-foot tall filing cabinet filled with documents that were supposed to be destroyed,' Gonzales said. Although the Pentagon first told the reporters they would not receive any documents regarding the selection process, Gonzales said they were successful because they placed FOIAs with the original agencies and stayed away from using the Tomb of the Unknown Soldiers reference. 'It wasn't hard to get the databases, but it was hard to get clean data,' he said, adding that the archive service tells users about potential obstacles. Data was put in wrong fields and names were spelled wrong. Gonzales used the Internet to find researchers and veterans who knew about Blassie's death and the following cover-up. He suggests that reporters doing military stories use variations on the names they want when searching government documents. He spent three months trying to find retired Col. Bill Parnell, whose troops recovered Blassie's skeletal remains and his identification card. The search was stalled because documents had the name spelled Darnell. Gonzales, teaming up with fellow reporter Eric Engberg and producer Dick Meyer, held off from contacting the Blassie family until the initial investigation was ending. 'We didn't want to drag their hopes up, but we needed to get them to sign affidavits allowing us to get military documents that usually only family members could obtain,' Gonzales said. On June 29, 1998, the United States Defense Department confirmed what the Blassie family and the CBS reporters had suspected for almost a year; DNA tests confirmed the remains buried at the tomb for 14 years were those of Michael Joseph Blassie. He is now buried at a veterans cemetery near the family's home in Florissant, Mo. 'It's what we had been working toward all along and the family were such great people you want to work even harder for them,' Gonzales said. The trio continues to investigate which military officials ordered Blassie's remains to be declared unidentifiable.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-june-counting-crime-right","slug":"1999-june-counting-crime-right","url":"https://dwillis.github.io/uplink/article/1999-june-counting-crime-right.html","year":1999,"month":"June","monthNum":6,"headline":"Counting crime right","kicker":"FBI REPORT PROVES PAPER'S FINDINGS","author_name":"Mark Fazlollah, Craig R. McCoy and Michael Matza","author_title":"","affiliation":"The Philadelphia Inquirer","summary":"The Philadelphia Inquirer's investigation into falsified crime statistics revealed that the city had systematically undercounted crimes for decades, with police finally admitting to nearly 3,000 uncounted property crimes in 1997 after the newspaper obtained and analyzed crime database files.","keywords":["crime","Philadelphia","statistics"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Access","Excel"],"full_text":"When the FBI released its 1998 crime report for major U.S. cities last month, the Philadelphia numbers confirmed the enormity of the city's past statistical falsification- the focus of a series of articles by The Inquirer over the previous year. Major crime dropped 7 percent in the rest of America, but it was up 11 percent in Philadelphia. In raw numbers, Philadelphia reported 2,603 more violent crimes and 9,211 more property crimes than in 1997.\n\nThe mayor and the police commissioner explained it was not a crime wave, just the result of better police reporting after The Inquirer had hammered away at the issue for the past year. It was more than just an issue of dry statistics. Inquirer reporters had interviewed dozens of victims who were shot; stabbed, seriously beaten, raped or robbed, but the police had declared in nearly every case that no crime had occurred.\n\nSome victims were denied state-funded compensation because the police said no crimes had been committed. And suspects stayed on the streets.\n\nFor decades, Philadelphia police engaged in a massive public relations conspiracy to make the city appear safer than it really was. The Philadelphia District Attorney recently characterized the falsification of crime stats as the city's \"dirty little secret.\"\n\nExposing the secret\n\nThe secret was exposed only after the city agreed to release zip drives with crime-by-crime data to The Inquirer and its sister paper, The Philadelphia Daily News. The database files provided a set of basic facts: dates, times, locations, crime-code classification, clearances information and more on 700,000 major and minor crimes from 1991 through 1997. Eventually, Tom Torok of the paper's computer-assisted reporting and analysis unit, put all that information on a fast Web site: http://home.phillynews.com/crime (See December 1998 Uplink).\n\nWhile the police department had resisted releasing this data for several years, Philadelphia Mayor Edward Rendell finally ordered it made public. Inquirer reporters sorted the data using the Access database and Excel spreadsheet programs, allowing them to produce crime analysis for each of the city's 23 precincts.\n\nSeveral precinct captains were able to help us find the flaws in the numbers for their individual districts. The captains had kept their own crime statistics for their precincts, and their numbers conflicted with the official figures. Pulling together the flaws in the precinct data, we were able to report that about 3,000 property crimes had apparently been deleted from the officials' count. The giveaway was a suspicious drop in car ripoffs that appeared in only one half of the year.\n\nThe police department, after initially contending that the precincts' captains were wrong, would eventually concede that 2,559 car thefts and 560 larcenies were never counted in 1997.\n\nIt was an enormous admission. The undercount in stolen cars alone exceeded the entire number of heisted vehicles reported for many cities in 1997. It was more, for example, than all the stolen cars reported for the year in Raleigh and Greensboro, combined population: 454,000). It exceeded the combined 1997 total for Norfolk and Virginia Beach, VA (total population: 650,000). The department's admission of under-reporting gave us credibility with town watch groups and victims' advocates - and even some police officials. They would all be important in the next phase of the project.\n\nHumanizing numbers\n\nFor our next round of stories, we wanted to put a human face on the numbers. We had to find victims whose crimes had been dumped. Among the steps we took was to compare court records for hundreds of cases with the crime-by-crime police data. We found that far more suspects were criminally charged with aggravated assault than police said they had ever arrested for the crime. The victims' advocates, rape counselors and town-watch groups gave us additional information about crimes they believed had never been properly reported by police.\n\nWe interviewed civilian crime-watch leaders who had kept their own personal records of crime in their neighborhoods. Politicians alerted us to victims who had complained of police inaction on crimes.\n\nSome police also helped us. They understood that dishonesty in crime reporting was hurting them. Because Philadelphia was supposed to be one of the safest urban areas in America, the bogus crime stats at times had been used to justify cuts in police funding.\n\nHelpful police showed us how to work the bureaucracy to obtain the original crime reports for incidents they knew had never made it to the official count. In getting this aid, we were able to exploit a kind of glasnost brought on by the arrival of a new reformist police commissioner, John F. Timoney, who had made accurate figures a cornerstone of his crime-fighting strategy.\n\nTimoney gyrated between two extremes as our reporting unfolded. On the one hand, he was blunt in his own lack of trust in Philadelphia crime figures. On the other, he often fought our reporting efforts when we pushed to illustrate the overall trends with concrete pictures of victims and concrete examples of police negligence.\n\nBack to documents\n\nIf an incident wasn't shown in the police data, we ordered the street cop's initial incident report. The city charges $15 for each copy of an incident report. We ordered hundreds.\n\nWhen we located victims, they often told us tragic stories of uniformed arrogance.\n\nRapes, for example, sometimes had been coded as \"investigate person\" if investigators questioned the reliability of the victims. Incidents of shooting and stabbing were coded as \"hospital case.\"\n\nEven a murder case was affected by the downgrading. Melanie Alge, an 86-year-old widow, was critically beaten by a robber in May 1998. Police initially characterized the incident as a property crime, purse snatching. Two weeks after the beating Alge died. \"Just because my sister was old is no reason to dismiss the incident as a mere purse snatch,\" Alge's sister would later write, demanding an investigation of police bungling.\n\nNo suspect has ever been arrested in the murder. On May 17, the day after the FBI released its report on crime in the largest American cities for 1998, the Inquirer published another crime story.\n\nThis time there was no zip drive. The police department had promised to give us new crime-by-crime data but did not complete the data before the story was published.\n\nStill, we were able to show some apparent flaws in Philadelphia's newest numbers by simply comparing them to the reports of other major cities.\n\nIn the new numbers, Philadelphia police reported 8,701 aggravated assaults - a 40 percent increase over 1997. That still seemed low.\n\nEven with the dramatic 40 percent jump, Philadelphia's figures were at serious variance with the number of aggravated assaults reported by other large U.S. cities.\n\nPhiladelphia was 22 percent below the average per-capita assault rate for the major U.S. cities in 1998.\n\nNine of the 10 major cities reported more aggravated assaults than robberies. Los Angeles, for one, reported twice as many assaults as holdups.\n\nThat makes sense, if you think about it. Surely, it is more likely to get beaten in a bar fight or other scuffle than to be held up.\n\nNot in Philadelphia. It was the sole exception among the top 10 U.S. cities. The numbers said 11,435 people were robbed last year but only 8,701 assaulted.\n\nUniversity of Maryland criminologist Lawrence Sherman, named by the police commissioner to lead an audit of the department's reports, had previously said the ratio of aggravated assaults and robberies was important.\n\nSherman said a city's statistics probably were unreliable if the relationship between the two crimes was far out of step with the national norms.\n\nAnd some officials never seemed to understand the impact of the chicanery. \"This isn't about how fast police respond to 911 calls, or how they deal with civilians, or how they fight crime. It's a matter of numbers,\" the mayor's spokesman, Kevin Feeley, told reporters when the scandal was unfolding last year.\n\nMark Fazlollah can be reached by e-mail at fazlo.pia@phillynews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-june-digging-up-submarines","slug":"1999-june-digging-up-submarines","url":"https://dwillis.github.io/uplink/article/1999-june-digging-up-submarines.html","year":1999,"month":"June","monthNum":6,"headline":"Digging up submarines","kicker":"CAR AND ESPIONAGE","author_name":"Sherry Sontag and Christopher Drew","author_title":"Authors","affiliation":"IRE","summary":"Journalists describe how they used computer databases and laptops to track sources and conduct interviews for their book on Cold War submarine espionage, demonstrating that traditional investigative reporting can be enhanced by modern technology even when analyzing classified information.","keywords":["database","sources","espionage"],"topics":["Investigative Methods","Technology"],"technologies":["dBase","FoxPro","Access","Profile","PC","modem/BBS","Internet"],"full_text":"Computer-assisted reporting: the phrase brings to mind complicated programs written to run analyses of miles of numerical data to create a litany of statistics to prove a trend, track the market, to do myriad tasks that have little to do with the old-fashioned investigative method of running around the country asking questions lots and lots of questions.\n\nCertainly when we began the six-year odyssey of researching and writing \"Blind Man's Bluff,\" our tale of submarine espionage during the cold war, we knew that there were no statistics to analyze, at least none we could get our hands on. Just about everything we wanted to write about was classified, and the Navy had made it clear that it was all going to remain classified. If this book was going to be written - and we were determined that it was - we were going to have to dig out 40 years of undersea espionage the old-fashioned way: one source at a time. Our analysis would be similarly low-tech: if a submarine came back to port damaged at the same time another submarine was reported missing, maybe the two incidents were related.\n\nStill, here are, writing an IRE article on computer-assisted reporting, and for good reason. Even for an old fashioned information dig, we quickly discovered that all the modern trappings of modems, databases, and laptops were going to prove crucial. Had the infant Internet been worth talking about when we began, that might have been a big help as well. Instead, we ultimately used the net at the end of our project, to announce rather than to dig for findings. But on to that in a bit.\n\nTracking interviews\n\nFirst our start. The very first thing we did was build a database - one of statistics but names, lots of names, all of them potential sources. This was a comprehensive list of just about every man who served on any submarine we had any reason to believe may have been involved in some incident that just might, maybe, if it panned out, end up in our book. There are all sorts of great database programs out there; D-Base, Foxpro and Access are among them. We didn't know about any of them at the time, and so we used an old DOS program called Profile. It worked great. It did just what we needed it to do: let us list men, their phone numbers, addresses, and just what subs they served on in which years.\n\nBy the way, when we say list, we mean type. Each profile was typed in by hand until we had thousands of names. We amassed those before we went out on a single interview. Then, as we traveled around the country - and we traveled a lot - we had the means of finding men by location, boat, by year, and we hoped these men just might be willing to tell us tales that they had never told their wives, children, parents or best friends.\n\nIt's worth noting that an awful lot of our interviews were typed directly into a laptop computer. Ultimately, the laptop proved to be no more distracting to sources than a pad and pen. In fact, often a bit of electronic show-and-tell was the perfect way to break the ice. It was also the perfect way to keep interviews sorted and to prevent scrawled notes from becoming more illegible with time. Later, when we were looking for that particularly great quote, the one we found two or three years before we were ready to write, a simple computer search dug up the tidbit every time. While the two of us ran around doing interviews, Chris' wife Annette Lawrence Drew scoured books and old newspapers, creating a time line of events on dry land. When were the summit meetings held, the treaties signed, when was talk of detente taking place on land while spying heated up at sea, how often did the rhetoric match the reality? All of it now rested on keyboard access and helped us create context.\n\nAs we wrote, we used a modem and a direct computer-to-computer connection to bounce ideas and drafts back and forth. And we used the computer to encrypt everything we sent. It wasn't that we really thought our phone lines were tapped. We just knew that it was possible. So we used Norton encryption - PGP is a better choice these days - to play what we called \"I-Spy,\" maintaining protection against the unlikely.\n\nNet spreads word\n\nBy the time our book was researched and written, the Internet had grown up enough to become our first marketing tool. We found new Web pages written by and for submariners. We found their bulletin boards. We created long e-mail distribution lists. We spread the word, a new book with tales long untold was coming out.\n\nWe know the effort was effective. We became one of the top sellers at Amazon.com before we were ever published. At least one man who received our e-mail on the fifth forward between submariners and spies turned around and organized a book signing that drew more than 200 people and where we signed nearly 500 books. This was the first book for submariners and their relatives that showed that the men did more than guard the Golden Gate bridge. Soon we found heated discussions about \"Blind Man's Bluff\" on bulletin boards dedicated to the CIA and to Tom Clancy.\n\nIt was also through the Internet that submariners came to find us, to write their concern about secrets that no longer were, to thank us for ending the silence, to offer their congratulations for a book by a couple of unknowns that surfed the best sellers list for longer than six months.\n\nSherry Sontag can be reached by e-mail at sherry@panix.ocm\n\nChristopher Drew can be reached by e-mail at drew6@.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-june-look-before-touching","slug":"1999-june-look-before-touching","url":"https://dwillis.github.io/uplink/article/1999-june-look-before-touching.html","year":1999,"month":"June","monthNum":6,"headline":"Look Before Touching","kicker":"STATS AND SUCH","author_name":"Neill A. Borowski","author_title":"","affiliation":"The Philadelphia Inquirer","summary":"Before diving into data analysis, journalists should visualize their data through histograms and other visual methods to understand its distribution and identify patterns or anomalies, much like how surgeons examine x-rays before operating.","keywords":["visualization","histogram","analysis"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["SPSS","SAS","CD-ROM"],"full_text":"You've arm-wrestled with the government agency for weeks. After haggling, they agree to make the data available to you. Finally, the shiny CD-ROM arrives in the mail.\n\nQuickly, you pour the file into a database. Like a surgeon, you wiggle your fingers, crack your knuckles and prepare to plunge your hands deep into the body of data.\n\nWait a moment. Drop the scalpel. Remove the gloves. No surgeon worth his license would operate without x-rays. The surgeon needs to look inside the patient and see before cutting.\n\nAnd the data analyst also should visualize the data before poking around. The best applied statisticians are the ones who, for example, run clinical trials for pharmaceutical companies, advise analysts to study pictures of their data before they begin any calculations.\n\nToo often, we start analyzing the data before sitting back and taking in the whole view. Taking in the view doesn't have to be time-consuming. And it can be even more revealing than any measurement you make.\n\nThe first step in visualization should be the histogram. The histogram is a bar chart, but the values are the frequencies of the key variable rather than the variable itself. Also, unlike the conventional bar chart, the histogram has no spaces between the bars. Here is a histogram showing full-time teachers by their salaries in the Philadelphia suburbs:\n\nThe data in the teacher-salary histogram are somewhat mound-shaped, but don't exactly follow the normal curve superimposed on the histogram. Many teachers are paid around $52,250. However, the bars to either side of this middle bar are shorter than they would be if the data approximated the normal curve. There's a sharp fall-off to the left of the $28,250 bar, but not as sharp a fall-off at the corresponding bar on the right side of the histogram. The $84,250 bar sticks out and might warrant some attention.\n\nOverall, the teacher salary data doesn't look too odd. You could design your histogram to exaggerate the oddities by adding more bars. And, conversely, by taking away bars, you mute the oddities. Generally, it makes little sense to have more than 15 bars in your histogram.\n\nThe data don't appear to skew heavily to the right or left. A right skew, for example, would suggest there is a disproportionate number of teachers earning high salaries. In fact, when we run the usual statistics on the data, we find the mean is $52,120 and the median is $51,610 - not a very large difference. If the mean were much higher than the median, the data would be right-skewed.\n\nNow take a look at another variable in the teacher salary database: total years of experience. These data aren't mound-shaped. They look almost bimodal (having two equally sized peaks). The two groups would be teachers with little experience (left mound) and teachers with a great deal of experience (right mound).\n\nUsually the near-bimodal distribution would indicate the need for more research and possibly a great story. Why would experience level plunge around year 14 and rise sharply in either direction?\n\nThe answer can be found in public-school enrollment totals. The data in this example are from 1997. Public school enrollment since the early 1960s bottomed out in 1984, according to U.S. Education Department data. A teacher starting work in 1984 would have had about 13 years of experience by 1997. That corresponds with the valley in the histogram. Enrollment peaked in 1971. A teacher starting in 1971 would have had 26 years of experience in 1997 - again agreeing with our histogram. Finally, there has been a steady increase in enrollments since 1984 (the peak is forecasted for 2006). Again, the growth in student numbers in the late 1980s and early 1990s is reflected in the left side of the experience histogram.\n\nBuilding the histogram\n\nIf you have SPSS or SAS, building a histogram can be relatively painless. In SPSS (v. 9.0), from the pull-down menu: \"Graphs\" \"Histogram...\" Set your variable (teacher salaries or experience, for example) Click on \"Display normal curve\" Click OK and SPSS will produce the histogram. To modify your histogram, double-click on it in the SPSS output window. This will open the chart in the SPSS chart editor. The histogram can be edited for color, line width, etc. You also can tell SPSS to increase or reduce the number of bars. To do this, click on the value labels under the X-axis (horizontal). This opens up the Interval Axis box. The default in this box is \"Automatic\" intervals. Click on \"Custom\" to change this and to make the \"Define\" button available. Click on this button and the \"Define Custom Interval\" box will open. In \"Definition,\" SPSS asks for the \"# of intervals.\" Try 10 to begin with. If too wide, try 6 or 7. If too narrow, try 11 or 12.\n\nIf your CAR shop doesn't have SPSS, you can build a histogram in Microsoft Excel. But it's much more cumbersome than statistical programs.\n\nIn Microsoft Excel (97), from the pull-down menu:\n\nYour data is on one worksheet (record limit is 65,536, based on the number of rows in the spreadsheet).\n\nSort the data and, after looking at it, determine how many bars you want in the histogram. Excel can automatically set the number of bars, but the program sometimes uses too many bars. Try to keep the number under 15. In the data, look at the smallest and largest records. Without sorting, you could use =min(range) and =max(range). Subtract the smallest from the largest and divide by one less than the number of bars desired. In the teacher salary example, the smallest is 18,500 and the largest is 86,828. The difference is 68,328. If we want 10 bars, each bar would be 7,592 wide.\n\nIf we start the first bin at 18,500, Excel will set this as the midpoint of the bar. We don't want this. We want the lowest number (18,500) to be at the extreme left of the histogram. The midpoint of the first bar, then, would be 18,500 + (7,592/2) or 22,296. The second bin would be 22,296 + 7,592 or 29,888 and so on. Go to another part of the worksheet. Fill in the midpoints of the bins for Excel.\n\nFrom the pull-down menu, click on \"Tools\" and then \"Data Analysis.\" This opens the Data Analysis box. Scroll down to \"Histogram\" and click OK.\n\nSet the \"Input Range,\" which is the spreadsheet range of the variable of interest, such as teacher salaries. Set the \"Bin Range,\" which is the spreadsheet range of the bins. Then select the \"Labels\" option and click OK. (Do not click the \"Labels\" box unless you've included column headers in the spreadsheet ranges you've highlighted).\n\nExcel will insert another worksheet, which will include the bin midpoints in the A column and the frequency in the B column. Use the frequency count to order a column chart from Excel. The A column (labeled \"Bin\") will be the X-axis labels.\n\nAfter building the column chart, notice that the bars are not touching. If you're simply trying to visualize the data, you don't have to make it a true histogram with the bars touching. If you're publishing the histogram, you'll want the bars to touch. To do so, click on the bars in the chart. Go to the Excel pull-down menu and click \"Format\" and \"Selected Data Series.\" This opens the \"Format Data Series\" box. Click on the \"Options\" tab. Set the \"Gap Width\" to \"0.\" The result will be no space between the bars.\n\nOther chart options will help to visualize the data. A good choice when comparing one variable between two groups is the box plot or \"box and whisker\" chart.\n\nIn the example, the variable on the X axis is whether the teacher is male or female. The Y axis is the salary level. The whiskers of the box plot show the end, or extreme, values.\n\nOther chart options also will help to visualize the data. A good choice when comparing one variable between two groups is the box plot or \"box and whisker\" chart.\n\nIn this case, they're nearly equal for each sex. The shaded box is the interquartile range - the range between the 25th and 75th percentiles. The line through the box is the median.\n\nAt a glance, the median for males clearly is higher than for females ($55,944 versus $49,616). In fact, the male median is just under the top of the female interquartile range, suggesting relatively sharp pay differences. The difference between the sexes might be explained in part by the experience levels (median 22 years for males versus 13 years for females). In SPSS, go to \"Graphs\" and \"Boxplot\" from the pull-down menu. The process is much more complicated in MS Excel. There is an explanation of how to do a box plot at the Microsoft site (http://support.microsoft.com/support/kbarticles/q155/1/30.asp) or search for \"whisker\" in the Microsoft Personal Support Center. For visualizing relationships between two variables before running a regression analysis, try the scatterplot. If we wanted to see a relationship between years of experience and salary, a scatterplot would look like this:\n\nThe data points suggest a linear relationship. The dark line is the fitted line (the line derived from the regression analysis), which you can order up as an option on the chart. In SPSS (9.0) you also can point to a datapoint on the scatterplot and click on it. It will embed the record number in the plot (that's what the 11,505 is above). The SPSS Data Editor also will automatically go to each record in question as you point.\n\nThe scatterplot in MS Excel can be ordered from the Chart Wizard as an XY (Scatter) chart. When the chart is complete, from the pulldown menu click on \"Chart\" and \"Add Trendline\" and a line will be fitted to the data.\n\nNeill Borowski can be reached by e-mail at nborowski@phillynews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-june-fips-fixes","slug":"1999-june-fips-fixes","url":"https://dwillis.github.io/uplink/article/1999-june-fips-fixes.html","year":1999,"month":"June","monthNum":6,"headline":"FIPS fixes","kicker":"TECH TIP","author_name":"Jeff South","author_title":"","affiliation":"Virginia Commonwealth University","summary":"FIPS codes are standardized federal identification numbers for geographic areas, but their inconsistent storage formats across government databases—sometimes numeric, sometimes character, with or without leading zeroes—creates significant challenges when trying to merge datasets in data analysis tools.","keywords":["FIPS","databases","standardization"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Access","FoxPro"],"full_text":"To you, it's Texas. But to the Census Bureau, it's \"48.\" You say New York's capital is Albany. Data heads call it \"3601000.\" The Seattle Seahawks' stadium is in King County, Wash. Geo-geeks say the Kingdome is in \"53033.\" Welcome to the digit-laden world FIPS codes - the numbers the federal government uses for every state, county, city and metropolitan area. These codes are not ony shorter, but they are also more precise: After all, is that South Texas county spelled La Salle, LaSalle or Lasalle? But FIPS codes can pose problems, too. Is the code for Alaska \"02\" or just \"2\"? Is California \"06\" or \"6\" - and within that state, is Orange County \"059\" or simply \"59\"? Moreover, what's the data type for these codes - numeric or character?\n\nThose are important questions if you're using Access, FoxPro and other tools for doing data analysis.\n\nIf one table has its FIPS codes as numeric fields and another table's codes are character strings, you can't automatically join the tables. FIPS codes - the acronym stands for Federal Information Processing Standards- are used in datasets on a variety of subjects, from population and home loans to pollution and traffic accidents.\n\nBut they're stored in an annoying array of ways by the Census Bureau, the IRS and other government agencies.\n\nIf one table has its FIPS codes as numeric fields and another table's codes are character strings, you can't automatically join the tables.\n\nSometimes FIPS codes are numbers; other times they're text. Even when the codes are character strings, they can have different formats, such as \"007\" (leading zeroes) or \"7\" (a number followed by spaces or blanks) or \"7\" (spaces before the number). Here are some tips on dealing with variations in FIPS code fields. One tip is a workaround: how to join on the fly tables that have their codes stored in different ways. The other tip is a fix: how to add permanent fields that present FIPS codes the right way.\n\nWhat is the right way?\n\nFIPS codes should be character fields. State-code fields should be two characters wide; county-code fields, three; metropolitan areas, four; cities, county divisions and other populated places, five.\n\nThe codes should have leading zeroes, so that Alabama is \"01,\" Alaska \"02,\" Arizona \"04\"\n\nThe codes are stored in an annoying array of ways by the Census Bureau, the IRS and other government agencies.\n\n(Curiously, there is no \"03\" among the state codes, and the codes for counties are all odd numbers.)\n\nBut what do you do when a government agency provides a table with FIPS codes stored differently? I encountered this recently when I got 1996-97 county-to-county migration data from the IRS: All the FIPS codes were numeric (California was \"6,\" and Los Angeles County was \"37\"). In years past, the codes were all character fields (California was \"06,\" and Los Angeles County was \"037\").\n\nTo join the new and old data - that is, to make the records match - I had to persuade FoxPro to consider the number \"6\" the same as the character string \"06\" and the number \"37\" the same as the character string \"037.\"\n\nThe workaround: the VAL() function. VAL() converts a character string into a numeric value. In the migration case, I wanted to match my FIPS-as-text fields (st_txt, cnty_txt) in my old data with my FIPS-as-numeric fields (st_num, cnty_num) in my new data. So I used this as my join condition: where new.st_num = val(old.st_txt) and new.cnty_num = val(old.cnty_txt) That says: Match the numeric FIPS code with the numeric value of the character FIPS codes. The VAL() function works in Access as well as in FoxPro. VAL() also can be used to join two text fields that have different FIPS code formats. A Virginia state agency once gave me data in which the county FIPS codes were character strings but without leading zeroes.\n\nOften you'll want to permanently change how FIPS codes are stored in your data. For instance, it will be faster and easier to join tables if your codes are consistent. Moreover, your codes must use the same format if you want to append records from one table into another table.\n\nI wanted to join this with Census Bureau data in which the county codes did have leading zeroes. So my join condition was: where val(table1.fips) = val(table2.fips) That way, \"1\" in the state's FIPS code field matched \"001\" in the Census Bureau's field, \"3\" matched \"003,\" and so on.\n\nThe fix: STR, LTRIM and PADL functions.\n\nOften I want to permanently change how FIPS codes are stored in your data. For instance, it will be faster and easier to join tables if your codes are consistent. Moreover, your codes must use the same format if you want to append records from one table into another table.\n\nLet's say you have a table in which the state FIPS code (st_num) and county FIPS code (cnty_num) fields are both numeric. In FoxPro's Table Designer, insert two character fields: st_fips, with a width of 2; and cnty_fips, with a width of 3. Then run this command:\n\nreplace all st_fips with; padl(ltrim(str(st_num)),2,'0') This takes a numeric \"1\" in the st_num field and turns it into the character string \"01\" in the st_fips field. Here's how the command works, starting from the middle of the second line:\n\nstr(st_num) means convert the number into a character string.\nltrim(str(st_num)) means trim all the blanks from the left side of this character string.\npadl(ltrim(str(st_num)),2,'0') means pad the string with zeroes so it fills up a width of 2 characters.\n\nYou can change the numeric cnty_num field into cnty_fips, a character field with leading zeroes, the same way: replace all cnty_fips with; padl(ltrim(str(cnty_num)),3,'0') Use a similar command to change a text field without leading zeroes (cnty_no0) into one with leading zeroes (cnty_fips): replace all cnty_fips with; padl(alltrim(cnty_no0),3,'0') This says: Take the cnty_no0 field; trim all the blanks at the beginning and end; then pad it with zeroes to a width of three characters.\n\nConversions in Access\n\nAccess does not support PADL, but it has an even easier function for converting the data. If your county FIPS codes are in a numeric field (cnty_num) and you want them in a text field with leading zeroes (cnty_fips), update the cnty_fips field to: Format(cnty_num, \"000\")\n\nThe same function will convert a text field without leading zeroes into one with leading zeroes.\n\nJeff South can be reached by e-mail at jcsouth@vcu.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-june-geo-spatial-sites","slug":"1999-june-geo-spatial-sites","url":"https://dwillis.github.io/uplink/article/1999-june-geo-spatial-sites.html","year":1999,"month":"June","monthNum":6,"headline":"Geo-spatial sites","kicker":"ON THE INTERNET","author_name":"Christopher Simpson","author_title":"","affiliation":"American University","summary":"A guide to online resources for accessing satellite imagery and geospatial data from government agencies and international sources, presented at the 1999 NICAR conference.","keywords":["satellite","imagery","geospatial"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["World Wide Web","Internet","FTP"],"full_text":"Simpson presented this handout at the 1999 NICAR National Conference. It suggests great web sites to get you started with satellite imagery and geospatial data.\n\nhttp://eduww.cr.usgs.gov/webglis\n\nThe U.S. Geological Survey's EROS DATA Center is the government's primary unclassified intelligence imagery from the 1960s and 1970s. GLIS is the primary index to the collection and is relatively easy to use considering the vast scope of the material. Declassification of certain intelligence satellite imagery from the later 1970s and 1980s is presently underway and will most likely be available via USGS site.\n\nhttp://gcmd.gff.nasa.gov and http://www-eosdis.ornl.gov/daacpages/otherdaacs.html\n\nThese NASA sites are gateways to a giant, evolving effort, known as EOSDIS, to link indexes to imagery around the world. The effort is by no means complete and sometimes operates awkwardly, but the sites permit access to a wide range of unclassified U.S. government imagery and scientific data gathered from outer space.\n\nhttp://waa.saanoaa.gov\n\nThe National Oceanographic and Atmospheric Administration's index of its weather imagery provides high-quality and, often, free imagery of continent-sized parts of earth.\n\nIn addition to overhead images of hurricanes and typhoons, it provides the wide-angle imagery necessary to see the scope of forest fires throughout the northern Amazon, environmental crises in northern Africa and similar very-large scale phenomena.\n\nhttp://terraserver.microsoft.com\n\nThis very large data collection combines USGS aerial photography of the continental United States with very high-resolution imagery gathered throughout the world by Russian Sovinformsputnik satellites. The USGS data can be downloaded for free; the high-quality Russian data is offered at a modest cost but is spotty in its coverage.\n\nhttp://www.crisp.nus.edu.sg/crisp.html\n\nThis Singapore-based ground station provides an index of imagery collected over Southeast and much of South Asia by satellites from the United States, Canada, Europe, SPOT, Japan and other sources.\n\nhttp://isis.dlr.delservices/ISIS/ISIS-query.html\n\nThis index, sponsored by the German Aerospace Center, provides convenient access to imagery of Europe from several sources, as well as links to other indexes worldwide.\n\nhttp://www.eurimage.itleinet/einet_home.shtml\n\nThe European Space Agency operates this user-friendly index imagery collected over Europe, much of the Middle East and the western territories of the former USSR.\n\nhttp://www.fas.orglirplindex.html\n\nOperated by the Federation of American Scientists, this index specializes in declassified \"spy satellites\" imagery and support for treaty-verification projects.\n\nhttp://www.spaceimaging.comlhomelprod-ucts/carterralworldviewlindex and http://www.spaceimaging.comlhomelproducts/carterralcarterra3dlindex.html\n\nThis is Space Imaging Corporation's Carterra imagery index, which provides access to commercial and unclassified government imagery gathered over the United States. The second URL provides the first access to \"assembly line\" (as distinct from custom-built) fly-through, moving satellite images that simulate the view that low-flying aircraft might see over, say, China's nuclear test range at Lop Nor.\n\nhttp://www.spot.com and http://wwww.spot.com/spot/home/sirius/welcome.htm\n\nSPOT Image's commercial DALI index presently costs $100 annually to subscribe. It provides an unusually well-organized and complete index from medium-resolution imagery (about 30 meters per pixel) for almost any point on earth.\n\nSPOT's imagery is often more complete and more current than its competitors, though not as useful for close-up shots of the ground. The new Sirius index promises to maintain DALI's functionality, offer new services and be available for free. The Sirius product should be available this spring.\n\nChris Simpson can be reached by e-mail at simpson@american.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-june-usable-data","slug":"1999-june-usable-data","url":"https://dwillis.github.io/uplink/article/1999-june-usable-data.html","year":1999,"month":"June","monthNum":6,"headline":"Usable Data","kicker":"HANDOUT OF THE MONTH","author_name":"Hal Straus","author_title":"database editor","affiliation":"The Washington Post","summary":"This article explains three key strategies for building effective web databases for news organizations: using appropriate application server software, breaking traditional relational database design rules for speed, and keeping interfaces simple to accommodate users' short attention spans.","keywords":["databases","software","performance"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["Perl","VBA/Active Server Pages","Oracle Application Server","Netscape Application Server","Access","ColdFusion","ODBC"],"full_text":"Usable Data\n\nBy Hal Straus\nThe Washington Post\n\nStraus provided this handout at the National Computer-Assisted Reporting Conference held in Boston this past March.\n\nThree Ways to Build Compelling Web Databases\n\nWeb databases aren't CAR databases. And the fact that you need different software to deliver them is only one of the reasons. Here are three un-CAR things you can do to keep your work from landing on the Web with a loud thud.\n\n1. Use the Right Software\n\nPerl and VBA/Active Server pages are good options for small and medium-sized sites. They don't work well when your volume is high (10 concurrent queries, 100,000 page views of a given feature). The problem: Forking.\n\nYou can speed things up a lot (and minimize forking) by building a Unix system from scratch. If you're AOL or Yahoo, with lots of programmers and a \"tech\" culture, that's a good plan.\n\nIf you're a newspaper or local broadcast Web site, a better solution is to go out and buy a Unix-based Web Application Server. The biggest names are Sapphire/Web and Net Dynamics. At Washingtonpost.com, we decided those cost too much. We bought Oracle Application Server and Netscape Application Server. From a database editor's point of view, they're about as easy to use as ASP or Cold Fusion, but they're faster.\n\n2. Forget All That Relational-Database Nonsense\n\nThis is simple stuff: You're probably already doing it. But if you're not, you can really speed up your Web databases and make them a lot more pleasant for users if you break some database design rules.\n\nDon't use joins. Joins work fine when you're using Access to do a CAR project. They're recommended in all the relational-database books. But even if they're perfectly indexed and you're using grossly overpowered hardware, they take too long on the Web.\n\nPut data for different Web pages in separate tables. Then, store the tables in system memory or put each of them on different hard drives. If possible, put the indexes your queries use on different drives as well. For example, if you're posting a school's database and have the test scores on a different page than the principal's name, make a different table for the scores that only has the scores and the basics that you need on that page (school name, etc.).\n\nTry to avoid ODBC drivers to access your database. Native drivers are faster.\n\n3. Keep It Simple\n\nMost of us in the news business got into using databases to produce stories. We're used to mining them to death to make sure that we've gotten every last bit of information out of them, then distilling the results down to understandable chunks when we write our stories. Or we are letting another reporter or an editor do it. That's great, but on the Web, you'll inevitably run into one big problem: 2.5 seconds.\n\nOur server logs tell us that site visitors spend less than three seconds on an average interior Web page, which is where your database will end up. If you want Web users to actually explore your databases, you've got to do a few simple things:\n\nDesign your features so users can go somewhere productive with a single click. You can give them lots of radio buttons and text boxes and search options, but you can't count on them because 2.5 seconds isn't a lot of time. Example of what not to do: The General Social Survey http://gort.ucsd.edu/gss/extract.html. Example of what to do: The Washington Post Poll Vault http://www.washingtonpost.com/wp-srv/politics/polls/vault/vault.htm.\n\nNumber-heavy pages don't work any better on the Web than they work in print, even if every number is a link. Example of what not to do: The Toxic Release Inventory Scorecard http://scorecard.org/env-releases/us.tcl#major_chemical_releases. Example of what to do: The Seattle Times School Guide http://texis.seattletimes.com/schoolguide.\n\nHal Straus can be reached by e-mail at Hal.Straus@washingtonpost.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-june-nicar-data-update","slug":"1999-june-nicar-data-update","url":"https://dwillis.github.io/uplink/article/1999-june-nicar-data-update.html","year":1999,"month":"June","monthNum":6,"headline":"NICAR data update","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR's Data Library has released updates to FBI Uniform Crime Reports (1980, 1990, 1993-97) containing detailed crime statistics across six databases with information on crime types, offender/victim demographics, and circumstances. The data offers valuable investigative resources but has limitations including self-reporting vulnerabilities and potential inconsistencies across jurisdictions.","keywords":["crime","data","FBI"],"topics":["Data & Databases","Crime & Justice"],"technologies":[],"full_text":"NICAR data update\n\nUpdates of the following data set are now available through NICAR's Data Library:\n\nFBI's Uniform Crime Reports (1980, 1990, 1993-97) contain detailed crime data gathered by the FBI from law enforcement agencies across the country. UCR comes in six tables:\n\nThe Return A database gives the number of crimes in 27 categories, including murder, rape, robbery, assault, burglary and larceny. Crimes are reported by month and broken down into actual offenses, offenses cleared by arrest and clearances under 18.\nSupplement to Return A provides greater detail on property crimes. Specific types of crimes (pocket picking, gas station robbery) are given, along with amounts stolen and recovered.\nThe Age, Sex and Race database provides male and female totals for various crimes. Race is given for juveniles and adults. The crimes are more specific, including drug offenses, gambling, DUIs, runaways and others.\nThe Supplemental Homicide Report. For each homicide, the data includes age, sex and race of offenders and victims, as well as their relationship and the circumstances of the crime. It also includes justifiable homicides by police and citizens.\nArson is set up like the Return A and Return A Supplement databases. Information includes fires in uninhabited structures and property damage estimates.\nThe Police database provides information about assaults and murders of police officers.\n\nStrengths: FBI Uniform Crime Reports are the only standardized source of detailed, agency-by-agency crime statistics for the whole United States. The reports don't identify criminals or victims, but they do provide interesting information about them, including age, sex, race, and sometimes crime circumstances. Do the numbers for your area look suspicious? Interesting stories have been written about discrepancies between what really happens and what gets reported to the FBI. (See \"Counting Crime Right?\" story, page 5).\n\nWeaknesses: All UCR data is self-reported; if a law-enforcement agency wants to fudge the numbers, it's easy to do. The FBI has integrity checks in place, but it mostly takes agencies at their word. Even the checks can cause problems: If the FBI deems an agency's (or entire state's) reports unreliable, it may delete them without telling anybody. Because different jurisdictions have different classifications for some crimes, these crimes are sometimes left out of the database without notation (i.e. Minnesota was listed as having no rapes for some years). So it is important to check values against local numbers.\n\nAlso coming this summer, HAZMAT (hazardous material spills), OSCAR (nursing home inspections), Federal Procurements (federal contracts), USDA (meat-packing plant inspections) and more.\n\nTo order, call NICAR at (573) 884-7332.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-inventing-a-school-system-busting-busing","slug":"1999-julyaugust-inventing-a-school-system-busting-busing","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-inventing-a-school-system-busting-busing.html","year":1999,"month":"July/August","monthNum":7,"headline":"INVENTING A SCHOOL SYSTEM: Busting busing","kicker":"","author_name":"Ted Mellnik","author_title":"Reporter","affiliation":"The Charlotte Observer","summary":"The story explores the implications of Charlotte's potential shift from race-based busing to neighborhood schools, post a Supreme Court challenge.","keywords":["education","busing","desegregation"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["GIS"],"full_text":"What if kids went to their neighborhood school? That's the big question in Charlotte. After the landmark 1971 Supreme Court decision upholding court-ordered busing here, the city was touted as the place where busing worked. But support has eroded over the past decade. Now a court challenge to race-based assignment policies threatens to dismantle Charlotte's system of busing for racial integration. To explore the issues and challenges that would accompany a move to neighborhood schools, The Charlotte Observer designed a mock school system. In our system, all kids went to the school closest to where they lived. Our model system is simplistic and probably impractical, but it gave us a way to spotlight potential changes and trends. We believed that if we fully explained our methods and assumptions, our model could contribute to our community's debate over neighborhood schools. The stories ran in January 1999 as the first five parts of 'Deciding Desegregation,' a continuing series. Reporter Debbie Cenziper and I described the forces of change acting on the schools. We compared schools in our system to existing schools, tracking ethnic diversity, concentrations of poverty, home-to-school distances, and campus crowding. The key ingredient of the Deciding Desegregation series was geographic data on each of about 94,000 students. We regularly use student data at The Observer because of the range of possibilities it offers for analysis. Most often, we deal with results of tests such as the SAT and North Carolina's end-of-grade and end-of-course exams. The data has an entry for each student with key facts, including information we used for the Deciding Desegregation series, such as a student's race, gender, grade in school, and school attended. But we still needed to find out where students lived. Geographic data is a set of coordinates, like latitude and longitude. These sorts of data would allow us to find a spot on earth near each student's home and display it as a dot in a computer mapping program. School officials have on hand geographic data for each student, drawn from their home addresses, in order to analyze attendance zones.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-julyaugust-dads-pay-little-welfare-fails","slug":"1999-julyaugust-dads-pay-little-welfare-fails","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-dads-pay-little-welfare-fails.html","year":1999,"month":"July/August","monthNum":7,"headline":"DADS PAY LITTLE: Welfare fails","kicker":"","author_name":"Jim Hopkins","author_title":"Reporter","affiliation":"The Courier-Journal","summary":"Investigation into welfare reform in Kentucky reveals challenges faced by former welfare recipients, focusing on issues like unemployment and unpaid child support.","keywords":["welfare","child support","Kentucky"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["AutoTrack"],"full_text":"For the first time since welfare reform began in Kentucky, a growing body of evidence shows that former welfare recipients in the state's chronically poor Appalachian region are failing to achieve one of reform's most important goals: finding jobs that pay a living wage. Welfare recipients in Kentucky's mountain counties also are more burdened by lax enforcement of child-support laws. Although welfare reform requires mothers to find work, many fathers who owe child support pay nothing, The Courier-Journal found in a six-month investigation. Our inquiry came down to people like Sheila Stone, a former welfare recipient trying to support four children on a $5.50-an-hour bank job. Her ex-husband hid from the law because he owed more than $40,000 in child support. Stone and the authorities didn't know where her ex-husband was. As part of our investigation, we tracked him down and eventually reported his return to jail. Reaching our conclusions was anything but simple, however. We fought for electronic databases from the state welfare agency responsible for reform. We relied extensively on data from the Census Bureau, the U.S. Bureau of Economic Analysis, and Auto Trak, a for-pay service that locates people. And we used paper records from the welfare agency and the state's employment services division to build databases of our own. From the beginning, we suspected the biggest challenge for Kentucky's welfare reform efforts would be the tens of thousands of women and children in the state's 49 Appalachian counties, home to some of the nation's poorest residents. Indeed, although Appalachia has 30 percent of the state's population, it has nearly half of all Kentucky's welfare cases. One of our chief sources of data was a statewide telephone survey of 522 former welfare recipients conducted by the University of Louisville. We persuaded Kentucky welfare officials to give us the raw data from the survey, which the university conducted under a state contract. The survey asked a host of questions, from the reasons recipients left welfare to whether life was better without government assistance.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-julyaugust-campaign-finance-kiddie-donors","slug":"1999-julyaugust-campaign-finance-kiddie-donors","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-campaign-finance-kiddie-donors.html","year":1999,"month":"July/August","monthNum":7,"headline":"CAMPAIGN FINANCE: Kiddie donors","kicker":"CAMPAIGN FINANCE","author_name":"Alan C. Miller","author_title":"","affiliation":"Los Angeles Times","summary":"Young teenagers are making large political donations to federal candidates, raising suspicions that wealthy parents are using their children to circumvent campaign contribution limits. Researchers analyzed nearly 9,000 student donations totaling $7.5 million between 1991 and 1998 to investigate the extent of this potentially illegal practice.","keywords":["donations","minors","loophole"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"They are the most precocious of political donors: youngsters who are not old enough to drive a car, never mind cast a vote, yet they dispense $1,000 checks to candidates for public office. The suspicion, of course, is that these school-age donors are the beneficiaries of a generous onetime boost in their allowance from parents looking to circumvent federal contribution limits. At $1,000 per election, family bundles are a tempting - and potentially illegal - way for wealthy givers to double or triple their largess to a favored candidate. Federal law places no minimum age on donors but requires that the funds be \"owned or controlled exclusively by contributors\" and that they \"give knowingly and voluntarily.\"\n\nConsultant helps\n\nOur curiosity about the extent of this practice was piqued by contributions from three students in a single family to a U.S. Senate candidate from California in 1998. So, we retained independent researcher Dwight Morris and his Campaign Study Group in Virginia, to provide a definitive set of numbers on federal contributions and a road map for further reporting.\n\nThe FEC regards student giving as such a potentially serious loophole that it has repeatedly urged Congress to ban donations by minors.\n\nStarting with the Federal Election Commission records of contributions of $200 or more between Jan. 1, 1991 and Dec. 31, 1998 - a total of 3.9 million donations - Morris searched for donations to federal candidates or party committees by those identified as \"students.\"\n\nHe then scrubbed the initial hits to locate additional money from those identified as students for some contributions but not others (donations often include no information listed for occupation).\n\nConversely, he eliminated other donors who were listed as students but who proved to be housewives taking a college course or professionals who had gone back to school.\n\nThe tally: 8,876 student donations totaling $7.5 million. And the trend lines showed the numbers were growing.\n\nUsing this database, Morris compiled a list of major student donors, including many of the 163 who gave a total of $5,000 or more. This was our starting point.\n\nSince the FEC does not list ages, we had to learn more about these donors to determine whether they were law school students who might be contributing themselves or preadolescents whose largess was less likely to be their own.\n\nOur strategy was to eliminate the older students through public records before approaching them and their parents.\n\nProfiling givers\n\nUsing online newspaper searches and voting and motor vehicle records, we built files on the top 50 donors who gave $13,700 to $93,250 each. This included a 15-year-old who donated $20,000 to the Democratic Congressional Campaign Committee.\n\nBut we still needed more truly young givers that we believed were out there.\n\nWe then focused on students who gave smaller sums and were therefore more likely to include youngsters in affluent California counties.\n\nThrough the FEC records, we identified those whose donations in recent election cycles came at about the same time that their parents gave the legal maximum to the same candidate. We then eliminated those who were eligible to vote by checking their names with county registrars of voters.\n\nUnderage donating\n\nThe final step was to nail down a specific age for those not registered to vote or licensed to drive.\n\nHere, we found a wide disparity in the national public availability of birth records. Fortunately, we were able to access them in Los Angeles County.\n\nDr. Scott Stolnitz, a Marina del Rey dentist, initially told me he wanted nothing to do with our story. When I subsequently called back to say that his daughter was going to be prominently featured, he said, \"You know nothing about my daughter.\"\n\nI replied that I knew she was 10 years old when she gave $1,000 to Republican presidential candidate Lamar Alexander the same day that both her parents also gave the maximum.\n\n\"OK,\" he said, \"Here's the story.\" (Which turned out to be that he decided to make the contribution on his daughter's behalf from a checking account that he funds.) With our cases in hand, further reporting revealed that the FEC regards student giving as such a potentially serious loophole that it has repeatedly urged Congress to ban donations by minors, based on the presumption that contributors below age 16 are not making contributions on their own behalf. Yet, since 1975, the oft-criticized watchdog has assessed just one fine arising from minors' donations.\n\nAlan Miller can be reached by e-mail at alan.miller@latimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-ambulance-response-times-cardiac-crapshoot","slug":"1999-julyaugust-ambulance-response-times-cardiac-crapshoot","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-ambulance-response-times-cardiac-crapshoot.html","year":1999,"month":"July/August","monthNum":7,"headline":"AMBULANCE RESPONSE TIMES: Cardiac crapshoot","kicker":"AMBULANCE RESPONSE TIMES","author_name":"Jack Dolan","author_title":"Investigative Reporter","affiliation":"Hartford Courant","summary":"A journalist uses computer-assisted reporting to analyze 911 call records and death data, revealing that a private ambulance company failed to meet response time standards for cardiac emergencies, with some patients waiting dangerously long periods.","keywords":["ambulance","database","investigation"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["FoxPro","ArcView"],"full_text":"When Mike McIntire handed a cardboard box of diskettes across the divider and said, \"I don't know if you can do anything with these,\" I knew I'd taken the right job.\n\nIt was my first day at the Hartford Courant, my first job since leaving NICAR McIntire, my new colleague on the investigative team, was working on a story about deathly slow response times by American Medical Response, one of the nation's largest private ambulance companies. AMR made several promises to Hartford officials to wrest the 911 business away from smaller, local companies including responding to 90 percent of serious medical emergencies within six minutes.\n\nDisks to database\n\nMcIntire had heard plenty of stories about AMR failing to meet that standard, but he knew he needed hard data to see if the anecdotes were part of a larger pattern. To that end, he got two years' worth of 911 call records from the Hartford police, who dispatch the ambulances. Each month of records came on a separate diskette as a tab-delimited text file. Those were the disks in the box Mike handed over the divider.\n\nBefore long, we'd appended all of the files into a single FoxPro table, calculated the response times, and sorted the 50,000 calls so that the one with the longest response time was on top. Then we picked through the data for what doctors say are indisputable emergencies: cardiac arrest, chest pains, and difficulty breathing.\n\nWe found that 34 percent of these most vulnerable patients waited more than six minutes for their ambulance - three times the number the AMR contract allowed. Hundreds of verified heart attack victims waited longer than ten minutes. Some waited longer than twenty minutes. Over two years, the number of people put at serious risk by long response times reached into the thousands.\n\nEmergency medicine experts across the country told us that dangerously long response times are bound to happen once in a while. That's why ambulance companies, when negotiating contracts, make sure to give themselves a cushion 10 percent of the time; 34 percent shocked the experts. We were told that if we wanted to discover what happened to those who waited the longest, the best place to start checking was the morgue.\n\nJoining death records\n\nSo we requested a database of all recorded deaths in Connecticut from the Department of Health. Among the fields in the death data were the name and address of the deceased and the date of death. The 911 records had no names of victims, but they did have the date of the call and the victim's address.\n\nOnce we had the hard numbers that showed just how long AMR's response times are, the rest of the CAR work just helped make an already strong story stronger.\n\nI joined the databases on two criteria. The first was where the addresses matched. The second was when a death was recorded within one day after a wait of more than 10 minutes after a 911 call for someone suffering from one of those three most serious emergencies. A sad roll call of silent victims filled the screen. Through interviews with these victims' families, police, EMTs and AMR executives, we discovered that, like everyone else who waited too long for an ambulance, most of the dead fell into three distinct categories.\n\nMapping the calls\n\nFirst, some seemed to have to wait too long for the ambulance because of where they lived. We grouped all of the addresses of calls that took longer than six minutes into census tracts, then mapped the tracts using ArcView. Each tract was a different shade depending on the percentage of calls that took too long - the darker the color, the longer the response time.\n\nThe darkest tracts were on the extreme northern, southern, and western edges of the city. We expected a lot of long response times in the north, since it is the poorest part of Hartford. But the west is the wealthiest, and the darkest-colored southern tract is home to the mayor and many long-time Democratic city bosses. So much for the old adage about city services being best on the mayor's own street.\n\nThe explanation, it turned out, is that AMR retrenched on an agreement to keep a certain number of ambulances based on the edges of the city at all times. Instead, the company reduced the number of ambulances available, rotating those that remain between the city's hospitals, which are located in the center of Hartford.\n\nA second apparent predictor of whether an ambulance would come quickly was the time of day. We expected to find that most of the longer calls happen in the early evening, when the 911 call volume is highest. Our logic was simple: if two people are waiting for one ambulance, somebody has to wait longer.\n\nInstead, we found response times are longest at mid-morning when 911 call volume is very light. Several EMTs explained that the peak hours for the lucrative business of shuttling non-emergency, well-insured patients home from the hospital are between 10 a.m. and noon, just after doctors have made their rounds and discharged patients. Therefore, AMR is loath to send an ambulance off on a 911 call during those hours since the company has a much better chance of getting paid for the high priced taxi service.\n\nA third contributor to long ambulance response times was the medical ignorance of some police officers thrust into the ambulance dispatcher role, often with very little medical training. We found 20 percent of confirmed cardiac arrest cases were not initially classified by police dispatchers as emergency calls, so the ambulance driver was not put on the highest alert.\n\nSince our investigation, the Courant ran several stories on AMR's poor performance in Connecticut and the company's problems holding onto contracts in other parts of the country.\n\nStory prompts suit\n\nFinally, in early June, the state of Connecticut reached an anti-trust settlement with AMR. The settlement forced the company to give up 30 ambulance licenses to its competitors and give up 40 percent of the Hartford market.\n\nIn Hartford, ambulances are dispatched by the police and not by the ambulance company. That was key to our investigation, because it meant the dispatch database was a public record.\n\nOnce we had the hard numbers that showed just how long AMR's response times are, the rest of the CAR work just helped make an already strong story stronger.\n\nJack Dolan can be reached by e-mail at dolan@courant.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-warning-homeowners-shoddy-shacks","slug":"1999-julyaugust-warning-homeowners-shoddy-shacks","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-warning-homeowners-shoddy-shacks.html","year":1999,"month":"July/August","monthNum":7,"headline":"WARNING HOMEOWNERS: Shoddy shacks","kicker":"WARNING HOMEOWNERS","author_name":"Ames Alexander","author_title":"Reporter","affiliation":"The Charlotte Observer","summary":"The Charlotte Observer's investigative series exposed widespread construction defects in new homes across the Carolinas, revealing that overworked building inspectors and inexperienced workers failed to catch serious flaws ranging from foundation problems to fire hazards.","keywords":["construction","inspectors","defects"],"topics":["Investigative Methods","Business & Economy","Data & Databases"],"technologies":["Access","World Wide Web"],"full_text":"Looking for a story that will bring about an avalanche of phone calls and e-mail from grateful readers? Have we got one for you. In December, The Charlotte Observer published \"Home Buyer Beware,\" a six-day series that detailed how Charlotte's housing boom had swamped builders, overwhelmed inspectors and left scores homeowners with costly and dangerous problems.\n\nAcross the Carolinas, thousands of families have discovered serious construction flaws in their new homes, ranging from shifting foundations to walls in danger of collapse. The homes belonged to people like Alan and Marie Nelsen, who vacated their new, half-million-dollar house in an exclusive lakefront neighborhood after an engineer reported that it was uninhabitable. People like Michael and Chona Hoy, who say serious construction defects led to a 1996 blaze that destroyed their new house.\n\nHow could it possibly happen? Our research showed that builders are routinely hiring workers with little or no experience. North Carolina's laws favor builders over buyers, and overworked building inspectors are missing serious flaws.\n\nChecking inspectors\n\nDuring our eight-month project, reporter Rick Rothacker and I accompanied building inspectors on their daily rounds, watched houses get built, pored through public records and interviewed more than 400 homeowners, builders, inspectors and others. And with the help of database editor Ted Mellnik, we used a variety of computer files to add power, authority and detail to our reporting.\n\nLocal officials said they couldn't provide precise statistics on the workloads of inspectors, so we got the raw data - four million computer records on all building inspections conducted in Mecklenburg County since the 1970s. Among the conclusions the computer data helped us draw:\n\nIn Charlotte, construction flaws were widespread. The data showed that public building inspectors were rejecting home builders' work in roughly 4 of every 10 inspections.\n\nBuilding inspectors were swamped. Inspectors in Mecklenburg were averaging 18 inspections per day. As a general rule, experts say, code officials should average about 10 inspections a day. On some days, code enforcers were doing more than 70 inspections and spending an average of less than eight minutes on each of them.\n\nWhether a house passed an inspection depended in part on the inspector. In 1997, one inspector failed 62 percent of the buildings he inspected. One of his colleagues, though, failed just 17 percent.\n\nOur research showed that builders are routinely hiring workers with little or no experience.\n\nThe Observer also conducted a survey of Charlotte-area residents who'd recently bought new homes and found that more than 30 percent wouldn't recommend their builder to a friend. To find all those homeowners, The Observer obtained databases from area counties with information about new house permits. Meanwhile, a North Carolina Department of Labor database allowed us to show exactly how brisk business had been for the region's home builders.\n\nHelpful sites\n\nWe also used a variety of Web sites. Among the handiest was one maintained by the National Association of Home Builders: www.nahb.com The group collects reams of information on new homes and the companies that build them - including survey data on how builders cope with the labor shortage and how much profit they're pocketing. Some advice for reporters who hope to delve into construction issues:\n\nBy all means request your county's building inspections data, but be prepared for some fancy computer work. For our project, Mellnik untangled the unwieldy hierarchical database provided by Mecklenburg County and put it in a form that allowed me to analyze it with Microsoft Access.\n\nBe ready to invest some time. It may make sense to attend a course on the building code, as I did. And there's no substitute for actually watching the nails get driven.\n\nYou may also want to get your graphics artists involved from the start. William Pitzer's superb series of graphics showed how houses are built and what often goes wrong, adding depth and context to the project.\n\nIf you tackle a story like this one, you'll probably find your time was well spent. After reading our series, Mecklenburg County officials agreed to hire more inspectors and take other steps to reduce the high inspection failure rate. State lawmakers introduced several bills to better protect home buyers. And more than 250 readers contacted the Observer to thank us and share their stories.\n\nAmes Alexander can be reached by e-mail at aalexander@charlotte.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-first-ventures-animals-to-go","slug":"1999-julyaugust-first-ventures-animals-to-go","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-first-ventures-animals-to-go.html","year":1999,"month":"July/August","monthNum":7,"headline":"FIRST VENTURES: Animals to go","kicker":"FIRST VENTURES","author_name":"Linda Goldston","author_title":"Reporter","affiliation":"Mercury News","summary":"An investigative reporter pursues animal disposition records from accredited zoos to track where surplus animals are sent after leaving facilities, eventually seeking access to the ISIS database used by zoos to track animal transactions.","keywords":["animals","records","investigation"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["Excel","CD-ROM","Internet"],"full_text":"The zoos were making me mad. I wanted animal disposal records from all of the accredited zoos so I could track their surplus tigers, monkeys and nyalas after they left the zoo grounds.\n\nMost of the zoos weren't cooperating, and I wasn't about to settle for a dik-dik studbook and records from a couple dozen of the smaller zoos.\n\nEven so, for a while I became known as \"Ms. Scanner\" in the office as I faithfully scanned every animal transaction record I'd been able to get my hands on into Excel thinking all the while I'd plead, pry, pout and use my Southern accent to get the rest.\n\nRecords from my FOIA requests to the U.S. Fish & Wildlife Service and the U.S. Department of Agriculture already filled enough boxes to turn my desk, my car and my living room into forts. But I was still missing huge parts of the puzzle: Where did all of the animals go?\n\nSeeking zoo records\n\nOne zoo director told me no reporter had asked for studbooks before, yet I was demanding them with a vengeance after learning all of them were public records, thanks to agreements signed with other zoological groups around the world. The studbooks recorded the genetic histories of animals in a species but they also contained such goodies as showing which animals left an accredited zoo and went to such dealers as Earl Tatum and Buddy Jordan.\n\nI became very fond of a three-letter initial recorded after many of those animals were listed as going to dealers in the studbook - LTF, or \"lost to follow-up.\" Lost to me, the public and the zoo itself. Lost in the regulatory black hole that governs zoo and exotic animals.\n\nEnough already!\n\nI had notebook after notebook of handwritten lists of where certain animals went after leaving the zoos, but the whole picture was far from complete.\n\nSo I decided, hell, I'll try to buy the same database the zoos use themselves to record their animal transaction records.\n\nThe first time I called the International Species Information System (ISIS) in Apple Valley, Minnesota, I simply said I wanted to buy a set of the CD-ROMs that hold all of the goodies.\n\nThe guy I spoke with said I didn't qualify - I needed to have a collection of animals. Ah, never fear, I said, I have a dog and a cat. The ISIS man laughed half-heartedly and said, um, no.\n\nFine. About that time, a pair of wild ducks had taken up residence in the moat surrounding two sides of the Mercury News and hatched more than a dozen ducklings to make their way in the chlorinated waters of the moat.\n\nSo I called the ISIS man back: \"Hey, now I have a dog, a cat, a mama duck, a papa duck and 14 babies,\" I said. \"Surely that qualifies as an animal collection.\"\n\nUm, no, that won't do, the ISIS man said, and there was no half-hearted laugh this time. But as soon as I checked the list of ISIS members on the Internet and saw a few of the same dealers listed that I was trying to track, I had one of those moments investigative reporters know well. It's the moment when you say, OK, by god, I don't know how, but I WILL get the information.\n\nSources offer data\n\nIt took a little while longer, but I obtained two copies of the database from separate sources. It enabled us to verify the integrity of the data on each set.\n\nNow all of you trained observers will have noticed that I slipped the first \"us\" into my narrative. I spent most of my career getting to the core of things by myself - well, me and all those boxes that accumulate. This time, I needed something I'd heard about but figured I'd never need to use - the expertise to make sense of those CD-ROMs. And luckily, I didn't have to go far.\n\nJennifer LaFleur, database editor at the Mercury News, and a former NICAR training director, took one look at my bewildered expression and said, OK, I'll help you, dummy. She actually said, \"Oh, OK,\" but you learn to joke early and often on these things.\n\nSgt. LaFleur drove me nuts for the first two weeks we were working with the data because I kept wanting to make printed copies of the 52 million animal transactions, which had to be made one copy, one transaction at a time. I wanted to hold them in my hands, add them to the forts of documents that now surrounded my desk, the inside of my car and my living room.\n\nBut Jennifer kept saying no.\n\nNo, no, no, you don't need to print them out, she said over and over, her voice rising each time.\n\nWell, dammit, I did need to, I still want to now and probably will print out copies of many of the animal transactions - one at a time - before it's all over. But I agreed to be a good kid and set about trying to identify who the recipients of zoo animals were in the computerized records. There was no manual that said so and so is a dealer or that NBJ was dealer Buddy Jordan's NBJ Ranch in Texas.\n\nExplaining the data\n\nI also located a terrific source from the zoo world who could help us understand the ISIS data. That made it possible for Jennifer and me both to make sense of this complex and quite literally awesome database. The source agreed that since we had the data, it would help no one if we misinterpreted them. And that's when Jennifer put this collection of millions of records into a database that we could search by individual zoo or pull out numbers for all of the accredited zoos. The data analysis derived from it was the crowning touch to the culmination of all that work: A four-day series called \"Zoo Animals To Go\" by yours truly in the San Jose Mercury News, published February 7-10, 1999.\n\nFor the record, this ol' warrior will add one thing: A database is a remarkable tool - one that I came to admire and deeply appreciate. I might even do it again.\n\nLinda Goldston can be reached by e-mail at lgoldston@sjmercury.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-nerd-notes-zoo-data-can-be-a-bear","slug":"1999-julyaugust-nerd-notes-zoo-data-can-be-a-bear","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-nerd-notes-zoo-data-can-be-a-bear.html","year":1999,"month":"July/August","monthNum":7,"headline":"NERD NOTES: Zoo data can be a bear","kicker":"NERD NOTES","author_name":"Jennifer LaFleur","author_title":"","affiliation":"San Jose Mercury News","summary":"Journalists describe their process of investigating animal transactions using databases from zoos and wildlife organizations, emphasizing the importance of data verification, manual cleaning, and cross-checking analyses multiple times to ensure accuracy.","keywords":["database","verification","animals"],"topics":["Investigative Methods","Data & Databases"],"technologies":["ISIS"],"full_text":"One key problem with the zoo project was that there was no government/public record database that kept track of animal transactions. But not every database is a government database - keep in mind that associations, non-profit agencies and other organizations keep databases. And although you may not have a public access right to the data, you may be able to use your sources to get data. One caution: VERIFY, VERIFY, VERIFY. We did this with the ZOO database by getting two different copies of the data from different sources and comparing them.\n\nOnce we had the data, the work began. We didn't receive a straightforward database with lots of documentation; we had to go into the \"canned\" software written by the ISIS people and find the databases we needed. And there were hundreds of data tables to sort through.\n\nAgain, by using sourcing, Linda was able to get some help understanding the database.\n\nAlso, the database simply included the names of entities that received animals. There was no field that designated someone as a dealer or an auction. All of that information came from documents and reporting. Linda had to go through the database and manually create a designation for dealers, auctions, hunting ranches, etc.\n\nReporting and sending samples of data runs to zoos helped us do a better analysis in the long run. Our first analysis looked at any animal transaction going to particular dealers, auctions or hunting ranches. After talking with some sources, we realized that any animal can have multiple transactions, so we narrowed the database to look at just the last transaction for each animal, which is basically where the animal ended up.\n\nOne caution: VERIFY, VERIFY, VERIFY. We did this with the ZOO database by getting two different copies of the data from different sources and comparing them.\n\nThe next set of analyses showed that some of the animals included things like \"domestic sheep\" and \"domestic pigs,\" and even though we really enjoyed \"Babe,\" we filtered out domestic animals. We wanted to concentrate the analysis on exotic animals.\n\nWe got another database from U.S. Fish and Wildlife that lists endangered and threatened species. We matched these data against the taxonomy of the animal transactions to find endangered animals.\n\nCAUTION: Matching data from two different sources can be extremely tricky and will still require some grunt work to double-check the matches by hand.\n\nOnce we had come up with our analyses by zoos, by recipients of animals and by species, we did the most important step to any analysis. We each went back to the very beginning and recreated every analysis. You can't take a chance that you missed something the first time. You must be able to duplicate your work.\n\nJennifer LaFleur can be reached by e-mail at jlafleur@sjmercury.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-erring-pharmacists-dealing-drugs","slug":"1999-julyaugust-erring-pharmacists-dealing-drugs","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-erring-pharmacists-dealing-drugs.html","year":1999,"month":"July/August","monthNum":7,"headline":"ERRING PHARMACISTS: Dealing drugs","kicker":"ERRING PHARMACISTS","author_name":"Bennett Cunningham","author_title":"","affiliation":"KTVT-TV (Dallas)","summary":"A hidden camera investigation reveals that pharmacists who make potentially fatal medication errors often escape disciplinary action because lawsuits target pharmacies rather than individual pharmacists, shielding names from state boards. Journalists used public records and courthouse documents to track pharmacists involved in lawsuits and compare them against state disciplinary records to expose this accountability gap.","keywords":["pharmacists","lawsuits","investigation"],"topics":["Investigative Methods","Data & Databases","Crime & Justice"],"technologies":["Access","Excel"],"full_text":"Pharmacists are among the most trusted professionals in the world. When we get sick, they dispense medicines to help us feel better. But when a pharmacist makes an error, it can be fatal.\n\nOur three-month hidden camera investigation found a disturbing case. A doctor in Texas prescribed a painkiller to a healthy mother. The few pills she ingested almost took her life.\n\nThe drug was not a painkiller; it was a drug for diabetics. Today, the patient is an invalid, unable to speak, eat and think for herself. The pharmacist dispensed the wrong drug. You might think the pharmacist who made the dispensing error was banished from his profession and leading an ignominious existence. You couldn't be more wrong.\n\nBoard unaware\n\nOur investigative report found that the Texas State Board of Pharmacy, which licenses and disciplines pharmacists, wasn't aware of many near-fatal mistakes. The reason is that when a patient sues, the lawsuit is against the pharmacy, rarely the pharmacist, which shields the pharmacist's name from the disciplinary board. Only if the patient files a complaint does the state get involved.\n\nThis story changed the way pharmacists in the state of Texas fill prescriptions\n\nTo make this story work, we needed a database. The idea was to track pharmacists found in lawsuits and determine if the state ever took disciplinary action. An open record request to the state yielded a plethora of great data. We asked for the full name, license number and date of birth of every pharmacist disciplined in the state of Texas since 1995. We requested the data on a floppy disk in either Access or Excel format. It wasn't a problem and a week later the data were cleaned and we were on our way. Sounds easy, but the bitter pill was just around the corner.\n\nThe basement of the Dallas County Courthouse is a musty old hovel. The ceiling is lined with fluorescent lights and the rows of yellow case files go on forever. The story was in here. I printed out a list of cases from the county computer. All of the lawsuits concerned large retail pharmacies. I whittled the list down by isolating the cases to large damage claims.\n\nPoring over paper\n\nFor two weeks, a researcher and I read the case files. We read every deposition, every interrogatory and every exhibit. Our mission: to find the name of each pharmacist involved. By the end of our research, we had volumes of legal pads filled with information on pharmacists. Now we had to figure out what to do with all of it.\n\nWe already had the data from the state. Next, we needed to find out if one of the pharmacists mentioned in any of the lawsuits was simultaneously disciplined by the state. After attending so many NICAR seminars, the phrase \"mapping out your data\" rang in my head. We kept it simple. We typed all the names of the pharmacists we found in the lawsuits into the database, then asked one basic question - which ones don't match? If there was a match between the state's list and our list derived from the lawsuits, the pharmacist was most likely disciplined from events arising from the case (we'd double-check this later).\n\nOnce the database determined whether there was a match, we asked other questions. Every time we entered a pharmacist's name, we entered other pertinent information on that same record. So, for example, we knew how many male pharmacists were addicted to controlled substances. We knew how many were convicted of crimes. The possibilities seemed endless. Only our thoroughness and our creativity limited the queries.\n\nComplex data\n\nThere were complications. The mass of data was so large that sometimes it made querying nearly impossible. Once, I had to print out the entire database on 500 sheets of paper because I couldn't get a query to work. After taping and stapling all the corresponding sheets together, I laid them out on my office floor. Then I did it the old fashioned way - I counted. I just couldn't get the computer to do it right. The mistake I made was organizing my data improperly. Remember mapping? Well, I didn't do a very good job. My data were not consistent, resembling a Word document rather than a well-oiled database. I cluttered the database with so much extraneous information that certain queries rendered the program useless.\n\nPlanning databases\n\nReporters trying to do a story like this should have an easy time if they plan their databases wisely. It is easy to plug in the names and license numbers for people and then check them against each other. The hard part comes in the details. That's where the database devil lives. He thrives in confusion and bad planning. He waits for you to find a new nugget for your story and then bam! You are toast. Gather, map and pray - three words to live by. No story should ever scare you. Always be ambitious in picking stories. Despite the fact that a poorly planned database can leave you in the database jail, good planning will land you on GO, and the rewards are endless.\n\nReporters trying to do a story like this should have an easy time if they plan their databases wisely.\n\nThe aftereffects of this story changed the way pharmacists in the state of Texas fill prescriptions. A new education program is underway and all pharmacists must triple-check every prescription before it leaves the counter.\n\nBennett Cunningham can be reached by e-mail at Bennettc@ktvt.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-ticket-scams-bad-drivers-fund-city","slug":"1999-julyaugust-ticket-scams-bad-drivers-fund-city","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-ticket-scams-bad-drivers-fund-city.html","year":1999,"month":"July/August","monthNum":7,"headline":"TICKET SCAMS: Bad drivers fund city","kicker":"TICKET SCAMS","author_name":"Bill Torpy","author_title":"Reporter","affiliation":"Atlanta Journal-Constitution","summary":"Pine Lake, Georgia raised 78% of its budget from traffic tickets, with a majority issued to African-American drivers on a single road section, sparking protests and investigations into alleged racial disparities and predatory policing practices.","keywords":["tickets","Georgia","revenue"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":[],"full_text":"Pine Lake, a town of about 800 residents some 10 miles east of downtown Atlanta, was the epitome of the age-old Southern ticket trap. However, it used computer technology to track its prey. The city raised 78 percent of its $1.1 million budget last year from traffic tickets - or put another way, the city pulled in a dollar from traffic tickets for every dime paid by residents in property taxes.\n\nPine Lake is not the only city in Georgia where traffic fines provide key revenue, according to an Atlanta Journal-Constitution study of city and county budget records. Statewide, there were 18 police departments that have operated in the black consistently since 1989, and several other metro Atlanta jurisdictions ranked high as vigorous traffic-enforcement towns.\n\nBut Georgia's undisputed traffic-ticket champion was Pine Lake, where enforcement on just one 1,500-foot section of road last year generated $860,000 worth of fines and fees. And a strong majority of the tickets were issued to African-Americans traveling through the predominantly white town.\n\nCity residents don't complain: the aggressive ticketing equates to low taxes and plush services. Residents don't even have to bag their leaves; a truck rolls by to vacuum them from the curb.\n\nDrivers protest\n\nAlthough residents love the arrangement, many motorists who pass through the town take offense. A protest encampment has attached itself to the traffic court. The district attorney was asked by black residents to investigate alleged racial inequities in the ticketing. And legislators have threatened to go after the city's charter.\n\nThe investigative package launched by the AJC into the Pine Lake ticketing situation is a perfect example of combining database research with traditional reporting to increase a story's impact and push for changes.\n\nA state legislator offered the tip after introducing me to three citizen-activists who had set up a semi-permanent protest against Pine Lake officials for what they termed as predatory policing. At first glance, the 1998 city budget numbers that the protesters provided (and the city verified) were startling and the story was a natural. I wrote about the protest to get it on the record and started on a longer news-feature.\n\nAfter the first report, I heard that our computer-assisted reporting editor, David Milliron, was digging into numbers concerning ticket traps throughout Georgia. We quickly married our efforts. Milliron's computer-based package provided the numbers I needed for authority and a yardstick for comparison's sake.\n\nA strong majority of the tickets were issued to African-Americans traveling through the predominantly white town.\n\nMilliron had been compiling data collected by Georgia's Department of Community Affairs. (Georgia has been tarnished in the past by ticket traps and the state compiles data on fines, forfeitures and court fees to keep tabs on those towns and counties that may be overzealous.)\n\nWith the records, we went back 10 years to show year-by-year statistical aberrations. Pine Lake turned out to be the poster child for aggressive, ticket-happy policing in Georgia.\n\nDicing the numbers\n\nThere were several ways to slice and dice the numbers - fines based on a per capita basis, fines as compared to a percentage of the police department's overall budget, fines as compared to the entire government budget, and so on. But no matter how we sliced the numbers, Pine Lake always came out on top.\n\nThe analysis gave us a benchmark: It showed the state average in per-capita fines was $35. Twenty-five cities collected twice that amount. But Pine Lake collected a whopping $1,055 per capita in 1998.\n\nOriginally Police Chief Mac McGlamery played down the issue of ticketing, saying his force was aggressive, but fair. When presented those figures, he kept saying \"Are you sure? No. 1? In the state?\"\n\nNegotiating for data\n\nTo go deeper on the story, Milliron also negotiated for computerized ticket data from a number of police agencies. In Pine Lake's case, none of the police or city officials had the technical know-how to retrieve the data from their new computer system. The police chief said the agency had nothing to hide, and that Milliron could have the data if he could figure out how to get them out of the computer system.\n\nSo Milliron worked with the city's software vendor in South Carolina, who sent him a software upgrade necessary to write a report on the system, then export the data to diskette. Milliron performed the necessary software upgrades and collected all of the data in one afternoon at the police department. (He even showed the chief's secretary and court clerk how they could do the same for future requests.)\n\nNow came the final data analysis. State records made it nearly impossible to break down who was being ticketed and for what offense. That's where the raw ticket data from the city of Pine Lake proved helpful. That data included the usual ticketing demographics such as age, sex, race, offense, date, time, fine amounts, issuing officer and even final disposition.\n\nNo tickets issued\n\nIt also showed that almost no tickets were issued for moving violations. That was too messy in court, the chief said. In fact, the department hasn't issued a speeding ticket since McGlamery took office in 1997. Instead, the 10-officer department targeted yes/no offenses like out-of-date license plates, no insurance, and no driver's license. And the fines meted out in the small municipal traffic court were much larger than fines for similar offenses in the neighboring cities and counties.\n\nTo aggravate matters, Pine Lake is predominately white, but 87 percent of those ticketed were black. That, however, may not be as sinister as it sounds, as the town is surrounded by almost an entirely African American population and a statistical count of cars found a 4-to-1 ratio of black drivers to white.\n\nAlso, the court disposition data - included in the department's ticket database - showed that blacks and whites fared similarly in court when facing similar charges.\n\nPublic responds\n\nThe articles drew an avalanche of e-mail and telephone calls. The city responded by saying it would try to be kinder and gentler but still firm - in how it deals with driving infractions.\n\nThe investigative package launched into the Pine Lake ticketing situation is a perfect example of combining database research with traditional reporting to increase a story's impact.\n\nBut there were more than just promises. The city also cut its police force in half and has replaced its white municipal court judge with a black woman. Also, the city has overhauled its fine structure and is vowing to sell the new expanded police department building.\n\nThe changes did not totally stop the controversy swirling around the tiny city. Protesters still seek the ouster of the city's police chief and demand that the city stop writing tickets on the busy street running on the city's southern edge. The city has refused to do that.\n\nBut as time has rolled by, the protests are less frequent.\n\nBill Torpy can be reached by e-mail at btorpy@ajc.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-supreme-court-and-car-justice-is-blind","slug":"1999-julyaugust-supreme-court-and-car-justice-is-blind","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-supreme-court-and-car-justice-is-blind.html","year":1999,"month":"July/August","monthNum":7,"headline":"SUPREME COURT AND CAR: Justice is Blind","kicker":"SUPREME COURT AND CAR","author_name":"Tony Mauro","author_title":"Reporter","affiliation":"USA TODAY","summary":"USA TODAY used computer-assisted reporting to create a database analyzing Supreme Court law clerks' demographics from 1972 onward, finding historically low minority hiring rates. The publication also analyzed the Supreme Court's docket to examine cases the court declined to hear, revealing confusion in various areas of law.","keywords":["Supreme","clerks","database"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Martindale-Hubbell directory","Internet"],"full_text":"The Supreme Court, which still hands out quill pens to lawyers who argue before it, does not lend itself easily to computer-assisted reporting. But for two projects, USA TODAY used information technology to help the reporting on two major enterprise projects.\n\nLegal prejudice?\n\nThe first was a March 1998 look at the law clerks who assist the justices in screening cases and drafting opinions. In addition to assessing their influence, we undertook the first-ever look at the demographic breakdown of the clerks hired by the current justices, dating back to 1972 in the case of William Rehnquist. While the names of the clerks were publicly available, nothing else about them was, and the court told us \"You're on your own\" in terms of tracking down this information.\n\nWe created a database with all the clerks' names and began, one by one, to figure out their background. Identifying gender was easy in most cases, and a small number of names (Asian and Hispanic) also made ethnic background obvious. But most names were not obvious, and we wanted to be doubly sure that our categorization was correct. Through online searches of legal directories and other sources, we were able to locate most of the clerks. One of the most useful free sites was the Martindale-Hubbell directory of lawyers around the country, at http://lawyers.martindale.com/marhublform\n\nWe called many of the clerks and asked them to give us their own ethnic or racial information and to do \"peer reporting\" on other clerks from their year at the court. We encountered some resistance but were eventually able to feel certain about our information.\n\nTo double-check, we sent our information to each justice individually. Only two justices responded. For the other seven, we did not interpret their silence as confirmation of our information, but after further checking we went ahead, and have not heard since that any of our information was wrong.\n\nThe survey indicated that the justices have hired extremely low numbers of minorities as law clerks. The stories triggered objections from civil rights groups and a debate over the reasons for and significance of the court's hiring practices. The following is a link to the original stories and follow-up stories on this issue.\n\nhttp://www.usatoday.com/news/court/clerk/clerk000.htm\n\nDocket declinations\n\nThe second project was a December 1998 look at the Supreme Court's docket, with special focus on the cases that the court decides not to decide-the cases the court passes up. Lawyers have complained with increasing frequency that the court, in reducing its caseload, has left many areas of the law in a state of confusion.\n\nThe court told us 'You're on your own' in terms of tracking down this information.\n\nTo document this, we solicited the help of U.S. Law Week, a BNA publication that lists, in capsule form, every case on the court's docket of paid cases. With their permission, we used their database to break down the cases the court declines to consider. This customized database indicated that the court declines substantially higher percentages of cases in certain categories than others.\n\nUltimately, we made only limited use of these data for two reasons.\n\nFirst, the wide variance in numbers of cases in different categories made the \"rejection rate\" seem exaggerated in some instances.\n\nSecondly, other reporting indicated that there are so many factors that go into the court's decision to decline a given case that relying on the numbers alone would lead to drawing unsound conclusions. The following are links to the main stories in this package.\n\nwww.usatoday.com/news/court/nscot933.htm\nwww.usatoday.com/news/court/nscot932.htm\n\nTony Mauro can be reached by e-mail at tonymauro@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-on-the-internet-backgrounding-by-web","slug":"1999-julyaugust-on-the-internet-backgrounding-by-web","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-on-the-internet-backgrounding-by-web.html","year":1999,"month":"July/August","monthNum":7,"headline":"ON THE INTERNET: Backgrounding by Web","kicker":"ON THE INTERNET","author_name":"Wes Williams","author_title":"","affiliation":"IRE","summary":"A 1999 IRE Conference handout provides journalists with practical guidance on backgrounding individuals online, starting with search engines and progressing to public records databases for more reliable information.","keywords":["backgrounding","records","research"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Internet","World Wide Web","AltaVista","InferenceFind","HotBot","Dejanews","modem/BBS","PACER","THOMAS","EDGAR","Social Security Death Index"],"full_text":"The following are excerpts from a handout presented during the \"Backgrounding the individual by Web and computer\" panel at the 1999 IRE Conference.\n\nStart with as much info as possible: Who are you searching for and why? Even if you're starting with just a name, it helps to know why you're looking for this person and what the story is about. This will give you other ideas on where to look for information.\n\nIs your subject a lawyer? Check an online legal directory. Does his bio say where he or she went to school? Check the school's alumni Web page for any stories on him or her. Every nugget of info you have will help, but look especially for approximate age, geography and occupations.\n\nThe computer is often the fastest way to background an individual. The following is a list of Internet sources of information. These are in the general order I go through, but it varies each time according to the amount of info I already have, the type of info I'm trying to find and how long I have to get it.\n\nSearch engines\n\nI've found good information before just by punching someone's name into a search engine. I often start with a quick keyword search through a few of my favorite search engine Web sites.\n\nPros: They're quick, free and can lead you to a lot of personal information if you're lucky.\n\nCons: You can end up with too many possibilities and get bogged down in trying to wade through all the John Does to find your John Doe. Good places to start include: AltaVista (www.Altavista.com) InferenceFind (www.inference.com) HotBot (www.hotbot.com) Dejanews (www.Dejanews.com)\n\nPublic records\n\nThe best and most accurate information is in public records. Some are available on the Internet, some on dialup bulletin-board systems and some you need the individual databases from the appropriate agency.\n\nIf I already have a good ID on someone (name, DOB, address), I go to these places first to look for info relevant to the story. Here's a list of the ones I hit most often.\n\nCounty court: Check for Web page or dialup BBS to get access to court docket information.\nCounty Recorder: Find property records and more in these like liens, tradenames, etc.\nCounty Assessor: See who's paying the taxes on a property or what it's worth.\nSecretary of State/USS: These help when trying to nail down business associated with someone's name or finding some lines\nState Corporation Commission: Maybe yours is on the Web. Ours is a dialup BBS that charges a small fee.\nProfessional registrations: Is your subject registered with a state agency (doctor, police officer, child care, center owner, contractor) Check for Web sites that have the info and get an electronic copy of the data so you can have a back up and do database-matching projects.\nFederal Bankruptcy District Court via PACER: This is slowly coming back to the Web, but for now, it's a dialup BBS in most places. See www.uscourts.gov/PubAccess.html or call 1-800-676-6856 for info. Feds charge 60 cents per minute for PACER access. You can get bankruptcy docket information and it's great on deadline.\n\nPublic record databases\n\nThese sites below have public record databases already online:\nTHOMAS, home of bills, laws and the Congressional Record (www.thomas.loc.gov)\nEDGAR Securities & Exchange Commission filings (www.sec.gov/edaux/searches.htm)\nInternet Non-Profit Center based on IRS records (www.nonprofits.org/locl/index.html)\nSocial Security Death Index (www.ancestry.com/ssdi/advanced.htm)\nAircraft Owners and Pilots. Watch for outdated info. (www.landings.com)\nDocfinder with links to state medical boards on the web. (www.docboard.org)\nMilitaryCity.com. Subscription-based search for military members and veterans. Free for Gannett companies.\nProperty Records Online, University of Virginia page. Lists where to find property record Web Sites (www.people.virginia.edu/~dev-pros/Realestate.html)\n\nPrivate people finders\n\nThese are great for locating hard-to-find people, relatives, crisscrossing addresses and finding non-local leads. If I'm in a hurry and it's worth the cost, this may be my first stop instead of my last. These services are usually not very good for finding people under age 22.\n\nAutotrack charges $2 a minute and an average in-depth search takes about 8 minutes. (www.dbtonlin.com)\nCBD Infotek charges $17 for an in-depth Discovery Plus! Report (www.cdb.com)\nKnowx Web-based service, search cost ranges from free to $1.50 per individual database, detail records range from 95 cents to $6.95 per record. (www.knowx.com)\n\nThese are the ultimate tools for quick and thorough research. The catch: They're very expensive and can waste your time when you come up with a hundred John Doe listings and you don't even have a middle initial.\n\nWes Williams can be reached by e-mail at williamsaz@worldnet.att.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-tech-tip-importing-pdf-tables","slug":"1999-julyaugust-tech-tip-importing-pdf-tables","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-tech-tip-importing-pdf-tables.html","year":1999,"month":"July/August","monthNum":7,"headline":"TECH TIP: Importing PDF tables","kicker":"TECH TIP","author_name":"Tom McGinty","author_title":"NICAR Training Director","affiliation":"NICAR","summary":"Reporters struggling to extract data from FBI crime statistics published in Adobe Acrobat format can use Aerial software to convert Acrobat documents into editable text for spreadsheets and databases.","keywords":["Acrobat","data","software"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["Adobe Acrobat","Aerial","Excel","Internet","World Wide Web"],"full_text":"There they are, staring you right in the face: the preliminary figures for the 1998 FBI Uniform Crime Report. It's all there on your screen city-by-city tallies of murders, aggravated assaults, robberies, you name it - and all of the figures appeared with the click of a hyperlink on the FBI Web site. You're salivating. Images of spreadsheets are dancing in your head. Your editor is going to love this!\n\nBut wait a minute. Something looks funny. What's up with the strange menu bar that suddenly appeared at the top of your Web browser? Why does the whole Web page look so odd, as if it's a picture of a document? Oh no! Could it be? Aaarghhh! It's true: The FBI has published the statistics using Adobe Acrobat, a format that Web publishers love but computer-assisted reporters loath.\n\nThe bad news is, it's a pain to get data tables out of Acrobat documents and into a crunchable format. The good news is, it's not impossible. And that's true because of Aerial, a piece of software developed by Ambia Corporation that turns Acrobat documents into text. This tech tip will go over the nuts and bolts of using Aerial. But first, here's a little background.\n\nThe way it was\n\nIn the good old days of the Web - say, four or five years ago - most tabular data on the Internet were published in plain text that could be quickly imported into spreadsheet programs or database managers and manipulated at will.\n\nThen along came those pesky HTML tables, which had pretty raised borders that gave them a 3-D look but were kludged up with HTML coding that made them nearly impossible to import. Microsoft's programmers solved that problem with Excel '97, which easily imports HTML tables.\n\nBut Acrobat also was on the scene and only growing in popularity with government agencies that put data-rich reports on the Web. It's an understandable, if annoying, phenomenon. With Acrobat software, bureaucrats can transform huge, typed reports and graphics into Acrobat Web documents with the click of a button. But unlike text or HTML tables, Acrobat tables cannot be imported into other programs. That's where Aerial comes in. And as the following directions will show, it's not too tough to install or use.\n\nThe bad news is, it's a pain to get data tables out of Acrobat documents and into a crunchable format. The good news is, it's not impossible. And that's true because of Aerial\n\nFirst, you have to be sure that you have Adobe Acrobat Reader 3.x installed on your computer. If you have a different version (either an earlier version or 4.0) the current version of Aerial will not work. Check your version of Acrobat and then:\n\nNew software\n\nGo to www.ambia.com, and click on the \"Products\" link, then click on the \"Adobe Acrobat Tools\" link. Next, click on the \"Aerial\" link. If you found that you didn't have Acrobat Reader 3.x, you can click on the yellow and red \"Get Acrobat Reader\" link on the bottom left of this page and download a free version of Reader 3.x. Install that software and then go back to the Ambia Web. Once you get to the Aerial page, you can either download the \"demonstration\" copy, which is free and good for just 30 days, or you can buy the software for $149 (unfortunately, Ambia recently increased the price from $49).\n\nIt's probably a good idea to try the demonstration version to determine if you will really want to use the software. Once you get the file downloaded, go to the folder where you stored it and double click on the file you downloaded to begin the installation.\n\nOnce the installation software starts up, one of the first steps is to set the path where it will be installed. Aerial fills in a default path that probably won't work, so you should hit the \"browse\" button and find the folder where Acrobat Reader 3.x is located.\n\nOpen that folder, then open the \"Reader\" folder and then open the \"plug-ins\" folder. Your path probably will look something like this: c:\\acrobat3\\reader\\plug_ins. Then just click on the Okay button and complete the installation.\n\nGrab the table\n\nNow that the program is installed, the fun part starts. Use the FBI Uniform Crime Report as your first test. Go to http://www.fbi.gov/ucr.htm and locate the link that reads \"1998 Preliminary Crime Statistics (.pdf).\" Instead of clicking on the link and opening the file in your Web browser, put your pointer over the link, click on your right mouse button and a drop-down menu will appear. Choose the option to \"Save link as\" (To get the same drop-down menu on a Macintosh computer, hold the control button down and then click on the link.) This will bring up a \"Save\" dialogue box. Choose the folder where you want to save the file and click on \"Save.\" Instead of opening the .pdf file in your browser, you have saved it to your hard drive.\n\nNow start up Acrobat Reader (you can find it by going through the \"Start\" button and \"Programs\"). Under the \"File\" menu, choose \"Open\" and then open the file you just saved. Now go to the page that has the table you want to move into a spreadsheet or database manager. You need to highlight the table first, so on the Acrobat Reader menu bar, click on the icon with the letters \"abc\" inside it. That will give you the text cursor. Starting at the top left corner of the table, hold your mouse button down and drag to the bottom of the page. The text of the table should become highlighted as you go. (Even if a table spans several pages, you can only highlight one page at a time, so you'll have to repeat these steps.)\n\nConverting tables\n\nOnce you have the entire page highlighted, go to the \"Plug-Ins\" pull-down menu, choose the \"Aerial\" option and then choose \"Copy table to Clipboard.\" A new window titled \"Select Clipboard Format\" will pop up. As you probably already know, the best choice is the default choice: \"Tab Delimited ASCII.\" Make sure that selection is chosen and then click \"OK.\"\n\nUsing Aerial, you have turned the mirage-like data of an Acrobat document into real numbers that will slake your thirst for crunching.\n\nYou have just converted the once-unusable Acrobat table into a tab-delimited ASCII table that can be used any way you see fit. You can either paste it into a word processor, save it as text and later import it into Excel, or paste it directly into Excel. Give both methods a shot and see which one works best for you. Now go back to the UCR Acrobat document and get the second page of the table, etc.\n\nBe cautious\n\nUsing Aerial, you have turned the mirage-like data of an Acrobat document into real numbers that will slake your thirst for crunching.\n\nA note of caution: Although Aerial has worked flawlessly for me every time I have used it, I have heard other reporters say that they encountered dropped characters in the translation. Of course, it's always a good idea to check the integrity of your data once it's imported.\n\nTom McGinty can be reached by e-mail at tmcginty@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-stats-and-such-raging-to-regress","slug":"1999-julyaugust-stats-and-such-raging-to-regress","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-stats-and-such-raging-to-regress.html","year":1999,"month":"July/August","monthNum":7,"headline":"STATS AND SUCH: Raging to regress","kicker":"STATS AND SUCH","author_name":"Neill A. Borowski","author_title":"","affiliation":"The Philadelphia Inquirer","summary":"A journalism educator explains the mathematical foundations of linear regression analysis, encouraging reporters to understand the statistical processes behind regression software rather than treating it as a black box. The article walks through the method of least squares using a practical example comparing school free lunch percentages to test scores.","keywords":["regression","statistics","journalism"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["SPSS","SAS","Excel"],"full_text":"How many times have you interviewed an \"expert\" who quickly described a process without going into detail? If you're any kind of interviewer, you made this expert stop and painstakingly explain how the process worked. The process might have been a controversial bond issue, a new communications technology or the treatment for a disease.\n\nNow let's consider the rage to regress. Only at a NICAR session will you hear journalists in the bar late at night discussing how they \"ran this independent variable with that dependent variable and came up with an r-square of\n\nYet, when we use this popular and insightful tool, we never press for an explanation of how the process works. We plug our variables into the black box (SPSS, SAS, Excel, etc.) and out pops the regression results. Many don't seem the least bit interested in how the black box did what it did.\n\nCall me a math nerd, but I like to understand the statistical underpinnings. Obviously you can't hand-crank a regression analysis of thousands of records. But it still makes sense to understand the process that's going on inside the black box.\n\nOur goal in simple linear regression analysis is to come up with a model: + random error. The \"y\" is your dependent variable (such as test scores). The \"X\" is your independent variable (such as poverty rates). The \"a\" is the slope of the regression line and the \"b\" is the constant (the intercept of the vertical axis). This is the equation of the regression line.\n\nThe regression line is fitted from, in essence, many averages. The model tries to plot the line that best fits the data that you've provided. The first step is a scatterplot, simply taking each data point and drawing a point where its X value and y value intersect. The regression line drawn through the points is the best fit, based on the average lengths of the data points from the regression line.\n\nThese two charts represent the percentage of students eligible for free lunch in a school (x) and that school's average test score (y). The other columns simply are X-squared, y-squared and X times y.\n\nFREE LUNCH % (x) | x2 | TEST SCORE (y) | y | xy\n0.55 | 0.3025 | 900 | 810000 | 495\n0.67 | 0.4489 | 900 | 810000 | 603\n0.03 | 0.0009 | 1365 | 1863225 | 40.95\n0.48 | 0.2304 | 1152 | 1327104 | 552.96\n0.12 | 0.0144 | 1218 | 1483524 | 146.16\n0.89 | 0.7921 | 856 | 732736 | 761.84\n0.76 | 0.5776 | 1010 | 1020100 | 767.6\n0.08 | 0.0064 | 1300 | 1690000 | 104\n0.26 | 0.0676 | 1193 | 1423249 | 310.18\n0.35 | 0.1225 | 1297 | 1682209 | 453.95\nSUMS\n4.19 | 2.5633 | 11191 | 12842147 | 4235.64\n\nNow the regression math - or \"method of least squares\" - begins.\n\nSxx = (The sum of the X squares) - (The sum of the x's, squared/n)\n\nOr, in math notation:\n\nSxx = - Ex-ex\n\nSubstituting from the table above:\n\nSxx = 0.80769\n\nNow the same for the y's (test scores):\n\nSyy = (The sum of the y squares) - (The sum of the y's, squared/n)\n\nOr, in math notation:\n\n=\n\nSubstituting from the table above:\n\nSyy = 12,842,147-11,191°-318,2\n\nFinally, we need a third \"average\" - the average of the x's times the y's.\n\nSxy = (The sum of the xs times the ys) - (The sum of the x's times the sums of the y's)/n\n\nOr, in math notation:\n\n=\n\nSubstituting from the table above:\n\n=\n\nWith these \"averages\" behind us, we now can find the components of the regression line.\n\n=-561.34\n\nSince a is the slope of the regression line, and because a is negative, the line is negatively sloped - meaning that it goes from the upper left to the lower right.\n\nIntuitively, this makes sense. As X grows, there are more (a higher percentage of) poor students in a district. So, as we go from left to right on the horizontal (x) axis, test scores drop.\n\nb = The average of the y's minus (a (from above) times the average of the xs)\n\nb=y-ax 1119.1 - (-561.34)(0.419) = 1354.3\n\nSo, the equation of the line, y ax + b, is:\n\ny = -561.34x + 1354.3\n\nPlug in the x's from the table above and the y's that are yielded in this equation will be the predicted scores.\n\nNow, we need to calculate the correlation coefficient (r) and, more importantly, the coefficient of determination (r2).\n\nWe already have all of the needed components, so we plug them in:\n\n-453.389 T r2 0.8\n\nThat's a coefficient that suggests a strong relationship between income and test scores. It means that 80 percent of the variability in test scores is explained by the variability in income, according to the regression model. The larger the coefficient (up to 1), the stronger the relationship between the independent and dependent variables.\n\nYour regression model still may be off base. It could be totally ridiculous. The formula does as it's told. As you can see from the math, there is no mystical transmutation of the numbers. It's all a matter of averaging, squaring, substracting, etc. - all calculations designed to simply minimize the distance of each actual data point from the estimated line. Keep this in mind in your rage to regress. Don't stop. Just keep this in mind.\n\nNeill A. Borowski can be reached by e-mail at lnborowski@phillynews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-handouts-of-the-month-sprawling-around","slug":"1999-julyaugust-handouts-of-the-month-sprawling-around","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-handouts-of-the-month-sprawling-around.html","year":1999,"month":"July/August","monthNum":7,"headline":"HANDOUTS OF THE MONTH: Sprawling around","kicker":"HANDOUTS OF THE MONTH","author_name":"Rose Ciotta and Greg Reeves","author_title":"Reporters","affiliation":"The Philadelphia Inquirer and The Kansas City Star","summary":"A journalism guide providing data sources and techniques for investigating urban sprawl, including census data, building permits, property records, and mortgage information to uncover how suburban expansion has affected local communities.","keywords":["sprawl","census","data"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Internet","World Wide Web","FTP"],"full_text":"Sprawling around\n\nThe following is a handout from the \"Investigating Sprawl: Techniques and Revelations\" panel at the IRE National Conference in June.\n\nBy Rose Ciotta The Philadelphia Inquirer and Greg Reeves The Kansas City Star\n\nSprawl is not new unless you haven't done the story in your town. Americans have been moving to the suburbs since before World War II, but the impact of this exodus has often been hidden. Here's a list of data sets to help you dig out how sprawl has affected your coverage area:\n\n1) U.S. Census - There's valuable info here even as we wait for the first Census 2000 data on Dec. 31 of next year. The Census Library in Washington, D.C. can send you population and other demographic data for your area back to pre-Civil War days. Charting county-to-county population moves by census tract since World War II gives the basic picture of sprawl for most areas.\n\nwww.census.gov/population/www/documentation/tups0027.html www.census.gov/population/www/estimates/countypop.html\n\n2) U.S. Census Building Permits - Place level data on the number of housing units from single family to doubles, triples and buildings with more than five units built every month or every year. Data are collected in a mail survey of localities that issue the building permits. Dates to 1980 easily; some earlier years are available on paper. Can be compared to units in structure in census. Annual and monthly data dating to 1980 are available on diskette from the bureau's Construction Division at (301) 457-1321.\nwww.census.gov/ftp/pub/const/www/c40index.html\n\n3) Local Property Roll including Sales Data - Tracking number of housing units and property values year by year in your area can provide real-life contrast between urban and suburban trends. Also, property taxes in some areas become grossly inequitable between newer and older homes because of the lack of updated reassessments. This is an important public policy question and can make for a high-impact story. Property assessment databases in most areas are public record, and should be part of your basic CAR data set. Data on the relationship of assessment to sales will help you pinpoint areas usually in the inner ring that are losing their value. The result is often that owners of lower valued homes pay a higher proportion of taxes than expensive homes. Look for public ownership of buildings. As population shifts, more residential property is abandoned in the inner city and inner ring. Use the roll to examine who is selling and who is buying the land where development is occurring.\n\nSprawl is not new unless you haven't done the story in your town but the impact of this exodus has often been hidden.\n\n4) HMDA (Home Mortgage Disclosure Act) Data - Who applies for home loans by census tract, income and race as well as who gets approved and denied. Issued annually by the FFIEC (Federal Financial Institutions Examination Council) - one loan application, one record. This data will allow you how to illustrate the impact of sprawl on minorities. The data shows where banks have loaned money for home purchases and the demographics of the buyer including race and income. The purpose of the disclosure law is to help the feds determine whether banks are serving local housing needs and to identify discriminatory lending patterns. Last year, 7,925 banks, savings associations and credit unions reported 16.4 million loan applications for 1997.\n\nwww.ffiec.gov\n\n5) FHA and VA Mortgages - Say hello to your local recorder of deeds office. Pick a subdivision that sprang up during the big move to the suburbs. Look up the mortgage on each parcel federally subsidized Federal Homes Administration and Veterans Administration mortgages are easy to spot. How much of a role did the federal government play in your area in contributing to the move to suburbia?\n\n6) IRS Migration Data - Track yearly movement of people from counties. These data are especially useful if your area has seen the movement of people from a city county to a suburban county. Provides demographic information on the persons moving, including age range and education. Issued yearly, with 1996-1997 the latest available. Includes the number of tax returns (which can be used to approximate the number of households), number of personal exemptions (which can be used to approximate the population), and, for the first time, adjusted gross income (AGI) and median AGI. (202) 874-0410.\nwww.irs.ustreas.gov\n\n7) Highway Volume Data - Most states and counties keep track of traffic volume. Looking at travel over time can reveal changing residential patterns. Also examine the type of road, what it was designed to carry and the volume it does carry.\n\n8) FBI Uniform Crime Reports - Annually issued data reveal the frequency of crimes by cities and towns. If you analyze these data over time, they will likely reveal increasing crime in the inner city. Suburban residents often cite increasing crime to justify their move to the suburbs.\nwww.fbi.gov\n\n9) Campaign Contributions - by builders to local officials making decisions about new developments, to county-level officials who often drag their feet on countywide planning and to state lawmakers who often refuse to impose planning requirements on localities. Also look for contributions from environmentalists lobbying for smart growth initiatives.\n\n10) Business Property Tax Breaks - for property data to show special exemptions for new businesses that locate in new residential areas. Calculate how much in local property taxes they are saving.\n\n11) County and ZIP Business Patterns - County-level data that reflect the number of businesses by category, number of employees and payroll value from 1988 through 1996 for county data. ZIP code level data have been published annually since 1964, with 1994-95 the latest year available on the Census Bureau Web site. These data are especially useful to show how businesses move from the inner ring to the outer ring and the growth of businesses in new residential areas.\nwww.census.gov/epcd/cbp/view/cbpview.html\nhttp://tier2.census.gov/zbp/zbpinfo.htm\n\n12) American Housing Survey - a wealth of information about the nation's housing - apartments, single-family homes, mobile homes, vacant housing units, household characteristics, income, housing and neighborhood quality, housing costs, equipment and fuels, size of housing unit, and recent movers. National data are collected every other year, and data for each of 46 selected metro areas are collected about every four years, with an average of 12 areas a year. The national sample covers an average of 55,000 homes. Each metropolitan area sample covers at least 4,800 homes. The Census Bureau conducts the survey for HUD. You can't get tract-level data from AHS, but you can divide your metro area into several parts, and report hundreds of data items, including estimated cockroaches-per-household.\nwww.census.gov/hhes/www/ahs.html\n\n13) Agricultural data - Track the (shrinking) agricultural acreage in your area over time. The U.S. Department of Agriculture puts out the Census of Agriculture with data on farms from which at least $1,000 of agricultural products were produced and sold in a year. Data is available on a county and state level. The census is taken every five years - during the years ending in \"2\" and \"7.\"\nwww.nass.usda.gov/census/\n\n14) Other Data - These may include school enrollment and cost per pupil, construction and commercial building permits, contributions by builders, voter registration data or censuses of governments and government finance by the Census Bureau. Local and regional planning departments and agencies often have much data on urban and suburban growth. Some newspapers have also conducted polls on public attitudes toward development and quality of life issues.\n\nRose Ciotta and Greg Reeves can be reached by e-mail at rciotta@phillynews.com or greeves@kcstar.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-handouts-of-the-month-editing-car-work","slug":"1999-julyaugust-handouts-of-the-month-editing-car-work","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-handouts-of-the-month-editing-car-work.html","year":1999,"month":"July/August","monthNum":7,"headline":"HANDOUTS OF THE MONTH: Editing CAR work","kicker":"HANDOUTS OF THE MONTH","author_name":"Jeffrey Kummer","author_title":"","affiliation":"St. Paul Pioneer Press","summary":"This article provides editors with practical guidance on managing computer-assisted reporting (CAR) projects through the concept, reporting, and writing stages, emphasizing thorough planning, field reporting, adequate time allocation, and compelling storytelling over data presentation.","keywords":["editing","reporting","analysis"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Here's a checklist of dos and don'ts that other editors - whether they are computer-savvy or not - may find helpful at each stage of a CAR story:\n\nConcept\n\nBrainstorm, discuss and critique stories thoroughly before you begin. CAR is time-consuming and expensive. Understand what you, your reporters and your newspaper or station is getting into.\nListen to all ideas. Beware of any proposal that begins with \"Let's take a look at\" Such stories are doomed unless they're better defined.\nDon't \"data dive.\" Just because there are million of records in those databases, it's easy to assume there has to be a story there some place.\nQuestions to ask: What do you think a computer analysis would show?\n\nWhat are you expectations based on? Who have you talked to? What other evidence do you have? What will CAR add to this story that we couldn't get in other ways?\n\nReporting\n\nKeep talking - perhaps daily, at least weekly, depending on the story's time frame. An occasional memo or draft that includes a working lede and nut graf can help keep the editor and reporter on the same track. It can also serve as an early warning system for a reporter gathering lots of information but losing sight of the story.\nGet reporters off the computer and out of the newsroom. Don't wait until the entire analysis is done to bounce findings off of sources, locate experts or talk with the real people behind the data you are examining. Those forays into the field can reinforce findings or poke holes in faulty conclusions.\n\nAllow enough time. CAR is not a shortcut. It is time consuming, especially for beginners. Trying to do CAR on deadline can convince novices their first CAR project should be their last.\nTeam when possible, especially on stories where timing is an issue. Dividing the work between two or more reporters can get the story into print or on the air sooner.\nDon't forget to involve copy editors, artists and photographers early. You can eliminate the \"deadly data\" syndrome from many CAR stories by using charts, summary boxes, etc.\n\nWriting\n\nLeave enough time for the writing and editing. When it's time to write, work with the reporter to take a final inventory of the information gathered. Evaluate the strengths and weaknesses of the material - what you will use and what you don't.\nBe on the lookout for CAR fatigue. Those nuggets of information that once seemed so revealing start losing their luster as the project goes on. Help reporters remember what will be of interest to someone getting this information for the first time.\nTell the story with people and events, not with data. If the reporter can't find examples of the trends revealed by the analysis, question the findings.\nDiscuss presentation, but resist formulas. At this point, it is no longer a CAR story. It's simply a story to be told in as compelling a way as possible.\nDon't burden readers or viewers with process. In most cases, they won't care how many thousands of electronic records we filtered and sorted. Our motto might be: We do CAR so you don't have to.\n\nEditing\n\nIf you have CAR skills, help be a backstop to the reporter's work. Most reporters welcome another set of eyes and hands on the data.\n\nAn experienced CAR reporter I recently hired told me: \"It drove me nuts that editors at my previous paper had zero CAR knowledge. As a result, every time I had a CAR story in the paper, I was hanging out there.\"\n\nSee the story through a reader's eyes. Is it interesting? Does it hold together? Are its points clear and its focus sharp?\nDo you understand the data the reporter has used and are you comfortable with the conclusion the reporter has reached?\nQuestion the reliability of the data the same way you would the veracity of any other key source that was to be the basis for a story. Why did you choose these records? Who has checked each calculation for errors? What data integrity checks were done? How current is the data? Have the data been used elsewhere in similar stories? Were there any problems or pitfalls that arose in those cases?\nIf you don't have CAR skills, ask the reporter to show you how he or she came up with key findings. If the reporter can't recreate those numbers with you sitting next to him or her at the computer, be concerned.\nAttack the numbers. Get as many of them out of the story as possible. Can some of them be delivered better in a graphic? Can percentages be recast in easier-to-read terms, such as \"four out of five doctors?\"\n\nAfter publishing\n\nDo the post-mortem. Sit down with the reporter and walk through the process. What would you each do differently next time? Where were all the flaws in the data? Who should we have talked to earlier? What did we wish we'd have gotten that we were unable to get this time around?\nIs there a plan for following the story? Can you come back in a year or two and see what has changed? Is the methodology documented well enough that another reporter could redo the analysis later and be able to compare the findings?\nAre there other uses for the data? Once the CAR story is done, work with the reporter or your CAR coordinator to make the data available to the rest of the newsroom. It's great to do a project on repeat drunk drivers. But now that you've got driving records in digital form, those could be an ongoing resource for other reporters and editors.\nJeffrey Kummer can be reached by e-mail at jkummer@pioneerpress.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-julyaugust-bits-bytes-and-barks-nicar-data-update","slug":"1999-julyaugust-bits-bytes-and-barks-nicar-data-update","url":"https://dwillis.github.io/uplink/article/1999-julyaugust-bits-bytes-and-barks-nicar-data-update.html","year":1999,"month":"July/August","monthNum":7,"headline":"Bits, Bytes and Barks: NICAR data update","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR announces updated datasets including Uniform Crime Reports, National Bridge Inventory System, and Federal Contracts Data, along with tipsheets and resources to help journalists analyze the information. The IRE Resource Center provides accompanying stories and guidance for using these databases, with an upcoming regional conference scheduled for September 1999.","keywords":["data","databases","journalism"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["FTP","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nNICAR data update\n\nUpdates of the following data sets are now available through NICAR's data library. Prices and years available can be found at www.nicar.org or by calling 573-884-7711.\n\nThe Uniform Crime report consists of six separate databases from the FBI gathers crime information from law enforcement agencies around the country. All of the databases provide the region, state, county, city, MSA and reporting agency identifier. Those who want to see a slice of the dataset can go to http://www.nicar.org/datalfbi and download a section.\nThe National Bridge Inventory System (NBIS) 1998 includes maintenance records collected by the Federal Highway Administration for assessing the nation's bridges. It is also used by the Department of Defense and the Federal Emergency Management Agency. State inspectors assess bridges and their reports are submitted annually to the FHA. This is a great database for CAR beginners. More information can be found at http://www.nicar.org/datalfhwal index.html#nbis\nFederal Contracts Data 1998 offers a database of Individual Contract Action Reports (ICARS) created by the Government Services Agency. Each record contains information about the agency granting the contract, the contractor, the place of performance and the contract amount.\n\nThis year, NICAR is including a FIPS code table and a table of U.S. Representatives so that reporters can match contracts to local representatives. The FIPS table isn't always filled in, so matches will not appear 100 percent of the time, but it's a good place to start. Upcoming in August: Hazardous Materials, National Boat Registration information and IRS 990 databases.\n\nTipsheets and stories\n\nIn conjunction with NICAR data updates, the IRE Resource Center has put together a list of stories and tipsheets. Not only do the stories help reporters see what kind of information is found in the data sets, but the tipsheets point out potential hazards and give good contacts to help put the information in perspective. We have tipsheets and stories to go with the FBI, Bridges and IRS 990 datasets. We also have stories to match with the Federal Contracts Data. You can search the resource center database at www.ire.org/resourcecenter to see what's offered and to order copies of the tipsheets.\n\nIRE Regional Conference\n\nIt's not too late for reporters and editors interested in the IRE Regional Conference coming up in Los Angeles on September 9-12, 1999.\n\nFor updates on that conference and other IRE or NICAR gatherings, see www.ire.org/resources/conferences.2","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-september-catch-and-release","slug":"1999-september-catch-and-release","url":"https://dwillis.github.io/uplink/article/1999-september-catch-and-release.html","year":1999,"month":"September","monthNum":9,"headline":"Catch and release","kicker":"CRIME AND CAR","author_name":"James Walsh","author_title":"public safety reporter","affiliation":"Star Tribune","summary":"Explores the issue of repeat offenders in Minnesota, focusing on the lack of prison sentences for felonies and crimes committed on probation. Uses data analysis to link multiple criminal justice databases.","keywords":["crime","probation","Minnesota"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["FoxPro"],"full_text":"Commit a felony in Minnesota and, chances are, you won't go to prison. Commit a crime while on probation and, chances are, you won't go to prison. We had heard it for years: that repeat offenders and even convicted offenders on probation can and often do re-offend with few consequences. In 1999, we decided to put it to the test. I applied for a newsroom fellowship that would allow me to temporarily jump from our public safety team to the projects team to work with Dan Browning, computer-assisted reporting editor. The move freed me from the daily grind of the police and courts beats to dig into the anecdotes and see if there were data to support them. Capturing crime data The first stop was Minnesota's Data Practices Act, the complex state statute that details which information collected and kept by public agencies is public - and which is not. We reviewed sections on arrest data, jail data and court services data regarding offenders on probation. The next step was to fire off letters to several metropolitan area counties, requesting the data we understood to be public. We wanted to first identify the people on probation in the Twin Cities area over the previous five years, then compare those data to jail bookings and court data to see if offenders on probation were being arrested and, if so, what happened afterwards. Getting the data took about three months. The state's two largest counties - Hennepin and Ramsey - refused to give us dates of birth of probationers, insisting the data were private under state law. The Hennepin County Sheriff's Office refused to give us dates of birth for offenders in the jail, again arguing that the data were private. Birth dates were critical information. The state has no standardized, unique identifier to track criminal suspects among counties or as they pass through the justice system. The tracking devices follow cases, not individuals. But by converting birth dates to strings and combining them with names to create key fields, we believed we could link several different criminal justice databases. Thus, birth dates were critical to ensuring that we were finding offenders who crossed county lines and to following individuals from arrest to court to prison and back into the community. The Star Tribune sued Hennepin and Ramsey counties for the data, but the lawsuit would take months. We decided to proceed the hard way, but were forced to change focus from probationers to repeat offenders. We wanted to examine who was booked most often, and what was happening to their cases after arrest. We had dates of birth for jail inmates in every Minnesota county but Hennepin. We figured we could match the Hennepin County data to other criminal records to capture dates of birth. We obtained millions of criminal justice records from the Minnesota Sentencing Guidelines Commission (results of every sentencing statewide, 1993-1997), the Minnesota Supreme Court (felony and gross misdemeanor court dockets statewide, 1993-1997), Department of Corrections data on adult offenders (1994-1998) and work release inmates (1994-1998), and Bureau of Criminal Apprehension conviction data (felonies and gross misdemeanors, 1984-1999). Recognizing repeaters We became the only organization in Minnesota to attempt to link these various databases to find comprehensive criminal histories of repeat offenders. Neither the state nor the counties had constructed anything to answer the questions we would be asking. By matching district court case numbers from the Hennepin County jail database to Hennepin County cases in the state Supreme Court database, we were able to fill in thousands of dates of birth from the court data. Doing individual court record searches on others allowed us to fill in more dates of birth for the most frequent offenders. And finally, using FoxPro, we created unique key fields using three different combinations of name (first-middle-last; first-middle initial-last; last) and date of birth for each table in our criminal justice archive. These matches enabled us to extract birth dates for thousands of other Hennepin County offenders. In the end, we filled in dates of birth for one-third of all people booked in Hennepin County over the five-year period we studied and, more importantly, the top 1,300 most frequent offenders. Those offenders accounted for the bulk of the bookings. One of our key findings was that the top 3.5 percent of the offenders accounted for 22 percent of the bookings in Hennepin County. We became the only organization in Minnesota to attempt to link these various databases to find comprehensive criminal histories of repeat offenders. Neither the state nor the counties had constructed anything to answer the questions we would be asking. To define \"chronic offender,\" we settled on a standard of more than 11 bookings in the five-year study period. Under that definition, we found at least 8,700 offenders getting booked over and over and over again. And we also found that the impact of chronic offenders was felt not only in the inner cities, but also in the suburbs and rural areas. The data backed up our hypothesis. But our work was just beginning. As with all good CAR projects, the data simply gave us a starting place. After about six weeks of data cleaning and writing string functions to query the various tables against each other, we flagged the top offenders in the seven-county metropolitan area of the Twin Cities. We then checked them against court and sentencing databases and began the task of finding the offenders and the victims to tell their stories. Meanwhile, my colleague Paul Gustafson worked on a separate but related project examining the work-release program run by the Minnesota Department of Corrections. Gustafson analyzed five years of data and found that the program was releasing more hardened criminals into the program, some of whom served as little as one-third of their sentence behind bars. We decided to publish this project together with the repeat offenders project because they are two ends of the same spectrum: a cheap, lenient corrections philosophy that has saved money but set criminals essentially free in the community. Many of these criminals escaped from supervision and some committed new crimes. Readers respond We published our series on July 18, 19 and 20. In the end, it was the very human stories of victims and repeat offenders that gripped many readers. But we also finally gave them a sense of the scope of the problem. Local television, local talk radio and National Public Radio picked up parts of our project. We've heard from dozens of readers, including some legislators and other elected officials who are calling for ways to monitor repeat offenders. And after publishing our series, Ramsey and Hennepin counties agreed to provide the dates of birth for probationers (and in Hennepin's case, for those booked into jail) for the five-year study period, thereby settling our lawsuit. James Walsh can be reached by e-mail jwalsh@startribune.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-september-school-sports-as-big-business","slug":"1999-september-school-sports-as-big-business","url":"https://dwillis.github.io/uplink/article/1999-september-school-sports-as-big-business.html","year":1999,"month":"September","monthNum":9,"headline":"School sports as big business","kicker":"FUNDING FOOTBALL","author_name":"Mike Fish","author_title":"Reporter","affiliation":"The Atlanta Journal-Constitution","summary":"Examines the significant role of high school football in Georgia communities, highlighted by financial investments and social status, with data collection revealing the economic impact of football programs.","keywords":["football","schools","Georgia"],"topics":["Business & Economy","Education","Investigative Methods"],"technologies":["Excel","Access"],"full_text":"In the small towns and one-school communities across Georgia, life has historically revolved around the fate of the high school football team. I was already kicking around the idea of a season-long package that would reveal just how important the game is to these communities - the role of parent-run touchdown clubs; the level of local and state tax monies spent on football, particularly for coaching salaries; and the status enjoyed by high school football coaches. Then, the letter came across my desk. It was an invitation from the pastor of a congregation in Macon, writing in his capacity as secretary/treasurer of the Westside High School football booster club. The club, he explained, wanted to show off its new football complex. And if The Atlanta Journal-Constitution newspapers were so inclined, the boosters would gladly take a reporter up in a helicopter for a birds-eye view of the complex. The thought of parents raising almost $2 million to build a football complex - not on school property but rather on land owned by its football booster club - was enough to convince the newspaper's editors that this was a story worth pursuing. The idea behind \"Pay Dirt! The Big Business of High School Football\" was to explore what separates the winners from the losers, the box-office successes from the failures. To do so, I teamed up with David Milliron, the newspaper's director of computer-assisted reporting. We built a comprehensive database of the nearly 2,400 high school football coaches and assistant coaches in the state. Forming a game plan Since teacher salary and personnel information is kept at the local school district level, the newspaper compiled an electronic database from information obtained from each of the state's 180 school districts - including payroll records, teaching certificates and teaching assignments. The AJC also obtained the state's annual school report card data, (in electronic form), which contained such information as school enrollments and average district salary data for teachers, administrators and support personnel. The database was built in Microsoft Excel and the analysis performed in Microsoft Access. The data were collected using the state's Open Records Act, which mandates the release of such information. Once each school district responded, the data were entered into the computer, and a \"proof sheet\" was then printed out for each of the state's 180 school districts. The \"proof sheets\" were then mailed back to the school district for verification of the information. The AJC sent out a third round of mailings and made numerous telephone calls to obtain 100 percent compliance. The last survey was faxed to the newspaper by a secretary just prior to leaving her office to attend her mother's funeral on the day before publication. While Milliron handled the survey and data collection, I spent my time interviewing coaches, administrators and booster club members. Many schools and administrators thought we were prying and refused to cooperate, but some expressed pride in their accomplishments. I eventually narrowed my reporting down to three schools to illustrate how the big business of high school football has evolved in Georgia. What the newspaper found was a startlingly well-financed support system for the game, one rivaling the model found on big-time college football campuses. Statistics suggest that interest in high school football is waning nationwide, but that certainly isn't true in Georgia. The established programs in southern Georgia continue to drum up huge financial and fan support. But increasingly, the Friday night fever has spread to the wealthier Atlanta suburbs, evidenced by the fancy stadiums - some with their own lounge boxes, spiffy weight rooms and well-appointed booster club headquarters. The final score Georgia may still struggle to shore up a comparatively weak public school academic record, but schools and taxpayers spare little expense to field competitive football teams. Our package questioned whether education is being compromised for playing games. Consider: $79.6 million in state tax dollars went to pay coaches' teaching contracts last year, although nearly a third have minimal or no teaching workload. Georgia taxpayers contribute through local school taxes another $9.6 million annually in supplemental pay for football coaches. Head football coaches make an average salary of nearly 55 percent more than the average for the state's teachers - $55,686 compared with $36,042. The earnings of 10 individual coaches each exceed the $75,724 annual pay for the state's lieutenant governor. Two coaches were found to earn higher salaries than their school principals, putting the schools at odds with certification standards of the Southern Association of Colleges and Schools. (It's likely both principals will be getting raises.) Seven percent of all state coaches are either ineligible to teach or hold conditional or probationary teaching certificates. Many parent-run booster clubs provide cash bonuses to coaches. The more successful coaches are rewarded with vacation trips and the keys to leased vehicles. These football booster clubs routinely operate as non-profits outside the jurisdictions of the schools, and at least one dub was found to have run a foul of IRS guidelines. The investigative package brought a huge readership response, both from the daily publication and the newspaper's Internet site. Some 300 e-mail messages, phone calls and letters showed up in the newsroom. And the newspaper's Internet site / which put up a searchable database of all the football coaches salaries and data - received an average of 1,300 hits an hour during the week of publication. Still today the database gets an average of 800 hits a day, with nearly 1 million searches conducted since the package was published last December. The package is accessible online at www.ajc.com/reportspagedirt Two coaches were foundto earn higher salaries than their school principals As a result of the AJC inquiry, the touchdown club at a suburban Atlanta high school which has an annual budget in excess of a quarter million dollars and which has raised $2.5 million for its new football complex retained an attorney and accountant to get the organization in compliance with state and federal tax laws. The Secretary of State previously dissolved the non-profit organization for failing to file annual reports and the AJC found the touchdown club hadn't filed tax returns with the IRS for at least three years. The package also alerted the IRS to the many taxable perks afforded some coaches from cash bonuses to vehicles to vacation trips. Stumbling blocks Gathering salary and teaching assignment information from all 312 public high schools proved to be a daily battle. Initially, some districts refused to comply or said it would cost hundreds of dollars for them to produce the information. We gathered information right up to the last hour. A lot of data were returned incomplete- either they didn't list coaches' names, omitted their teaching responsibilities or failed to differentiate football earnings from base teaching salaries. We constantly faxed and mailed requests to the districts. Finally, we shipped our data to the individual school districts and had them sign off on the accuracy of what we planned to publish regarding their coaches. The booster or touchdown clubs posed a different problem because they are set up outside the school as private non-profits. We got a grasp on budgets and annual income from reviewing IRS Form 990 tax records. But in almost all cases, it came down to piecing together information from a variety of current and past club officers, coaches and athletic directors. In the end, trying to get reactions from educators and public officials proved a surprisingly daunting chore. Outgoing Gov. Zell Miller, who championed education and promoted the state lottery to fund the state Hope Scholarship Program, said it was too hot a subject to take on in his last days. Lt. Gov. Pierre Howard didn't respond to interview requests. And Linda Schrenko, head of the state Education Department, turned down repeated interview requests on the subject, including a personal visit to her office two days before the articles were published. Tips for new players Next time, we'd allow more lead-time to gather and obtain the relevant documents and records requests. It requires a great deal of diligent follow-up to get a 100 percent response when requesting information from 312 schools. It's also important to clearly define early on what you're seeking in the open records request. We'd probably have been better served to start with a small school sample to get a better idea of what kind of information we could expect to get back. Then we could perfect our request for data relatively easily. Mike Fish can be reached by e-mail at mfish@ajc.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-september-out-to-the-ball-game","slug":"1999-september-out-to-the-ball-game","url":"https://dwillis.github.io/uplink/article/1999-september-out-to-the-ball-game.html","year":1999,"month":"September","monthNum":9,"headline":"'Out to the ball game'","kicker":"LOBBYING LAWMAKERS","author_name":"Kit Wagar","author_title":"database reporter","affiliation":"Kansas City Star","summary":"Investigates lobbyist gifts to Missouri legislators, highlighting the state's lack of restrictions on such gifts and the resulting influence on lawmakers.","keywords":["lobbyists","gifts","Missouri"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel","FoxPro"],"full_text":"Missouri lawmakers are an avaricious lot. Many states prohibit or severely limit gifts from lobbyists on the premise that no one gives you something for nothing. But Missouri is one of 19 states that put no restrictions on lobbyists' gifts. As a result, a spending spree surrounds every legislative session, feeding a culture that sees no conflict in conducting the public's business while on a corporate tab. The one nod the law makes to public interest is a requirement that lobbyists report their spending on individual lawmakers to the state Ethics Commission. Though filled with loopholes, the monthly reports offer a glimpse at the gifts used to appease lawmakers: Cases of wine and beer. Tickets to see pop singer Alanis Morissette and tenor Luciano Pavarotti. Season passes to the St. Louis Cardinals. One lawmaker took $780 in tickets to the St. Louis Blues. Build a database To get a handle on the extent of the spending, the Star built a database of all 3,438 gifts given to lawmakers during this year's 4 1/2-month session. The conclusion: A lobbying technique that began as a bit of clubby salesmanship has evolved into a legislature on the dole. The Starput the master database on its Web site so readers could call up the gifts each lawmaker received. The analysis determined that lobbyists gave individual lawmakers gifts worth $139,219. That includes only one-on-one gifts and does not include the money spent on the constant stream of receptions, parties and meals for groups of legislators. The top recipient personally received gifts worth $5,843, or nearly $1,300 a month. Only once during the entire session did he go a week without taking something. The analysis also showed that: House Democrats took 63 percent of the gifts; 30 percent of the gifts involved tickets, lodging or transportation; One in four lawmakers took more than $1,000 in gifts; Three lawmakers received more than $1,000 in freebies from a single lobbyist. The analysis was done using Excel 5.0 and FoxPro 2.6. The first problem was the volume of the gifts, which are listed only on handwritten reports. The newspaper hired two people to enter the early data, but interns and reporters had to finish it. Start with Excel The resulting spreadsheet listed the lawmaker, his or her title and party, the lobbyist, the date of the transaction and a description of the gift and its value. The biggest weakness in the data was the way lobbyists described their gifts. While some lobbyists listed \"beverages,\" other lobbyists listed the same gift as \"food and/or beverages.\" Others combined freebies, describing them as \"tickets/parking/meal.\" Move to FoxPro I used Excel's Filter command - the Extract command in earlier versions to divide the gifts into different categories, and pasted them into other spreadsheets in the same workbook. I eventually combined them into five general categories: food & drink; tickets; lodging, transportation and entertainment; gifts; and all other. I then went through each category to eliminate redundancies. Gifts described as two things were picked up twice, others three times. To eliminate those, I sorted the spreadsheets by lawmaker, date and amount so the same records would be in adjacent rows. I then used a logical function to find rows that exactly matched the row next to it. After eliminating the redundant records, I saved each of those spreadsheets and the master list in Excel 4.0. Then I imported them into FoxPro databases. That allowed me to total all the gifts by lawmaker, lobbyist, party, chamber or date. The biggest expenditures were on the first day of the session; many of the other big days came in the last three weeks of the session. One thorny problem was revisions in the records. State law allows lawmakers to have the record of the gift expunged if they dispute the expenditure or repay the lobbyist. The paper began building the database in March. By May, a few dozen gifts were no longer listed in the records. We decided to base the articles on the final versions. One representative, for example, had taken his family on a lobbyist-paid, four-day vacation to Branson, Mo., in March. After he was interviewed about the trip in June, he repaid the lobbyist, so the trip was not listed on the final reports. The entire project took a little more than three weeks - a week of data entry, three days to crunch the numbers and about eight days to report and write. The Star put the master database on its Web site so readers could call up the gifts each lawmaker received. Three lawmakers received more than $1,000 in freebies from a single lobbyist. All the lobbyists and the recipients of large amounts insisted that the gifts had no effect on legislation. Then why, the biggest recipient was asked, do lobbyists spend all that money? \"What kind of stupid question is that?\" he replied. \"I'm not going to answer that. How would I know?\" Kit Wagar can be reached by e-mail at kwagar@kcstar.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-september-tipsheets-on-crime","slug":"1999-september-tipsheets-on-crime","url":"https://dwillis.github.io/uplink/article/1999-september-tipsheets-on-crime.html","year":1999,"month":"September","monthNum":9,"headline":"Tipsheets on crime","kicker":"","author_name":"","author_title":"","affiliation":"Star Tribune","summary":"This article provides guidance on crime reporting resources, including a Star Tribune investigation on repeat offenders and multiple tipsheets covering crime mapping, investigation techniques, and court coverage.","keywords":["crime","reporting","investigation"],"topics":["Crime & Justice","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"You can read the full series on repeat offenders published by the Star Tribune at www.startribune.com/news/projects/\n\nFor further reporting on crime, check out the following tipsheets:\n\nTipsheet #858, \"Mapping crime,\" from the 1999 National Computer-Assisted Reporting Conference\n\nTipsheet #1059 from the panel \"Quick Hits: How to Develop Fast investigative Stories on City Hall, Crime and Education\" at the Unity '99 Conference\nTipsheet #874, \"Furloughs Linked to Crimes,\" from the 1999 National CAR Conference\nTipsheet #778 from the panel \"Covering Cops and Courts\" at the 1998 National CAR Conference.\nAll tipsheets are available online at www.ire.org/resourcecenter","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-september-tipsheets-on-lobbying","slug":"1999-september-tipsheets-on-lobbying","url":"https://dwillis.github.io/uplink/article/1999-september-tipsheets-on-lobbying.html","year":1999,"month":"September","monthNum":9,"headline":"Tipsheets on lobbying","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"This article directs journalists to IRE tipsheets that provide government contact information and resources for researching lobbying activities, ethics laws, and lobbyists' involvement in elections across different jurisdictions.","keywords":["lobbying","government","tipsheets"],"topics":["Government & Politics","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"Take a look at Tipsheet #116, a one-page source list of government office contacts that have Information on lobbying activities.\n\nTipsheet #100 provides state-by-state listings of agencies that administer ethics and lobby laws.\n\nAnother helpful tipsheet regarding lobbyists is Tipsheet #923, \"Lobbyists and local elections,\" from the 1999 National CAR Conference.\n\nAll tipsheets are available online at www.ire.org/resourcecenter","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-september-pac-power","slug":"1999-september-pac-power","url":"https://dwillis.github.io/uplink/article/1999-september-pac-power.html","year":1999,"month":"September","monthNum":9,"headline":"PAC Power","kicker":"CAMPAIGN FINANCE","author_name":"Jeff Porter","author_title":"Staff Writer","affiliation":"Arkansas Democrat-Gazette","summary":"Looks into how high school football programs are financially maintained in Georgia, involving taxpayer money, booster clubs, and coaching salaries.","keywords":["finances","taxpayer","football"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["FoxPro","floppy disk"],"full_text":"Most political action committees are lit- erally - big business. In our case, the best way to make use of PAC data is \"joining\" - that is, using a computer to match up material in separate data files. Staff Writer Noel Oman alerted us that for the first time, the Arkansas secretary of state's office could provide electronic data for PAC contributions. The General Assembly was in session - perfect timing for a story. The result was a fairly quick-hit story, written by Oman and accompanied by graphics, for Arkansas Democrat-Gazette readers with more details than ever reported about PAC contributions to Arkansas lawmakers. Free data But we needed more information. We obtained data kept by legislative staff, including information on lawmakers and their committee assignments. To their credit, both arms of state government - the secretary of state's office with PAC data and the Arkansas Bureau of Legislative Research with lawmaker information - had no problem handing over the data tables on diskettes. Cost: nothing. Getting the files was just the beginning. We had to put things together, wanting to accomplish several things. First, we wanted to identify the biggest PAC contributors, so we asked Visual FoxPro to show a total of contributions from each PAC. Instantly, we saw the biggest players. We also examined the top recipients. Thanks to Oman's experience and an electronic file from the secretary of state's office, we could identify PACs related to one another, representing the same companies or business interests. Now the fun began: trying to match up lawmakers who got PAC money and their committee assignments. This year's legislative session had issues including electric utility deregulation, telemarketing restrictions and highway spending and tax plans. So we wanted to look at both contributors and recipients on key committees. Matching PACS to committees Here's a list of things we did: First, in both PAC data and legislative membership data, we broke the files in two, creating separate files for each legislative chamber. Why? Because we planned later to match PAC contributions with committee membership tables, using two factors to make the match: the lawmaker's last name and the district number the lawmaker represents. In just a few cases, Senate and House members might have the same district number and the same last name. Say, Rep. Brown and Sen. Brown with identical district numbers, even though they serve in different chambers. We obtained data kept by legislative staff, including information on lawmakers and their committee assignments .both arms of state government ... had no problem handing over the data tables on diskettes. That gave us a list showing how much PAC money legislative committee members received. With those files, we could easily pull up, examine and analyze contributions from various PACs to specific committee members. One caveat for this file that we had to observe: the contributions were repeated, because lawmakers had more than one committee assignment. But since for this file we wanted to break contributions down by committee, it's exactly what we needed. We used the previous files for looking at lawmaker PAC contributions overall. The results We found the usual \"suspects\" and no hidden skeletons. However, Oman's reporting, accompanied by graphics, did inform and educate readers. Here's a clip: Bob Balhorn sat in the audience last Wednesday when the House Insurance and Commerce Committee took up a bill to enable Arkansans to ban telemarketing calls to their homes. His was not just another face in the crowd. Balhorn, 63, of Little Rock is executive vice president of the Arkansas Realtors Association, the state affiliate of the National Association of Realtors. It is a post he has held since 1985, a tenure that covers eight regular sessions of the Arkansas General Assembly. Before that, the native Texan held a similar post in South Dakota, where he also sold real estate. His association, 6,000 members strong, does not immediately come to mind when the most powerful special interests at the state Capitol are assessed. But in the 1998 elections, no other single trade organization or corporation gave more to candidates for state office in Arkansas, according to an Arkansas Democrat-Gazette review of political action committee reports at the Secretary of State's office. The two committees registered to the association gave $78,800 to candidates for state offices in 1998. Of the total, $72,800 went to legislative candidates, the given to legislative office-seekers by any state-registered political action committee. The association's generosity in 1998 was no anomaly. In the 1996 election, the Realtors' association gave $63,350 to sitting lawmakers, second only to the nursing home industry, which reported giving $84,450. The nursing home industry's state umbrella organization, the Arkansas Health Care Association, reported giving $57,000 in 1998. And for a plus, we placed the PAC contribution file on our intranet, The Source, so all our editors and reporters could easily look up contributions for state candidates from the 1998 campaign season. Jeff Porter can be reached by e-mail at jporter@ardemgaz.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-september-tipsheets-on-campaign-finance","slug":"1999-september-tipsheets-on-campaign-finance","url":"https://dwillis.github.io/uplink/article/1999-september-tipsheets-on-campaign-finance.html","year":1999,"month":"September","monthNum":9,"headline":"Tipsheets on campaign finance","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"A collection of journalism tipsheets on campaign finance research provides guidance on accessing resources from the National Institute on Money in State Politics, analyzing campaign finance data to uncover relationships between candidates and contributors, and conducting online searches for campaign finance information.","keywords":["campaign","finance","resources"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["World Wide Web"],"full_text":"Check out the following tipsheets on campaign finance:\n\nTipsheet #992, \"www.followthemoney.org,\" from the 1999 IRE National Conference, provides a list of resources available from the National Institute on Money in State Politics.\n\nTipsheet #937, \"Campaign Cash: Is it better to give than receive?\" from the 1999 National CAR Conference suggests ways to look at campaign finance data and discover relationships among candidates, contributors and issues.\n\nTipsheet #827 from the panel \"Uncovering the secrets of campaign finance and lobbyists\" at the 1998 IRE National Conference gives tips on online searching for campaign finance information.\n\nAll tipsheets are available online at www.ire.org/resourcecenter","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-september-sas-vs-spss","slug":"1999-september-sas-vs-spss","url":"https://dwillis.github.io/uplink/article/1999-september-sas-vs-spss.html","year":1999,"month":"September","monthNum":9,"headline":"SAS vs. SPSS","kicker":"TECH TIP","author_name":"Jessica Larson","author_title":"NICAR Staff","affiliation":"NICAR","summary":"This article compares SPSS and SAS statistical software programs used in computer-assisted reporting, explaining that SPSS is more user-friendly and affordable for beginners while SAS offers greater power and flexibility for advanced users with larger budgets.","keywords":["SPSS","SAS","statistics"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["SAS","SPSS"],"full_text":"There is but one argument in computer- assisted reporting that rivals that of PC versus Mac. It gets blood boiling and inspires people to write opinionated tomes on the NICAR-L listserv. It's the debate over the merits of SAS versus SPSS. Users of the statistical programs each trumpet the advantages of their pet program (\"SPSS is easier!\" \"SAS is more powerful!\") but occasionally one can hear them muttering over the disadvantages inherent in each: \"SPSS can't do what I want it to.\" \"SAS is just too expensive.\" So which is the better program? The experts say it really depends on your needs in your newsroom. The pluses According to most, SPSS is usually best for beginners in CAR. One important factor is its relatively low cost - about $1000 for commercial use, half that for academic use and even less for individual students (both graduate and undergraduate versions are available). It also requires less complex commands and is quick to learn. Philip Meyer, Knight Chair and professor of journalism at University of North Carolina-Chapel Hill, is confident in the program's usefulness for CAR work: \"SPSS will handle every statistical problem that I can think of that a journalist might encounter. Including some weird-science stuff like logistic regression and factor analysis.\" Finally, SPSS is easy to understand Meyer calls this factor \"transparency.\" \"Transparency means being able to see right through the complicated computer operations as though they weren't even there. All you see is your input and the output. Another expression for transparency is 'ease of use, he says. \"Every computer system designer has to choose between transparency and flexibility. SPSS leans toward transparency, SAS toward flexibility.\" Which brings us to SAS's main advantage: it can do more than SPSS, a benefit that the SAS users tout, though their voices may be drowned out by those of the more numerous SPSS users. SAS can bring in a number of different data types, and it creates detailed logs each time programs are run, making it easier to dig out any bad data. Because of its ability to handle very large data sets, oftentimes SAS is used by the professional agencies that supply reporters with data. Steve Doig, professor at the Cronkite School of Journalism at Arizona State University, describes SAS as \"extremely powerful and full-featured. \"You just won't outgrow this, Doig says. The minuses Despite their preferences for one or the other, users of both programs are honest about the disadvantages inherent in each, which are sometimes related to the advantages. SAS's power, for example, comes at a cost - financial as well as the time required to learn the more complex procedures. \"You have to write programs,\" Doig says. \"There are pulldown menus, but they don't do many of the programming functions that the SPSS interface does.\" Also, the monetary cost can be prohibitive, especially for smaller newspapers or television stations. A license must be purchased to run SAS, rather than just a neat, cellophane-wrapped package of software. A first-year license fee for a basic SAS program will run you $2,200-2,300. The yearly renewal cost, though, is much cheaper - about half the initial amount. Any stats modules licensed on top of that basic program cost an extra $1,100-1,200. But the base program, which Doig says can handle chi-squares, frequency tables, etc., should be sufficient for most work. Expense and complexity both limit the number of CAR reporters who become SAS users, which might be a problem if you want to ask questions about your work. But Doig says there is help available: \"We few SAS zealots are so tickled when someone does want to use it that we're generally happy to chime in.\" SPSS is cheaper, but has its own drawbacks, It can't handle large data loads as SAS can, nor can it do complex file-management tasks as well. A few NICAR-L complaints about SPSS have included its limited ability to handle text fields, a lack of some key diagnostics for basic statistical procedures and less efficient data cleaning than SAS. Though he recommends SPSS for beginners in CAR, Meyer acknowledges its shortcomings in terms of flexibility. \"Bottom line: you can do especially file-management tasks, with SAS,\" he says. \"For those with the guts to stick out the SAS learning curve, it will last them for- ever. Those fainter of heart should start with SPSS. They can always get an outside contractor to do the slice-and-dice chores.\" What's right for you? In the end, it all boils down to a few basic questions: a) How much intensive statistical work do you need to do? b) How much statistical experience do you have? How much can you or your organization afford? If you answer \"A lot\" to all three, SAS may be your best bet. If it's \"Not much\" across the board, SPSS is probably more suitable. Any combination of the two responses means you need to weigh the pros and cons of each and maybe post a question or two to NICAR-L in hopes that a fervent SAS or SPSS user can sway you to one side or the other. Jessica Larson can be reached by e-mail at jessica@nicar.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-september-returning-to-regress","slug":"1999-september-returning-to-regress","url":"https://dwillis.github.io/uplink/article/1999-september-returning-to-regress.html","year":1999,"month":"September","monthNum":9,"headline":"Returning to regress","kicker":"STATS AND SUCH","author_name":"Neill A. Borowski","author_title":"","affiliation":"","summary":"This article explains key concepts in regression analysis, including how to handle outliers, interpret t-tests, and make data transformations, while emphasizing the importance of understanding the underlying mathematics rather than relying on computer programs as a 'black box.'","keywords":["regression","outliers","analysis"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Excel"],"full_text":"This is the second of two parts on how to do a regression analysis. This issue's column deals with regression outliers, the t-test and making transformations. In the last issue, the main advice was not to succumb to the \"black box\" syndrome - being awed by the ease of running regression analyses through computer programs with no appreciation for what is going on behind the curtain. The math behind the regression process was explained in detail. The regression model seeks to fit a line through - or close to - all of your data points. In a plot of the data, the x-axis holds the independent variable and the y-axis holds the dependent variable. That line is one that you might recall from high school algebra: y = ax + b + random error. The \"y\" is your dependent variable (such as test scores). The \"x\" is your independent variable (such as poverty rates). The \"a\" is the slope of the line and the \"b\" is the constant (the intercept of the vertical axis). This is the equation of the regression line. You may have seen the equation stated in different ways. Some statistics texts prefer the notation of y = bx + a. Ory=mx+ b. Or, even fancier, the equation y + Blx E Select whichever you're most comfortable with. The goal in regression is to show how much the dependent variable is related to the independent variable. This quest to find the truly predictive model is one undertaken by social scientists, physicists, epidemiologists, quality-control managers and others in nearly every field. If you're interested in practicing regression, there are plenty of free, downloadable datasets available on the Internet. Two great sites are the Journal of Statistics Education Data Archive at www.amstat.org/ publications/jselarchive.htm and the Data and Story Library at http://lib.stat.cmu.edu/DASL/, where you'll even find datasets covering acorns and oak trees and passenger car mileage. In most cases, the size of the dataset is small, making regression with MS Excel and other spreadsheet programs relatively easy. Here are some points to remember in the rush to regress. Dealing with outliers An outlier is a data point (observation, record, etc.) that represents an extreme value when compared with other values in the data. Outliers can be gremlins in all data analysis, whether you're calculating the mean or running a complex regression. If you're going to be \"hanging your hat\" on your regression results understand the t and the tests. If you don't pay attention to these tests, your whole hypothesis may be rejected without your knowledge. The simple way of thinking about an outlier is in terms of its effect on the average. Consider a study of speed in fatal automobile accidents. The estimated speeds of the cars involved in such accidents on one stretch of road in one year are 65, 55, 95, 60 and 65. The average speed is 68. Yet, all but one speed is less than 68. The 95-miles-per-hour speed skews the data. The average without the extreme value is 61 - certainly more representative of the group. Extreme values or outliers also can affect the outcome of your regression analysis. Many of us have seen the outliers in a scatterplot of test scores, with the percent of students eligible for free lunch as the independent variable. We see relatively poor schools with unusually high grades (the \"overachievers\") and relatively wealthy schools with unusually low grades (the \"underachievers\"). In these cases, the news may be in the outliers. What secrets might you find in the poor schools that do well on tests? However, a common practice in regression analysis is to throw out the outliers to try to achieve a higher coefficient of determination (r2). With the outliers included, maybe your r2 is 0.53, meaning that 53 percent of the variability in test scores is explained or predicted by the variability in income. By eliminating the outliers, your r2 may increase by several points - making you feel even more confident about a relationship. How far off are some data points? In SPSS, SAS and EXCEL, for example, you can save and study the residuals, which are the differences between the actual value of each observation and the predicted value from your regression model. You also can calculate the residuals by hand by substituting each \"x\" into the regression equation to solve for \"y\" (y = ax + b). Then find the difference between the actual \"y\" (for example, the test score) and the predicted \"y\" from the regression equation. (You can visualize residuals in a scatterplot of your data by drawing vertical lines from the datapoints to the regression line. The lengths of the vertical lines represent the residuals.) Watching the t-test The t test and the F test are part of the regression output in statistical programs and Excel. Yet, many people may pay no attention to these tests in their zeal to achieve the highest-possible r2. Pay attention to these tests, which report both the t and the F values, as well as their significance levels. These tests will indicate if there is a serious problem with the regression model. The ideal situation is to see \".000\" or something close to zero in the significance box. If you see much more than zero, this is cause for concern. If you're going to be \"hanging your hat\" on your regression results, spend some time with statistics texts to understand the t and the F tests. They are not difficult to understand, but they are involved. If you don't pay attention to these tests, your whole hypothesis may be rejected without your knowledge. Making transformations If you plot the data and find there are curves or bends in the fitted line, you may want to express the data differently, or \"transform\" it. I remember discussing this process with a colleague at an IRE and NICAR seminar and he gasped at the thought of changing the data. However, transformations are uniform and change all of the data by the same amount-much like multiplying both sides of an algebra equation by 2 to get rid of the 1/2 on one side of the equation. Depending on the severity and direction of the curve in the fitted line, you may want to transform either the independent (\"x\") or the dependent (\"y\") variable. Methods include substituting the square or square root of \"x\" or \"y\" or taking the natural logarithm of one of the variables. The goal is a better fit - a higher r2 and a straighter regression line. Clearly, there is a lot to know about regression analysis. It takes more than simply running a regression using SPSS or Excel and relying on each program's online Help facility to explain what you are doing. Pick up a statistics text or two - particularly those used in graduate school. Many college statistics departments have put up Web sites that explain the basics. Some actual statistics classes are online and available to the public. A good place to start is in Yahoo at http://dir.yahoo.com/ SciencelMathematics/Statisticsl But, most important of all, know that there are a lot of gears whirring in the black box and that it's worthwhile understanding a bit about how they mesh (or fail to mesh). Neill Borowski can be reached by e- mail at nborowski@phillynews.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-september-sick-slackers","slug":"1999-september-sick-slackers","url":"https://dwillis.github.io/uplink/article/1999-september-sick-slackers.html","year":1999,"month":"September","monthNum":9,"headline":"\"Sick\" Slackers","kicker":"FIRST VENTURES","author_name":"JAMES H. CAMPANINI","author_title":"reporter","affiliation":"","summary":"A journalist uses computer-assisted reporting to investigate teacher absenteeism in Lowell schools, discovering data processing errors and systemic issues hidden within 20,000 school records that led to a three-part exposé on sick-day abuse and its impact on students.","keywords":["absenteeism","investigation","data"],"topics":["Investigative Methods","Education","Data & Databases"],"technologies":[],"full_text":"When the president of the United Teachers of Lowell denounced on local radio my investigation into teachers' absenteeism, saying, \"You can interpret the numbers to say whatever you want,\" he reminded me of similar words spoken by IRE's executive director Brant Houston. At NICAR boot camp, Houston repeatedly warned journalists to do the legwork and check for the story behind the numbers. I chuckled as the union president spun his damage control. My computer-assisted reporting training had helped produce an ironclad three-part series that turned never-seen-before records into a powerful exposé on teachers' sick-day abuse and its impact on students. I found data processing errors in nearly every file I received from the School Department It's always easy when it's over, but as an editor-turned-CAR reporter I faced the constant pressure of an editor breathing down my neck to get the project done. After every computer analysis update he would declare, \"OK, you've got the numbers, can't you write it now?\" But the numbers were only the lead-in to the story. The numbers, in fact, were a Rosetta stone. As I deciphered their meaning it became clear there was more to teacher absenteeism than just sick days. I had to examine budget issues, the contract, sick leave, accumulated sick leave (teachers can pile up a bank of unused sick days from year to year), and reasons for the absences. Only then could I put into context the statistics from 20,000 Lowell School Department records that threatened to overwhelm me. It didn't help that I was doing this project at home after my regular workday, since the newsroom lacked the necessary computer and software resources. Moreover, my editor's frustration grew as I put more energy into the CAR investigation. The genesis of the project came in casual conversation with the Lowell City auditor in January. He said he was processing 70 to 100 paychecks a week for substitute teachers - the most in his two decades on the job. I then filed a Freedom of Information Act request, one of four FOIAs involved in the project, for the substitute payroll accounts. I found that for 1997-98, Lowell had spent $1.2 million on substitute teachers. I was curious, and that's when I began examining the system of teacher absences. Three months into the project, when I began reporting, I commandeered two general assignment reporters, Jennifer Fenn and Heidi Perlmann, to help conduct interviews and write sidebars. Fenn uncovered 3,000 classroom absences caused by professional development workshops that the School Department had failed to document. Along the way, I adopted a mantra that aided my reporting and writing and led me to the promised land of publication: how does the system work? Titled \"Absent With Leave: Who's Teaching Your Child Today?,\" the three-part series took four months to complete and chronicled teachers' sick-leave abuse and a union contract that helped fuel it, the lust for professional development grant money that literally pushed teachers out of the classrooms and an overall lack of administrative oversight. It concluded that teacher absences were straining the Lowell Public School System financially and taking a toll on the educational opportunities of its 16,000 students. 'Rithmetic The investigation covered the 1997-98 Lowell school year, which was the most recent year for which statistics were available. The major findings were: Lowell's 1,138 teachers had accounted for 16,224 classroom absences, for an average of 14 days each over the 180-day school year. The highest teacher absenteeism due to sick leave occurred on Mondays and Fridays. The days before and after holidays and school vacations also reflected high absentee rates. Teachers called in sick two and three days in advance of their purported illness. Attendance records received little or no School Department scrutiny. Professional development workshops for teachers accounted for more than 3,300 documented classroom absences, with as many as 60 teachers out of the classroom on any given day. The School Department paid out $3.2 million in lost wages to teachers and was forced to hire 10,540 substitutes at a cost of $1.2 million, or $215,000 more than was spent to purchase textbooks for the entire school system. The School Department couldn't keep up with the demand for substitutes, with 1,433 classes going uncovered due to teacher absences. This forced students to double up with other classes, oftentimes eliminating their original course of study, and furthered a dilution of regular instruction. Writing I tried not to make the investigation a numbers story, and I got help in putting teacher absenteeism in perspective by reading articles from the ERIC Database. The writing was reduced by 11 illustrative charts, all but one compiled by using Excel. The most compelling, however, was a full-page calendar of the school year showing the daily breakdown of teacher absences for sick leave, personal days and professional workshops. This was done through Access. A fever chart that revealed the spikes of absences on Mondays and Fridays, as well as holidays and vacations, accompanied the graphic. I showed a preliminary version of the calendar graphic to all my interview subjects, and it was effective in getting them to comment. They couldn't duck what they saw in black-and-white. The first part, published June 13, ran 100 inches and highlighted the patterns of abuse associated with teachers' sick leave. It had a sidebar disclosing that Lowell teachers had accumulated more than $21 million in unused sick-day pay which was an uncompensated liability facing the city. By contract, the city was required to buy back the sick days at 33 percent of their value when teachers retired, meaning taxpayers were on the hook for a budget-busting $7 million. As an editor-turned-CAR reporter I faced the constant pressure of an editor breathing down my neck to get the project done. After every computer analysis update he would declare, \"OK, you've got the numbers, can't you write it now?\" Part Two focused on reaction from School Department officials and the union president, offering details on how the teachers' contract had tied the hands of principals to deal effectively with sick-leave abuse. The contract allowed each teacher 15 paid sick days and two paid personal days a year without excuse. Part Three dealt with the professional development program, which was largely built on state and federal grants. It uncovered the School Department's neglect in evaluating the effectiveness and efficiency of the workshops teachers were attending; $17 million in grant money had forced school officials to adopt a \"use it or lose it\" philosophy. Research The teacher absenteeism investigation was my first in-depth computer-assisted reporting project since my January \"graduation\" from NICAR boot camp. As a senior editor who hadn't authored a major piece since the Reagan administration, the project tested my research and writing skills, both of which were rustier than Noah's nails. But my CAR convictions were greater than my doubts and deficiencies. Instead, the biggest obstacle I faced was my own skepticism as to my self-taught computer skills. As a CAR novice I didn't trust my computer skills. I agonized over making a mistake that would taint the analysis. But I proceeded slowly, beginning the first week in February and working 2-3 hours a night for three months. Overall, I spent 90 hours researching 20,000 School Department records, both electronic and paper. The computer records amounted to four disks stored in three different software programs, none of which matched up with a common identifier for a clean import and merge. The key information - a transactions file for every documented teacher absence - was stored on an old Wang VS system which I got converted to an ASCII file. I found data processing errors in nearly every file I received from the School Department, the most glaring involving 1,014 teachers who were inadvertently listed as missing school for a religious holiday when they were in the classroom. In the end, I had to build a database in Access to contain the information I needed. I also relied heavily on Excel to build an updated assignment roster that gave me the whereabouts of every teacher in Lowell's 28 public schools and what they taught. This helped me later when I cross-checked absênces by individual schools and by teacher specialization. It led to two explosive charts, one ranking the schools by absences and the other ranking teachers as to their specialization (fourth-grade teachers averaged 17 sick days a year, the most of any group). Each night I printed out reams of analysis and brought them to the office the next day where two newsroom secretaries and available copy editors checked them. If something didn't add up, that night I was back on the computer trying to figure it out. Results There was no shortcut to the final conclusions. I had to study the teachers contract, pester the city's chief labor negotiator and interview two dozen school officials, including administrators from outside Lowell who offered valuable insights as to what patterns of abuse I should be looking for. In addition, I had to dispel my editor's perception that I was punching computer keys without a sense of purpose. In the end, we both learned that CAR projects require trust and patience. James Campanini can be reached by e- mail at jcampanini@lowellsun.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-september-healthcare-by-color","slug":"1999-september-healthcare-by-color","url":"https://dwillis.github.io/uplink/article/1999-september-healthcare-by-color.html","year":1999,"month":"September","monthNum":9,"headline":"Healthcare by color","kicker":"WHEN PREJUDICE KILLS","author_name":"Dave Davis","author_title":"","affiliation":"The Plain Dealer","summary":"A journalist investigates racial disparities in death rates from preventable and treatable diseases, finding that Black men die at nearly three times the rate of white men, using CDC mortality data to examine the gap across different illnesses and demographics.","keywords":["mortality","disparities","healthcare"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["CDC Wonder system"],"full_text":"Bernard Hedge was screwed. I knew it the moment I laid eyes on him. The former bodybuilder was stuffed into a wheelchair in a two-room apartment with barely enough room for him to maneuver. His kidneys were failing. His blood pressure and diabetes were uncontrollable even with the dozens of pills he took each day. \"Life is not guaranteed for any of us,\" Hedge told me. \"I could live another year, five years, 10 years. Nobody knows.\" Within a few months of the publication of my stories on the gap in death rates between whites and blacks, he would lose a leg to the diabetes. Then, a few months later, like too many black men, he'd lose his life early. He was 55. Death rate data I had decided to examine the gap in death rates for preventable and treatable diseases after a nurse friend introduced me to Hedge and after hearing Dr. Patrick Remington, an epidemiologist at the University of Wisconsin School of Medicine, speak at an IRE medical reporting conference. Hedge was screwed, but he wasn't pathetic. His struggle to finish his life with dignity impressed me. Our stories found that black men were dying from these diseases in astonishing numbers.. rate nearly three times greater than white men. Remington introduced me to an invaluable resource for mortality data - the Centers for Disease Control and Prevention online \"Wonder\" system (http://wonder.edc.gov). With it, I was able to calculate death rates by race, gender, and disease for every county and state in the nation. With it, I learned just how screwed Bernard was. After experimenting with Wonder, I decided to look at death rates for people under 65 years old suffering from seven treatable or preventable diseases. The idea was that these diseases - female breast cancer, cancer of the cervix and uterus, pneumonia, prostate cancer, colorectal cancer, high blood pressure and diabetes - might be a crude measure of people's access to health care, or lack of access to it in this case. Our stories, published by The Plain Dealer in May 1998, found that black men were dying from these diseases in astonishing numbers, with more than 140 deaths for every 100,000 black men, a rate nearly three times greater than white men. The death rate for black women was double that of white women. For some of the illnesses, the death rate for blacks was now where it was for whites two decades ago. And the gap between the races was widening. Pursuing prejudice A short time after my stories came out, I was blown away by a five-day series by Ford Fessenden - then of Newsday and now of The New York Times - that examined many of the same issues, but in much greater detail. Ford had attended the same IRE medical reporting conference I had. His eight-part series, \"The Health Divide,\" published in Newsday in November and December of 1998, is available from IRE and is a must-read for anyone interested in health care reporting. For my stories, I also used death certificate data that I purchased for $100 a year from the Ohio Health Department. I found these records (one for every person who died in the state that year) useful for finding relatives of people who died of specific illnesses. In the past, I've also used them to analyze C-section rates at hospitals across the state, find premature babies who died in hospitals not properly equipped to care for them, and uncover deaths at dangerous heart cath labs, among other things. According to the National Center for Health statistics, 19 states make death records publicly available. They are California, New Jersey, Connecticut, Florida, Georgia, Iowa, Kentucky, Maine, Massachusetts, Michigan, Minnesota, North Carolina, Ohio, South Carolina, South Dakota, Vermont, Washington and Wisconsin. Most of them also make birth certificate records available, too. For all but eight states, some restrictions apply or state officials will make you jump through some hoops to get the data, such as producing a research proposal on how you'll use the records. But I think you'l find the records worth the trouble. Vital Resources for Vital Statistics: National Center for Health Statistics: (301-436-8500). A great resource for reporters working with vital statistics data. National Association of Health Data Organization: (703-532-3282). Keeps tabs on which states make available vital statistics and hospital discharge data and how the public can get it. Centers for Disease Control and Prevention's online \"Wonder\" data system: (http://wonder.cdc.gov). Allows you to run queries on mortality data for counties, states, or the nation. You can e-mail the results to yourself and then import the data into a spreadsheet for sorting and making nifty graphics. Dave Davis can be reached by e-mail at ddavis@plaind.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-september-bits-bytes-and-barks","slug":"1999-september-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1999-september-bits-bytes-and-barks.html","year":1999,"month":"September","monthNum":9,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"This journalism newsletter announces upcoming Computer Assisted Reporting (CAR) training seminars in Columbia, Missouri covering database design and mapping software, along with information about an IRE regional conference, job listings, and updated hazardous materials data.","keywords":["training","data","journalism"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["FoxPro","Access"],"full_text":"Bits, Bytes and Barks\n\nUpcoming advanced CAR training\n\nIntermediate CAR Camp Sept. 17-19, 1999 - Columbia, Mo.\n\nThis seminar will cover data cleanup, database design, and construction of front-ends. We will use FoxPro and Access for our software while concentrating on the principles that can be used in database manager software. Actual data will be used. This is an intense, hands-on seminar for those who have mastered querying in database managers and have some knowledge of building databases.\n\nMapping CAR Camp Oct. 21-24, 1999 - Columbia, Mo.\n\nThis three-day seminar will offer intensive hands-on training using mapping software for stories. We will also look at past stories that have used mapping programs. The sessions will include address geocoding, thematic mapping, overlaying and spatial analysis.\n\nAll lessons are based on government data. Participants are asked to have a basic knowledge in using relational database programs such as Access or FoxPro. Participants are also encouraged to bring local data to work on during open lab time.\n\nIRE Regional Conference\n\nIRE's next Regional Conference will be in Washington D.C. on Jan. 14-16. There will be more than 50 panels featuring experts from print and broadcast sharing tips and techniques for better stories. The NICAR staff will hold daily sessions on CAR topics in the Computer Demo Room. Watch www.ire.org/training/conferences for more details.\n\nNew job?\n\nIRE's Job Center (www.ire.org/jobs) lists some of the hottest jobs in journalism, especially news reporting, editing, producing and researching Employers like the fact that they reach the best in the business through this site - and they have the space to offer lots of details at no extra cost. Also, take a look at the latest in fellowships, scholarships and grants aimed at journalists and journalism educators.\n\nNICAR Data Update\n\nAn update of the following data set is now available through IRE and NICAR's data library. Prices and years of all available data sets can be found at www.nicar.org or by calling (573) 884-7711.\n\nThe Hazardous Materials Incident Report Subsystem database (Hazmat) now extends from 1970 to 1998. It includes nine tables with incident reports of unintentional releases of hazardous materials for all modes of transportation.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-kennedy-crash","slug":"1999-october-kennedy-crash","url":"https://dwillis.github.io/uplink/article/1999-october-kennedy-crash.html","year":1999,"month":"October","monthNum":10,"headline":"Kennedy crash","kicker":"CAR HELPS UNCOVER THE DETAILS","author_name":"Andy Lehren","author_title":"","affiliation":"Dateline NBC","summary":"Following JFK Jr.'s 1999 plane crash, aviation databases provided crucial context for understanding the accident, revealing spatial disorientation as a likely cause and highlighting broader safety patterns in general aviation. The incident demonstrated both the value of in-house data analysis for accurate reporting and the importance of using databases as a starting point rather than an end point for journalism.","keywords":["aviation","crash","data"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"When John F. Kennedy Jr.'s plane went down July 16, aviation experts had a good idea what went wrong.\n\nAt the same time, aviation databases were invaluable for providing context, breaking news and getting facts right.\n\nThe tragedy also offered another lesson for journalists that Web sources are still not as good as having the data in-house, and that we must use the data as a starting point for more reporting, an end in itself.\n\nA quick recap: A pilot, not yet rated to fly just using instruments, crashed amid fog at night on his way to Martha's Vineyard. The plane, a single-engine Piper Saratoga, was found off the New England island. Dead were the celebrated son of the president, along with his wife and her sister.\n\nBy noon the next day, aviation experts offered a good estimate about what went wrong. Experienced pilots, former officials from the National Transportation Safety Board, and others familiar with general aviation outlined how Kennedy seemed to have lost his bearings. In the night and fog, he could have become disoriented and unwittingly steered his plane into his grave. If he were rated to fly using instruments, he would have been better able to navigate without any reference points below. Those reference points - lights, the horizon, buildings, and other landmarks - are keys for non-instrument rated pilots. While the NTSB has yet to determine the probable cause, key evidence has not detracted from this outline. Radar showed a sharp descent, the landing gear was retracted, and the propeller was still turning - all signs that he lost his way rather than suffered from a mechanical problem.\n\nKennedy was not the first to suffer from spatial disorientation. Since 1995, at least two other pilots flying the same kind of plane crashed with the NTSB later listing spatial disorientation as the main probable cause. Two died June 7, 1996, in Goleta, Calif. (the data allowed us to track down footage of the aftermath), and one died Aug. 26, 1996, in Medford, Minn. In fact, most of the time, pilots are the ones responsible for crashes in that type of airplane, and the numbers mirror the pattern in general aviation.\n\nNTSB data provided a wealth of information about those who have died in this kind of plane, and the number of crashes (more than two dozen fatal crashes during the last three and a half years). The data allowed comparisons of crashes involving instrument-rated versus visual flight-rated pilots.\n\nOne key is to know that any analysis should begin by looking at an airplane's group. Boeing 737's, for instance, are a group. In this case, the group is known as PA-32. Aviation experts, including New Piper's own safety head, who is a former NTSB official, confirm this is the right way to begin. The number of fatal crashes, and the causes, allowed Dateline NBC to pose more pointed questions about general aviation safety, and gave viewers a more detailed context for Kennedy's crash. Dateline NBC, from what I've been told by other reporters and from what I've gleaned from the competition, appears to have been the first news organization to give this information. And several newspapers greatly undercounted previous accidents.\n\nReporters who relied just on Web sites, like the one run by the NTSB, would have missed details on previous crashes such as pilot names - vital for pulling clips and archived footage. They also would have missed knowing whether pilots were rated to fly solely with instruments.\n\nIn addition, we were able to show the aircraft was not flawless. Service difficulty reports documented one repair to JFK Jr.'s plane that included a maintenance worker complaining about the plane's design; how a propeller control bolt rubbed against the engine baffle, and could jam the plane at less than full speed. (Several Web sites that carry SDRs were either down or missing this report, a recurring problem that I and other reporters have noted in NICAR circles).\n\nOf all the airworthiness directives, the most interesting called for replacing air filters in a number of New Piper planes - including JFK Jr.'s - because they had been known to crumble, with the debris getting sucked into the engine. That could cause an engine to quit.\n\nIndeed, aviation records show people have perished in this kind of plane because of mechanical problems. While mechanical problems were not the likely cause of JFK Jr.'s crash, these facts were still important to underscore this was not a perfect aircraft.\n\nBy working closely with Dateline producer Colleen Halpin, and correspondent Victoria Corderi, we were able to weave details from these public records in interviews on the Saturday after the crash, and much of this remained unreported by the time Dateline aired a day later.\n\nThere are other ways that computerized aviation records helped us report the story. Ownership records allowed us to track down previous owners. Registration records, along with incorporation records, allowed us to quickly verify airplane photographs being pitched by freelancers to NBC News. Aviation Safety Reporting System records, plus computerized maps of aviation facilities, allowed us to quickly check the kinds of problems along the route flown by JFK Jr.\n\nThere was a lot of dirty data to weed out, but that's for another Uplink issue.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-transplant-disparity","slug":"1999-october-transplant-disparity","url":"https://dwillis.github.io/uplink/article/1999-october-transplant-disparity.html","year":1999,"month":"October","monthNum":10,"headline":"Transplant disparity","kicker":"ANALYZING MEDICAL DATA","author_name":"Kevin McCoy","author_title":"Reporter","affiliation":"The New York Daily News","summary":"A New York Daily News investigation used data analysis and computer tools to expose disparities in organ transplants, revealing that African-American and Latino patients in the New York City area received fewer heart, liver, and lung transplants despite higher disease rates and greater need.","keywords":["transplants","disparities","minorities"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["Excel","FoxPro","Internet","World Wide Web"],"full_text":"The New York Daily News' headline on Aug. 8 was dramatic: \"Deadly Disparity in Transplants, Blacks and Hispanics Deprived.\" Behind it lay months of research and reporting, some of it aided by computer.\n\nThe two-part series showed that African-American and Latino patients in the New York City area got fewer heart, liver and lung transplants during the last three years even though they suffer higher rates of diseases that might make them need a transplant to survive.\n\nThe stories also showed that African-Americans have far greater need for kidney transplants than other patients because they suffer higher rates of kidney failure. Yet African-Americans in the New York City area face the longest wait in the nation for a donated kidney.\n\nThe long road that led to the series began with a suggestion from a health care official who pointed me to the Web site of the United Network for Organ Sharing (www.unos.org), the agency that coordinates all organ transplants in the United States.\n\nWhat caught my attention was UNOS' \"critical data\" on the race of donors and recipients for specific types of transplants at hospitals in New York City. The statistics for heart, liver and lung transplants seemed to show a large disparity that favored whites. Unfortunately, the data was outdated.\n\nI contacted UNOS and asked if the agency had more updated transplant data, by race and by organ, for New York City-area hospitals. I also posed the question to the New York Organ Donor Network, the agency that works with UNOS to coordinate transplants in the city and eight surrounding counties.\n\nThe agencies declined to release the most recent hospital-by-hospital data. But through negotiations the officials agreed to provide a breakdown of transplant donors and recipients, by race and by organ, for the entire New York Organ Donor Network coverage area from 1996-1998.\n\nThis data, analyzed with U.S. Census population estimates and disease and mortality rates from the National Center for Health Statistics, provided partial confirmation that minority patients got certain kinds of transplants less often than their white counterparts.\n\nVerifying the evidence\n\nSeveral transplant doctors who agreed to analyze the data confirmed the conclusion. However, I wanted additional evidence. I suspected there was other data, not available on the UNOS Web site, that might help test the conclusion.\n\nI asked UNOS if they could further break down the data for me by providing the zip codes of each organ donor and recipient. They sent it a few weeks later in an Excel spreadsheet.\n\nZip code data would tell me whether the organ recipients came from white or minority neighborhoods.\n\nI imported it into Visual FoxPro, then started to analyze transplant recipients by neighborhood.\n\nNext, I contacted the New York City Department of Health and asked for neighborhood breakdowns of deaths caused by major heart, liver, lung and kidney diseases. The agency readily agreed to provide the data.\n\nFocusing the search\n\nOnce it arrived, I focused in part on deaths from liver cirrhosis and hepatitis. African-Americans and Hispanics suffer far higher rates of these diseases than whites.\n\nMoreover, there had been enough liver transplants performed in the metropolitan area during the last three years to allow me to conduct a statistically valid analysis.\n\nThe Health Department data showed that residents of an 18-zip code section of the South Bronx and East Harlem, areas that feature a largely African-American and Latino population, suffered the city's highest rates of deaths from liver disease.\n\nDid residents of these communities get the highest share of liver transplants? No. In fact, the FoxPro analysis showed that residents of these areas got less than 4 percent of all liver transplants performed in the metropolitan area during the last three years.\n\nThat percentage was virtually identical to the share of liver transplants received by residents of a far smaller, four-zip code area of the city. Unlike the South Bronx and East Harlem, these four areas had relatively high percentages of white residents.\n\nNeither doctors who analyzed the data, nor the transplant centers challenged the data or the series' conclusions.\n\nWhile my research was under way, UNOS officials announced several changes that should make it easier for journalists and others to analyze transplant data. Starting this fall, more recent hospital-by-hospital data is scheduled to be made public by the agency.\n\nOnce UNOS makes the data available, journalists should be able to pursue a wide range of local and national stories on transplant issues like death rates and survival times, as well as race.\n\nKevin McCoy can be reached by e-mail at kmccoy@edit.nydailynews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-whats-next","slug":"1999-october-whats-next","url":"https://dwillis.github.io/uplink/article/1999-october-whats-next.html","year":1999,"month":"October","monthNum":10,"headline":"What's next?","kicker":"","author_name":"MaryJo Sylwester","author_title":"Staff","affiliation":"IRE and NICAR","summary":"Uplink newsletter will feature computer-assisted reporting (CAR) topics including school violence coverage, data negotiation, and Census 2000, along with practical guidance for journalists of all skill levels and market sizes. The publication will include standing features on statistics, beginner experiences, technical tips, and campaign finance data.","keywords":["reporting","journalism","data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":[],"full_text":"In the coming months, each issue of Uplink is going to centerpiece a prominent issue in computer-assisted reporting. Our goal is to provide answers to difficult questions and hands-on suggestions for both reporters and editors, in markets of all sizes.\n\nThe issues will show how everyone - no matter what level of training or available resources - can find and carry out quality stories using computer-assisted reporting.\n\nCenterpiece topics in upcoming issues include:\n\nUsing CAR to cover school violence.\nNegotiating for data.\nGetting ready for Census 2000.\nUsing mapping software.\nCAR on deadline.\nEditing of CAR stories.\nHow small markets incorporate CAR.\n\nIn addition to these topics, we'll continue to have several standing features such as Stats and Such, a column devoted to the use of statistics; First Ventures, tales from beginners about their first foray with CAR; Tech Tip, featuring technical advice for working with data in various software programs; and On the Internet, offering useful tips for navigating the Web.\n\nA new standing feature will be a page devoted to using campaign finance data. We hope you enjoy the upcoming issues. Anyone interested in writing an article about your experience in any of these topics or offering story suggestions please contact me at IRE and NICAR either by e-mail at maryjo@nicar.org or by telephone at (573) 882-0684.\n\nIn addition, we are always looking for reporters or editors to write about their experience on a recent CAR project.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-uplink-survey","slug":"1999-october-uplink-survey","url":"https://dwillis.github.io/uplink/article/1999-october-uplink-survey.html","year":1999,"month":"October","monthNum":10,"headline":"Uplink survey","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR invites readers to complete a survey about their preferences for the Uplink newsletter, including questions about computer-assisted reporting experience level, software use, and desired content topics.","keywords":["survey","journalism","software"],"topics":["Journalism & Newsroom"],"technologies":["Excel","Lotus 1-2-3","Access","FoxPro","Paradox","FileMaker Pro","GIS"],"full_text":"We're interested in knowing more about what you'd like to see included in Uplink to make sure our content is meeting your needs. Please take a few minutes to fill out this survey, either in the online version at the NICAR Web site, www.nicar.org, or by mailing this form to IRE and NICAR, Attention: MaryJo Sylwester, 138 Neff Annex, Missouri School of Journalism, Columbia, MO 65211.\n\nIf you have any questions, feel free to contact MaryJo Sylwester at maryjo@nicar.org or at (573) 882-0684.\n\nHow would you describe your experience level with CAR?\n\nBeginner\nIntermediate\nAdvanced\n\nDo you use a PC or Macintosh computer?\n\nPC\n\nMacintosh\n\nWhat software do you use? (check all that apply)\n\nExcel\nLocus1-2-3\nAccess\nVisual FoxPro\nFoxPro 2.6\nParadox\nFilemaker Pro\nmapping software\nstatistical software\nother (include name)\n\nHow frequently do you use CAR skills for your job?\n\nEvery day\nOnce or twice a week\nA few times a month\nA few times a year\n\nHow would you rate Uplink in terms of quality of content?\n\nExcellent\nGood\nFair\nPoor\n\nHow helpful are Uplink articles to you in general?\n\nExcellent\nGood\nFair\nPoor\n\nWhat types of articles would you like to see in Uplink? (check all that apply) Hands-on tips How-I-did-it type stories Internet tips Advanced CAR tips Campaign finance Using statistics, statistical software Other\n\nSoftware information\n\nBasic CAR tips\n\nWhat would you like to see changed in Uplink?\n\nAny suggestions for future stories?","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-being-prepared-pays","slug":"1999-october-being-prepared-pays","url":"https://dwillis.github.io/uplink/article/1999-october-being-prepared-pays.html","year":1999,"month":"October","monthNum":10,"headline":"Being prepared pays","kicker":"WHEN PLANES CRASH","author_name":"Elizabeth A. Marchak","author_title":"","affiliation":"The Plain Dealer","summary":"Journalists covering plane crashes should develop expertise in aviation by learning how pilots think, visiting local airports, building source networks, and studying accident databases to ask informed questions about the causes and context of incidents.","keywords":["aviation","pilots","accidents"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["CompuServe","CD-ROM","World Wide Web"],"full_text":"On any given weekend, thousands of private planes take off. Every once in a while one crashes.\n\nThe cause is usually pilot error. But the story is always far more interesting, if you know what questions to ask.\n\nPlane crashes are really about humans' use or abuse of technology. In our booming economy, more people are becoming private pilots, buying or leasing planes. Some pilots - and their planes - will never show up on your radar. Others are accidents waiting to happen. To add credibility to your news organization's coverage of the next accident, start learning now how the aviation world operates.\n\nLurking online at chat groups like CompuServ's AvSig site, it's easy to get the impression that pilots are a chatty bunch. They are - until they discover you are a reporter. Few reporters have developed enough expertise to pass their technical muster. We don't even know how to ask good questions, they say.\n\nThey have a point. Pilots spend thousands of dollars to put their planes in the air. They have had to learn to use some pretty sophisticated devices like radar, automatic pilots, fuel and navigation systems. Many are proficient in maintenance. They are exacting people. They make life and death decisions. They know they can't make a mistake. And, they reason, neither should reporters.\n\nWhere to start\n\nStart by taking plane rides at local airports. They are not expensive. They are a great way to win sources, especially if you decide to go to ground school, the precursor to flight school.\n\nNext, find out what businesses are at the airport. Build a spreadsheet of companies, addresses, phone numbers, names of employees, description of what they do. Make sure you know who handles security, janitorial services, baggage, fuel, maintenance, snowplows, bird control and the control tower.\n\nSince your newsroom probably has limited its aviation coverage to earnings and plane orders and a few service problems at local airlines, you may have a lot to learn. To get a handle on how many pilots and planes there are in your area, get the pilot and aircraft registration databases from NICAR or another suitable source.\n\nJane's \"All The World's Aircraft,\" provides detailed information about 1,400 civilian and military airplanes around the globe. Through the online site, the yearbook costs $445, the CD-ROM $1,145. (The book is also available at most public libraries.) With the information, you can build files and start another spreadsheet listing popular planes.\n\nAccident data\n\nHow often are these small private planes in accidents?\n\nYou'll find the answer at the National Transportation Safety Board's Web page. For starters, study the accidents and incidents by using the query form so you can look for specific airplanes. You can even look for things like \"probable cause.\" Then do like many veteran aviation reporters do - track the accidents on a daily basis.\n\nFor example, between Friday, Aug. 28 and Sunday, Aug. 30, there were 20 accidents listed. In four of those accidents, a total of seven people died. More than half those accidents involved Cessnas, Pipers and Beeches - three of the most widely used private airplanes in the United States.\n\nAfter a crash, safety board officials examine the scene and debris. Sometimes they can make immediate pronouncements. Most often they can't. This creates an uproar with reporters who want juicy new details for deadline.\n\nSometimes it takes months or years to figure out which part failed first and why. That's why the safety board won't speculate on deadline and why anyone who covers aviation should have a broad general knowledge to fill in those deadline gaps.\n\nPrivate pilots are one part of aviation's biggest safety problems, according to both the safety board and the Transportation Department's Office of the Inspector General. The pilots, through inexperience or inattention, end up where they don't belong including in the flight paths of incoming jets. These are often called runway incursions.\n\nIf an accident is big enough the safety board opens up a docket that includes a stunning array of documents like tests on broken equipment and statements from people involved in the accident. The dockets for some accidents are available on CD-ROM or in report form.\n\nThe FAA assists, but is rarely in charge of an accident investigation. But the agency is useful for other reasons. Its job is to enforce the FARs federal aviation regulations - that govern general aviation. Most of the regulations affecting private pilots are contained in Part 91 of the FARs, often the least restrictive of all the FAA regulations.\n\nIf a crash involves some kind of maintenance problem, the FAA may have already issued information to pilots in the form of an advisory circular or a mandatory order to fix the problem called an airworthiness directive. They may be searched on the FAA's Web site.\n\nThe FAA's Accident and Incident database is also useful, because it tracks events involving individuals, as well as major aircraft. But don't look for any service difficulty reports - private pilots don't have the reporting requirements that commercial airlines do. That data may be obtained through NICAR or another processing service.\n\nFreedom of Information requests should be filed with the FAA to get enforcement action against the pilot or mechanic or the airline. While there are some privacy issues that limit disclosure, few news organizations file a FOIA for enforcement documents so there is almost always a story.\n\nIf your airport is one of the 100 biggest in the U.S., information about it is available on the CD-ROM \"The 1998 ACE Plan - Aviation Capacity Enhancement CD-ROM and Airport Database.\"\n\nThe airport layout is a nice graphic to keep in a computer file for emergencies like crashes or collisions.\n\nGeneral aviation issues affecting private pilots and their planes get lots of ink, if you know where to look. The Flight Safety Foundation, which issues monthly reports on aviation safety, frequently focuses on private pilots. The reports are worth reading and saving because they frequently reprise safety board reports in a less technical, more easily understandable format.\n\nFrom time to time, the General Accounting Office, the investigative arm of Congress, examines a private pilot issue, or an issue relating to small single and twin-engine aircraft.\n\nQuestions to ask\n\nWith this basic information, here are some questions to ask when a private plane crashes:\nWhat is the tailnumber? Serial number?\nWhat is the make and model of the plane?\nWhat is the make and model of the propeller or engine?\nWas there a flight plan?\nWhat were the pilot's ratings?\nHow many hours in this model plane?\nHow many other ratings did the pilot have?\nHow did the pilot load the airplane?\nIs there a possibility that the plane was overloaded or that its weight and balance was not properly calculated?\nHow much fuel was in the plane when it took off? Was there enough?\nWas there any past disciplinary action against the pilot?\nWere any service bulletins issued by the engine or plane manufacturer?\nHow many hours/cycles since the last engine overhaul or repair?\nHas the FAA issued any advisory circulars or airworthiness directives?\nHow often has this plane/model been in an accident before and what were the causes?\nWhat did the pilot do in the preceding 24-48 hours before the crash?\nIf there is a manned tower, was there any conversation between the pilot and the controllers?\nWas this plane tracked on radar? What do the readings show?\nWhat were the plane's last known coordinates?\nHow did it descend? Quickly? Did the pilot level off? Seem disoriented?\nWas there any kind of runway or airspace incursion?\nBased on what you see here, what is the piece of equipment you want to examine first?\nWas the rescue response appropriate? Did it contribute to the deaths?\nIf the plane broke up what came off first? Where was it found?\nHad the pilot or a previous owner modified the airplane in any way?\nHow many previous owners of the plane and engines? Why did they sell?\nWhat was the weather at the time of the accident?\nWas the pilot assigned a specific altitude?\n\nWhen the plane goes down, your competition will report that a plane crashed. You will be able to tell readers or viewers about the most common problems with the planes, what the safety board has recommended to solve the problem and whether or not the FAA did anything about it safety issues people care about.\n\nElizabeth Marchak can be reached by e-mail at emarchak@mail.digizen.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-tales-from-the-field","slug":"1999-october-tales-from-the-field","url":"https://dwillis.github.io/uplink/article/1999-october-tales-from-the-field.html","year":1999,"month":"October","monthNum":10,"headline":"Tales from the field","kicker":"MORE ON THE KENNEDY CRASH","author_name":"","author_title":"","affiliation":"","summary":"Reporters covering JFK Jr.'s plane crash used computer-assisted reporting techniques and online databases to produce compelling stories that went beyond basic coverage, including context about aviation safety and comparisons to other crashes. The Record demonstrated how access to FAA and NTSB databases combined with traditional reporting could create authoritative, detailed journalism without requiring advanced statistical analysis.","keywords":["reporting","databases","aviation"],"topics":["Investigative Methods","Public Safety","Technology"],"technologies":["Internet","World Wide Web","Nexis","Aviation Safety Reporting System"],"full_text":"Tales from the field\n\nWhen John F. Kennedy Jr.'s plane crashed the night of July 16, several reporters were able to take their coverage beyond the rest of the pack using some simple computer-assisted reporting techniques.\n\nIt didn't take number-crunching expertise. It didn't take a lot of powerful software. Access to the Internet and a little digging helped produce some powerful and compelling stories.\n\nHere are a couple tales of how they did it:\n\nRobert Gebeloff, The Record\n\nIn North Jersey, the Kennedy story was a huge affair. Not only is The Record in the New York media market, but Kennedy's plane took off from a local airport.\n\nNeedless to say, Gebeloff was called in to work early on Saturday (July 17) morning, along with two transportation reporters, police reporters, a news aide vacationing on Martha's Vineyard, and anyone else lucky enough to be home when phone rang.\n\nWhile most people think of CAR as a number-crunching endeavor, Gebeloff said they crunched zero numbers for the Kennedy story, yet managed to make key contributions throughout the paper's exhaustive coverage.\n\nGebeloff used various aviation databases like a well-placed source - a source who provided leads to key information that was used to make stories more interesting and authoritative.\n\nThat Saturday, The Record staff decided to produce a piece about aviation safety, putting this crash in context. How often do single engine planes go down? What are the causes? How does this compare to other modes of transportation?\n\nSome may have tried to compile all of this from the Federal Aviation Administration or National Transportation Safety Board aviation databases, but many of these numbers are already summarized in various government reports - which The Record already had on hand.\n\nWhat made The Record story more distinctive was its ability to cite other recent plane crashes - in detail, Gebeloff said. They downloaded data from the FAA's Office of Accident Investigation (www.faa.gov/avr/aai/aaihome.htm), which provides preliminary accident reports from the prior 10 days.\n\nWhile the data are preliminary, there's a narrative in there so you get an idea of what happened. Using Lexis/Nexis, they found news accounts of the accidents.\n\nAs the story unfolded, some began wondering aloud whether the search-and-rescue effort was more extensive for Kennedy than in other crashes. The Record staff decided to find the families and associates of previous fatal accidents and let them answer the question.\n\nTo find those people, Record reporters used three aviation databases to pull information on fatal plane crashes over water: FAA, NTSB and the Aviation Safety Reporting System.\n\nGebeloff said the key to the coverage was in the details.\n\n\"Using the narratives in the reports, news accounts of the accidents, and, most importantly, interviews with victim family members and associates, we were able to produce a compelling read that was fact-based and answered a question on everyone's mind,\" he said.\n\nRobert Gebeloff can be reached by e-mail at gebeloff@bergen.com\n\nSusan Milton, Cape Cod Times\n\nWhile other reporters gathered information from the scene, Cape Cod Times reporter Susan Milton turned to the Internet for a few answers.\n\nAnd she found plenty. The plane registration papers that washed up on shore didn't have John F. Kennedy Jr.'s name, but by plugging into a commercial Web site, www.landings.com, she found the name of the company and address the plane was registered to. (Landings has a great deal of information about planes and pilots not found elsewhere, but reporters who have used it suggest using caution with the information found there).\n\nAt the same site, Milton pulled up Kennedy's pilot's license information. The addresses matched.\n\nAt the NTSB Web site, she collected reports for all the fatal crash reports for all airports on Cape Cod and the area islands over the past 10 years. That, combined with clips from the paper's archives, led to a story on the high number of accidents due to errors by inexperienced pilots in bad weather.\n\nAfter the plane was recovered, some media outlets were reporting that the Kennedy family was considering a burial at sea.\n\nMilton turned to a search engine and typed in \"burial at sea.\" That led to a copy of the Navy Burial Service (a portion of which became the story's lead) and a phone number to the U.S. Navy's Office of Mortuary Affairs.\n\nThe representative there yielded a wealth of information: what it takes to get a burial at sea through the Navy; history of burials at sea and a rough estimate of how many have been conducted; and a tip that the Coast Guard also performed similar rites. Much of the information was useful in determining whether the Kennedy family was getting special treatment, Milton said.\n\nMilton also used the Internet to find the Environmental Protection Association's requirements for a burial at sea and statistics on cremations and burials from a national funeral association.\n\n\"(The Internet) enabled us to put a lot of what happened to John F. Kennedy Jr. and the two sisters into perspective and context for readers,\" Milton said. \"It was a great source for background information on the registration that washed to shore and allowed us to write a lot of informed stories about burials at sea and other accidents in the area.\"\n\nFor reporters covering similar crashes in the future, Milton suggested looking at the aviation accident reports available online to see if there are any similarities or differences and put the latest crash into context.\n\n\"Don't hesitate to ask a question on the Internet,\" Milton said. \"It might lead you to a direction you might not have considered.\"\n\nSusan Milton can be reached by e-mail at smilton@capecodonline.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-back-to-school","slug":"1999-october-back-to-school","url":"https://dwillis.github.io/uplink/article/1999-october-back-to-school.html","year":1999,"month":"October","monthNum":10,"headline":"Back to school","kicker":"MINING EDUCATION DATA","author_name":"MaryJo Sylwester","author_title":"Staff","affiliation":"IRE and NICAR","summary":"Education reporters can use publicly available school enrollment, budget, and employee data as investigative tools to analyze district performance and funding disparities, though accessing detailed student and teacher records often requires negotiating confidentiality restrictions with state officials.","keywords":["enrollment","budget","confidentiality"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"With students back in school, there should be a host of data available to help analyze how school districts in your area are faring.\n\nMost states require schools to conduct a head count at the beginning of the school year, and often these reports include information on gender, ethnicity and other variables. Data on the number of teachers at each grade level, the number of students signed up for free or reduced lunches, district employee lists and payroll data should also be available.\n\nThese basic data sets can prove to be gold mines throughout the year.\n\n\"It's no different than doing a library search on old stories. It's the same kind of tool,\" said Linda Johnson, the computer-assisted reporting coordinator and former education reporter at the Lexington Herald-Leader in Lexington, Ky.\n\nData gold mines\n\nBasic enrollment data can provide clues to whether the school or district as a whole is gaining or losing students. Combining that with teacher information can show student-teacher ratios. A simple district employee list can be valuable if police arrest somebody who is rumored to be a school employee - especially over the weekend when administrative offices are closed.\n\nJeff Porter, a reporter with the Arkansas Democrat-Gazette who previously covered education, recommends reporters get \"everything they can get their hands on\" in terms of data.\n\nFor beginners, the school budget is a great place to start. And having financial information is always useful, Porter said.\n\nUse financial data to compare year-to-year changes in funding, particularly at the department level. How does special education funding compare to athletic funding? How much is being spent per student? What are teachers being paid and is it effective?\n\nPorter said an increasing number of states are producing detailed education data, dealing with things such as characteristics of every student, school and teacher. This year, Arkansas will also include data that will make it possible to analyze how many library books and computer stations each school has per student.\n\nOne of the challenges that remains, however, is overcoming the confidentiality problem associated with student and teacher data.\n\nNegotiating roadblocks\n\nJohnson fought Kentucky education officials for two months before settling for less data than she wanted. She asked for 10 years worth of teacher information, including a unique identifier (no names), the college he or she graduated from, the certification test taken, score received, race, gender, date of birth and school district (not specific school).\n\nBut state officials still feared someone would be identifiable, especially in smaller districts.\n\n\"They argued that even if we trusted you and made it available, we'd have to make it available to anybody who wanted it,\" Johnson said. \"And I couldn't really argue with that one.\"\n\nShe ended up with two tables, with data going back just four years. The first table had testing information, the year the person was born and the college attended. The second had the same testing data, year born and school district. Data from prior years simply weren't available electronically.\n\nThe data led to a series last year, \"The Learning Gap\" that showed the passing score on teacher certification tests was significantly lower than other southern regional states.\n\n\"Confidentiality can be a big block,\" Johnson said. \"You've just got to keep plugging away.\"\n\nPorter recommends starting with the person who works with the data or the lowest level of bureaucracy, and then keep going up when negotiating for data.\n\n\"If the district feels like it has to aggregate numbers, instead of giving specifics, then you should negotiate,\" Porter said. \"Some schools will argue that because they don't want you to have that information.\"\n\nHandouts on using CAR to cover education, available at the IRE Resource Center, include:\n\n\"Some education tips from Alabama,\" by Jay Reeves of the Associated Press (#890).\n\n\"CAR on the education beat\" by David Heath of the St. Louis Post-Dispatch and Jeff Porter of the Arkansas Democrat-Gazette (#956 and #957).\n\n\"School report cards: How to go beyond test scores,\" by Heather Newman of the Detroit Free Press (#891).\n\n\"Resource list from The Learning Gap,\" the Lexington Herald-Leader's series on teacher certification (#892).\n\n\"Analyzing capital spending to find discrimination,\" by Neil Reisner of The Miami Herald (#896).\n\nHaving the details, however, is much more useful in the long run, Porter said. With raw data, reporters can also analyze things such as:\n\nDemographics across a district, area or state.\nHow teacher ethnicity in a school compares to student ethnicity.\nTeacher experience levels.\nCollege entrance exam results, including percentage of students who took the test.\nFacility information, such as school overcrowding.\n\nDetails in the data\n\nJosh Freed, education reporter for the Post-Bulletin in Rochester, Minn., used persistence with education officials in his area to obtain payroll data for the St. Charles, Minn., School District. An auditor had reported that school officials were receiving \"bonuses\" from the district's \"leftover\" money each year, but the audit didn't divulge any further details.\n\nAfter receiving 1995-1998 data in an Excel spreadsheet, Freed manually added the payroll records for 1994 - ending up with about 800 records. A few simple queries in Access summed up the total bonuses each employee received over the seven years and Freed quickly had the basis of a solid story showing two principals had received bonuses totaling more than $22,000 and $21,000, respectively during that time period.\n\nA simple story by CAR standards, Freed says, but it made a difference. The small Minnesota school district isn't giving out bonuses anymore.\n\nMaryjo Sylwester can be reached by e-mail at maryjo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-college-crime","slug":"1999-october-college-crime","url":"https://dwillis.github.io/uplink/article/1999-october-college-crime.html","year":1999,"month":"October","monthNum":10,"headline":"College crime","kicker":"HANDOUT OF THE MONTH","author_name":"Kit Lively","author_title":"Reporter","affiliation":"The Chronicle of Higher Education","summary":"Federal law requires colleges to report annual crime statistics including violent crimes and drug/weapons violations, though the self-reported data should be verified against police logs and approached with skepticism due to widespread underreporting, particularly for sexual offenses.","keywords":["reporting","campus","underreported"],"topics":["Education","Crime & Justice","Investigative Methods"],"technologies":[],"full_text":"Federal law requires all colleges that receive federal funds, including student aid money, to report crime statistics annually. This is a public record, and each report should include statistics for the most recent three years. The crimes include murder, manslaughter, serious sex offenses, aggravated assault, robbery, arson and vehicle theft. Colleges also must report the number of arrests for alcohol, drug and weapons violations.\n\nThe law was amended in 1998 to require colleges to make public alcohol, drug and weapons violations that are handled in confidential campus judiciary hearings. It also required colleges to provide more detail about hate crimes and added a $25,000 fine for failure to comply.\n\nEven with the tougher law, these annual reports should be approached with some skepticism, since the data are self-reported and no federal agency monitors the reporting regularly. One way to check the numbers is to go back to the original police logs and compare the totals.\n\nWhen comparing crime statistics for several institutions, it's best to factor in enrollments. A school with 10,000 students and 10 rapes isn't necessarily more dangerous than one with 1,000 students and two rapes.\n\nAnother source for campus crime statistics is the FBI's annual Uniform Crime Report. Colleges aren't required to participate but a few hundred do voluntarily.\n\nSex offenses\n\nSex offenses are greatly under-reported on campuses, as elsewhere. Colleges are supposed to include all crimes that are reported to non-police authorities, like counselors and rape crisis centers, but very few do. These people are good to interview to get a fuller picture of what's actually happening on campus. I've found some tension between these officials and campus police, because their standards for deciding when to report a crime differ. If that tension creates inaccurate reporting, it's a good story.\n\nSome colleges avoid reporting crimes by referring incidents to the campus judicial system. This happens most often with infractions like alcohol and drug offenses - which the 1998 changes in the federal law were intended to address. But there also have been examples of colleges failing to include rapes on the annual published reports because they were handled internally. Getting information on campus judiciaries usually is frustrating because most colleges claim that the proceedings and records are protected by the federal Buckley Amendment (also called FERPA, for Family Educational Rights and Privacy Act). But things may be changing. Several state supreme courts have ruled that campus judiciary records are subject to state sunshine laws and should be open.\n\nColleges are supposed to report crimes that occur in any building owned by the school, including fraternity houses and off-campus housing, although compliance is uneven. Many schools, however, do not report incidents that occur on public sidewalks or streets around campus buildings.\n\nChecking statistics\n\nOne way to get an idea of whether the reported statistics are accurate is to interview students about whether they've been crime victims and whether they reported those crimes to officials. Researchers have found that students are very unlikely to file reports when they've been victims, especially if drugs or alcohol were involved.\n\nLook at campuses to see what physical characteristics affect crime. Although crime rates on campuses generally are lower than in surrounding neighborhoods, urban campuses tend to have more crime. So do colleges with dormitories and features that bring the public to campus, like hospitals and major public streets.\n\nThe U.S. Department of Education has investigated a few campuses for which it has received complaints about inaccurate crime reporting. The DOE has been very slow and reluctant to exercise this regulatory power.\n\nKit Lively can be reached by e-mail at kit.lively@chronicle.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-using-sat-scores","slug":"1999-october-using-sat-scores","url":"https://dwillis.github.io/uplink/article/1999-october-using-sat-scores.html","year":1999,"month":"October","monthNum":10,"headline":"Using SAT scores","kicker":"BEST OF NICAR-L","author_name":"","author_title":"","affiliation":"NICAR-L listserv","summary":"Journalists discuss the challenges of comparing SAT scores across states, noting that participation rates and demographic factors significantly affect results and that simple statistical adjustments may not effectively normalize the data.","keywords":["scores","states","participation"],"topics":["Data & Databases","Investigative Methods","Education"],"technologies":["ArcView"],"full_text":"Using SAT scores\n\nThese are a few takes from a recent discussion on the NICAR-L listserv regarding using SAT scores. See page fifteen for information on subscribing to NICAR-L.\n\nI'd like to run a map showing SAT scores by state, but I seem to remember that state scores are affected by how many students actually take the test. More students, lower scores. So, is there a way to adjust scores using the information provided - score and percent taking the test? Anna Byrd Davis, The Commercial Appeal, Memphis, Tenn.\n\nProbably the best way to adjust these scores is by running a regression with number of students testing as the independent variable. But I've heard doing this takes so much variation out of the scores that your map would be all one color, or close to it.- Geoff Dougherty, St. Petersburg Times, Fla.\n\nGeoff (Dougherty) is right about the result of using regression, and that's because the larger the population, the lower the variability. The key question may be why, implicitly, larger states have lower scores. The ACT reports a similar phenomenon. In our one high school here and statewide, there is a higher score for those whose coursework points toward college prep. Bring in those who may not have thought about college until later in high school, not completed work, or are simply not up to snuff, and those can also explain the population-score relationship. -George E. Schwarz, Rio Grande Sun, Española, N.M.\n\nThere are a bunch of other factors that may affect SAT/ACT scores. For instance, if your state has a high school graduate scholarship program that uses one of the tests as an eligibility requirement, you end up getting more kids taking the test who may not have been college-bound in previous years, or compared to other states. -Mark Schleifstein, The Times-Picayune, New Orleans, La.\n\nYes. It's not the number who take the test that matters, it's the proportion of high school seniors. The effect comes from the fact that states whose public colleges do not require the test get high scores because those who do take it are trying for competitive colleges. - Philip Meyer, University of North Carolina.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-numerical-accuracy","slug":"1999-october-numerical-accuracy","url":"https://dwillis.github.io/uplink/article/1999-october-numerical-accuracy.html","year":1999,"month":"October","monthNum":10,"headline":"Numerical accuracy","kicker":"STATS AND SUCH","author_name":"Neill A. Borowski","author_title":"","affiliation":"The Philadelphia Inquirer","summary":"Software programs like SPSS and Excel often produce different numerical results for statistical calculations like percentiles, and economists warn that journalists and analysts should not blindly trust software accuracy since many programs fail basic numerical tests.","keywords":["software","percentiles","accuracy"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["SPSS","Excel"],"full_text":"Few things are as alarming as analyzing numbers for the second or third or fourth time to check on them before publication only to find you're getting different answers. This probably has happened to all of us, regardless of the story or computer program. For years, it has happened with regularity when I compute percentiles.\n\nThe scenario usually goes something like this: I set out to find percentile \"breaks\" for school test scores. My first run is in SPSS. In Frequencies, you can select the number of \"cut points\" (for example, five cut points yield the 20th, 40th, 60th, 80th percentiles). Or, you can specify each percentile in SPSS. My second run is in Excel. I use the percentile function and more often than not come up with answers different from those found with SPSS.\n\nAvoid believing that the software must be correct. The evidence shows there is no cause for blind faith.\n\nAfter overcoming the churning feeling in my stomach, I check the breaks and find they cut the data into the same groups. In one simulation, SPSS set the 20th percentile break at 95.5 while Excel set it at 98.6. A visual inspection of the data shows the break comes after 94.5 and before 99.7.\n\nDifferent algorithms? Probably. I'm still waiting to hear from the software companies. The point is: The answers were different. Could there be other conflicts between programs that aren't as readily apparent?\n\nShould we, as journalists and analysts, worry?\n\nIf we worry, we won't be alone.\n\nEconomists B.D. McCullough of the Federal Communications Commission and H.D. Vinod of Fordham University sounded the alarm in the June issue of the Journal of Economic Literature, the widely respected and refereed scholarly journal of the American Economic Association.\n\nTheir 30-page article details the authors' benchmarking of econometric software packages and similar work in the past by other econometricians. \"Apart from cost considerations,\" the authors note, \"economists generally choose their software by its user-friendliness or for specialized features. They rarely worry whether the answer provided by the software is correct.\"\n\nThe authors surveyed five journals that published more than 120 reviews of econometric software between 1990 and 1997. \"All but three paid no attention to numerical accuracy, and only two applied more than a single test of numerical accuracy,\" McCullough and Vinod found.\n\nThey tested four econometric software packages and found varying answers. (They didn't identify which packages, saying that was the job of software reviewers.) Their tests and descriptions of others were complex and economics-heavy. In one test, only one of four programs correctly calculated the correlation coefficients in a problem. Only two of the four correctly calculated the standard deviation in the same problem.\n\nRecall that a correlation coefficient of 1 is \"perfect\" correlation variable moves in lockstep with the X variable. One software package tested by the authors found correlation coefficients greater than 1 (1.127 and 1.137).\n\n\"Many textbooks convey the impression that all one has to do is use a computer to solve the problem, the implicit and unwarranted assumptions being that the computer's solution is accurate and that one software package is as good as any other,\" the authors wrote.\n\nThe authors call for more rigorous testing of software. They call on scholarly journal editors to require authors to identify their software, including the version number. And they call on other economists to demand accurate software rather than shopping only for \"speed, user-friendliness and the latest econometric features.\"\n\nA fundamental tenet of research is replicability - the ability of researchers to come up with the same results of other researchers given the same methodology and data. Software accuracy problems undermine this basic truth, the authors contend. \"An unfortunate consequence of two different packages providing two different answers to the same problem is a lack of replicability in economic research,\" they wrote.\n\nNow, back to journalism and our brand of data analysis.\n\nIf we don't worry, we should at least be aware. Test your findings with at least two different computer programs. When possible, calculate one of the statistics by hand and compare your results with the software's.\n\nAnd avoid believing that the software must be correct. The evidence shows there is no cause for blind faith.\n\nNeill Borowski's can be reached by e-mail at inborowski@phillynews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-desktop-site-improved","slug":"1999-october-desktop-site-improved","url":"https://dwillis.github.io/uplink/article/1999-october-desktop-site-improved.html","year":1999,"month":"October","monthNum":10,"headline":"Desktop site improved","kicker":"ON THE INTERNET","author_name":"Duff Wilson","author_title":"","affiliation":"Seattle Times","summary":"The Reporter's Desktop, a free collaborative website hosted by IRE, provides U.S. and Canadian journalists with curated links to essential reporting tools like mapping services, research databases, and search engines designed for speed and serious work.","keywords":["journalists","tools","desktop"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web","MapQuest","MapBlast","Medline","Hoover's","Northern Light","Ask Jeeves","Yahoo","Google","AllTheWeb"],"full_text":"The best place to start work on the web just got better. The Reporter's Desktop, a noncommercial and collaborative Web site aimed at making U.S. and Canadian journalists' work faster and easier, moved from a personal server to www.reporter.org/desktop a site hosted by IRE.\n\nFor those who have 18 million bookmarks, this is for you. For novices, this is for you, too. The Reporter's Desktop, or RD, is a slim, flat site, lean and mean, built for speed. There is only one layer beneath the desktop, an optional page of details about the great tools on top.\n\nAll of the tools themselves are free. I will be frank. These tools are borrowed. I do not say stolen, because I am quite sure companies don't mind. No one has objected so far.\n\nI hate to sound like your boss, but: This is for serious work. The RD has no headlines, stocks, weather or sports scores. This is some of what the Reporter's Desktop offers:\n\nEasy door-to-door directions with the two best forms from MapQuest and MapBlast.\nThe single best White Pages, Yellow Pages, and e-mail lookup I could find.\nMedline, 9 million scientific and medical studies, abstracted. A gold mine for facts and sources.\nHoover's, a great resource to sketch a company on deadline and find news, finances and competitors.\nNorthern Light, still my favorite search engine with its folders system and special documents. Others: Ask Jeeves! seemed silly but it's great. Yahoo is the old standby. The new Google is huge. I'm evaluating www.alltheweb.com\nWhat's not there is just as important. I weed the RD ruthlessly. It needs to load fast. If you want comprehensive or specialized lists, check out the mega-bookmark sites instead.\nAt a reporters' conference, somebody asked me suspiciously, \"What's in this for you?\" So cynical! I get nothing but the best Web page I can design to launch my own work, and the pleasure of trying to keep up with the best reporting tools on the Web.\n\nDuff Wilson can be reached by e-mail at dwilson@seattletimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-political-coverage","slug":"1999-october-political-coverage","url":"https://dwillis.github.io/uplink/article/1999-october-political-coverage.html","year":1999,"month":"October","monthNum":10,"headline":"Political coverage","kicker":"CAMPAIGN FINANCE","author_name":"Cindy Eberting","author_title":"Staff","affiliation":"IRE and NICAR","summary":"IRE and NICAR are launching a series of hands-on workshops across Midwestern state capitals to teach journalists data analysis skills for covering campaign finance, funded by a Joyce Foundation grant. The workshops will equip reporters with the technical abilities to independently analyze contribution databases and track money flows in elections.","keywords":["campaign","finance","journalism"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["spreadsheet","World Wide Web"],"full_text":"Knowing how to truly analyze contribution data and track the cash flow to campaigns is becoming as essential to basic election coverage as having candidates' home phone numbers.\n\nAlmost any serious candidate - for state races as well - are keeping their contributions in a database. Journalists without the data analysis skills to pore over these records are left to report the view of a candidate or political watchdog groups.\n\nHence the mission of the campaign finance project at IRE and NICAR: to quickly spread the data-crunching skills specific to covering federal, state and local elections.\n\nIn August, the Chicago-based Joyce Foundation awarded IRE and NICAR a 16-month grant to accomplish this plan. Here's how we're doing it and how you can help.\n\nWe're planning a series of two-day hands-on computer-assisted reporting workshops starting in January located at the state capitols in Illinois, Indiana, Ohio, Michigan, Minnesota and Wisconsin.\n\nEach campaign finance seminar will be specific to its location. For example, the training exercises at an Illinois seminar will use the most recent Illinois campaign finance data. Reporters will also learn the caveats of working with federal data, including which resources are the most useful, as well as how each organization (Center for Responsive Politics, Public Disclosure, Inc., Federal Elections Commission) massages the data.\n\nWe're looking to train with data from the Center for Responsive Politics and the National Institute of Money in State Politics, which each offer databases with contributor names coded by occupation, employer or industry.\n\nEach tailored seminar will feature that area's experts in campaign finance laws and data. The experts will come from state elections boards as well as local political watchdog groups. These panelists will give reporters a chance to interact with those who built the databases and work with the data.\n\nThe hands-on classes will include exercises in:\n\nspreadsheets, databases, and mapping\n\nimporting campaign finance data from the Web and other file types into both spreadsheet and database formats\n\njoining relational databases\n\nperforming enterprise joins, including how to match campaign finance databases to state contract databases\n\nIn addition to the data-crunching lessons, reporters will also learn how to spot the latest loopholes in campaign finance laws that politicians are using to raise more money. These loopholes include the ever-increasing practice of transferring soft money donations from national parties to state parties.\n\nFor each of these workshops we are looking for reporters who want to help with organizing, promotion and training or coaching the hands-on computer-assisted classes.\n\nIf you're interested in helping out or can offer suggestions on programming please contact us at the Campaign Finance Information Center (573) 882-1982 or cficcomments@ire.org.\n\nWe are planning the first set of money in politics workshop being offered by IRE and NICAR. Here are the locations and months that are tentatively scheduled:\n\nLansing, Mich. January\nColumbus, Ohio February\nSpringfield, Ill. February\nIndianapolis, Ind. March\nMadison, Wis. March\nMinneapolis, Minn. April\n\nPlease watch for more details on our Web Site at campaignfinance.org/seminars.html\n\nCindy Eberting can be reached by e-mail at cindy@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-delimiter-dilemmas","slug":"1999-october-delimiter-dilemmas","url":"https://dwillis.github.io/uplink/article/1999-october-delimiter-dilemmas.html","year":1999,"month":"October","monthNum":10,"headline":"Delimiter dilemmas","kicker":"TECH TIP","author_name":"Jason Grotto","author_title":"Staff","affiliation":"IRE and NICAR","summary":"The article discusses methods for handling pipe-delimited text files in FoxPro 2.6, which cannot natively support pipe delimiters, offering solutions including using FoxPro's built-in text editor or external search-and-replace tools.","keywords":["FoxPro","delimiter","import"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["FoxPro","Access","Xlate","Notepad"],"full_text":"A few months ago, a discussion about how to handle pipe-delimited text files in FoxPro 2.6 appeared on the NICAR-L listserv. Unlike most database managers, FoxPro versions below 6.0 can't handle pipe delimiters when appending (importing) text files into an existing structure. The good news is the tips outlined here can be used to deal with other delimiter problems. Technically, any character can be a delimiter. Paul Walmsley of IRE and NICAR has even seen carriage returns used as delimiters and page returns used to mark the end of records. The goal is to find a search-and-replace tool that meets your needs.\n\nThe important thing to realize is there are many ways to work around strange or incompatible delimiters.\n\nyou'll need to be aware of ways around the problem if you have a pipe-delimited file and plan to use FoxPro 2.6 as your database manager.\n\nThe first tip comes courtesy of Tom Boyer, research editor for The Seattle Times. Tom suggests using the text editor in FoxPro 2.6 when dealing with pipe-delimited files. Just open the text file in Fox, go to \"Edit\" on the \"Menu Bar,\" and choose \"Find.\" In the window that appears, use a pipe to fill in the \"Look For\" box. Then type It in the \"Replace With\" box. The It is FoxPro's way of representing a tab with characters. When you're done, hit the \"Replace All\" button on the right side of the box. Then go grab a cup of coffee because it could take a while if you have a large file.\n\nThis technique can also be used in word processing applications, but you have to know how your application represents tabs with characters. For instance, Microsoft Word uses ^t to represent tabs.\n\nAnother option when trying to import a pipe-delimited file into FoxPro 2.6 is to use one of the many search-and-replace applications available on Web sites like www.download.com. Web developers use many of these applications to ggtrings of HTML code, but these applications work just as well on simple search-and-replace actions.\n\nNICAR uses a DOS search-and-replace program called Xlate to create tab-delimited files from pipe-delimited Service Difficulty Reports downloaded from the FAA's FTP site. The nice thing about these DOS applications is you can make batch files in Notepad that allow you to search and replace on many files at once. For instance, NICAR uses the following batch command on the FAA data:\n\nc:\\util\\xlate \"|\" \\x09 SDRC9932.TXT SDRC9932.tab\nc:\\util\\xlate \"I\" \\x09 SDRC9933.TXT SDRC9933.tab\nc:\\util\\xlate \"I\" \\x09 SDRC9934.TXT SDRC9934.tab\nc:\\util\\xlate \"]\" \\x09 SDRC9935.TXT SDRC9935.tab\nc:\\util\\xlate \"I\" \\x09 SDRC9936.TXT SDRC9936.tab\nc:\\util\\xlate \"I\" \\x09 SDRC9937.TXT SDRC9937.tab\n\nThe first command (\"c:\\util\\xlate\") tells DOS we want to use XLATE in the util directory on the C drive. The \"I\" is the character we want to replace, and the \"Ix09\" is the hexadecimal value for a tab. SDRC993X.TXT are the original files, and SDRC993X. are the files we create after searching and replacing.\n\nOf course, you also could import the data into Access and then export it into a .DBF if you really want to use Fox. The problem with that tactic is the field lengths will default to 255 characters.\n\nThe important thing to realize is there are many ways to work around strange or incompatible delimiters. The problem isn't restricted to pipe-delimiters and FoxPro. But luckily the solutions listed here can handle just about any delimiter riff that throws you off.\n\nJason Grotto can be reached by e-mail at jason@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-story-update","slug":"1999-october-story-update","url":"https://dwillis.github.io/uplink/article/1999-october-story-update.html","year":1999,"month":"October","monthNum":10,"headline":"Story update","kicker":"Bits, Bytes and Barks","author_name":"","author_title":"","affiliation":"The Atlanta Journal-Constitution","summary":"The Atlanta Journal-Constitution's investigation revealed that a Georgia town disproportionately targeted African-Americans for traffic tickets on a single road, generating $860,000 in fines with 87% of tickets issued to Black drivers. The exposé prompted significant changes including the resignation of the police chief, appointment of an African-American replacement, a new Black female judge, and a reduction in the police force.","keywords":["discrimination","investigation","policing"],"topics":["Investigative Methods","Demographics & Census","Crime & Justice"],"technologies":[],"full_text":"The July-August issue of Uplink featured the results of The Atlanta Journal-Constitution's coverage of a Georgia town's targeting of African-Americans in distributing traffic tickets. Bill Torpy and David Milliron found that only 1,500 feet of road generated $860,000 in traffic fines, and that 87 percent of those ticketed were black.\n\nCitizen protests and the AJC's exposure of the apparent discriminatory and excessive ticketing practices preceded the resignation of the police chief and his subsequent replacement with an African-American man, who may officially become the new chief of police upon completion of training. A female African-American municipal judge has also been appointed, and the town's police force was cut in half.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-nicar-data-update","slug":"1999-october-nicar-data-update","url":"https://dwillis.github.io/uplink/article/1999-october-nicar-data-update.html","year":1999,"month":"October","monthNum":10,"headline":"NICAR data update","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR has released updated datasets including the MAUDE medical device reporting database and IRS 990 tax-exempt organization database, available through their data library.","keywords":["databases","medical","taxation"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"NICAR data update\n\nUpdates of the following data sets are now available through NICAR's data library. Prices and years available can be found at www.nicar.org or by calling (573) 882-0684.\n\nThe Manufacturer and User Facility Device Experience Database (MAUDE), or Medical Device Reporting Program (MDR) database contains information on defective or potentially defective medical devices that are already on the market. The three tables are made up of voluntary reports, user facility reports, distributor reports and manufacturer reports, and the data are from the Center for Devices and Radiological Health. The MDR database began when Congress enacted regulations on Dec. 13. 1984, requiring manufacturers and importers of medical devices to report to the FDA whenever a marketed device causes or contributes to the death or serious injury of a person, or has the potential to harm someone due to a malfunction.\n\nThe IRS 990 database contains information on organizations that are exempt from federal income taxes. An included pdf file explains the rules and procedures for organizations seeking tax exemption.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-october-upcoming-advanced-car-training","slug":"1999-october-upcoming-advanced-car-training","url":"https://dwillis.github.io/uplink/article/1999-october-upcoming-advanced-car-training.html","year":1999,"month":"October","monthNum":10,"headline":"Upcoming advanced CAR training","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Two upcoming journalism training events: a three-day mapping software seminar in Columbia, Missouri in October 1999 focused on geocoding and spatial analysis, and an IRE regional conference in Washington, D.C. in January 2000 featuring 50+ panels on reporting techniques and CAR topics.","keywords":["mapping","conference","training"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Access","FoxPro","GIS"],"full_text":"Mapping CAR Camp Oct. 21-24, 1999 - Columbia, Mo. This three-day seminar will offer intensive hands-on training using mapping software for stories. We will also look at past stories that have used mapping programs. The sessions will include address geocoding, thematic mapping, overlaying and spatial analysis. All lessons are based on government data. Participants should have a basic knowledge in using relational database programs such as Access or FoxPro. Participants are also encouraged to bring local data to work on during open lab time.\n\nIRE Regional Conference Jan. 14-16, 2000 Washington, D.C. IRE's next regional conference will have more than 50 panels featuring experts from print and broadcast sharing tips and techniques for better stories. The NICAR staff will hold daily sessions on CAR topics in the Computer Demo Room. Watch www.ire.org/training/conferences for more details.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-behind-school-walls","slug":"1999-november-behind-school-walls","url":"https://dwillis.github.io/uplink/article/1999-november-behind-school-walls.html","year":1999,"month":"November","monthNum":11,"headline":"Behind school walls","kicker":"KIDS AND VIOLENCE","author_name":"John Kelly","author_title":"Reporter","affiliation":"Indianapolis newspaper","summary":"An Indianapolis journalist obtained Indiana's school discipline database and analyzed expulsion data school-by-school, discovering both valuable insights and important limitations in how schools report disciplinary actions to the state.","keywords":["expulsion","database","schools"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"The news stories cited a federal survey of the states, which reported a drop in gun-related expulsions during the 1997-1998 school year.\n\nThe timing was bizarre for me in Indianapolis.\n\nFor months, I had been eyeballing a similar school discipline database I knew the Indiana Department of Education was compiling. Just days before the national report came out, I finally got the disk with the Indiana schools' data.\n\nOne dataset listed every expulsion at every public school along with fields showing the student's grade, gender, date of birth and the reason for expulsion. Another tallied the number of suspensions by reason for each school.\n\nWe were going to be able to do exactly what I had hoped: provide parents with a school-by-school accounting of the discipline happening behind the local schoolhouse walls.\n\nWithin days, we gleaned from the database how many students were kicked out of each public school for bringing a handgun to campus, using drugs, alcohol or tobacco or just acting out. Then, we went to work reporting.\n\nThe data was easy to confirm and hard for school officials to dismiss. That's because it comes straight from the local principals' offices.\n\nAnnual reports\n\nEach school year, every public school in Indiana must file paper reports with the Indiana Department of Education documenting their disciplinary actions – some of which is tallied for an annual report the states must submit to the U.S. Department of Education under the Gun-Free Schools Act.\n\nThe paper reports are certainly public records. Better yet, Indiana keys the information into a database. Other states probably do the same.\n\nIn our analysis and reporting, we found some weaknesses in the data. First, the tallies for each school only showed how many times students got caught and punished for bringing guns to school or breaking other rules.\n\nThere is no way to know how often offenses actually occurred on campus. Some schools may crack down on certain offenses while others turn a blind eye.\n\nSecond, like most crime data, you are going to find inconsistencies when you ask hundreds of school officials to lump incidents into broad categories based on vague definitions.\n\nIn Indiana, for example, one school confiscated what looked like a grenade from a student and promptly expelled him, state officials said. The grenade turned out to be a cigarette lighter, but the school reported the expulsion to the state under the category of \"other firearms.\"\n\nBeyond data\n\nWe overcame those weaknesses with reporting. We talked to state and local educators, students and others. We also carefully studied the paper reports that schools filed and the instructions they were given in filling out those forms.\n\nAll offered insight that helped us explain what conclusions people could and could not draw from the figures. Interviews with school principals gave us added opportunities to check our data against the paper reports they had filed with the state. We checked many schools' figures against individual reports provided by the state.\n\nAlong with our stories, we filed charts listing the expulsions and suspensions at every public school in Indiana for various offenses. That allowed local newspapers and broadcasters to do their own stories.\n\nWe filed the package within days of the federal report's release and just as Hoosier children were headed back to school.\n\nUnder-reporting\n\nAlong the way, we uncovered a problem with the national report: it was at least partly wrong. Indiana had reported 62 gun-related expulsions to the feds. But our database included 129 gun-related expulsions.\n\nWe rechecked our database work, and I called the state Department of Education official in charge of collecting and reporting this data and asked why the totals were different. He called back to say the state totals were right and the ones in the federal report were wrong.\n\nWe continued working on our stories, while the state education staff met to figure out what went wrong. The official called back to say the state had under-reported their figures and said, \"We just have to own up to this one.\"\n\nEducation officials in other states reported discrepancies in gun expulsion figures too, and the AP moved a national story highlighting the national report's errors.\n\nAs with most CAR stories, there was more traditional reporting involved here than number-crunching. The whole project took less than a week from the day I popped the floppy disk into my computer.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-conviction-rate","slug":"1999-november-conviction-rate","url":"https://dwillis.github.io/uplink/article/1999-november-conviction-rate.html","year":1999,"month":"November","monthNum":11,"headline":"Conviction rate","kicker":"CRIME AND COURTS","author_name":"Bill Wallace","author_title":"","affiliation":"San Francisco Chronicle","summary":"A journalist investigates allegations that San Francisco's District Attorney Terence Hallinan is dismissing serious criminal charges through excessive plea-bargaining, using California's Offender Based Transaction Statistics data to analyze case dispositions.","keywords":["plea-bargaining","dismissals","prosecution"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Offender Based Transaction Statistics"],"full_text":"More than a year ago, a source in the San Francisco District Attorney's office told me the city's new DA, Terence Hallinan, was plea-bargaining criminal cases at a disturbing pace, and many serious criminal charges were being dismissed in the process.\n\nThe tip was intriguing, but I needed to find some way of checking it out short of simply examining every case Hallinan's office had handled – a daunting task given the thousands of cases filed annually.\n\nLate last year, I discovered a type of data called Offender Based Transaction Statistics (OBTS) that was used by the California Attorney General's office to generate a section of the state's annual report, \"Crime and Delinquency in California.\" The information is known as \"Adult Felony Arrest Dispositions,\" available from the Criminal Justice Statistics Center of the state Department of Justice.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-campus-crime","slug":"1999-november-campus-crime","url":"https://dwillis.github.io/uplink/article/1999-november-campus-crime.html","year":1999,"month":"November","monthNum":11,"headline":"Campus crime","kicker":"KIDS AND VIOLENCE","author_name":"John Kelly","author_title":"","affiliation":"AP (Associated Press)","summary":"Journalists investigated school expulsion data by combining official statistics with on-the-ground reporting, uncovering significant discrepancies between state and federal gun-related expulsion figures in Indiana. The analysis revealed data collection inconsistencies and led to a national story about under-reporting errors in the federal report.","keywords":["expulsion","data","reporting"],"topics":["Investigative Methods","Education","Data & Databases"],"technologies":["Access"],"full_text":"ing guns to school or breaking other rules. There is no way to know how often offenses actually occurred on campus. Some schools may crack down on certain offenses while others turn a blind eye.\n\nSecond, like most crime data, you are going to find inconsistencies when you ask hundreds of school officials to lump incidents into broad categories based on vague definitions.\n\nIn Indiana, for example, one school confiscated what looked like a grenade from a student and promptly expelled him, state officials said. The grenade turned out to be a cigarette lighter, but the school reported the expulsion to the state under the category of \"other firearms.\"\n\nBeyond data\n\nWe overcame those weaknesses with reporting. We talked to state and local educators, students and others. We also carefully studied the paper reports that schools filed and the instructions they were given in filling out those forms.\n\nAll offered insight that helped us explain what conclusions people could and could not draw from the figures. Interviews with school principals gave us added opportunities to check our data against the paper reports they had filed with the state. We checked many schools' figures against individual reports provided by the state.\n\nAlong with our stories, we filed charts listing the expulsions and suspensions at every public school in Indiana for various offenses. That allowed local newspapers and broadcasters to do their own stories.\n\nWe filed the package within days of the federal report's release and just as Hoosier children were headed back to school.\n\nUnder-reporting\n\nAlong the way, we uncovered a problem with the national report: it was at least partly wrong. Indiana had reported 62 gun-related expulsions to the feds. But our database included 129 gun-related expulsions.\n\nWe rechecked our database work, and I called the state Department of Education official in charge of collecting and reporting this data and asked why the totals were different. He called back to say the state totals were right and the ones in the federal report were wrong.\n\nWe continued working on our stories, while the state education staff met to figure out what went wrong. The official called back to say the state had under-reported their figures and said, \"We just have to own up to this one.\"\n\nEducation officials in other states reported discrepancies in gun expulsion figures too, and the AP moved a national story highlighting the national report's errors.\n\nAs with most CAR stories, there was more traditional reporting involved here than number-crunching. The whole project took less than a week from the day I popped the floppy disk into my computer.\n\nWe got the data for free, but Indiana's education department is accustomed to forking over large databases to us for nothing. Once we had the database, the computer work was no more sophisticated than importing a text file into Microsoft Access and running a half dozen sum and count queries.\n\nReporters in other states, which collect and compile the same data from local schools, could easily replicate the reporting.\n\nJohn Kelly may be reached by e-mail at jkelly@ap.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-columbine-aftermath","slug":"1999-november-columbine-aftermath","url":"https://dwillis.github.io/uplink/article/1999-november-columbine-aftermath.html","year":1999,"month":"November","monthNum":11,"headline":"Columbine aftermath","kicker":"KIDS AND VIOLENCE","author_name":"Scott Smallwood","author_title":"","affiliation":"Albuquerque Journal","summary":"An Albuquerque newspaper obtained and combined three years of crime data from school district and city police records to reveal that district crime reports significantly underestimated violence and that incidents were unevenly distributed across schools. The investigation uncovered that district officials had never tracked crime statistics by individual school, prompting them to acknowledge that principals and parents should have access to this breakdown.","keywords":["schools","crime","data"],"topics":["Data & Databases","Investigative Methods","Education"],"technologies":["Excel","Access"],"full_text":"A few days after the Columbine High School (Colo.) shooting last April, we asked our local school district for statistics about violence at individual schools around Albuquerque. At first, we were planning to turn around a simple daily, but the district's foot-dragging eventually prompted us to turn the story into a small project.\n\nOfficials first told us that the district had never tallied any of its crime and violence data by school. They said they didn't want to embarrass principals of the more crime-ridden schools. Instead, each year the district sends the state a general report listing the total number of assaults, gun violations, drug charges and incidents of vandalism. We were welcome to that, they said.\n\nWe imagined that in Albuquerque Public Schools, a district of 120 schools with 85,000 students, district-wide numbers probably told just a small part of the story. We asked the district for its database of criminal reports. (The district has its own police force that patrols the elementary schools and middle schools. City police monitor the high schools.) They wouldn't give it to us electronically and our state law seemed to support them on that.\n\nStarting from paper\n\nThey did give us paper printouts of three years of data, which we were able to scan in. Using a fax server that's generally used to scan classified ads and death notices, we were able to quickly scan dozens of pages at one time. Luckily, the paper reports were very clean and a couple of Excel macros later, we had recreated their database.\n\nWe then went to the city police to get records of every criminal report filed at the city's 11 high schools for the same time period. We were the first to combine the two data sets – a total of 13,000 criminal reports over three years – to get a more complete picture of crime in city schools.\n\nWhile district officials knew of most gun violations in the schools and correctly reported those to the state, they didn't have any record of most of the city police reports, for everything from aggravated assault to rape.\n\nWe found that the district's annual reports underestimated nearly every category, especially assaults and drug violations. We also found, to no surprise, that in a district with 11 high schools and about 30 middle schools, reports of crime and violence weren't spread evenly across the city. Some schools reported four or five times the number of incidents as others.\n\nThe superintendent acknowledged that principals and parents should have this information broken down by school site.\n\nAfterthoughts\n\nThere were a number of things we probably should have done differently.\n\nWe didn't realize going in that the underreporting would be such a large part of the story. We should have gone back to the city and requested reports for the three years from the 79 elementary schools and 30 or so middle schools, but by that time we were well into the project and wanted to get it into the paper. Thus, we focused mostly on high schools.\n\nAlso, we probably should have fought longer on getting the data electronically, despite the poorly worded state law.\n\nThe stories appeared in the paper a month after we first asked for the violence statistics. About two weeks were wasted negotiating with the district about the data. Once we had created the database, we spent a few days using Access and Excel to analyze the data and another week reporting and writing the stories. We combined the crime data with other school data in our files, including drop-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-a-few-of-the-stories-on-school-violence-at-the-ire-resource-center","slug":"1999-november-a-few-of-the-stories-on-school-violence-at-the-ire-resource-center","url":"https://dwillis.github.io/uplink/article/1999-november-a-few-of-the-stories-on-school-violence-at-the-ire-resource-center.html","year":1999,"month":"November","monthNum":11,"headline":"A few of the stories on school violence at the IRE Resource Center","kicker":"","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"The IRE Resource Center houses several investigative journalism pieces examining school violence in the 1990s, including stories on firearm access, school arsons, and rising violence among teenage girls.","keywords":["violence","schools","firearms"],"topics":["Education","Investigative Methods"],"technologies":[],"full_text":"A few of the stories on school violence at the IRE Resource Center:\n\nAfter the string of multiple school shootings in the early and mid '90s, Education Week reporter Jessica Portner examined easy access to firearms as one of the main causes in explaining why one student killed another on the steps of a Savannah, Ga., school (File 14853).\n\nJeff Meader, an Education Week journalist, reported in November 1998 that \"during the months that Americans were transfixed by the tales of students killing other students, there were nearly as many stories – if not more – of children setting fire to their schools.\" (File 14883).\n\nA 1998 story by Royal Ford, a Boston Globe staff writer, examines possible causes of the increasingly violent nature of teenage girls, as well as solutions such as neighborhood and school programs. (File 14905).\n\nTo order, call the Resource Center at (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-inconsistent-data","slug":"1999-november-inconsistent-data","url":"https://dwillis.github.io/uplink/article/1999-november-inconsistent-data.html","year":1999,"month":"November","monthNum":11,"headline":"Inconsistent data","kicker":"KIDS AND VIOLENCE","author_name":"MaryJo Sylwester","author_title":"Staff","affiliation":"IRE and NICAR","summary":"Journalists across the country have discovered major inconsistencies in school violence reporting data, with schools using different definitions and methods to record incidents, requiring reporters to verify statistics through multiple sources and interviews.","keywords":["reporting","schools","data"],"topics":["Data & Databases","Investigative Methods","Education"],"technologies":[],"full_text":"Reporters across the country who have looked at school violence statistics say there is one big problem with the data: inconsistent reporting.\n\nIn Raleigh, N.C., WRAL-TV reporter Stuart Watson found cases of sexual harassment reported as sexual assault and bullets reported as guns.\n\nIn Virginia, the state's first-ever \"report card\" had numbers that seemed to defy reality for three school districts of roughly the same size, according to Jeff South, a journalism professor at Virginia Commonwealth University. Here's what the state said: City of Richmond, 188 reported incidents; Henrico County, a suburb, 859 incidents; and Chesterfield County, a suburb, 11 incidents.\n\nThe Richmond Times Dispatch published articles at the beginning of April telling the public about the discrepancies in the data.\n\n\"Educators have cautioned that some of the numbers – particularly those that record incidents of violence – are subject to the interpretation of the schools reporting them and may not be reliable as points of comparison from school to school and division to division,\" one of the Times Dispatch articles said.\n\nSo how do you overcome this problem?\n\nSeveral reporters from around the country – both print and broadcast – chimed in with some suggestions, based on their own experiences. What follows is an overview of what they had to say.\n\nThe first step is to study the data, looking for \"numbers that seem to defy reality,\" and ask for written guidelines on how school officials defined or categorized violent incidents, South recommended.\n\nWatson suggested running an internal check by comparing school data against alternative sources such as police logs or reports and other school reports. Even if the school has its own police force, city police or sheriff's deputies may also take reports of criminal activity at schools.\n\nBe sure to ask data collection or analysis people to explain any discrepancies and call the schools directly to double check the numbers.\n\n\"We found keypunch errors and principals who did not read definitions or follow simple instructions,\" Watson said.\n\nScott Smallwood of the Albuquerque Journal, said it's important to ask how crimes are reported – whether the principal tells someone or students go directly to officers on campus.\n\nHe also found those schools with more serious crime and drug problems tended to be more diligent in reporting.\n\n\"(Under-reporting) is probably more likely with the lesser incidents – from vandalism to simple assault to disorderly conduct. Our feeling was it would be harder to keep quiet about robberies, rapes and serious assaults with officers patrolling the campuses,\" he said.\n\nBut ultimately the only way to overcome discrepancies in the data is an old-fashioned technique called interviewing people. Talk to students, educators, police officers, state officials and anyone else who can shed some light on the \"reality\" behind the numbers.\n\nAnd finally, be extremely careful in how you report the numbers. Be sure to clarify any discrepancies and throw out any numbers or comparisons that you can't verify as accurate.\n\nMaryJo Sylwester can be reached by e-mail at maryjo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-more-resources-on-school-violence","slug":"1999-november-more-resources-on-school-violence","url":"https://dwillis.github.io/uplink/article/1999-november-more-resources-on-school-violence.html","year":1999,"month":"November","monthNum":11,"headline":"More resources on school violence","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This resource guide provides journalists with links to government databases and reports on school violence, including juvenile justice data, education statistics, and school crime indicators from multiple perspectives.","keywords":["violence","statistics","schools"],"topics":["Education","Crime & Justice","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"More resources on school violence:\n\nNational Criminal Justice Reference Service (www.ncjrs.org/ojjhome.htm) – Here you can access reports generated by the National Juvenile Court Data Archive, which collects, stores, and analyzes data about young people referred to U.S. courts for delinquency and status offenses. Their data include demographic information about the juveniles, the reasons for their referral to court and the court's response.\n\nA second NCJRS site (www.ncjrs.org/jjvict.htm) offers a long list of available reports and links to other sites regarding juvenile justice, including school violence.\n\nNational Center for Education Statistics (www.nces.ed.gov) includes reports and statistics, such as:\n\n• Violence and Discipline Problems in U.S. Public Schools, 1996-1997 (released March 1998)\n\n• Digest of Education Statistics, 1998, also includes some stats on school violence.\n\nBureau of Justice Statistics Clearinghouse (www.ojp.usdoj.gov/bjs/abstract/iscs98.htm)\n• Includes the Annual Report on School Safety 1998, which has national school crime data, successful strategies to reduce school violence and a local safety checklist for parents.\n\n• Also includes Indicators of School Crime and Safety 1998, which presents data on crime at school from the perspectives of students, teachers, principals and the general population using a comprehensive array of surveys as sources.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-school-violence-resources","slug":"1999-november-school-violence-resources","url":"https://dwillis.github.io/uplink/article/1999-november-school-violence-resources.html","year":1999,"month":"November","monthNum":11,"headline":"School violence resources","kicker":"","author_name":"","author_title":"","affiliation":"Bureau of Justice Statistics Clearinghouse","summary":"The Bureau of Justice Statistics Clearinghouse provides comprehensive reports on school safety and crime, including national data on school violence, prevention strategies, and perspectives from students, teachers, and principals.","keywords":["safety","crime","schools"],"topics":["Crime & Justice","Education","Data & Databases"],"technologies":["World Wide Web"],"full_text":"Bureau of Justice Statistics Clearinghouse (www.ojp.usdoj.gov/bjs/abstract/iscs98.htm)\n• Includes the Annual Report on School Safety 1998, which has national school crime data, successful strategies to reduce school violence and a local safety checklist for parents.\n\n• Also includes Indicators of School Crime and Safety 1998, which presents data on crime at school from the perspectives of students, teachers, principals and the general population using a comprehensive array of surveys as sources.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-learning-the-hard-part","slug":"1999-november-learning-the-hard-part","url":"https://dwillis.github.io/uplink/article/1999-november-learning-the-hard-part.html","year":1999,"month":"November","monthNum":11,"headline":"Learning the hard part","kicker":"FIRST VENTURE","author_name":"Jodi Nirode","author_title":"","affiliation":"The Dispatch","summary":"A journalist describes the challenges of obtaining and analyzing 911 dispatch records for a Columbus police response time investigation, including data formatting issues and missing arrival time logs that complicated the analysis.","keywords":["dispatch","response","data"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Access","Excel"],"full_text":"I was so idealistic. After finishing IRE and NICAR boot camp this summer I thought the hardest part about doing my first solo computer-assisted reporting project would be writing Access queries. How quickly I learned how many pitfalls there can be along the way.\n\nThe project was looking at the response times of Columbus police.\n\nOn Mother's Day, a 48-year-old man was shot and killed in front of his children and grandchildren – 30 minutes after a neighbor warned a police dispatcher that a drunk man was firing shots from his front stoop.\n\nPolice called the delay a fluke – that rare case when more important calls bogged them down from getting to a run that should have been the division's second highest priority.\n\nThe Dispatch wanted to know for sure.\n\nI requested a year's worth of the 911 dispatch records. In my request, I stated I wanted nothing omitted and I wanted them in a comma-delimited text format. Simple, right?\n\nHowever, the woman placed in charge of my request gave me separate files for each day – 366 in all. So I'd have to input and link each file for every Access query.\n\nThe dispatchers comments weren't included, I later learned, because they are stored separately.\n\nSo, I had to make the request again.\n\nAnd again, the files came back in separate databases.\n\nThe head of the department gave me a technical argument about the information being too large to store in one text file; if I wanted it in one file, he could do it, but in Access only.\n\nGod, did I feel dumb.\n\nI should have asked for them to give it to me in Access originally. I just never dreamed they had it. It taught me the first of many valuable lessons that would resurface later – never assume.\n\nThe data work\n\nAlas, finally after about a month of waiting and re-requesting, I had my data. I was so excited. I signed out a new IBM Winbook and examined the 897,870 records or calls that Columbus police took that year. (By the way, I was defining response time as the time it took between the time the call was dispatched to when the first police cruiser arrived on the scene.)\n\nI queried the data every which way. Average response times for gun runs, domestic calls, dead bodies, burglaries, stabbings – you name it.\n\nI looked at medians. Then I looked at response times by shift and precinct using Excel's pivot table. (What a marvelous invention.)\n\nI knew going into the project that nearly 50 percent of police in Columbus weren't logging their arrival times. It was disappointing, but we planned to use the data anyway and note the problem.\n\nAs I saw it, the information I had to use was the only information the division had to make important decisions such as staff allocations, requests for more manpower and citizen complaints. The response times might not be dead on, but I thought they would still show disparities in response times between the larger and smaller precincts. (Of course, with the help of data processing I did some more sophisticated queries first to see if the lack of reporting was similar across the city's 19 precincts and it remained fairly constant for all types of calls.)\n\nWhat I didn't anticipate was a programming quirk that plugs in the same time for when the call was received, dispatched and when the officer arrived.\n\nThe self-initiated run, as Columbus police call it, is supposed to be used when an officer witnesses a crime or calls the run into a dispatcher shortly after it happens.\n\nThose runs, though, comprised 28.9 percent of calls and couldn't be used. Factoring those out, we had about 21 percent of the total calls to use.\n\nThree independent statisticians assured us the data would still be fine to use if we stipulated that the averages were averages of the data available – and not averages of the division as a whole.\n\nA new direction\n\nAs controversial as it might have been, the thought of using it didn't last long. Doing queries on the data without the self-initiated runs showed how skewed the data can get when you start slicing it too much.\n\nResponse times for officer in trouble – a call officers use when their life or well-being is in danger – jumped to 8 minutes. (Without self-initiated runs, the average was about 3 minutes.) I knew the 8 minute-average was not only absurd, it was false.\n\nThough a young reporter, I've covered police on and off for six years and knew that was the call that police likely responded to the quickest.\n\nGoing back to police communications I got the answer I dreaded. The 911 system plugs in arrival times for the first car on scene who logs his arrival time.\n\nSo for example, if an officer shows up an hour later to handle traffic and is the first on scene to hit a button on his cruiser console signifying his arrival, that's the time the 911 system records. I knew then, the project as planned was dead.\n\nOf course, there was still a very important story: that the division has no idea how long they take to get to crime scenes. They can't.\n\nOf course, in retrospect I felt dumb for assuming the arrival time listed in the database was for the same officer listed on the department's run. Next time, I'll be very thorough in finding out how information gets plugged into each column.\n\nIn the end, the story prompted outrage by the City Safety Director and others, but no change.\n\nThe director ordered the officers be forced to record their arrival times, but the police contract has a provision that the city can't alter any past practices, unless changed in the contract.\n\nAnd the Fraternal Order of Police didn't see the lack of response times as a problem, so the director's order was moot.\n\nThe division is caught up in a battle too with the U.S. Department of Justice over claims that they abuse citizen's rights, so even with follow-ups, the issue is still likely going to be lost in the shuffle. It was a lot of work to uncover a record-keeping problem, but I sure learned a lot along the way.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-tipsheets-available-from-the-ire-resource-center","slug":"1999-november-tipsheets-available-from-the-ire-resource-center","url":"https://dwillis.github.io/uplink/article/1999-november-tipsheets-available-from-the-ire-resource-center.html","year":1999,"month":"November","monthNum":11,"headline":"Tipsheets available from the IRE Resource Center","kicker":"","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"This resource guide provides tipsheets on analyzing police staffing and covering crime using computer-assisted reporting (CAR), including guidance on story ideas, data sources, and television presentation techniques.","keywords":["police","crime","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Crime & Justice"],"technologies":[],"full_text":"\"Tips on Analyzing Police Staffing,\" by Geoff Dougherty, of the St. Petersburg Times, from the 1999 National CAR Conference. (Tipsheet #872)\n\nA tipsheet by Paul Adrian, of WAVE-TV in Louisville, Ky, includes answers to several important questions about CAR and the crime beat, including what kinds of stories you can do, where you can get data and how to get all of those data across on TV. (Tipsheet #439, from the 1995 National CAR Conference) Audio tape available through Sound Images, Inc., (303)649-1811, for $10. Ask for tape #CAR95-29.\n\nTo order these tipsheets or other items, contact the IRE Resource Center by phone at (573) 882-3364 or e-mail: rescntr@nicar.org and refer to the tipsheet numbers.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-uplink-story-ideas","slug":"1999-november-uplink-story-ideas","url":"https://dwillis.github.io/uplink/article/1999-november-uplink-story-ideas.html","year":1999,"month":"November","monthNum":11,"headline":"Uplink story ideas","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The NICAR newsletter invites journalists to submit stories using computer-assisted reporting techniques, technical solutions, and story ideas for publication, with contact information for the managing editor.","keywords":["journalism","reporting","submission"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Have you or one of your colleagues recently published a story using CAR that has not been done before or has involved particularly difficult data work?\n\nDo you know of a technical problem (or its solution) that others may like to hear about?\n\nIs there some issue or beat that we haven't covered? If you have a story idea, we'd like to hear from you.\n\nPlease contact managing editor MaryJo Sylwester either by e-mail at maryjo@nicar.org or by phone at (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-aftermath","slug":"1999-november-aftermath","url":"https://dwillis.github.io/uplink/article/1999-november-aftermath.html","year":1999,"month":"November","monthNum":11,"headline":"Aftermath","kicker":"KIDS AND VIOLENCE","author_name":"Scott Smallwood","author_title":"","affiliation":"Albuquerque Journal","summary":"A newsroom uses computer-assisted reporting to analyze school district violence and vandalism data, discovering that electronic data access and time management are crucial for timely investigation of educational crime statistics.","keywords":["data","schools","investigation"],"topics":["Investigative Methods","Crime & Justice","Education"],"technologies":[],"full_text":"out rates and test scores, to examine any possible correlation. Those went into a sidebar.\n\nOur immediate editors were helpful in freeing up some time for us to finish the project. But because the newsroom doesn't have a long history with computer-assisted reporting, others thought the stories would be ready the day after the school district gave us 400 pages of printed reports.\n\nA few weeks ago, the district released its newest violence and vandalism report. A new state law requires them to break it down by school site and for the first time officials gave individualized reports to each principal about crime and violence in his or her school. Also, after our fight in the spring, we were able to get the data electronically that same day.\n\nScott Smallwood can be reached by e-mail at ssmallwood@abqjournal.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-the-bush-files","slug":"1999-november-the-bush-files","url":"https://dwillis.github.io/uplink/article/1999-november-the-bush-files.html","year":1999,"month":"November","monthNum":11,"headline":"The Bush files","kicker":"CAMPAIGN FINANCE","author_name":"Aaron Diamant","author_title":"staff","affiliation":"IRE and NICAR","summary":"George W. Bush's campaign finance data is publicly available online but only in a massive PDF format that is difficult to work with, requiring specialized software and methods to convert into usable spreadsheet or database formats. The article outlines two approaches journalists can use to extract and analyze this data: a complex technical method used by the New York Times and a simpler manual method using PDF conversion tools.","keywords":["campaign","finance","data"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["PDF","PostScript","Excel","Access","Adobe Exchange","Monarch","Adobe Acrobat","Aerial","Word"],"full_text":"Amidst a great deal of self-congratulatory hype, George W. Bush has made available on his Web site (www.georgewbush.com) a complete list of every single campaign contribution he has ever received. Sounds great, except it's nearly impossible to work with.\n\nThe problem is that the data are only available as one giant .pdf file (and I mean a giant 1,750 pages and counting as of this writing and that's only for the 3rd quarter of 1999). So although the data don't cost you a dime, you really can't do anything with them unless you have lots of time and specialized software to convert them into a workable format.\n\nA few of you have written to the NICAR-L listserv in the last few months asking for help converting the monster .pdf file into a format that can be imported into Excel or Access.\n\nAnd as the 2000 election gets closer, more of you will be asked to take a look at, and produce stories on, the Bush campaign finance data from your area.\n\nThe following are some methods others have used:\n\nMacGyver method\n\nThis is the method used by reporters at The New York Times.\n\nStep 1. Write two PostScript programs that go automatically to the Bush Web site. One version should read the page and download the .pdf files you need. However, you don't want to download all the files after the first time. A second version of the PostScript goes to the directory and checks for the most recent additions.\n\nStep 2. Load the data into Adobe Exchange with a \"Redwing\" plug-in designed by Monarch. This extracts text from a .pdf file. Use the \"Extract all to Monarch\" command that parses the text. You may run into a problem with the multi-line format of the original data (sometimes there is a second line added for comments which can be lost). To solve it, go to step 3.\n\nStep 3. Run a PostScript that checks for spaces at the beginning of lines. If there is a space, write the script such that it will save the comment as a separate field.\n\nStep 4. Export as Tab Delimited.\n\nStep 5. Load into Access.\n\nDegree of difficulty: 9.5. Not recommended for anyone except the most serious programmers.\n\nMother Theresa method\n\nFor those of you who are fairly patient, this method may be for you. It is much simpler than the previous method, but there are more limitations.\n\nThere is an Acrobat Reader plug-in called Aerial, which a number of you may be familiar with. You can use Aerial to convert a .pdf to an .rtf document. This is a good tool for copying a table at once and pasting it into Excel.\n\nIt can convert the whole .pdf document at once, but is not infallible. This is where the Mother Theresa part comes in. Once you convert the .pdf to an .rtf document you can use a word processing program such as Microsoft Word to clean the data and convert it to the format of your choice for import into a spreadsheet or database.\n\nYou can download a trial version of Aerial from www.ambia.com/aerial.htm. For help using Aerial you can refer to an article written by IRE's Training Director Tom McGinty in the July/August edition of Uplink.\n\nDegree of difficulty: 5.\n\nFed Ex method\n\nIf you need to get data from a .pdf file that's posted on the Web, you can send a message to pdf2txt@adobe.com with the URL of the .pdf in the body of the message. The contents of the .pdf should be returned to you as a text message. You should get a response in a few minutes (sometimes a lot longer for larger files like the Bush .pdf). But be warned, you are going to have to clean the data yourself.\n\nDegree of difficulty: 4\n\nIndiana Jones method\n\nIndy was known for hunting for things that are already out there. Believe it or not there are actually a number of sites that","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-bush-data-sources","slug":"1999-november-bush-data-sources","url":"https://dwillis.github.io/uplink/article/1999-november-bush-data-sources.html","year":1999,"month":"November","monthNum":11,"headline":"Bush data sources","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This article provides a curated list of resources for accessing and downloading George W. Bush's campaign contribution reports and related financial disclosures in various digital formats.","keywords":["campaign","finance","downloads"],"topics":["Data & Databases","Government & Politics"],"technologies":["PDF","Excel"],"full_text":"www.georgewbush.com – George W. Bush's Web site, where the public can access his campaign contribution reports in .pdf format.\n\nwww.ambia.com/aerial.htm – At this site, you can download a trial version of Aerial, a software program that can help translate .pdf documents.\n\npdf2txt@adobe.com – An e-mail address to Adobe; send .pdf files here to be returned as a text message.\n\nwww.tray.com – Web site of Public Disclosure, Inc., where electronically filed presidential data is easily downloaded.\n\nwww.foodnews.org/bush.html – Food News' searchable database of Bush's campaign finance reports; however, data can't be downloaded.\n\nhttp://metalab.unc.edu/javafaq/bush – Elliotte Harold's Web site includes Bush's original data as tab-delimited documents, available for downloading.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-confidence-rates","slug":"1999-november-confidence-rates","url":"https://dwillis.github.io/uplink/article/1999-november-confidence-rates.html","year":1999,"month":"November","monthNum":11,"headline":"Confidence rates","kicker":"BEST OF NICAR-L","author_name":"","author_title":"","affiliation":"The Gazette","summary":"Journalists and statistical experts discuss how to properly interpret Census Bureau income and poverty data, including the correct confidence levels for state-level estimates and the proper statistical interpretation of confidence intervals.","keywords":["Census","confidence","statistics"],"topics":["Data & Databases","Demographics & Census"],"technologies":[],"full_text":"No doubt some of you are working today with the Census Bureau's just-released income and poverty rates. Naturally, I'm looking at Colorado's numbers in comparison to the other states.\n\nThe state-by-state estimates of median income and of poverty rates are accompanied by measurements of standard error. Does anyone know the confidence level associated with these standard errors? The appendices dealing with how to work with standard error seem to indicate that the confidence interval, at least for U.S.-level estimates, is 90 percent. Does that same confidence interval hold for the state-level estimates?\n\nA specific example: Colorado's three-year average median income is $44,349. The standard error is $1,075.\n\nWould I, then, be correct in saying something like this: \"The Census Bureau is 90 percent certain that Colorado's median annual income ranged between $43,274 and $45,424.\" ?\n\nA corollary question: Which of the median income (or poverty rate) measures is best to use? The three-year averages? The most recent 2-year rolling average? – Jeff Thomas, The Gazette, Colorado Springs, Colo.\n\nThe state-level data also is reported at the 90% c.l. [confidence level] This is noted in Table D in the footnote for the percent change column (those percent changes that are statistically significant). The Census release uses the three-year average for income (see the bulleted item on Alaska's income). This gives them a dartboard that's a bit larger. – Neill Borowski, The Philadelphia Inquirer, Philadelphia, Penn.\n\nThe Census Bureau usually reports error margins at one standard error, which means the confidence level is 68 percent. I don't have the report to which you refer, so I can't evaluate the indication that it is 90 percent. But for 90 percent confidence, according to sampling theory, you need an error band that is 1.65 standard errors wide. – Philip Meyer, University of North Carolina, Chapel Hill, N.C.\n\nThis is a common mistake in interpreting confidence intervals. We cannot be certain that the true mean (or whatever parameter) falls within the confidence interval. The correct way to interpret a confidence interval is to say: \"In repeated sampling, our range (confidence interval) would encompass the true mean 90% of the time.\" Thus, if we were to sample from a population 100 more times and determine the mean and respective intervals for each of those samples, then 90 of the intervals would contain the true population mean. – Mary A. Davis, Center for Environmental Studies, Virginia Commonwealth University, Richmond, Va.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-november-y2k-on-the-web","slug":"1999-november-y2k-on-the-web","url":"https://dwillis.github.io/uplink/article/1999-november-y2k-on-the-web.html","year":1999,"month":"November","monthNum":11,"headline":"Y2K on the Web","kicker":"","author_name":"Russell Clemings","author_title":"","affiliation":"The Fresno Bee","summary":"A tipsheet from The Fresno Bee's Russell Clemings provides resources for investigating Year 2000 (Y2K) conversion issues, highlighting the President's Council on Year 2000 Conversion website as a key official source.","keywords":["Y2K","investigation","resources"],"topics":["Investigative Methods","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Below is an excerpt from a handout compiled by Russell Clemings of The Fresno Bee, for a regional Investigative Reporters and Editors conference in Los Angeles in September. A copy of the full tipsheet is available by calling the IRE Resource Center at (573) 882-3364 and asking for Tipsheet #1095.\n\nSome good Y2K Web sites:\n\n• President's Council on Year 2000 Conversion: www.y2k.gov\n\nThe most official of the official sites. Contains links to Y","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-in-pursuit-of-data","slug":"1999-december-in-pursuit-of-data","url":"https://dwillis.github.io/uplink/article/1999-december-in-pursuit-of-data.html","year":1999,"month":"December","monthNum":12,"headline":"In pursuit of data","kicker":"DEALING FOR DATA","author_name":"Brant Houston","author_title":"Executive Director","affiliation":"IRE and NICAR","summary":"The article provides guidance on obtaining government databases through freedom of information requests, recommending a two-phase approach: first attempting friendly negotiations with agencies, and if unsuccessful, escalating through humor, legal experts, and appeals to higher authorities.","keywords":["FOIA","databases","negotiation"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"You believe in Sunshine laws and the Electronic Freedom of Information Act. You might believe those laws work. And sometimes they do. But there are many times when all you get for your trouble is:\n\nA) A simple rejection largely unsupported by law or regulation.\nB) Endless delays.\nC) A $25 million bill for 10 hours of data work.\n\nThere are good times to avoid freedom of information requests and there are other times, as Sarah Cohen points out in her article (see page four), you may have no other choice than to go the long arduous route of FOIA. In either case, you will probably enter that familiar, gray area of negotiations.\n\nHere are some basic steps that many have followed in the pursuit of federal, state and local databases.\n\nIf it's not on your beat, get a good feel for the agency before you approach it. Check their Web site, if they have one. Check with fellow reporters on how the agency responds to requests. Try to get its annual report and an audit of it. You get more respect if you show up knowing the agency's real business, what databases it definitely has, and how it has fared in its last audit.\nBe pleasant. Be casual. Don't be rushed. Set aside some time for your first visit there. If the agency is out of state, work the phone the same way.\nTry the low-key approach. You know you add to their burden. You know they are understaffed and underfunded. You want to make the transaction as quick and painless as possible. You are understanding. You will narrow your request in exchange for rapid response. Make sure to get the preliminary information about the database. (See Dawn Fallik's article on page three).\nFigure out who actually knows the databases. If necessary, arrange a meeting with the public information officer and the data processors.\nKnow what you really want and how it might differ from what they may have. This will allow you to give up getting certain data for expediency's sake. (Just don't allow it to be a precedent.)\nTry to work with them so that if you have to make a written request it is exactly the one they need to give you the database.\n\nPlan Two\n\nNo success with the easy, nice way? No databases forthcoming? It's time for Plan 2. Most negotiations with recalcitrant agencies begin with their absurd requests for money such as $45 million or $50,000. Don't be shy about using that absurdity for a humor story or to relate humorously to other officials. Have freedom of information experts on hand to comment about the ridiculous behavior.\n\nTake it seriously, but also think of it as sport with a little contest awaiting you each day. Your blood vessels will explode if you get mad every time they put you off.\n\nThen get them to tell you what laws they are citing. Then call the attorney general, commissions, the governor's office, the supervisors and ask them if the rejection sounds reasonable?\n\nYou have to use your instincts, but a daily call not two or three daily calls- is not unreasonable for dealing with unreasonable people.\n\nGiven that the agency is stonewalling, it is perfectly legitimate to start asking for hard copies of records, budgets, audits, reviews and consultant reports to learn more about this obstacle to freedom of information. Don't do it to harass. Do it because they are withholding public information. Call other agencies for information, audits and reports on that agency. Find out what the agency is really up to. It's a good way to get stories while you wait. Call staffers who are not authorized to speak to you. Call former employees. Ask for more meetings to work out the dispute. Write many more FOIAs, if necessary. Many times reporters seeking databases have been asked after several months: \"What will it take to make you go away?\" Often, the officials forgot it all started with a simple database request. They may be shocked when you remind them of that. \"Is that all?\" they say. Yes.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-hidden-dangers-airlines-and-hazmat","slug":"1999-december-hidden-dangers-airlines-and-hazmat","url":"https://dwillis.github.io/uplink/article/1999-december-hidden-dangers-airlines-and-hazmat.html","year":1999,"month":"December","monthNum":12,"headline":"Hidden Dangers Airlines and HazMat","kicker":"HIDDEN DANGERS","author_name":"Elizabeth Marchak","author_title":"","affiliation":"The Plain Dealer","summary":"The article investigates ongoing issues with hazardous materials on airlines following the 1996 Valujet crash and evaluates the effectiveness of FAA's enforcement.","keywords":["airlines","hazardous materials","FAA"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"In 1996, the Federal Aviation Administration promised to change the way it oversaw aviation, including the problem that was believed to have caused the May 11, 1996 Valujet crash - hazardous materials. In March 1997, the FAA started issuing press releases when the agency proposed fines of $50,000 or more for HazMat violations. Meanwhile, there were tips from sources that the FAA's press releases didn't tell the whole story. One person offered as proof an internal document which laid out some FAA officials' interpretation about enforcing HazMat training laws. Fifteen months after the crash, the document said, the FAA's Southern Region was still at odds with the rest of the agency's enforcement officials about whether airlines that didn't carry hazardous materials even needed to train their workers to recognize the stuff. With the disparity between the FAA's press releases and my newly-acquired internal document, I turned to the data. I was spurred on by Congressional testimony that said passenger airlines accounted for 60 percent of the cargo flown on airplanes. Ten months later, on Jan. 17-18, 1999, we published a two-part series called 'Safety on Hold: The hidden dangers in airline cargo'. We made extensive use of two government databases. I examined 20 years' worth of FAA enforcement data, studying trends by violator, frequency, charge, year, region, FAA office, and resolution. My findings filled one of those fat, three-ring binders. The data showed the FAA's enforcement peaked in the six months following the much-publicized crash, assessing $3.2 million in fines for the whole year. In 1997 and for the first half of 1998, the number of violations and the fines decreased. Confronting the agency with its own data - analyzed the way they analyze it - put them in a box. Then I spent several more weeks studying HazMat reports to DOT's Research and Special Programs Administration. RSPA's data is in tables which cannot all be linked because the comments field produces a Cartesian product. But some data can be linked on a far smaller scale if the record numbers from the main table are linked to the table with the comment field. I did this to track down airlines' reports, which gave me anecdotes and information about all the efforts the airlines make to contain leaks and spills. Meanwhile, the number of HazMat reports of spills and leaks had increased 623 percent for the decade through June 1998, far outstripping the growth in cargo and way ahead of other transportation modes for the same 10-year period. But the data my editors found most interesting was how enforcement varied among the nine FAA regions.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"1999-december-privatized-data","slug":"1999-december-privatized-data","url":"https://dwillis.github.io/uplink/article/1999-december-privatized-data.html","year":1999,"month":"December","monthNum":12,"headline":"Privatized data","kicker":"THE VENDOR QUESTION","author_name":"Brant Houston","author_title":"Executive Director","affiliation":"IRE and NICAR","summary":"Privatization of public data by government contractors creates barriers for journalists seeking access to records, with vendors charging excessive fees despite legal protections for fair pricing. The article offers strategies for journalists to obtain data directly from agencies or through alternative sources.","keywords":["privatization","data","vendors"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"Privatized data\n\nBy Brant Houston\nIRE and NICAR Executive Director\n\nWhat do you do when government contracts with a private vendor to handle public data?\n\nYou suffer even more when you try to get that data.\n\nPrivatization of data appears to be an increasing problem for journalists trying to purchase copies of public databases.\n\nNow you not only have to battle with public officials, but you also have to deal with businessmen eager to make a buck.\n\nRecently, a reporter from a New Jersey weekly wrote to the NICAR listserv, saying that a private vendor wanted $425 (or 5 cents per property) for the property records in two small towns.\n\nWhile other New Jersey reporters gave good advice the records (unfortunately a year old) from the state for free - it's a problem that is not easy to circumvent.\n\nFor example, when some states passed laws earlier this decade on access to electronic information, they provided that the public should be able to get the information for a fair and reasonable price. Nonetheless, vendors are stubborn about releasing the data and agencies often offer little help.\n\nHere are some possible open avenues:\n\nMake sure to ask the agency if they keep their own copy of the data. (Many journalists forget to do that.) It can be a long shot, but it has worked in the past. If so, then ask them to make a copy for you when they back it up.\nFind out if the agency gives a copy of the data to another agency. This is what the New Jersey reporters suggested on the NICAR-L thread. Check both the state and federal levels.\nKnow what the law says and if it says the vendors can't overcharge, then write a letter to them citing the law and \"cc-ing\" the agency and any other appropriate officials such as the attorney general.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-successful-e-foia","slug":"1999-december-successful-e-foia","url":"https://dwillis.github.io/uplink/article/1999-december-successful-e-foia.html","year":1999,"month":"December","monthNum":12,"headline":"Successful E-FOIA","kicker":"E-FOIA","author_name":"Dawn Fallik","author_title":"database librarian","affiliation":"NICAR","summary":"Journalists requesting data from government agencies should first gather detailed technical information about the dataset and then submit a comprehensive FOIA request that specifies the exact format, fields, and documentation needed to properly analyze the data.","keywords":["data","FOIA","database"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Access","FoxPro","9-track tape","CD-ROM"],"full_text":"At the NICAR database library, we get a lot of calls that sound like this:\n\n\"I finally got some data from our local police force, but I don't know what's on it. Can I just send it to you and you tell me what it is?\"\n\nUnfortunately, the answer is no. Without some basic information about data on the tape, disk or CD-ROM, it's nearly impossible to decipher what the agency delivered.\n\nWhether you're analyzing the data yourself or sending it to NICAR, the best approach for avoiding this dilemma is to first do a little homework, then write a detailed Freedom of Information request to make sure you get the proper paperwork.\n\nThe right questions\n\nBefore you even order the data, call the tech support or data person for the agency - whomever is in charge of data collection. Try not to talk to the public relations person because they usually have no idea what a database is, never mind which ones the agency oversees.\n\nOnce you get to the right person, ask what the exact name of the dataset is, request a record layout in advance (which will tell you exactly what fields are in the dataset) and ask how they keep the records.\n\nMany agencies are now using common computer programs such as Access or Foxpro, but others will want to send you the information on a 9-track tape, 3480 cartridge or Jaz cartridge.\n\nNine-track tapes will be the most likely media you might run into.\n\nHere's a checklist of questions you should ask when ordering data on nine-track tape (thanks to Brant Houston's book, \"Computer-Assisted Reporting: A Practical Guide.\")\n\nWhat is the tape's density? 6,250 bpi, 1,600 bpi or other?\nWhat is the record length? Is it fixed or variable? Are the records standard or defined?\nWhat is the blocking factor or block size?\nIs the tape labeled or unlabeled?\nIs the database in ASCII or EBCDIC?\nWhat kind of machine is producing the tape? IBM, Hewlett-Packard, Digital Equipment Corporation or other?\nHow many records are contained on the tape?\nHas the information been compressed or packed?\n\nIf you don't have the capability of converting data from these formats, NICAR can usually help. But please give us a call to make sure before you send anything to us.\n\nIn addition to the above information, NICAR also likes to have an agency contact, just in case we have questions.\n\nThe FOIA letter\n\nNow you are ready to write your FOIA request. Here is a list of 10 things you should include in your eFOIA to better your chances:\n\n1. Mention the exact name of the dataset.\n2. Request all the fields collected in the database.\n3. Request all the code sheets used with the database.\n4. Request the record layout.\n5. Ask for a record count so you can make sure you've gotten the entire database.\n6. Ask for a printout of the first 100 records.\n7. If the dataset includes more than one table, ask for a relational chart of how all the tables join.\n8. If the record layout does not include a description of the fields, ask for a data dictionary, which gives more details.\n9. Specify how you would like your data. You should know what is available from your discussion with the tech person. Say \"We would like this data in dbf format on a CD-Rom\" or \"We would like this data in text format on a zip disk.\" Sometimes you will have to supply the medium.\n10. If it is state data, cite the local Sunshine Law. If it is a federal database, the FOIA citation is U.S. Freedom of Information Act, 5 U.S.C. 522.\n\nBe prepared to argue over field elimination and price. The more you know about the data to begin with, the stronger position you'll have when haggling.\n\nDawn Fallik may be reached by e-mail at dawn@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-federal-data-fights","slug":"1999-december-federal-data-fights","url":"https://dwillis.github.io/uplink/article/1999-december-federal-data-fights.html","year":1999,"month":"December","monthNum":12,"headline":"Federal data fights","kicker":"DEALING FOR DATA","author_name":"Sarah Cohen","author_title":"database editor","affiliation":"The Washington Post","summary":"A Washington Post database editor's attempt to obtain immigration hearing data from the Justice Department starts with a promising meeting but devolves into bureaucratic delays and FOIA denials, illustrating how federal agencies use procedural tactics to avoid releasing information.","keywords":["FOIA","bureaucracy","immigration"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Access","FTP"],"full_text":"In the second week of my job as database editor for The Washington Post, a reporter asked for help in finding data on immigrants' hearings for asylum.\n\nHe knew the agency operated under strict secrecy, but had in his hands data produced for a similar response from a local lawyer. The database seemed like a gold mine anyway, so I figured I'd try for both the summary data and the underlying case management system.\n\nIt started pleasantly enough. Within a week or so, I had an appointment to visit the Justice Department's Executive Offices of Immigration Review. Meeting with the head of the systems department seemed like a dream come true. I left with a deal: She'd have my case management system ready on the following Monday, stripped of sensitive information. And I got to leave with the user manual that showed all of the codes I needed. Everyone agreed this would be simpler for them than to run the statistics we'd asked for.\n\nI was shocked. I'd heard stories of years of negotiations to get simple federal databases. This one even had some sensitive information in it, and they were willing to create something I could use without disclosing the exempt data.\n\nNot so simple\n\nMy naivete worked in the agency's favor. Nine months later, we don't have the database. Four months after my visit, the reporter who needed the data for his story won a FOIA appeal and the agency eventually produced the statistics he sought.\n\nThe story of how we went from friendly cooperation to a strong denial has been repeated in other agencies here. And these FOIA officers are better than their state and local counterparts in understanding reporters' short attention spans and using every day they can to avoid releasing data.\n\nOur friendly meeting with EOIR went downhill pretty quickly. The following Monday, I was told that the systems analyst was ready to run my extract, read it into an Access database and FTP it to us as soon as the agency's general counsel glanced at the list of fields we'd get. Problem was, the lawyer was out sick and no one else could take her place.\n\nAfter two weeks of illness, she returned to \"higher priority\" items. Daily calls to the agency produced nothing, so I offered to write a FOIA request, which would at least put the request in their list of higher priority requests. I still wonder if this was a mistake. The public information officer I was dealing with was delighted - \"Sure, go ahead!\" he said. The clear implication: It would be taken out of his office, and put into the bureaucratic hopper where I'd be lost forever.\n\nDenials\n\nSoon the FOIA calls started. Did they get it? Do they plan to respond to it? And on the 20th business day, I got my rejection. The reasons were ludicrous. First, the case number incorporated an alien's ID number, which the agency likened to a Social Security number. That, they said, could be used to identify people at risk. The problem is, only that agency \"has\" that ID number, and it would have to be the one to give out the information that it unlocked. Second, it refused to encode those numbers.\n\nFinally, it rejected the entire package because, it said, the records were compiled for law enforcement purposes and would interfere with their ability to conduct investigations. No one could think of how, other than that \"lenient\" judges might be chosen by defendants if possible during appeals.\n\nMy second FOIA was eventually slightly more effective, but it took eight months of near-constant bickering to produce just a portion of the firearm trace database maintained by Alcohol, Tobacco and Firearms.\n\nThis may sound familiar, but it raises the level of secrecy and the lack of accountability to a new level that I've not seen in even some of the strictest states.\n\nBut the fights are worth undertaking. For any reporter who works in a border area, or who needs to compare trends from state-to-state, getting the baseline federal data is still important. Just be prepared to wait even longer, fight even harder and be even more persistent than usual.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-foi-resources","slug":"1999-december-foi-resources","url":"https://dwillis.github.io/uplink/article/1999-december-foi-resources.html","year":1999,"month":"December","monthNum":12,"headline":"FOI resources","kicker":"DEALING FOR DATA","author_name":"David Wethe","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists seeking help negotiating for data access can turn to several organizations and resources, including the Reporter's Committee for Freedom of the Press, NICAR, IRE, and the Society of Professional Journalists, which provide guides to open records laws, tipsheets, and FOIA assistance.","keywords":["data","FOIA","records"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Negotiating for data can seem like a daunting task if you have to face troubles all on your own. Fortunately for reporters, though, there are several places you can turn for help.\n\nHere's just a few of the possibilities: The Reporter's Committee For Freedom of the Press Web site (www.rcfp.org) offers a guide to open records and meeting laws in all 50 states. Each state has slightly different standards that record custodians are held to, and each Sunshine Law is explained in depth on this site.\n\nThey also have a section on electronic records (www.refp.org/elecress), which includes the statute from each state specifying access to electronic records.\n\nFurther questions about each state's Sunshine Law can be directed to Byron Brown at (703) 807-2100. Rebecca Daugherty can also answer more broad questions about the national Freedom of Information law.\n\nThe NICAR listserv archive includes many messages with questions about negotiating for data, with answers posted by fellow list members. There is a good chance you might find some suggestions from another journalist who has faced the same brick wall. Even if you don't subscribe to the list, you can check the archives through the NICAR Web site (www.nicar.org). Click on the phrase at the bottom \"Join NICAR-L, a mailing list on CAR techniques.\" The link to the archives is at the bottom of the page.\n\nIRE's Resource Center (www.ire.org/resourcecenter) is also a helpful place to find tipsheets that take you step-by-step through the negotiating process.\n\nThese tipsheets, written by journalists who faced similar hurdles in their negotiations, are archived by subject index.\n\nThe Society of Professional Journalists maintains an extensive FOIA Resource Center on its Web site, where you can find a state-by-state listing of FOI contacts, information on the latest Freedom of Information court cases and legislation, and instructions on writing a FOIA.\n\nAnd if you're getting some strange excuses from public officials, check out the Poynter Institute's Web site.\n\nThere, Nora Paul has included her compilation of the top 38 excuses from government agencies, and suggestions for overcoming them. That page is located at www.poynter.org/car/cg_excuse.htm\n\nDavid Wethe can be reached by e-mail at c742109@showme.missouri.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-overcoming-excuses","slug":"1999-december-overcoming-excuses","url":"https://dwillis.github.io/uplink/article/1999-december-overcoming-excuses.html","year":1999,"month":"December","monthNum":12,"headline":"Overcoming Excuses","kicker":"DEALING FOR DATA","author_name":"Jennifer LaFleur","author_title":"database editor","affiliation":"St. Louis Post-Dispatch","summary":"This tipsheet provides journalists with strategies for overcoming common excuses from data custodians who resist providing access to databases, including tactics for dealing with claims about complexity, cost, unavailability, and proprietary restrictions.","keywords":["databases","journalism","data"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["ASCII"],"full_text":"Overcoming Excuses\n\nThis is an excerpt from a handout created by Jennifer LaFleur, currently database editor at the St. Louis Post-Dispatch, for the 1997 National CAR Conference. The full version of the tipsheet is available at the IRE Resource Center. Ask for tipsheet #671.\n\nExcuse #1: Our database is very complicated and you probably won't understand it.\nAnswer: \"Oh really, are you using some kind of variable-length or redefined records in your relational database design that won't just export to an ASCII comma-delimited file?\" If you're not dealing with someone who knows the data, this usually gets you there.\n\nExcuse #2: Our computer system can't do that.\nAnswer: \"Really? That's unusual for today's database systems. What system do you use?\" This should eventually get you to someone who knows what's going on. You may also run into cases where they truly do not understand how to do anything with their computer except run the reports they know how to run. Find out who the vendor is and talk with them. I've also run into cases where they used software I was familiar with and could talk them through the process over the phone.\n\nExcuse #3: The person who knows how to do that is on vacation for two weeks/doesn't work here anymore.\nAnswer: \"No one else knows how to do it? What if there was an emergency?\" This sounds like an even better story.\n\nExcuse #4: It will cost you $20,000.\nAnswer: Ask for an itemized estimate of charges. Three out of four times, this gets the cost down at least some. Offer to pay reasonable programming fees. You should only have to pay the programmer their hourly rate, not overhead.\n\nIf they do programming, ask for a copy of the program or at least have them put in writing that they will save the program in case you need the data again next year.\n\nSee if there is a rate charged by state agencies to other agencies.\n\nProvide your own tape. Ask for a backup tape. In some cases, this actually might be the price charged to that agency from a larger state data processing center.\n\nDo a records request for the data they've provided and what they charged other organizations.\n\nExcuse #5: That uses proprietary software.\nAnswer: You don't want software. You want the data.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-coming-soon","slug":"1999-december-coming-soon","url":"https://dwillis.github.io/uplink/article/1999-december-coming-soon.html","year":1999,"month":"December","monthNum":12,"headline":"Coming soon","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Uplink's next issue will feature stories on analyzing Census 2000 race classifications, applying computer-assisted reporting to religion coverage, and teaching CAR techniques in academic settings.","keywords":["census","religion","education"],"topics":["Demographics & Census","Investigative Methods","Education"],"technologies":[],"full_text":"Coming soon\n\nNext month's issue of Uplink will include stories on:\n\nDealing with new race classifications in the Census 2000 data\nUsing CAR to cover religion\nCAR in the college classroom","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-foreign-planes","slug":"1999-december-foreign-planes","url":"https://dwillis.github.io/uplink/article/1999-december-foreign-planes.html","year":1999,"month":"December","monthNum":12,"headline":"Foreign planes","kicker":"AVIATION DATA","author_name":"Maryjo Sylwester","author_title":"","affiliation":"IRE and NICAR","summary":"The article provides resources for journalists to access data on foreign aircraft registrations following the EgyptAir crash, including commercial databases like Aviation Data Service and Airclaims, as well as free online registration databases for various countries.","keywords":["aircraft","databases","aviation"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["World Wide Web"],"full_text":"Foreign planes\n\nBy Maryjo Sylwester\nIRE and NICAR\n\nWhen the EgyptAir flight 990 crashed into the Atlantic Ocean on Oct. 31, reporters across the country were scrambling to find data about foreign airlines.\n\nNICAR, which has a substantial collection of aviation data, only has records on planes registered in the United States. (NICAR does have some enforcement data on foreign planes) So where else can you turn the next time something like this happens? There are at least a couple of places for datasets. But be prepared to pay big bucks. Aviation Data Service, in Witchita, Kan., claims to have one of the most extensive databases of both U.S. and international planes. They collect the data from a network of sources, including civil registries, plane and engine manufacturers, insurance companies, dealers and others.\n\nThey sell three databases: one consisting of business jets and turbo jets; another of helicopters and one of commercial airlines. You can get just U.S. registered or international planes, or both. The information is updated monthly.\n\nA query on their prices just after the EgyptAir crash came back with these estimates:\n\nAll three datasets, international planes only: $50,000 to purchase one time, or $3,000 per month on a subscription basis.\n\nJust the commercial dataset, international planes only: $30,000 one time, or $2,000 per month on a subscription basis.\n\nAviation Data Service can be reached at (316) 262-1491.\n\nAnother source for international airplane data is Airclaims in London, England. The organization's database, called \"CASE,\" has full histories of more than 60,000 aircraft including commercial jets, turboprops and business jets worldwide. Cost is $10,000 annually on a subscription basis.\n\nMore information is available on the company's Web site, www.airclaims.co.uk/index.htm or by calling their North American account manager Robert Grundy in Costa Mesa, Calif., (714) 437-3103.\n\nThere are a few Web sites that offer online databases of airplane registrations for foreign countries that might be worth checking-with caution, however.\n\nDutch registrations: www.scramble.nl/ciol phlnedreg_main.htm\n\nIrish registrations: www.web-ie.com/irishav/\n\nLandings, at www.landings.com, also includes links to several official and unofficial Web sites for registries of other countries, including Bermuda, Canada, Italy, Singapore, South Africa and Sweden.\n\nMaryJo Sylwester can be reached by e-mail at maryjo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-down-the-road","slug":"1999-december-down-the-road","url":"https://dwillis.github.io/uplink/article/1999-december-down-the-road.html","year":1999,"month":"December","monthNum":12,"headline":"Down the road","kicker":"FIRST VENTURE","author_name":"Alfonso Chardy","author_title":"Reporter","affiliation":"The Miami Herald","summary":"A journalist describes how computer-assisted reporting (CAR) tools, particularly Microsoft Excel, enable deeper data analysis and discovery rather than just surface-level coverage, using traffic volume data from Miami-Dade County as a practical example.","keywords":["reporting","data","journalism"],"topics":["Investigative Methods","Technology","Business & Economy"],"technologies":["Excel"],"full_text":"When I first heard about computer-assisted reporting, I thought a super smart computer had finally been invented to either write stories or - at the very least - make the phone calls and do all the reporting needed for a story.\n\nJokes aside, computer-assisted reporting may well be the best thing to happen to journalism since objectivity became the non-negotiable standard in the profession.\n\nThe reason: CAR helps all journalists write the type of stories that have never been written before- stories with the precision of scholarly studies.\n\nThe power of the computer to order and group figures and information on the screen in ways not possible on paper enable reporters to discover hidden information that was available before only to a limited number of government officials or business managers.\n\nWith CAR, journalists can go from merely covering to discovering information.\n\nAllow me to tell you how I've been able to put my newly-acquired skills in CAR to use. Keep in mind that I'm no expert by any means. I only went to my first computer-assisted class - aptly called boot camp August at the University of Missouri School of Journalism in Columbia, Mo.\n\nOn the beat\n\nIt was actually during the IRE and NICAR boot camp that I realized, on the very first day, that I could use these CAR skills on my beat.\n\nI cover transportation in Miami-Dade County - and one of the biggest parts of my job is to monitor and write about traffic congestion on the highways.\n\nSo, when Brant Houston, the IRE and NICAR executive director, described the properties of Microsoft's Excel spreadsheet program, I knew instantly that I could apply the software to traffic figures.\n\nThat's because one way of measuring whether a road is congested is to determine how many vehicles it's carrying and the only way to determine that is to count the number of vehicles that pass at any given time at a pre-determined spot on the road.\n\nThe state Department of Transportation publishes the results of their counts in thick documents listing the average daily traffic volume on all state roads.\n\nBut those reports are difficult to understand or analyze. However, once I understood how to use Excel, I knew I could apply the software to the traffic volume reports and develop stories with unusual angles.\n\nSo as I participated in the NICAR boot camp in August, I began to develop my first Excel database of average daily traffic figures for major expressways in Miami-Dade County.\n\nWith CAR, journalists can go from merely covering to discovering information.\n\nI had to enter by hand into a new Excel file the DOT traffic numbers because they were not available in electronic form.\n\nTo make it simple, I entered data only for the major expressways like Interstates 95 and 75, among others. So every day, after boot camp finished, I would come back to my bed and breakfast hotel room and enter traffic figures for each of the 10 or so expressways I picked at random to determine whether traffic had gone up or declined in the county. I had sets of figures for 1997 and 1998.\n\nA better view\n\nBy the end of the week of boot camp a clearer picture emerged. Something I had never realized before. Traffic was up - way up - not only on all major expressways in the county but particularly on those that link Dade to neighboring Broward County.\n\nOn one roadway, I-75, traffic was up a whopping 99 percent from about 60,000 to almost 120,000 annual daily vehicles near the Broward-Dade line.\n\nThis statistic, though it was in the 1998 DOT traffic volume report, was not immediately apparent to me until I used Excel to compare the figure with the 1997 report.\n\nThat way I was able not only to see the difference between one year and the next but also actually calculate the difference.\n\nThe software gave me the ability to add, subtract, average and do percentages that before would have been difficult since I'm not a math wizard by any means. Often I still resort to my trusty fingers to do or confirm an operation.\n\nAt any rate, learning Excel and how to apply it to DOT numbers led to the production of three stories for The Miami Herald within the first couple months after boot camp.\n\nThe first was a simple one that told readers how traffic volume had gone up on major expressways in the county from 1997 to 1998, just to show off my new skills.\n\nThe second story was much more complex. By calculating traffic volumes, I was able to discern that the heaviest traffic increases were on roads linking Miami-Dade County with neighboring Broward County.\n\nThe significant discovery here is the explanation why traffic had increased on these cross-county roads: that a large percentage of people who once lived in Miami-Dade County had moved to new homes across the county line in Broward but kept their jobs in Dade.\n\nThe \"reverse commute\" began after Hurricane Andrew devastated South Dade in 1992 and has accelerated in the last two or three years.\n\nWhile I think I've become pretty adept at manipulating information in Excel, dealing with serious database programs like Access is still a challenge.\n\nIt may be a different story down the road, so to speak.\n\nAlfonso Chardy can be reached by e-mail at Achardy@herald.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-fatal-accidents","slug":"1999-december-fatal-accidents","url":"https://dwillis.github.io/uplink/article/1999-december-fatal-accidents.html","year":1999,"month":"December","monthNum":12,"headline":"Fatal accidents","kicker":"NICAR DATA LIBRARY","author_name":"Maryjo Sylwester","author_title":"","affiliation":"NICAR","summary":"The Fatal Accident Reporting System (FARS) dataset, maintained by the U.S. Department of Transportation since 1975, contains detailed information on fatal traffic accidents and can be used by journalists for enterprise stories and breaking news reporting. The database includes comprehensive data on accident conditions, vehicles, drivers, and fatalities, with examples of how newsrooms have used it to investigate highway safety and accident trends.","keywords":["accidents","database","journalism"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"The Fatal Accident Reporting System (FARS) dataset, which includes fatal accidents on U.S. public roads where at least one person died within 30 days of the accident, can be used for a variety of enterprise stories and as a resource for breaking news. NICAR has data available from 1975 through the end of 1998.\n\nThe data, collected by the U.S. Department of Transportation since 1975, includes an impressive array of details on each accident, including:\n\nWeather, light and road conditions.\nWhether a school bus, police or emergency vehicle was involved.\nPrevious speeding, drunk driving and other convictions of drivers.\nAlcohol and drug test types and results.\nSeat belt use and air bag deployment.\nPerson killed was on the job at the time.\nAccident following a police pursuit.\nAccidents in designated school zones.\nDrivers cited for inattentiveness, drunk driving or other violations.\nVehicle make, model and body.\n\nNew in the 1998 data are fields identifying the motor carrier identification number (which is required for trucks and buses crossing state lines) and driver height and weight.\n\nThe Roanoke (Va.) Times used the FARS data, along with state data on vehicle miles driven, to show that an interstate in Virginia was not as deadly as many people had come to believe.\n\nThe Star-Ledger in Newark, N.J. used FARS to report that fatal accidents had increased on a highway after the speed limit was increased to 65 mph. But they also discovered the increase was not unusual, after comparing the change to historical data.\n\nHaving the dataset readily available in your newsroom could also pay off for breaking news. For example, the next time someone is killed on the local interstate, you could be the first to say exactly how many people have been killed on that highway in the past 15 years. (The DOT didn't start including the exact name of the highway until the 1982 data).\n\nThe database consists of four relational tables representing the four levels of information police collect: accident, vehicle, person and driver.\n\nMore information is available on the NICAR Web site at www.nicar.org/datadfars MaryJo Sylwester can be reached by e-mail at maryjo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-bundling-bucks","slug":"1999-december-bundling-bucks","url":"https://dwillis.github.io/uplink/article/1999-december-bundling-bucks.html","year":1999,"month":"December","monthNum":12,"headline":"Bundling bucks","kicker":"CAMPAIGN FINANCE","author_name":"Dana Wilkie","author_title":"","affiliation":"The San Diego Union-Tribune","summary":"A journalist describes investigating McCain's campaign contributions using database analysis to uncover bundling practices, where coordinated donations from employees of the same company circumvent federal campaign finance limits.","keywords":["bundling","contributions","McCain"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["Access","World Wide Web"],"full_text":"I first got the idea for this story during a computer seminar for the Paul Miller Washington Reporting Fellowship. We were using Microsoft Access, learning how to more efficiently import large databases from the Web and how to manipulate that data.\n\nOur instructor mentioned that the Microsoft program was excellent for unearthing large numbers of checks written to candidates on the same day - a practice known as \"bundling.\" The practice allows companies and industries to get around federal limits on their own gift-giving by having many of their employees give money instead.\n\n\"Many of McCain's contributions came at the urging of CEOs who were close to McCain, who had done business before the senator's Commerce Committee.\"\n\nShortly afterwards, my editor assigned me to look at campaign contributions to presidential candidates and to see if any good stories came of it. It struck me that since Sen. John McCain was setting himself up as a champion for campaign finance reform, it might be interesting to see if he took advantage of any fundraising loopholes himself, such as bundling.\n\nImporting data\n\nWe imported presidential fundraising data from the FEC Web site directly into Microsoft Access. We separated out McCain's contributions, then asked the program to show each donation by date. Since \"bundling\" typically involves coordinated contributions from people working for the same boss or company, we also asked the computer to show the \"occupation\" and \"employer\" of each contributor beside the date of the donation. This allowed us to easily see if there were large numbers of donations that appeared on the same day, or within a few days of one another, and from people working for the same company or industry.\n\nAs the story points out, we found several contributions written on the same day by employees of the same company. We wanted to see if these employees' relatives had given money on the same day, too, so we searched for addresses that matched the addresses of those we knew had given to McCain. We found several matches.\n\nFilling in by phone\n\nWe were able to fill in the details about the contributions through phone interviews - where the checks were collected, who hosted the event, how the host invited his employees, whether there was pressure of any kind to attend, etc. While speaking with campaign-finance-reform-minded groups, we learned that the Center for Responsive Politics had recently done an analysis of \"bundled\" contributions and published a report naming McCain as one of the campaign's top bundlers. We conducted interviews with CRP staff and referred to CRP's report. Clearly, many of McCain's contributions came at the urging of CEOs who were close to McCain, who had done business before the senator's Commerce Committee, and who had given the senator all the money they were allowed to give under federal election law. The large number of checks showered on McCain - at the same time and by people at the same company - sent McCain the unmistakable message that if the senator won the presidency, it behooved him to remember the generous donations of those who helped put him in the White House.\n\nDana Wilkie can be reached by e-mail at dana.wilkie@copleydc.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-drivers-records-access","slug":"1999-december-drivers-records-access","url":"https://dwillis.github.io/uplink/article/1999-december-drivers-records-access.html","year":1999,"month":"December","monthNum":12,"headline":"Drivers' records access","kicker":"PUBLIC RECORDS ALERT","author_name":"Maryjo Sylwester","author_title":"","affiliation":"NICAR","summary":"The U.S. Supreme Court is reviewing the constitutionality of the 1994 Driver's Privacy Protection Act, which restricts public access to drivers' license records, with implications for journalists who rely on such records for reporting.","keywords":["privacy","journalists","Supreme"],"topics":["Investigative Methods","Government & Politics","Journalism & Newsroom"],"technologies":[],"full_text":"The availability of drivers' license records is once again in the spotlight as the U.S. Supreme Court is considering the constitutional validity of a 1994 law that limited the public's access to the records.\n\nThe court's decision, which is expected sometime after the New Year, will also determine the fate of a provision in an appropriations bill passed by Congress - and signed by President Clinton - in October that changes the 1994 Driver's Privacy Protection Act (DPPA) to limit access further.\n\nEffect on journalists\n\nFor journalists, this could mean either a return to full access to drivers' records or a complete roadblock, depending on the Supreme Court's ruling.\n\n\"If they want to use the big 'P' word - privacy - to invoke all this unnecessary legislation, we need to speak up as journalists,\" said Kyle Elyse Niederpruem, president of the Society of Professional Journalists and a writer for the Indianapolis Star. \"The big deal now is waiting on the Supreme Court to figure out the mess - whether that be colored by privacy interests, state rights issues or compelling public safety concerns - here's one person who hopes the Supreme Court sees through this mess.\"\n\nNiederpruem said the DPPA, which went into effect in 1997, merely made \"everybody's life a nightmare,\" including both journalists and DMV officials. The exemptions and variances in state laws made for a patchwork across the country, and many journalists were unaware of the changes until they tried to access the data after the laws passed, Niederpruem said.\n\nWording of law\n\nThe DPPA restricted state departments of motor vehicles from releasing personal information contained in drivers' license records, unless they established an \"opt-out\" provision for citizens to withhold their information from public release.\n\nAbout twenty-nine states have since established an \"opt-out\" and a few have adopted a media exemption. News media exemptions fall under a provision of the federal law that keeps records open for matters relating to vehicle and driver safety.\n\nLatest proposal\n\nHowever, a provision in the Department of Transportation appropriations bill, passed in October, would require states to have an \"opt-in\" provision instead, and allows the government to withhold federal highway funding for states that don't comply. This would mean individuals would have to agree, in writing, to allow their personal information to be released to the public.\n\n\"If they want to use the big 'P' word - privacy - to invoke all this unnecessary legislation, we need to speak up as journalists,\" said Kyle Elyse Niederpruem.\n\nThe state of South Carolina challenged the DPPA on the grounds that it interferes with the states' sovereignty under the Tenth Amendment by requiring states to enforce a federal program. The Federal Appeals Court ruled the law unconstitutional there.\n\nThree other states also challenged the law - Alabama, Wisconsin and Oklahoma - and appeals courts upheld the law in the Wisconsin and Oklahoma cases, and invalidated the law in the Alabama case.\n\nWhen the Supreme Court heard oral arguments in the case in November, justices questioned whether the federal government was encroaching on state functions, but said striking down the DPPA might also result in a need to revisit a 1985 law regarding fair labor standards.\n\nMaryjo Sylwester can be reached by e-mail at maryjo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-mapping-conversions","slug":"1999-december-mapping-conversions","url":"https://dwillis.github.io/uplink/article/1999-december-mapping-conversions.html","year":1999,"month":"December","monthNum":12,"headline":"Mapping conversions","kicker":"TECH TIP","author_name":"Andy Lehren","author_title":"Technology reporter","affiliation":"NBC","summary":"A tech guide explaining how to properly convert latitude and longitude data from degrees/minutes/seconds format to decimal degrees for use in mapping programs, with practical examples and SQL formulas.","keywords":["coordinates","conversion","mapping"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["MapInfo","ArcView","FoxPro","Access"],"full_text":"Earlier this year I was on the phone with a government official, asking about some of the undocumented features that came with one of his agency's databases. What came next was right out of the 'Blues Brothers' movie. It's the scene where Elwood asks a waitress, \"What kind of music do you usually have here?\" She replies, \"Oh, we've got both kinds we've got country and western.\" I asked him how the agency recorded latitude and longitude. He replied, \"We keep it in latitude and longitude.\" OK, so maybe that's funny only if you're nerdy. But you are reading Uplink. And a Tech Tip at that. I took another stab at my question: do you keep latitude and longitude in degrees or decimal degrees? The wrong answer could put a town like Halifax Estates, Fla., about 25 miles into the Atlantic Ocean. And we wouldn't want to do that to Halifax Estates. There are lots of government databases that come with latitude and longitude fields. That means there's an opportunity to map your data, and find trends you could not catch without a GIS program like MapInfo or ArcView.\n\nQuestions to ask\n\nWhen you get a database like this, there are a few questions you should ask. With national databases, it should be pretty obvious whether the data is in degrees or decimal degrees. But it may be harder to tell with certain local data just by looking at it. So ask. Here's the difference: We're all used to seeing globes, where zero degrees longitude goes through Greenwich, England, making it Y2K party central. And zero degrees latitude is also known as the equator, a belt running around the middle of our planet. Every time you move a degree along the equator, you're moving about 111,320 meters. Move about 74 degrees from Greenwich, England, and you're in line with New York.\n\nThe tricky part is what happens when you're dealing with fractions of a degree. Degrees are divided into minutes and seconds. Sixty seconds makes a minute. Then sixty minutes makes a degree. Just like an hour on your watch.\n\nBut mapping programs want latitude and longitude stored in decimal degrees. That's just like it sounds.\n\nHalifax Estates, Fla., is 80.98 decimal degrees from Greenwich, England. Go a couple hundredths more, and you're 81 decimal degrees away.\n\nConversions\n\nLet's assume you have three columns of data: degrees, minutes and seconds. You want to keep degrees the same. Divide minutes by 60. That's dividing the minutes by the number needed to make a degree. Divide seconds by 3,600. That's dividing the seconds by the number needed to make a degree. In FoxPro, assuming all the fields are numeric, and longitude field is called x_long, the result would almost be a line like this: Replace all X_long with degrees + (minutes/60) + (seconds/3600). In Access, the update query would almost look like: Update MyTable set MyTable.x_long = degrees + (minutes/60) + (seconds/3600) We're almost done, but not quite. There's one more thing to do. In decimal degrees, a longitude west of Greenwich, England, in our hemisphere, is a negative number. A latitude above the equator is positive. Sometimes these are marked with a W or an N in databases. So make sure you multiply your North American longitudes by -1.\n\nSome mapping programs come with functions that allow you to do conversions before you attempt to map a database. But running data conversions in mapping programs can be painfully slow. Here's a quick way to let your database manager do the hard work fast.\n\nAndy Lehren works at -74.3833 longitude and 41.55 latitude and can be reached by e-mail at alehren@nbc.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-arcview-themes","slug":"1999-december-arcview-themes","url":"https://dwillis.github.io/uplink/article/1999-december-arcview-themes.html","year":1999,"month":"December","monthNum":12,"headline":"ArcView themes","kicker":"BEST OF NICAR-L","author_name":"","author_title":"","affiliation":"","summary":"GIS users troubleshoot why three map themes (streets, census tracts, and precincts) cannot be displayed simultaneously in ArcView, with experts diagnosing the issue as a projection mismatch between datasets from different sources.","keywords":["projection","GIS","themes"],"topics":["Technology","Investigative Methods"],"technologies":["ArcView","TIGER"],"full_text":"I have three themes in the table of contents of my view: Streets, Precincts, and Census Tracts. When I turn on the Streets and Census Tracts themes, they are drawn in the view window, streets appearing on top of tracts.\n\nWhen I turn on the Precincts theme, nothing is drawn in the view window. The only way to see the precincts is to make the theme active, then zoom to the active theme. Streets and tracts disappear. Neither streets nor tracts (even though they are turned on), are drawn in the view whenever the precincts are drawn. Any ideas? - Jeff Thomas, The Gazette, Colorado Springs, Colo.\n\nAre the themes all in the same projection? I've run into the same problem with one theme being in projected meters and the others in decimal degrees. I believe you can use the Projector extension to convert between projections to make things line up. - Matthew Ericson, The Philadelphia Inquirer, Philadelphia, Penn.\n\nYou're describing exactly what happens when the data is in different projections. There is a \"projector\" extension in ArcView 3+. It can only re-project data that is in unprojected decimal degrees.\n\nI'm guessing you have tracts and streets in decimal degrees (which is the way ESRI distributes data and the way TIGER shows up), but the precincts you got from the government were projected somehow. So you need to find out what projection the precincts came in. Most local agencies use the appropriate \"state plane\" in meters or feet. Make sure you know whether it's state plane based on the NAD83 or NAD27.\n\nThen you can reproject your tracts and streets to the same projection. Then put all three of them into a View and you should be OK. If your streets/tracts and precincts are all already projected, then projector won't help you, because you need something in decimal degrees to start with.\n\nIf you have the just-released ArcView 3.2, it has a re-projector that *can* re-project data that isn't in decimal degrees. So in that case you could re-project each of the themes into whatever you want them to be. It can be run in batch mode if you have several themes to change. For those of us who get a lot of GIS data from local agencies that shows up already projected in a cacophony of projections, this improvement in ArcView 3.2 is a huge help. - Dan Keating, The Washington Post, Washington, D.C.\n\nThe story so far: Three themes in my view's table of contents: streets (line), tracts (polygon), precincts (polygon). The problem: Two of the three, streets and tracts, may be viewed simultaneously. But Precincts can be viewed by itself only. And if I move between views of streets/tracts and precincts, I must first \"zoom to theme\" before the theme may be viewed.\n\nMatthew, Tim and Dan quickly diagnosed this as a projection problem. It appears they are correct. I am told (but am uncertain of how to verify for myself, instruction here is welcome) that both the streets theme, which I got through ESRI's TIGER data service, and tracts, which I got from CEISIN, are in decimal degrees. The precincts theme is in state plane feet, Colorado Central Zone (is this the same as UTM zone 13?), NAD83.\n\nSo, based on their most excellent and timely advice, I need to use the projection extension to make the three themes compatible.\n\nUnfortunately, I'm working with a demo version of ArcView. It's a nearly complete version, except, of course, in that it does not have the projection extension and will not accept the addition of any extensions. Jeff Thomas.\n\nI think you can find the projection utility (called Project!) on ESRI's web site. It's definitely included in ArcView 3.2 - David Herzog, Providence Journal, Providence, R.I.\n\nMy friends, the NICAR list is the best deal in journalism. Your quick diagnosis of projection incompatibility sent me back to El Paso County's GIS dept., which provided confirmation that the projection of the voting precinct shapefile was not in decimal degrees (as were the census tract and street shapefiles from TIGER and CEISIN), but in \"state plane feet, Colorado Central Zone, NAD83.\"\n\nDan Keating generously offered to run a projection conversion with his full-blown version of ArcView, but for reasons unknown, the files choked up his attempt. In the meantime, the county's GIS team leader took it upon himself to run a conversion, shipped me the converted files, and all is well. Jeff Thomas.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-the-1999-ire-awards","slug":"1999-december-the-1999-ire-awards","url":"https://dwillis.github.io/uplink/article/1999-december-the-1999-ire-awards.html","year":1999,"month":"December","monthNum":12,"headline":"The 1999 IRE Awards","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"The 1999 IRE Awards accept entries until January 14, 2000 for journalism published or aired during 1999, featuring categories for investigative reporting, freedom of information, crime reporting, and student work.","keywords":["journalism","awards","deadline"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"The 1999 IRE Awards:\n\nContest deadline is Jan 14, 2000\n\nEntries must have been published or aired between Jan I and Dec. 31, 1999 The awards include:\n\nIRE Medal for Outstanding Investigative Reporting or an IRE Certificate of Recognition for Newspaper, TV and Other Media.\nIRE FOI Award honoring an individual or organization whose actions further open records or open government.\nTom Renner Award for outstanding crime reporting.\nStudent awards for outstanding investigative reporting.\nEntry forms are available to download on the IRE Web site (www.ire.org) or call (573) 882-2042 for more information.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"1999-december-bits-bytes-and-barks","slug":"1999-december-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/1999-december-bits-bytes-and-barks.html","year":1999,"month":"December","monthNum":12,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Investigative Reporters and Editors (IRE) is hosting regional and national conferences in 2000, along with computer-assisted reporting boot camps and training programs for journalists. The NICAR data library is also evaluating its database collection and seeking input on which databases to keep or add.","keywords":["conference","training","journalism"],"topics":["Journalism & Newsroom","Investigative Methods","Data & Databases"],"technologies":["spreadsheet","Internet"],"full_text":"IRE Regional Conference\n\nInvestigative Reporters and Editors Regional Conference 2000 will be held Jan. 14-15 in Washington, D.C.\n\nThe conference, which will be held at the National Press Club, will include panels featuring experts from print and broadcast sharing tips and techniques for winner stories, a computer demo room with daily sessions on computer-assisted reporting topics, vendors and exhibits and reprints of the top investigative work in the country.\n\nSee page 15 of this issue for a registration form.\n\nRegistration and hotel information for all conferences is available on the IRE Web site at www.ire.org. More information is also available by calling (573) 882-2042.\n\nIRE National Conference\n\nThe IRE National Conference 2000 will be held June 1-4, 2000 in New York.\n\nThe conference will include an optional CAR day, a computer demo room, TV Show and Tell Room, reprints, vendors and exhibits and a variety of panels featuring some of the top investigative reporters in the country.\n\nRegistration and hotel information for all conferences is available on the IRE Web site at www.ire.org. More information is also available by calling (573) 882-2042.\n\nNICAR data library\n\nThe NICAR data library is evaluating its collection of databases, and is considering eliminating some of the less popular. They are also looking for new national databases to add to the collection.\n\nIf you know of any national databases that you've found useful or think would be a good addition to NICAR, they'd like to hear from you.\n\nThe least popular databases being considered for elimination are the National Endowment for the Arts (NEA), NHTSA Vehicle Recalls and Complaints, CDC HIV Surveillance data and NASA Air Safety Reporting Program.\n\nIf you would like to put in a plea to save any of these datasets, please contact the data library administrators, Dawn Fallik at dawn@nicar.org or Jason Grotto at jason@nicar.org. They are also available by phone at (573) 884-7711.\n\nCAR boot camps\n\nIRE and NICAR has several computer-assisted reporting boot camps scheduled for 2000. These week-long camps provide hands-on training in the use of spreadsheets, database managers and Internet searching from NICAR staff.\n\nThe dates for the camps, which are held at the Missouri School of Journalism in Columbia, Mo., are: January 5-9, March 26-31, May 7-12, July 21-26, and August 6-11.\n\nContact John Green at IRE, (573) 882-2772 to check availability. More information about the camps, including registration forms and hotel information, is available on the NICAR Web site, www.nicar.org, under the training section.\n\nA boot camp for editors is also scheduled Feb. 24-26.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-religion-with-car","slug":"2000-januaryfebruary-religion-with-car","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-religion-with-car.html","year":2000,"month":"January/February","monthNum":1,"headline":"Religion with CAR","kicker":"ON THE BEAT","author_name":"David Crumm","author_title":"journalist","affiliation":"Detroit Free Press","summary":"A journalist describes how computers and the internet have become essential tools for researching and reporting on the increasingly diverse religious landscape in America and globally, utilizing websites, email lists, and CD-ROMs to track religious movements and developments.","keywords":["religion","internet","research"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Internet","World Wide Web","CD-ROM","Encarta"],"full_text":"Searching for God at the turn of the millennium requires more than prayer and meditation. It takes a computer.\n\nThe subject is vast and is growing more diverse by the decade. In the United States, 9 out of 10 Americans tell pollsters that they pray and are part of some religious group. While the vast majority of Americans claim to be Christian, most Christians feel increasingly free to switch back and forth among dozens of different denominations - or, if they're feeling restless, to break away entirely and form their own independent sects.\n\nThe influences of the world's other major faiths are surfacing more frequently in contemporary music, television, the Internet, movies and magazines.\n\nOutside the United States, religion also remains a major force for good or ill. It can be a powerful catalyst in ethnic strife, a fuel for political aspirations and, in many cases, a helpful inspiration toward resolving conflicts and building communities.\n\nTo keep up with this amazingly complex landscape, I use my computer in four basic ways.\n\nInternet\n\nAs a journalist, I don't idly surf the Internet, because there are thousands of religious Web sites and most of them are not factually reliable.\n\nOver the past five years, I've developed my own list of reliable religious Web sites, most of them sponsored by well-established groups. The list changes constantly as Web sites appear, evolve and disappear. A good place to start sorting out your own list of useful sites is the Yahoo index to religion, which you'll find linked off Yahoo's main page (as a sub-category of \"Society & Culture\").\n\nA few of the sites I use most frequently are the Vatican's information-rich Web site at www.vatican.va, a large evangelical site at www.gospelcom.net that links to nearly 150 religious groups and a helpful site at www.wfn.org where many Protestant denominations post news releases. A superb place to background yourself on religious history and traditions is the recently upgraded www.britannica.com, sponsored by the encyclopedia.\n\nBy carefully sifting sites, the Internet has played a major role in my reporting on everything from the annual Muslim fast of Ramadan to the plight of the Chinese meditation movement Falun Gong.\n\nI also subscribe to e-mail lists from many of the major religious groups I cover. If you look around the major religious sites on the Web, you'll find places where you can sign up to receive almost daily e-mails from places as diverse as the Evangelical Lutheran Church in America, the National Council of the Churches of Christ, the Council on American-Islamic Relations, informal neo-pagan networks and even the supporters of imprisoned Falun Gong activists. Many of these e-mails are quickly dispatched to my \"trash\" folder each morning - but a quick scan of the dozens of \"subject lines\" I find in my inbox often sparks a future story idea.\n\nCD-ROMs\n\nI use a variety of religion-related CD-ROMs I've collected over the past five years. My CD-ROM library includes a tour of the Vatican, a guide to holy places in the Middle East, a scholarly Bible dictionary, a history of the Holocaust, a text of the Dead Sea Scrolls and a collection of Jewish reference books. These titles tend to be released by publishers and then quickly go out of print, so watch for the release of helpful CDs and snap them up as soon as they go on sale.\n\nOne excellent CD-ROM to consider is \"On Common Ground: World Religions in America\" by Columbia University Press, an in-depth look at many faiths by Harvard University's Pluralism Project. And, because writing about religion often involves references to history and traditional culture, it's very useful to keep a simple copy of an Encarta encyclopedia handy.\n\nThe most basic software for religion writers is a searchable Bible to quickly find key references that may crop up in stories on deadline. But remember that English translations vary widely and you'll probably want at least a King James Version as well as a more modern English version. I have four sets of Bible software on my PC and use the \"New Oxford Annotated Bible\" most frequently.\n\nDatabases\n\nI use a variety of databases to quickly organize and search the gargantuan amounts of information that flow from the world of religion.\n\nMS Access is excellent for building customized databases. For example, the Access database I've created on members of the College of Cardinals - the group from which the Catholic church selects its popes - lets me track this influential group season after season.\n\nI also use askSam-brand SurfSaver software on the Internet, which allows me to save key Web pages in categorized folders on my PC. Later, I can search the entire text of these voluminous holdings.\n\nI use MS Word to create a crude but amazingly effective database out of reams of electronic data I've received about religion. I save important wire stories, significant e-mail press releases and other similar information by copying/pasting these texts into 10 ongoing Word files I have created. They represent categories, such as, \"Catholic,\" \"Politics and Religion\" and \"Islam.\" As an individual file reaches about 1.5 MB, I archive it onto a Zip disk and create a new file. When I'm reporting and want to pinpoint a particular story or document, I simply open up the appropriate Word file and use the Edit/Find function to search for key words.\n\nE-mail\n\nBecause reporting on religion almost always involves an attempt to connect news with the lives of ordinary people, I use e-mail as a lifeline to keep in touch with people I've encountered over the years.\n\nI also maintain specialized lists of people by category related to my beat. For example, when the Catholic hierarchy is considering a change in church rules that might affect local parishes, I will fire off a quick e-mail to my list of Catholics and ask whether they know of anyone especially concerned about such a change. People who e-mail me with an intelligent comment - readers, academics, clergy and others - are potential future sources. I save their e-mail addresses and may include them in the lists I'm building.\n\nUltimately, covering religion at the turn of the millennium is like covering presidential politics - if there were 1,000 parties fielding candidates. The subject can be overwhelming, unless journalists use all the available RAM at their disposal.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-race-tabulations","slug":"2000-januaryfebruary-race-tabulations","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-race-tabulations.html","year":2000,"month":"January/February","monthNum":1,"headline":"Race tabulations","kicker":"CENSUS 2000","author_name":"Steve Doig","author_title":"","affiliation":"Arizona State University","summary":"The 2000 Census introduces significant methodological changes in how race and ethnicity are measured, making it crucial for journalists to understand these differences when reporting on demographic trends and race relations in America.","keywords":["Census","demographics","race"],"topics":["Demographics & Census","Investigative Methods"],"technologies":[],"full_text":"No problem in American society has been more chronic and vexing than that of race relations. Add to that the strains of demographic change caused by the immigration to the United States of millions leaving their countries of birth to seek a better life here. The resulting frictions and adjustments are the source of many important stories for journalists, and Census data is one of the best ways of measuring those changes.\n\nHowever, the actual tallying of persons by race and ethnic origin has changed greatly over time, and the differences in definitions and methodology from one Census to the next can make it difficult to do accurate comparisons. One of those big changes in methodology is occurring with the 2000 Census, and it will require careful attention from those of us who the best advice I have now is, 'Stay tuned!'","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-reapportionment","slug":"2000-januaryfebruary-reapportionment","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-reapportionment.html","year":2000,"month":"January/February","monthNum":1,"headline":"Reapportionment","kicker":"CENSUS 2000","author_name":"Griffin Palmer and Steve Doig","author_title":"","affiliation":"The Daily Oklahoman and Arizona State University","summary":"The 2000 Census will determine how the 435 House seats are reapportioned among states based on population changes, with some states gaining seats while others lose them through a mathematical method called equal proportions that has been used since 1940.","keywords":["census","reapportionment","population"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"The 2000 Census isn't just a good idea — it's the law of the land. The U.S. Constitution mandates a national headcount every ten years for one purpose, and the specified reason isn't to help fast-food franchisers decide where to build their next burger joint. Rather, the census is required for fairly apportioning the 435 seats in the House of Representatives among the 50 states. All other census applications, whether drawing political district lines, allocating federal funds or siting taco stands, are just constitutional gravy.\n\nThe decennial reapportionment of Congressional seats means great stories, and they're the very first ones that will come out of actual 2000 Census data. By law, the Secretary of Commerce (who oversees the Bureau of the Census) will tell the President in late December of 2000 the official population of each state, and the President will then notify Congress. At that point, the bloodletting will officially begin.\n\nHowever, it's no trivial task to translate the states' populations into the number of seats each will get, as the two of us have learned.\n\nBecause the number of Congressional seats is capped at 435, some fast-growing states will gain new seats at the expense of those with slower growth. Reapportionment means political musical chairs in the losing states, and a big scramble for the newly-open seats in gaining states.\n\nHowever, it's no trivial task to translate the states' populations into the number of seats each will get, as the two of us have learned. A simple proportional allocation (in which, for instance, a state with five percent of the nation's population gets five percent of the 435 seats) doesn't work for two reasons. For one, there are no fractional seats in Congress (although some of the occupants certainly are fractious), and so a proper rounding scheme would have to be devised. And more important, the Constitution guarantees that even the smallest state will get at least one House-seat even if its share of the nation's population is \"worth\" less than half a seat.\n\nEqual proportions\n\nNumerous apportionment schemes have been devised, and five different ones actually have been used at various times in the nation's history. (See www.census.gov/population/censusdata/apportn.pdf for a brief history of those methods, and www.census.gov/srd/papers/pdf/rr92-6.pdf for a more exhaustive discussion of the history and mathematics of apportionment methods.) We'll only concern ourselves here with the so-called \"method of equal proportions\" which has been used since 1940 and was upheld unanimously in 1992 by the U.S. Supreme Court.\n\nSimulation\n\nWe won't try to explain the mathematical reasoning that underlies the method of equal proportions, because then we'd have to pretend that we actually understand it ourselves. But we have devised an Excel spreadsheet that accurately simulates the method and thus will let you figure out where your state stands in the reapportionment shuffle. To satisfy yourself that it works, first we'll build the spreadsheet using the official 1990 populations. Here's how:\n\nOpen a new spreadsheet.\nPut these labels in the first row on the specified columns: A: \"State\", B: \"Population\", C: \"Seat #\", D: \"Multiplier\", E: \"Priority value\".\nFill range A2:A51 with the names of the states.\nFill range B2:B51 with the 1990 apportionment populations of the states (which you can find at www.census.gov/population/censusdata/table-a.pdf)\nFill range C2:C51 with the number 2.\n\n(We start this with seat 2 because every state is guaranteed at least one seat.)\n\nPut this formula in Cell D2: =1/SQRT(C2*(C2-1)). The formal name for this multiplier is the \"reciprocal of the geometric mean,\" in case you get asked on \"Who Wants to be a Millionaire?\"\nPut this formula in Cell E2: =D2*B2\nPut this formula in Cell A52: =A2\nPut this formula in Cell B52: =B2\nPut this formula in Cell C52: =C2+1\nCopy range A52:C52 all the way down to Row 2951. You'll know you did it right if \"Wyoming\", \"455975\" and \"60\" are on that row.\nNow select range D2:E2 and copy it all the way down to Row 2951, too. It worked if you see \"0.0168073\" and \"7663.716\" down there.\n\n(Okay, okay, here's what's happening. The method of equal proportions requires that you calculate a \"priority value\" for at least as many seats as each state could possibly get. As it turns out, the easiest thing to do is calculate such values for seats 2-60 for every state, on the grounds that not even California will get as many as 60 seats. And 50 states times 59 priority values means 2,950 rows.)\n\nThe decennial reapportionment of Congressional seats means great stories, and they're the very first ones that will come out of actual 2000 Census data.\n\nNow open a new sheet and follow these steps:\n\nGo back to your first table and copy the entire thing, all 2,951 rows and five columns.\nUse \"Paste Special Values\" to paste it onto the new sheet.\nSort that second sheet by descending order of Priority Value\nInsert a blank row at Row 387. This cuts off the table at 385 seats.\nInsert 50 blank rows into Range A2:A51.\nCopy the names of the 50 states into Range A2:A51.\nPut a one (1) in Cell D2, and copy it down to Cell D51\n\nWhew. You now have a table with 435 records, one for each seat in Congress. To find out how many seats each state gets, simply do a pivot table by dropping the State variable into both the Row and the Data boxes.\n\nThe results\n\nThe main table also is worth a detailed look because it shows the order in which the seats were parceled out. Notice, for instance, that California got 42 of its 52 seats before Idaho and Rhode Island got their second seats. Notice also the first few states in the remainder below your main table. Those are the ones that just missed getting another seat (at someone else's expense, of course) and where much of the wailing was heard after the 1990 Census.\n\nNow that you've done one for 1990, repeat the process using projected populations for 2000, which you can find at www.census.gov/population/projections/state/stpjpop. You'll see plenty of winners and losers in the data. And if your state is near the bottom of the main table, or near the top of the leftovers, you'll know how important a good census count will be to your readers — and your politicians.\n\n(A final note for FoxPro mavens: Griffin first developed this apportionment simulation using FoxPro. If you want a copy of his method, contact him by e-mail.)","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-uncovering-secrets","slug":"2000-januaryfebruary-uncovering-secrets","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-uncovering-secrets.html","year":2000,"month":"January/February","monthNum":1,"headline":"Uncovering secrets","kicker":"MILITARY AVIATION","author_name":"Russell Carollo","author_title":"","affiliation":"Dayton Daily News","summary":"The Dayton Daily News obtained military aviation accident databases through FOIA requests to examine safety patterns across the Air Force, Army, and Navy, discovering that analyzing the complex, acronym-laden data required careful fact-checking to distinguish actual causes from symptoms of accidents.","keywords":["databases","military","accidents"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":[],"full_text":"The Dayton Daily News began its 18-month examination of military aviation safety with a fishing expedition. I was fishing for computer databases that hadn't been worked over by 100 reporters when I learned that no one had ever asked for any of the databases on military aviation accidents.\n\nSo I filed Freedom of Information Act requests with the safety centers for the Air Force, Army and Navy, which includes the Marine Corps. (I gave up on the Coast Guard after numerous telephone calls and requests went unanswered.) Each service has its own database, and, though there are some similarities, the databases differ significantly from one another, even though some aircraft are flown by more than one service.\n\nMajor civilian aviation accidents are investigated by an independent body, the National Transportation Safety Board, and the results of NTSB investigations are made public. But major military aviation accidents usually result in two separate internal investigations: One done by one or more people who produce a public report, and the other, a more through investigation, done directly through one of the military safety centers.\n\nInterviews, findings, recommendations and conclusions of the second investigation the one done through a safety center - are secret, and the courts have upheld this secrecy for years under the so-called Machan Privilege. Factual information, however, is not protected under the Machin Privilege.\n\nSo from the beginning, getting information from the safety centers was not easy. I negotiated for months over what was factual and what was protected under the Machin Privilege.\n\nIn the end, I got much more than expected. Though the databases differed greatly from one another, each service provided similar types of information: Dates and times of accidents, locations, types of aircraft, cost of accidents, types of injuries, names of crewmembers and even the names of parts believed to have failed. With the identity or parts suspected of failing, I had indications of what the secret safety investigation believed caused accidents - information usually kept from the public under the Machin Privilege.\n\nUnder the surface\n\nBut reading the databases wasn't easy. Anything military usually is filled with acronyms and coded language, and military aviation is padded with additional layers of technical jargon. Because of this, stories on military aviation require additional fact-checking, which should include a review by a military aviation expert.\n\nDave Gulliver, who assisted with some of the data analysis, and I discovered that what appeared to be true on the surface often wasn't necessarily the case. For example, one part that appeared to have been linked to dozens of Army helicopter accidents turned out to be landing skids, something that frequently breaks during crash landings but is rarely the cause.\n\nUsing the parts tables, we were able to link a single part or a single type of problem to dozens of accidents, concentrating on parts that eventually caused major accidents. In one case, we found that hydraulic problems in the Sea Knight helicopters had caused more than 71 emergencies and accidents from 1988 through July 1998. Nearly all of the cases involved the utility hydraulic pump, and in at least three cases, helicopters were destroyed and crews barely escaped with their lives.\n\nThe Navy database even had a \"reoccurring problem narrative,\" tracking problems such as those in the Sea Knight that occurred more than once. One field actually totaled the numbers of times a problem had occurred, and the reoccurring problem narrative added comments, sometimes from commanders concerned that the problem had not been fixed.\n\nThe Air Force database had an engine table, complete with serial numbers of engines involved in accidents. With this table, we were able to track engines that had been in more than once accident. We found one engine that had been on three F-16s in less then four months. Twice the engine was removed after pilots complained of smoke and fumes in the cockpit, and finally the engine was put on a third jet, which crashed into a South Carolina neighborhood, killing one man and injuring seven others.\n\nBeyond numbers\n\nUsing the information we gathered from the database, we then sought the lengthy reports on the accidents, filing more than 150 Freedom of Information Act requests with the various services. We also spent months tracking down crewmembers, relatives of victims, attorneys and other with information about the accidents. One of the most interesting aspects of the reporting came when we shared what we had learned from the computer records and accident reports with pilots, accident investigators and other people who had investigated the accidents. Their explanations greatly enhanced our ability to search the databases for the more significant problems and disregard other problems that - although they appeared significant - actually were minor. Pilots who survived accidents and the relatives of pilots who were killed frequently had no idea of the findings we had discovered on the databases. Several people explored legal action as the results of telephone calls from the Dayton Daily News concerning our findings, and at least one attorney changed his lawsuit, adding to it a company our records showed produced parts the Army had questioned.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-cost-of-living","slug":"2000-januaryfebruary-cost-of-living","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-cost-of-living.html","year":2000,"month":"January/February","monthNum":1,"headline":"Cost of living","kicker":"ECONOMIC DATA","author_name":"Jennifer LaFleur and Michelle Quinn","author_title":"","affiliation":"Mercury News","summary":"A Mercury News analysis of Silicon Valley's wealth distribution reveals economic paradoxes: while roughly 1 in 9 Santa Clara County households are millionaires, most residents aren't particularly wealthy by net worth measures, and only 29 percent can afford median-priced homes compared to 55 percent nationally.","keywords":["wealth","Silicon Valley","housing"],"topics":["Business & Economy","Data & Databases","Demographics & Census"],"technologies":["Claritas","Ferret","Census Current Population Survey"],"full_text":"When most people think of Silicon Valley, they think of two things: technology and money. If a few national reports are to be believed, nearly everyone in Metro San Jose is a millionaire. Earlier this year, a team of Mercury News reporters set out to find out just how much money is being made in Silicon Valley, how it was distributed and how it has affected people's lives. What our analysis of several private and public databases found was that the valley is out of whack economically compared with the rest of the nation, riding a boom filled with paradoxes. According to one of the most aggressive estimates, 65,000 Santa Clara County households - roughly 1 in 9 - are worth more than $1 million, not including the value of their homes. And yet, only 29 percent of Santa Clara County's households can afford the median-priced home, far below the national rate of 55 percent. And two out of five county households would need to stretch beyond advisable limits to rent the average two-bedroom apartment.\n\nThe bottom line of using any of these types of data sources is to compare the results to other information.\n\nPerhaps most startling: although many people here earn exceptionally high pay compared with the rest of the country, most aren't particularly wealthy. Look at the net wealth of Santa Clara County households - their assets minus their debts - and the distribution closely mirrors the nation's as a whole.\n\nData sources\n\nThis series used several sources of data. Our biggest concern was that the market data models truly reflected Silicon Valley. Because many market surveys cap home values and incomes low values for our area, we compared data from several sources.\n\nWe ended up using wealth measures from Claritas, an Arlington, Va.-based marketing information firm, which estimates income and wealth distributions for the United States, including all ZIP codes and small areas. Estimates are based on a combination of factors: extrapolation of census data, Claritas' Market Audit Survey and other area-specific sources of demographic data. The Market Audit Survey annually collects information on household assets, liabilities, demographics and financial behavior for approximately 120,000 U.S. households. We used Claritas figures for some of the income data. A note of caution: If you plan to use any market data, make sure you fully understand the methodology used in their model and that you can verify it with other data sources. Also, these data are good for large metropolitan areas, but not as good for smaller areas. Additional sources of data we used to look at income included: Census Current Population Survey data (downloadable off www.census.gov using the Ferret extraction tool), for which we had to combine years to get a valid sample; and California state franchise tax board tax filing data. Using these data, we were able to show that the mean income was increasing faster than the median income, showing that the higher end was pulling up the average.\n\nHousing data\n\nAlthough many things may seem expensive in Silicon Valley, the one driving factor behind most people's struggles is housing. We used California Association of Realtors data and data from the Real Estate Research Board of Northern California to look at trends in housing costs. Depending on how assessments are done in a particular area, property tax data might be useful for this as well. Because Proposition 13 froze California assessment data at sales prices, it's not a useful tool for looking at housing costs. Rent data was harder to find. A few private companies do rent analysis, but they vary depending on how the data are gathered.\n\nHUD Fair Market Rent measures are a good start, although they estimate rents lower than the median for an area. Another source for rent data is RealFacts.\n\nThe RealFacts data inventory has been assembled over a period of years, and provides complete and current information on more than 4,000 apartment complexes in 10 Metropolitan Statistical Areas (MSAs), including California, Oregon, Washington, Nevada, Utah and New Mexico. These tend to be slightly higher because they are for apartment complexes.\n\nIncome data\n\nTo get a handle on how much money insiders in Silicon Valley made, we used a database of SEC corporate insider trading files from Thomson Financial Wealth Identification (formerly CDA). We used this to get an estimate of the number of people with large amounts of wealth.\n\nTo figure out the value of stock options held by employees of select Silicon Valley companies, Institutional Shareholder Services did an analysis based on 10-K filings and several other company factors such as turnover.\n\nAfter analyzing many economic measures for Silicon Valley, we had a starting point. As with any data-based project, the data are the beginning. The bottom line of using any of these types of data sources is to compare the results to other information.\n\nThe large amounts of data that we gathered were not all included in the stories, but we were able to make many of them available online so that individuals could look up wealth, income and other measures for their neighborhoods.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-disciplined-doctors","slug":"2000-januaryfebruary-disciplined-doctors","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-disciplined-doctors.html","year":2000,"month":"January/February","monthNum":1,"headline":"Disciplined doctors","kicker":"HEALTH CARE","author_name":"Richard J. Dalton, Jr.","author_title":"computer analyst","affiliation":"Newsday","summary":"Newsday's investigative series revealed that managed-care companies employ doctors who have faced serious disciplinary action, despite promising quality care to patients. The investigation required a landmark lawsuit to access state health databases and systematic searching of multiple websites to uncover 132 disciplined doctors working in HMO networks.","keywords":["investigation","discipline","healthcare"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["SPSS","World Wide Web"],"full_text":"A landmark lawsuit, a monstrous database and rigorous searching using healthcare Web site search engines lie behind a recent Newsday series on disciplined doctors working in managed care.\n\nNewsday's series, reported over a 10-month period, revealed that doctors who have been punished for serious or even fatal wrongdoing often continue to work for managed-care companies despite the healthcare industry's promise to screen their doctors and offer the best care to patients.\n\nWhile many papers have examined HMOs' denial of treatment, this series broke ground by demonstrating that managed-care companies employ disciplined doctors, effectively breaking their promise of quality care.\n\nStaff writer Thomas Maier reported and wrote the series, with computer analysis by this reporter and Dave Ewalt and assistance by Bob Fresco.\n\nThe Health Department regularly releases information from the database to researchers and reporters but removes data identifying doctors to preserve their privacy.\n\nBefore starting on the series, Newsday and The New York Times put its lawyers to work, suing the New York State Department of Health to obtain a database of information about the performance of doctors in treating patients in New York hospitals.\n\nThe Health Department regularly releases information from the database to researchers and reporters but removes data identifying doctors to preserve their privacy. That meant key information remained secret, including the license numbers of attending and operating physicians working on the patient, the admission date, payor identification number and the hospital name.\n\nThe omission of such information made it impossible to determine the quality of care by individual doctors and hospitals.\n\nThe newspapers argued that physicians had no such right to privacy but agreed that the data about individual patients should remain private. In June 1998, a state appeals court agreed with a lower-court ruling siding with the media, allowing access to the database of millions of hospital admissions.\n\nWeb sites\n\nWhen the legal battles subsided, the scene of action turned to the World Wide Web for a Health Department Web site listing disciplined doctors, managed-care company sites listing doctors in the networks and the American Board of Medical Specialties' site revealing doctors with board certification.\n\nThe healthcare companies and agencies probably never envisioned their Web sites being scrutinized in the manner Maier examined them. From the board-certification site, he found doctors who erroneously claimed board certification in managed-care listings.\n\nFrom the HMO sites, he didn't find a new doctor, he uncovered disciplined doctors-132 of them disciplined in New York state in the 1990s.\n\nUsing brief descriptions of the disciplinary actions from the Health Department Web site, we created a database of disciplined doctors, including their names, license numbers, addresses, discipline dates and whether the doctors were board certified. We also noted the doctors' offenses, such as patient mistreatment, sexual misconduct or billing errors, and the disciplinary actions, such as suspension, probation or license revocation.\n\nNext, we began to quantify how many procedures these doctors oversaw after they were disciplined, and how much money patients or insurers paid for care overseen by these disciplined doctors.\n\nData work\n\nThe hospital admission data contained millions of procedures performed from 1990 through 1998. Using SPSS, we extracted the procedures overseen by doctors disciplined anytime in the 1990s.\n\nThat database contained 130,000 records, but only some procedures were performed after the attending or operating physicians were disciplined. We then matched the database with the list of disciplined doctors to uncover procedures performed after the discipline date.\n\nThe discipline date field was straightforward. But, as is common with most government data, the database of hospital procedures from the Statewide Planning and Research Cooperative System, which contained the hospital admission date, was quite quirky.\n\nThe admission date in the database was divided into three fields: a year, a month, and a day - not a day of the month, but a day of the week.\n\nWe wanted to ensure that we only counted patient visits and hospital charges that occurred after the doctor was punished to demonstrate that taxpayer money was being spent on medical care overseen by disciplined doctors. But the unusual format of the admission date presented a problem when a patient visited a doctor in the same month the doctor was disciplined.\n\nSo we decided to turn every admission date into the first of the month. If a patient was admitted in November 1998, we turned that into Nov. 1, 1998, using a module written in Microsoft Access and an update query.\n\nSuppose the patient had been treated in November 1998 and the attending physician was disciplined on Nov. 20 of that month. Using Nov. 1, the visit would have preceded the discipline date and wouldn't be counted. We gave the doctor the benefit of the doubt because it was impossible to be more specific about the admission date.\n\nOnce we cleared up the date dilemma, we began to calculate the amount of money that the state had spent on managed-care doctors through Medicare and Medicaid.\n\nFurthermore, Maier discovered the actual charges to Medicare and Medicaid turned out to be about 60 percent of the total cost.\n\nAt first, the task seemed easy - a field indicated the charges. Then came the caveats. The field represented the total cost of the hospital visit, including the room, surgery and payments to the attending and operating physicians and others.\n\nSo we discounted the figures by 40 percent, and could only say that the disciplined doctors \"oversaw\" the hospital visit, explaining why the data prevented us from determining how much each doctor actually received.\n\nLicense problem\n\nEach procedure record also contained three licenses of doctors associated with the treatment, listing the licenses for the attending, operating and \"other\" physician.\n\nWe chose to use only the attending and operating physicians. Still, multiple licenses for each record presented a quandary.\n\nIt's impossible to link the two licenses one from the attending physician and one from the operating physician - to the one license field in the doctor table.\n\nTo explain why, here's an analogy. Suppose I have a lookup table named States that translates U.S. Postal state abbreviations (e.g., State.c contains \"NY,\" \"NJ,\" etc. into state names, and State.nam contains \"New York,\" \"New Jersey,\" etc.).\n\nSuppose I have an address book that lists someone's work address and home address, and I want to link that to the lookup table of state names. Suppose further that a friend lives in New Jersey and works in New York.\n\nThat state code for the work address, \"NY,\" and home address, \"NJ,\" cannot simultaneously point to the state code in the lookup table for state names. If I did, then what would the field State.name represent: New York or New Jersey?\n\nTo resolve that dilemma, add the doctor table to the query twice by clicking twice on the \"add\" option in the query/show table window. That adds another table called State_2.\n\nTo use it, link the home address to the state table and the work address to the State_2 table. Now State.name is \"New York\" and State_2.1 is \"New Jersey.\"\n\nBy the way, Access doesn't really create another table. It merely creates another pointer to the same table.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-where-to-look-for-data","slug":"2000-januaryfebruary-where-to-look-for-data","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-where-to-look-for-data.html","year":2000,"month":"January/February","monthNum":1,"headline":"Where to look for data","kicker":"HEALTH CARE","author_name":"Maryjo Sylwester","author_title":"Managing Editor","affiliation":"Newsday","summary":"A guide for journalists covering managed care and healthcare reporting, detailing key databases and sources including hospital procedures, disciplined doctors, financial records, and certification information that can be accessed through Freedom of Information requests and public agencies.","keywords":["healthcare","databases","journalism"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"The Newsday series, \"Managed Care and Doctors: The Broken Promise,\" tackled a topic that is becoming increasingly important for journalists to analyze as the health care industry continues to grow larger every year. Whether you're at a large or small news organization, looking at health care in your community is possible. There were three primary databases that Newsday used for its series:\n\nHospital procedures performed at local hospitals. Use this to calculate the number of hospital visits and charges related to disciplined doctors. (Newsday and The New York Times sued to gain access to this database.)\nDoctors who have been disciplined. Each state has a regulatory agency for licensed professionals, which would be able to provide this information. (A Web site in New York lists the doctors, but it is not in database format. Newsday received this database electronically through a Freedom of Information Act request.)\n\nThere are plenty of industry sources where you can get averages or typical numbers for hospital finances, such as how much debt they have or how much money they make from outpatient versus inpatient procedures.\n\nA list of board-certified doctors. One place to check is the American Board of Medical Specialties, an umbrella group that represents 24 specialty boards. (www.certifieddoctor.com/verify.html)\n\nFinances\n\nAnother area of the medical industry that could be scrutinized, without requiring a great deal of resources, is finances of local hospitals or clinics. Most health care organizations have some sort of government regulation, and therefore financial reporting requirements. Each state should have a government agency overseeing health care, where you could obtain a great deal of information. Some hospitals are non-profit organizations, which are required to file Form 990s. The Internal Revenue Service Web site (www.irs.gov) lists organizations that have filed the tax-exempt forms, including some basic information for each one. However, the forms are not available there.\n\nRecently, some IRS-990s have been posted at the Guidestar site (www.guidestar.org/news/features/990status.html) or on the National Center for Charitable Statistics site (http://nccs.urban.org/9901).\n\nOther sources\n\nThere are plenty of industry sources where you can get averages or typical numbers for hospital finances, such as how much debt they have or how much money they make from outpatient versus inpatient procedures.\n\nThe Health Care Financing Administration may be able to provide some good information. Their Web site (www.hcfa.gov) has some statistical data, such as payments to hospitals and national statistics you can use for comparison purposes.\n\nOn its Web site, the Department of Health and Human Services (www.dhhs.gov/progorg/oig/cumsan/index.htm) has a list of people who can't take Medicare or various other federal payments for services, and other useful information.\n\nBefore launching into a story of this type, you might also want to check the IRE Resource Center. They have a large number of not only stories that have been done by other news organizations on health care topics, but also some tipsheets that could be useful. You can search the Resource Center database at www.ire.org/resourcecenter.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-the-money-trail-tracking-campaign-cash","slug":"2000-januaryfebruary-the-money-trail-tracking-campaign-cash","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-the-money-trail-tracking-campaign-cash.html","year":2000,"month":"January/February","monthNum":1,"headline":"The Money Trail: Tracking Campaign Cash","kicker":"CAMPAIGN FINANCE","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering workshops on campaign finance data analysis across multiple Midwest cities from January through March, with registration details available on the Campaign Finance Information Center website.","keywords":["workshops","campaign","finance"],"topics":["Government & Politics","Journalism & Newsroom","Investigative Methods"],"technologies":["World Wide Web"],"full_text":"These IRE and NICAR workshops on using campaign finance data will be held over the next few months in various cities. More information is available at the Campaign Finance Information Center Web site, www.campaignfinance.org/seminar.html\n\nHere are the dates and locations:\n\nJan. 22-23: Lansing, Mich.\n\nFeb. 5-6: Columbus, Ohio\n\nFeb. 26-27: Springfield, Ill.\n\nMarch 4-5: Madison, Wis.\n\nMarch 25-26: Indianapolis, Ind.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-ceos-giving-habits","slug":"2000-januaryfebruary-ceos-giving-habits","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-ceos-giving-habits.html","year":2000,"month":"January/February","monthNum":1,"headline":"CEOs' giving habits","kicker":"CAMPAIGN FINANCE","author_name":"Richard Dunham","author_title":"White House and national political correspondent","affiliation":"Business Week","summary":"A Business Week correspondent describes his strategy of analyzing CEO political donation patterns to gain a competitive edge in presidential campaign reporting, detailing the meticulous data compilation and verification process used to track contributions from Fortune 1000 executives.","keywords":["donations","analysis","executives"],"topics":["Business & Economy","Government & Politics","Investigative Methods"],"technologies":["Excel"],"full_text":"In the hyper-competitive world of presidential campaign reporting, it's a constant challenge to stay ahead of the pack - particularly if you work for a magazine that is published weekly.\n\nBut the best way to get noticed, I've discovered as White House and national political correspondent for Business Week, is to create a specialized niche in the overcrowded marketplace of political stories. For me, that's been the political donation habits of America's top CEOs.\n\nEarly in the year, before the presidential candidates filed their first disclosure forms with the Federal Election Commission, I wrote a story about the early giving habits of prominent business figures. This was done the old-fashioned way: a lot of phone calls to a lot of executives and to all of the campaigns.\n\nAs the campaign heated up after Labor Day, I devised a plan to take the next step: to comprehensively study the donation patterns of big businesses' top corporate executives. To beat the competition, we needed to do the analysis as soon as the 1999 third-quarter campaign spending reports-due on Oct. were made public.\n\nProject goals\n\nAt the start of the project, I composed a list of reporting goals: A simple scoreboard of the candidates' total number of CEO donors, an analysis of the extent to which CEOs \"hedged their bets\" among different candidates, a breakdown of corporate givers by economic sectors (such as technology, telecommunications, entertainment, finance and energy) and a review of the 2000 donations made by CEO backers of the 1996 Clinton-Gore team.\n\nAs a reporter with good (but not state-of-the-art) computer skills, I knew that I'd need a little bit of help to complete my analysis within days of the data becoming available. I got that assistance from Campaign for America, a nonpartisan watchdog group that enlisted two of the most knowledgeable money-in-politics experts, Kent Cooper and Tony Raymond.\n\nWe compiled a list of the top 1,000 corporations by market capitalization, with their addresses and chief executive officers, and arranged it in Excel. Cooper and Raymond checked the list against the new FEC reports and came up with more than 5,000 possible hits. We had to pare down the list by hand, eliminating people with identical names living in different states or people with different middle initials. Just under 600 potential donors made the cut. We \"scrubbed\" the data one more time to make sure we didn't confuse fathers with sons, such as \"Smith Jr.\" with \"Smith IV.\" Our final list had 538 contributions by CEOs. Some of these executives had donated to more than one candidate (some to as many as four White House wannabes) and some had sent multiple contributions to a single candidate. We even found four individuals who were CEOs of two different companies, so their contributions were listed twice in our database. Again, we adjusted our totals. In the end, we found 389 contributors.\n\nEconomic sector\n\nTo analyze the contributions by economic sector, Business Week's Fred Jespersen sent us the magazine's computerized listing of corporations coded by economic sector. This is proprietary data, so it can't be replicated by outsiders, but a reporter can create a reasonable facsimile by looking up each corporation in a business directory like Standard & Poor's publication.\n\nWith the sectoral data now merged in our Excel files, we rearranged our data by sector and tallied the results.\n\nAs expected, George W. Bush had a huge lead among energy company CEOs. But he had an unexpectedly large lead over Al Gore among the Vice-President's beloved technology companies. In fact, Gore placed third behind Bill Bradley in techland. What's more, Bradley matched Gore in contributors among the big Hollywood CEOs.\n\nThere are computerized lists available of anything from lawyers to bankers to professional athletes. A regional reporter could pare down a list of CEOs (or bankers or lawyers) to include only those from a certain state or region.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-online-tax-calculator","slug":"2000-januaryfebruary-online-tax-calculator","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-online-tax-calculator.html","year":2000,"month":"January/February","monthNum":1,"headline":"Online tax calculator","kicker":"TAX CUTS","author_name":"Genevieve Anton","author_title":"Statehouse Reporter","affiliation":"Colorado (appears to be a Colorado news organization)","summary":"Colorado journalists created an interactive tax calculator to help voters understand how proposed tax cuts would personally affect them, making complex fiscal policy accessible by moving beyond numbers and political language to show real financial impacts.","keywords":["calculator","taxes","Colorado"],"topics":["Business & Economy","Government & Politics","Technology"],"technologies":[],"full_text":"Tax cuts are extremely popular among most voters, but difficult to write about in a way that hits home with your readers. These are classic pocketbook issues, yet our stories are usually heavy on numbers, bureaucratic language and political spin. Colorado had a $704 million surplus this year, a state record that prompted historic tax cuts. The question wasn't whether to cut or spend - constitutional spending limits mean surpluses must be returned to taxpayers - but who would get the tax breaks. When the proposed tax package emerged from the Colorado General Assembly, my statehouse colleague Michele Ames and I scrambled to make sense of it to readers. Who would be the real winners and losers if this passed? With only three days to write the first analysis for the Sunday edition, we decided to find several real people in our area, each representing a different socio-economic situation, and figure out how the specific tax proposals would affect them.\n\nWhat we needed was a way to let individual taxpayers see how the proposed tax cuts would affect them directly.\n\nBased on information they provided about their age, marital status, taxable income and investment exclusions, we calculated their individual tax bill under the new plan. The same type of analysis was done for a small business and a large manufacturer in our area.\n\nWe bolstered these findings by creating imaginary taxpayers and plugging their financial data into algebraic formulas for each tax cut proposal to get a larger sense of how it would impact various taxpayers.\n\nIt was a crude and tedious process, one that we desperately wanted to expand and refine.\n\nBut there's a tight lid on information about taxpayers. The Colorado Department of Revenue said it had no reason to track certain information - even on an aggregate level such as how many people currently file for certain exemptions, exclusions or deductions.\n\nWhat we needed was a way to let individual taxpayers see how the proposed tax cuts would affect them directly.\n\nThat's when I heard about a creative approach to tax coverage being tried by New Hampshire Public Radio: a Web site devoted entirely to the issue in that state. Although their Web site was a massive project that had taken months to pull together, there was one element we could copy: an online calculator that lets people see how the tax cuts affect them directly.\n\nSince the New Hampshire Web site hid the coding and formulas for the calculator, we had to start from scratch with little to go on.\n\nFortunately, our editors loved the idea. Unfortunately, they had no clue how difficult it would be and gave us only one week to pull it all together.\n\nCalculator creation\n\nIn creating the calculator, we had no experience or examples to follow, and not even the state's top budget or tax experts had attempted anything like this - although some did offer to look over our work when done.\n\nThe idea was to give people a rough estimate of what they would save under each proposed tax reduction and a comparison of their final tax bill under the entire tax package versus the alternative - a straight refund of the surplus.\n\nSo we took each proposed tax cut and broke it down into an algebraic equation based on revenue formulas that ranged from one line to two pages in length.\n\nFor example, a proposed reduction of the state income tax rate from five to 4.7 percent looked like this: (your taxable income .05) - (your taxable income .0475) = tax savings.\n\nNo, we didn't make the public walk through these tortuous calculations.\n\nWe simply figured out what information we needed to know and narrowed it down to Feedback forms also helped generate great stories and new ways of thinking about how we could cover the tax debate in the future.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-weighting-data","slug":"2000-januaryfebruary-weighting-data","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-weighting-data.html","year":2000,"month":"January/February","monthNum":1,"headline":"Weighting data","kicker":"PROBLEM SOLVING","author_name":"Griff Palmer","author_title":"","affiliation":"The Daily Oklahoman","summary":"The Daily Oklahoman used geocoding and census data to assign demographic indicators like median income and educational attainment to Oklahoma City students by linking their addresses to census block groups. The staff discussed how to properly weight block group data when calculating school-level medians, discovering that weighting occurs naturally when ranking individual cases rather than grouped summaries.","keywords":["geocoding","demographics","median"],"topics":["Data & Databases","Education","Investigative Methods"],"technologies":["MapMarker","Atlas GIS","FoxPro","SPSS","Excel"],"full_text":"In pursuing a recent education project, The Daily Oklahoman staff writer Bobby Ross persuaded Oklahoma City schools administrators to give us an electronic file containing all students' addresses and school assignments.\n\nWe wanted to come up with demographic indicators for the students. We explored linking student addresses to county assessor data, using home values as an indicator of economic status. We gave up that idea, though, after realizing that dealing with students living in apartments and rental properties would be impossible, given the limitations of the Oklahoma County assessor's data.\n\nWe settled on using 1999 block group-level median income estimates and 1990 educational attainment data from Claritas as indicators for students. The question of how to properly weight the data generated lots of interesting discussion on NICAR-L, NICAR's electronic mailing list for CAR practitioners.\n\nHow we did it\n\nTo link students up to census block groups, we first geocoded students' addresses with MapMarker software. We used Atlas GIS to assign each geocoded address to its appropriate block group. It was then a simple matter with FoxPro to write the median income and educational attainment data for the corresponding block group to each student's record.\n\nOnce we had assigned a block group level median income to each student, we wanted to calculate a median for each school in the district, and here, the question of weighting came into play. At most schools, most students live in one or two block groups. Those block groups should carry more weight when calculating a school-level median than do block groups in which just a few students live.\n\nUsing a program like SPSS, it's simple to apply a weighting factor to a variable.\n\nAs the statisticians in The Oklahoman's marketing department explained to us, though, it's not necessary to apply a weighting factor to determine a weighted median if you have a dataset with every case in it and not grouped summaries. The weighting occurs naturally as medians are calculated. When ranking a set of values to determine the central value, a category (in this instance, a block group) encompassing a large number of cases (students, in this instance) automatically carry more weight in the rankings.\n\nTo give a simple example, say you have three cases with block group level medians assigned, and you want to calculate a median from those three cases:\n\n1 24,586\n2 29,483\n3 32,771\n\nThe median of the three values is 29,483. But say you have five students, three of whom live in the same block group and therefore have been assigned the same representative median income. If you rank the cases, the value that occurs more than once weights the median:\n\n1 24,586\n2 24,586\n3 24,586\n4 29,483\n5 32,771\n\nTo double-check, I threw all 40,200 cases into SPSS and ran case summaries, calculating the median of median incomes, grouped by schools. Then, I used FoxPro to group the 40,200 cases by school and block group. The query created a table listing, for each school, all block groups from which the school was drawing students and the number of students it was drawing. I opened the resulting table into SPSS, weighted the data by number of students in each block group, and then calculated the grouped median for each school.\n\nThe results were the same both ways.\n\nSPSS's ability to weight grouped data by frequency is one of the many features of this program that make it a powerful tool for data analysis. Calculating weighted medians of grouped data in Excel can be done, but that's another article.\n\nCommercial vendors\n\nAs we worked with our schools project, and discussed the nuances of weighting data, we also found ourselves grappling with the question of whether and how to use marketing data from a commercial vendor.\n\nExperienced CAR practitioners approach commercial data with cautious skepticism. After deciding to use census block group median household income as an indicator of students' economic status, we were faced with a choice: use the by-now-badly-outdated 1990 census data, or use updated estimates from a commercial data vendor. After giving it a good bit of thought, and weighing comments offered up on NICAR-L, we opted to use 1999 estimates from Claritas. Whenever the subject comes up on NICAR-L, the comments offered are thought-provoking. The Miami Herald's Tim Henderson views commercial data with a jaundiced eye. Tim points out that annual estimates put out by commercial marketing data firms are just that: estimates. He further points out that the commercial firms, for competitive reasons, don't like to share a lot of information about how they arrive at their estimates.\n\nThe fact that estimates are being drawn using vague methods should be enough to give sober journalists pause. But then, so should using 1990 census data to try to talk about the characteristics of our community in 1999.\n\nOn one point there is strong agreement among most who have weighed in on the topic on NICAR-L: it would be risky, indeed, to use annual estimates, whether from the U.S. Census Bureau's Current Population Survey or from commercial vendors, to try to measure changes over time at the local level and more dangerous still to try to use such data to measure year-to-year changes at the tract level.\n\nUsing small-area data to compare different parts of your community within the same year is a different matter than using the data to try to measure changes over time (though the rationale for using 1999 Claritas estimates - the assumption that it is more up-to-date - presumes that the commercial source has made some sort of reasonably accurate assessment of changes over time).\n\nUse with caution\n\nA consensus emerges among many NICARians as to how to handle the use of commercial data (and these principles apply more broadly, to data other than commercial products): tell your readers/viewers as completely as possible what data you're using, how you're using it, and what its limitations are. Look for other sources of data to either corroborate or question the commercial data.\n\nThis is the approach we're following at The Oklahoman as we use this data. We've cross-referenced Claritas' block group household income data, aggregated to the school level, with school-level free and reduced-price lunch figures, and we've found a rough, if imperfect correlation. As the median income figure for a school goes down, the percentage of students receiving free and reduced-price lunches tends to go up.\n\nIn instances where the techniques we're using show obvious weakness, we're pointing out those weaknesses to our readers.\n\nOverall, we believe the data we're using is allowing us to measure what we set out to measure. Even though it's imperfect, it's allowed us to take a deeper, more meaningful look at the story than we could without the data.\n\nAs Steve Doig, Knight Chair in Journalism at Arizona State University points out, before too long we'll have a solid benchmark against which to compare the data we're now using: the 2000 census.\n\n\"I think a really interesting story to be done shortly after the 2000 data is released will be a comparison of how closely the various vendors were to reality,\" Doig says.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-the-future-of-car","slug":"2000-januaryfebruary-the-future-of-car","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-the-future-of-car.html","year":2000,"month":"January/February","monthNum":1,"headline":"The future of CAR","kicker":"IN THE CLASSROOM","author_name":"David Kesmodel","author_title":"","affiliation":"IRE and NICAR","summary":"Student journalists are increasingly learning computer-assisted reporting (CAR) skills to analyze data and uncover stories, with journalism schools adopting various approaches to teach these skills either as standalone courses or integrated throughout their curricula.","keywords":["journalism","data","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Excel","Access"],"full_text":"When student journalists Katie Crecente and Summer Parrish got a tip last year that prices for the same drug prescriptions varied widely among pharmacies in their community, they thought they might have a story. An analysis of an Excel spreadsheet confirmed their hunch.\n\nThe University of South Carolina students asked the pharmacies for prices on four types of drugs and built their spreadsheet in a computer-assisted reporting class. Crecente reported the story on the local cable TV system and won first place in the student competition for the Southern Regional Emmy Awards.\n\nCrecente and Parrish are not alone. Students in dozens of U.S. journalism and mass communication programs are learning to analyze information in spreadsheets and database managers, and to find the story behind the numbers.\n\nMany of the schools now teaching CAR based their approach on the program at the Missouri School of Journalism, in Columbia, Mo., which just completed its 10th year. The program has also led the way by training dozens of professors at boot camps offered by NICAR, a joint venture between the school and Investigative Reporters and Editors.\n\nThe issue many journalism educators say they are now wrestling with is not whether to teach CAR skills, but how and to what degree. Increasingly, they are trying to decide whether to teach the skills in a stand-alone class, weave them into other reporting and writing classes, or do both.\n\nAccording to an informal e-mail survey of 100 accredited journalism and mass communication programs for this story, educators appear to be evenly split in their approaches to teaching CAR. One half of the 64 respondents said they offered a stand-alone CAR class, although some do not offer the class every semester.\n\nOf the 32 programs that don't teach a stand-alone CAR class, 25 said the skills are taught in other courses. And most of the schools that offer a freestanding class say they also try to integrate the skills throughout the curriculum.\n\nSteve Doig, the Knight Chair at Arizona State University's Cronkite School of Journalism and Telecommunication, said the goal of trying to teach all journalism students basic CAR skills is difficult to achieve, particularly in large schools like his. \"I have 15 students each semester in my (CAR) class, 30 in a year,\" he said. \"There are 1,400 students in the Cronkite school. I can't teach them all.\"\n\nDoig said the school is tinkering with ways to circulate CAR throughout the curriculum. His current tactics include introducing students to online resources in his basic news writing class and presenting guest lectures on CAR in his colleagues' classes.\n\nThe classes\n\nThe names that are given to stand-alone CAR classes vary, including precision journalism or computer-assisted reporting. Some schools spend a lot of time teaching online research skills, while others concentrate on spreadsheets and database managers.\n\nAt the Missouri School of Journalism, CAR is taught in a stand-alone class and incorporated into an investigative reporting class taught by Brant Houston, executive director of IRE and NICAR.\n\n\"Part of our mission here is to make sure this tool is blended into good reporting,\" said Houston, author of the textbook, \"Computer-Assisted Reporting: A Practical Guide.\"\n\nOne of his students, Mark Greenblatt, used National Bridge Inventory data to report on failing bridges in Missouri.\n\nFor the stand-alone class, Jo Craven trains students to use Microsoft Excel and Access and to negotiate for data from government agencies. In addition to weekly lessons, students work on two semester-long projects.\n\nCraven wants students to take away from her class the ability to logically go about their reporting, whether the story involves a dataset or not. \"I want them to learn to respond to those bells in their head that say, 'Something here is not right.' The worst thing they can do as reporters is have a funny feeling and not follow up on it,\" she said.\n\nOne of her students, Charles Choi, recently built a spreadsheet of data on check fraud in Columbia and wrote a front-page story for the Columbia Missourian. Choi said journalism schools with stand-alone CAR classes are giving aspiring journalists more reasons to get a degree in journalism, as opposed to a liberal arts degree.\n\n\"This makes journalism school more important,\" he said. \"Not everybody needs a V-8 engine, but it's really nice.\"\n\nFactors\n\nA lot of factors affect a school's approach to teaching CAR, including faculty members' philosophical views about the subject, whether adequate resources are available, and whether faculty members are qualified to teach CAR.\n\nThe bulk of the accredited programs offering a stand-alone CAR class teach it as an elective, because under the accreditation rules, students must take about three-fourths of their classes outside their major, and the schools can offer only so many core journalism classes.\n\nAccreditation has influenced the decision of the journalism department at Washington and Lee University not to offer a freestanding CAR class, said Ham Smith, department head.\n\n\"In the highly restricted accredited curriculum, it does not seem wise to require a CAR course,\" he said.\n\nMeanwhile, he argued, teaching CAR as an elective is problematic.\n\n\"Isolating CAR in a non-required course unfairly limits the number of students knowledgeable\" in CAR, he said, \"so we dribble it out everywhere as a standard reporting activity.\"\n\nPhilip Meyer, a professor of journalism at the University of North Carolina-Chapel Hill and a pioneer in using computers to enhance reporting, feels strongly that his school should not offer a stand-alone CAR class. At UNC, the skills are taught in other classes.\n\n\"Teaching CAR as a separate class perpetuates the illusion that computer competence is an isolated and esoteric specialty accessible only to a few,\" Meyer said.\n\nVirginia Commonwealth University also has subscribed to the theory that CAR should not be taught as a separate class, said Jeff South, an associate professor in the School of Mass Communications.\n\nBut what South and some other educators say they have discovered is that when you don't teach a specialized class in CAR and depend on faculty members to integrate the skills in other courses, the job doesn't always get done.\n\n\"When everyone is responsible for CAR, it can turn out that no one is responsible for CAR,\" South said. At VCU, he said, \"some faculty members did not have adequate CAR training and did not feel comfortable teaching CAR.\"\n\nNow, the school is looking into requiring a course in information gathering, which would introduce students to online resources and perhaps to spreadsheets and database managers.\n\nWith time, say many educators, stand-alone classes will be phased out as CAR becomes a more routine reporting tool. Schools may teach advanced classes in CAR, but more and more students won't blink when handed data in a spreadsheet.\n\n\"Hopefully the concept of using a spreadsheet and going online will become so ingrained that treating it as a separate weird specialty will no longer be important,\" Doig said.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-troubled-bridges","slug":"2000-januaryfebruary-troubled-bridges","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-troubled-bridges.html","year":2000,"month":"January/February","monthNum":1,"headline":"Troubled bridges","kicker":"FIRST VENTURE","author_name":"Mark Greenblatt","author_title":"","affiliation":"University of Missouri","summary":"A journalism student uses computer-assisted reporting to investigate bridge conditions in Missouri, discovering that while database analysis provided initial leads, direct expert consultation and inspection data proved essential for uncovering the actual state of disrepair in local bridges.","keywords":["bridges","database","investigation"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["National Bridge Inventory"],"full_text":"Troubled bridges\n\nBy Mark Greenblatt\nUniversity of Missouri\n\nI wanted to find an easy database to work with for my first computer-assisted story. I had taken a CAR course at the University of Missouri-Columbia earlier, but had put off applying what learned. My only goals were to use the CAR skills I learned and also to play around a little bit with my brand new computer.\n\nSince bridges have always fascinated me, I thought I'd take a look at the conditions of the ones in my area for a project in an investigative reporting class, taught by IRE and NICAR Executive Director Brant Houston.\n\nI wanted to play it safe the first time and avoid getting a database with dirty data, so I obtained my copy of the bridges inspection database for Missouri directly from NICAR. I had no idea what I was about to get myself into.\n\nIt turned out we could only use the database as a tipsheet to finding bridges of interest, and also to find the bigger picture.\n\nThree months later, the story I uncovered would have Missouri lawmakers calling for quick reform. The state of the bridges in our area turned out to be in dismal shape.\n\nStarting out\n\nKOMU TV's news director Stacey Woelfel and I originally thought we could do a quick story to see if any bridges open to traffic in Columbia and around our viewing area were in bad shape. We had heard rumors of bad bridges but had nothing nailed down. It would be easy, I thought, to simply run a few queries on the database, come up with a list of the worst bridges, and broadcast the story after confirming a few facts with local bridge inspectors. I couldn't have been more wrong, and that turned out to be good.\n\nJust learning about the database took me a couple weeks of playing around on my spare time. The record layout of the National Bridge Inventory was more than 100 pages long, and it included a lot very technical terms like sufficiency rating formulas, structure numbers and deck geometry.\n\nThe sufficiency rating is a computer-generated formula that is supposed to represent each bridge's overall condition. I ran different queries showing the bridges in mid-Missouri with the lowest sufficiency ratings. With that list in hand, I contacted a local bridge inspector with the Missouri Department of Transportation to learn more.\n\nHe immediately told me that using sufficiency ratings was the wrong way to do this study. The only way to look at bridges is to look directly at the safety ratings for their substructures, superstructures and decks, he said. He took me out on bridge inspections for a day to teach me about his job.\n\nBy the end of the day, I had determined there was no accurate way we could use the numbers in the National Bridge Inventory to point us towards any list of the 10 worst bridges in mid-Missouri. It turned out we could only use the database as a tipsheet to finding bridges of interest, and also to find the bigger picture.\n\nNew direction\n\nI went back to my computer and started running more queries. I ranked the mid-Missouri bridges with the worst substructures and the worst decks, and for the first time grew concerned about the sheer numbers of bridges in bad shape in our area. Far too many bridges were rated in poor condition, and many of them carried thousands of cars a day on major highways.\n\nAfter examining the written records of countless bridges to match them with the computer data for accuracy, I noticed a common trait. All the bridges the computer data pointed to as in bad shape were all determined to be structurally deficient. That means they can't carry the legal loads they were designed to hold.\n\nA query we ran showed that close to one-third of the 3,000 bridges in mid-Missouri were in that bad shape. The break in this story came when the computer data pointed us towards one bridge that seemed like it should be closed based on its ratings. The written report said the bridge was in \"exceptional\" condition.\n\nIt turned out the Missouri Department of Transportation keeps a list of 19 bridges in mid-Missouri that are \"exceptionally\" deficient, yet still open to traffic. One of a few inside sources I had developed anonymously passed that list on to KOMU.\n\nThe list was remarkably frightening. It showed the Missouri Department of Transportation knew the bridges in mid-Missouri were falling apart in some cases, but it had no immediate plans to fix them. One bridge we focused on was described as having \"concrete periodically falling into traffic lanes\" on Interstate 70.\n\nWe queried the database again, and found out it would take 49 years to fix all the bridges that needed repairs in mid-Missouri at the rate the state was going. The problem was, state officials told us the average lifespan of a bridge is only 50 years.\n\nIn the end, the computer database we used added depth to the story and made it possible to more quickly uncover larger, specific problems. After the story aired, the Speaker of the House of the Missouri House of Representatives requested a copy of the story to use in the Legislature to push for reform. Other lawmakers also called, and an internal MoDOT e-mail was forwarded to me that showed MoDOT was \"putting everything else on the backburner\" for a while to examine what they can do about their bridges in mid-Missouri.\n\nI think it's safe to say I'm hooked on CAR after doing this project. I can't wait to get ahold of another set of data.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-parsing-nightmare","slug":"2000-januaryfebruary-parsing-nightmare","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-parsing-nightmare.html","year":2000,"month":"January/February","monthNum":1,"headline":"Parsing nightmare","kicker":"TECH TIP","author_name":"MaryJo Sylwester","author_title":"Managing Editor","affiliation":"IRE and NICAR","summary":"Robert Gebeloff sought solutions for parsing a semicolon-delimited ASCII data file lacking record separators, receiving eight different approaches ranging from Perl scripts to Word macros to custom programming code.","keywords":["parsing","data","delimiter"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Perl","Word","Visual Basic"],"full_text":"Many times there is more than one solution to a problem - as Robert Gebeloff discovered in late November when faced with an unusual parsing dilemma. His post to the NICAR-L thread elicited about eight different solutions. Here's the problem: a state agency had grudgingly turned over some data in ASCII, delimited with semicolons, with each record ending in the person's name. The big problem, though, was that there wasn't a delimiter between records. Here's a sample: GRADY,SELMA ;8M MIGLIORE MANOR ;ELIZABETH;NJ;07206-64; NB00010500; Z;99;19340127;SELMA GRADY Gebeloff wanted to know if there was a way to construct a search/replace that would recognize the name reversal at the end of the record and insert some kind of delimiter. Here's a few of the suggestions, including the ultimate solution Gebeloff used:\n\nSuggestion one:\n\nMatthew Ericson, of The Philadelphia Inquirer, suggested using Perl to construct a search-and-replace expression that will put a line break after each record: perl -p-e's/((.+?),(.+?)\\s*;+?)\\3 \\2\\s*/\\1\\n/g;\"datafile.txt More information about Perl is available www.perl.com\n\nSuggestion two:\n\nPaul Skolnick, of Thunder & Lightning News Service in Belleair Bluffs, Fla., suggested this solution: run the text file through a macro in a (newer version) word-processing program to parse the name fields. Since the first and last words in the records are the same, you would insert a carriage return-line feed after the second appearance and set that off as one distinct record. The macro recorder in Word 2000 is in the Tools menu. It has a Record function in it, but you would have to doctor the recorded macro some.\n\nSuggestion three:\n\nOffered by David Heath, of The Seattle Times, this solution preserves the original data, so commas before first and middle names are kept. This program takes bits of the file to work with at a time, which requires more code, but ultimately makes the program run faster.\n\nDeclare variables, including an array for the nine fields. Dim intReadFile, intWriteFile As Integer Dim strLine, strTempLine As String Dim strField(1 To 9) As String Dim X, ctr As Integer Dim lngCheckName, lngLengthOffields As Long\n\nOpen the text file and the output file intReadFile = FreeFile Open \"d:\\data\\parse\\newhha1.txt\" For Input As #intReadFile intWriteFile = FreeFile Open \"d:\\data\\parse\\output.txt\" For Output As #intWriteFile\n\nPrint the field names in the first line of the output file Print #intWriteFile, \"Name\" + \"Address\" + + \"State\" \",\" + \"ZipCode\" + \"LicenseNo\" \"Code\" + + \"Expires\" Start a loop to read the text file. In actuality, it will loop only once. Do While Not EOF(intReadFile) Stuff the whole file into a variable \"strLine\" Line Input #intReadFile, strLine\n\nStart a second loop to read smaller chunks of the file This program is made more complicated by efforts to speed up the processing. It obviates the need to resave the whole file into memory after parsing each field. Do While Len(strLine) > 0 Initialize the counters 1 ctr = ctr + 1 lngLengthOffields = 0\n\nCut a chunk of 5000 bytes off the file and manipulate it. This makes the program run much faster. strTempLine = Mid(strLine, 1, 5000)\n\nFeed the nine fields into an array, using semi-colons as the delimiter For X = 1 To 9\n\nIf PositionOfSemiColon(strTempLine) = 0 Then Exit Do strField(x) = Mid(strTempLine, 1, PositionOfSemiColon(strTempLine) - 1) strTempLine = TruncateLine(strTempLine) lngLengthOffields = lngLengthOffields + Len(strField(x)) + 1 Next X\n\nCut off the start of the file already fed into the array strLine = LTrim(Mid(strLine, lngLengthOffields + 1))\n\nMeasure the length of the extraneous data and cut it out of the next record If ctr = 1 Then lngCheckName = Len(strField(1)) If ctr > 1 Then strField(1) = LTrim(Mid(strField(1), lngCheckName)) lngCheckName = Len(strField(1)) End If\n\nWrite the data to a comma-delimited file Write #intWriteFile, LTrim(strField(1)), LTrim(strField(2)), strField(3), strField(4), strField(5), strField(6), strField(7), strField(8) Loop Loop\n\nClose the input and output files Close Notify the user when the program is done retval = MsgBox(\"All Done\", vbOKOnly, \"Go Home\") End End Sub\n\nCreate a function to locate semi-colons Private Function PositionOfSemiColon(String1 As String) PositionOfSemiColon = InStr(1, String1, \";\", vbTextCompare) End Function\n\nCreate a function to cut data from the file once read Private Function TruncateLine(String1 As String) TruncateLine = Mid(String1, PositionOfSemiColon(String1)) End Function\n\nSolution:\n\nA Visual Basic script, drafted by Tim Henderson, of The Miami Herald, and modified slightly by Gebeloff, creates a text file and nine variables.\n\nIt then creates a loop that goes through the data looking for field delimiters. The contents of the first nine fields are assigned to the variables, and then it stacks the next nine records underneath, and so on until it runs out of data. The result is theoretically a tab-delimited text file that has line breaks after each record.\n\nGebeloff discovered, however, that his software didn't recognize tabs, so he changed the script to insert a comma between fields in the new table.\n\nHe also had to swap in commas for semicolons, which meant getting rid of extraneous commas in the original data. The script (run from Word's VB Editor):\n\nSub ParseDelimited() variables for the two text files we need Dim outfile As Integer Dim infile As Integer\n\nvariables for the fields we create Dim add As String, city As String, st As String, zip As String, licno As String, code As String, expires As String, dob As String, name As String\n\nOpen the file to receive the parsed fields outfile = FreeFile Open \"c:\\temp\\hhc2.txt\" For Output As outfile Open the original datafile infile = FreeFile Open \"c:\\temp\\healthaides\\hhc2.txt\" For Input As infile\n\nLoop through the file inputting the variables Do While Not EOF(infile) Input #infile, add, city, st, zip, licno, code, expires, dob, name Each time you get all 9 fields put them in the new file separated by commas Print #outfile, add & \",\" & city & \",\" & st & \",\" & zip & \",\" & licno & \",\" & code & \",\" & expires & \",\" & dob & \",\" & name Loop Clean up Close #infile Close #outfile End Sub\n\nMaryJo Sylwester can be reached by e-mail at maryjo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-census-glossary","slug":"2000-januaryfebruary-census-glossary","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-census-glossary.html","year":2000,"month":"January/February","monthNum":1,"headline":"Census glossary","kicker":"CENSUS 2000","author_name":"Paul Overberg","author_title":"reporter","affiliation":"USA Today","summary":"A glossary explaining key Census Bureau terms and procedures, including non-response follow-up methods, accuracy measurement techniques, and local outreach strategies used during the decennial census.","keywords":["census","methodology","accuracy"],"topics":["Demographics & Census"],"technologies":[],"full_text":"Census glossary (From handout #809, by Paul Overberg of USA Today):\n\nNRFU: Non-response follow-up. If you don't mail back your census form, a live human comes to interview you. Or eventually, your neighbors, if you can't be found.\n\nICM: Integrated Coverage Measurement. A national survey of 750,000 households that will be done just after and matched to the census, duplicating its core in miniature. It will be used to double-check the accuracy of the data and, Congress willing, to adjust totals.\n\nCCC: Complete Count Committee. Local advisory groups assembled to get the word out and cue the Census Bureau to local circumstances.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-predicting-next-pope","slug":"2000-januaryfebruary-predicting-next-pope","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-predicting-next-pope.html","year":2000,"month":"January/February","monthNum":1,"headline":"Predicting next Pope","kicker":"ON THE BEAT","author_name":"David Crumm","author_title":"Reporter","affiliation":"Free Press","summary":"The Free Press created a comprehensive MS Access database of current and historical cardinals to analyze potential papal successors to the aging Pope John Paul II, discovering 107 eligible candidates and tracking demographic shifts in the College of Cardinals since 1978.","keywords":["pope","cardinals","database"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["Access"],"full_text":"Since the current pontiff, Pope John Paul II, is 79 and in poor health, many Catholics are concerned about the identity of his possible successors. What kind of man will emerge as the next worldwide leader of the Catholic Church?\n\nThat's what the Free Press hoped to explore in an in-depth examination of leading papal candidates.\n\nThe problem is that, if John Paul died tomorrow, there would be 107 qualified candidates: all of the current cardinals under age 80.\n\nBeyond interviewing Vatican-watchers, church scholars and our local cardinal, we wanted to offer our own analysis of the current College of Cardinals. The first question was the most basic: exactly how many cardinals were under age 80 and thus were eligible candidates?\n\nBeyond that, we wanted to know more about the background of the cardinals. For instance, could another non-Italian be elected to follow John Paul? How heavily were various countries and continents represented in the current group?\n\nTo resolve these questions - and to be able to accurately track the exact makeup of the College of Cardinals in future stories - I created an MS Access 97 database of all current cardinals. To provide a comparison with the makeup of the college that elected John Paul II, I also built a database of all cardinals involved in the 1978 conclave.\n\nI collected data on more than 200 cardinals: their names, birthdates, countries of origin, current assignments, etc. Building the database took several full days of painstaking work, because the information was available only in a printed format - and had to be extracted from a series of annual almanacs. In this case, I could not directly import electronic data.\n\nI wound up with more than a dozen categories of data about each man - including a category to identify those cardinals in our database who had played a role in the church since the 1978 conclave, but who had died.\n\nImmediately, we were able to identify 107 men as eligible for election - as of our publication date. Looking deeper into the data, we found that, since John Paul's election, he has thinned the ranks of Italian cardinals from 23 to 17 - and has spread a few of those votes to new cardinals from Africa and the United States.\n\nWe also determined that John Paul has personally appointed nearly 90 percent of the current College of Cardinals, an indication that most of the candidates tend to agree with the current pope on major issues.\n\nThe database was the key to writing our story with authority and specificity about the numbers - and, overall, to pushing our analysis of the candidates to a higher level.\n\nThe valuable bonus in this case is that, as we continue to follow the issue of papal transition, we can quickly check in on the accurate list of candidates at any point in the future.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-religion-web-sites","slug":"2000-januaryfebruary-religion-web-sites","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-religion-web-sites.html","year":2000,"month":"January/February","monthNum":1,"headline":"Religion Web sites","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A compilation of religion-focused websites curated for journalists covering religious topics, including Christian organizations, churches, and interfaith resources.","keywords":["religion","resources","websites"],"topics":["Technology"],"technologies":["World Wide Web"],"full_text":"Here are just a few religion Web sites from Shawn McIntosh's beat page (www.reporter.org/beat/religion.html):\n\nChristian Coalition: www.cc.org/\n\nCovenant Baptist Church -Texas: http://CovenantBaptist.org/\n\nReligion News Links (Zondervan Publishing House): www.zondervan.com/newslink.htm\n\nNation of Islam Online: www.noi.org/\n\nInternet for Christians: www.gospelcom.net/ifel\n\nThe Islamic Herald: www.ais.org/~bsb/Herald/links.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-bits-bytes-and-barks","slug":"2000-januaryfebruary-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-bits-bytes-and-barks.html","year":2000,"month":"January/February","monthNum":1,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"MaryJo Sylwester","author_title":"","affiliation":"NICAR","summary":"The Uplink newsletter survey reveals that readers across all skill levels want more technical how-to content, prompting the publication to improve its coverage and incorporate more resources tailored to small news organizations.","keywords":["survey","technical","journalism"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nUplink survey\n\nThank you to everyone who took the time to answer our survey on the NICAR Web site during the fall. We've compiled the results and will use them to provide content that better suits your needs.\n\nOverall, we found Uplink has a wide range of readers in terms of skill levels. At least from the survey (which, of course, isn't statistically valid), our readership appears to consist of an equal mix of those who consider themselves beginner, intermediate and advanced.\n\nThe biggest finding of the survey was that readers of all skill levels want more technical stories - with specific \"how-to\" examples. This is something we're going to address immediately, and I hope you'll see gradual improvements in this area in the coming issues. And keep in mind that you can help by offering to share your solution to a tough technical problem. Eventually, somebody else may offer a solution that would help you as well.\n\nWe're also working to incorporate more information pertinent to small news organizations - since doing CAR with limited resources is very different.\n\nAs always, we encourage you to share the results of your work, no matter how large of a news organization you work at or whether it's print or broadcast. Your examples and tales (of both success and failure) can help others learn.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-nicar-data-library","slug":"2000-januaryfebruary-nicar-data-library","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-nicar-data-library.html","year":2000,"month":"January/February","monthNum":1,"headline":"NICAR data library","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"The U.S. Coast Guard's Marine Safety Information System database has been updated through June 30, 1999, containing registration information on over 265,000 commercial and recreational vessels that journalists can use to investigate shipping accidents and identify boat owners. The dataset is available for purchase at tiered pricing based on news organization size.","keywords":["vessels","registration","database"],"topics":["Data & Databases","Investigative Methods","Public Safety"],"technologies":["Marine Safety Information System"],"full_text":"The boat registration dataset has been updated through June 30, 1999. This database, the U.S. Coast Guard's Marine Safety Information System, contains information on commercial and recreational vessels.\n\nIt can be used mainly to find out information about a specific boat or ship, such as finding the owner. Reporters have used this dataset when a shipping accident occurs or to find out information about U.S. Coast Guard boats.\n\nThere is documentation for 265,584 vessels registered with the Coast Guard, including the company name and address. A second table includes information on boats with expired registration, foreign vessels and recreational boats.\n\nThe dataset is available for the entire United States at a cost of $60 for small news organizations, $80 for those between 50,000 and 100,000 circulation (or 25-50 market) and $100 for large news organizations.\n\nMore information is available on the NICAR Web site, www.nicar.org/data, or by calling the data library at (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-januaryfebruary-census-workshops","slug":"2000-januaryfebruary-census-workshops","url":"https://dwillis.github.io/uplink/article/2000-januaryfebruary-census-workshops.html","year":2000,"month":"January/February","monthNum":1,"headline":"Census Workshops","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are planning Census 2000 data workshops with expert trainers Steve Doig and Paul Overberg, with dates and locations to be announced soon.","keywords":["Census","workshops","training"],"topics":["Journalism & Newsroom","Demographics & Census"],"technologies":[],"full_text":"IRE and NICAR will be holding Census workshops both before and after the data is released. We're working with Steve Doig, of Arizona State University, and Paul Overberg, of USA Today, to provide hands-on training on working with the Census 2000 data. Dates and locations haven't been set yet.\n\nStay tuned here or at the IRE Web site, www.ire.org. for more information.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-around-the-globe","slug":"2000-march-around-the-globe","url":"https://dwillis.github.io/uplink/article/2000-march-around-the-globe.html","year":2000,"month":"March","monthNum":3,"headline":"Around the globe","kicker":"INTERNATIONAL CAR","author_name":"David Wethe","author_title":"","affiliation":"IRE and NICAR","summary":"Computer-assisted reporting is expanding globally, with journalists in Europe and South America using data analysis to create compelling stories, though access to electronic information remains a significant barrier due to high costs and limited freedom of information laws outside the United States.","keywords":["journalism","data","information"],"topics":["Investigative Methods","Journalism & Newsroom","Government & Politics"],"technologies":["spreadsheet"],"full_text":"Although the use of computer-assisted reporting is growing in countries outside of the United States, the issue of access to information - particularly electronic information - remains a concern.\n\nFrom South America to Scandinavia, journalists say more and more information is becoming available electronically, but it is either aggregated at a municipal or county level or is high-priced from a private company.\n\nTo remedy the situation, journalists in European Union member countries are pushing for the EU to develop a Freedom of Information law similar to that in the United States.\n\nIn the meantime, aggressive and dedicated journalists continue to publish and broadcast compelling and powerful stories using computer-assisted reporting.\n\nIn this issue, we've included snapshots of some of the work being done around the world. Here's a glimpse:\n\nNetherlands\n\nThe origin of CAR in the Netherlands came in 1994 when the NRC Handelsblad became the first in its country to publish the results of an election mapping project.\n\nThe Handelsblad displayed in full color graphics how each party finished according to individual provinces. The idea drew the attention from other media outlets in radio, TV and magazine and is now considered a standard for publishing election results. Yet despite the recent success, the public is still slow to catch on to the importance of CAR projects. Sometimes projects fail to draw any remarks at all from readers, said Dick van Eijk of NRC Handelsblad.\n\nA big problem for Dutch journalists is the lack of independent oversight in making sure public data is accessible to everyone. Although most government information is offered in electronic format, it is still too expensive for most newspapers to obtain. The most expensive data is anything on companies or homeowners, van Eijk said.\n\nA recent coup for Dutch journalists, however, was winning a court case that deemed public schools data as public information.\n\nVan Eijk has used CAR to look at topics ranging from royal decorations to attendance and voting of European Parliament members. He analyzed where the 2,000 recipients of Queen's Day Royal Decorations lived and used maps that show that the higher decorations were granted in different areas than lower ones.\n\nPrior to the European Parliament elections, Van Eijk analyzed how often members of the parliament had been present at meetings and how they voted during the past five years. National background turned out to be a key factor. The Dutch were present and voted most often while Italians were least often present.\n\nDenmark\n\nThree Danish journalists first learned how to incorporate databases and spreadsheets into their everyday work when they attended a 1996 CAR bootcamp in Columbia, Mo. Last April, the trio started the Danish Institute for Computer-Assisted Reporting. Since then, DICAR has attracted more than 200 members.\n\nNils Mulvad, who helped start DICAR, said that although most Danish editors are reluctant to change, CAR is starting to catch on. Mulvad said his current goal is to forge a joint initiative among other journalists to fight for electronic access to documents. (Read more about DICAR on page three.)\n\nBrazil\n\nReporter Jose Roberto de Toledo of the Brazilian newspaper Folha de Sao Paulo said most journalists are still slow to catch on to CAR in his country. In fact, many government agencies are more advanced with electronic files than journalists, he said. Despite that, de Toledo has published several crime stories with the help of CAR. He constructed a database of more than 100 cases of multiple homicides for 1998 as a way to profile the victims. By using other crime data, de Toledo was able to offer for the first time a list of the Brazilian cities with the most homicides. Like in other foreign countries, Brazilian readers' interest in CAR stories has been limited to such hot topics as violence and unemployment. \"If the article has too many numbers, rates, percentages and little ideas, it tends to be boring,\" de Toledo said.\n\nDavid Wethe can be reached by e-mail at davidw@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-lake-wobegon-effect","slug":"2000-march-lake-wobegon-effect","url":"https://dwillis.github.io/uplink/article/2000-march-lake-wobegon-effect.html","year":2000,"month":"March","monthNum":3,"headline":"Lake Wobegon effect","kicker":"SCHOOL TEST SCORES","author_name":"Drew Sullivan","author_title":"","affiliation":"The Tennessean","summary":"The Tennessean investigated Tennessee's claim of above-average student test scores and found significant discrepancies between norm-referenced tests and independent assessments like NAEP and ACT, revealing the state's scores were inflated. The investigation discovered that 80% of states using the same TerraNova test showed above-average results, but only 50% scored above average on the NAEP test, illustrating how test selection and presentation can distort educational performance claims.","keywords":["testing","scores","inflation"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["spreadsheet","TerraNova"],"full_text":"Each fall, the state of Tennessee releases its school test scores and every year The Tennessean dutifully runs three open pages of scores and graphs showing how local schools are doing. One perennial result is that Tennessee's students score above the national average in most subject areas, a fact trumpeted by the state department of education.\n\nLast fall, we decided to look into that claim. What resulted was a quick one-week story that called into question the basis of this claim. We also learned an important lesson on how information is presented and the importance of questioning basic assumptions.\n\nAt the heart of the matter is the way standardized tests are done nationwide. State assessment programs often include two elements: a norm-referenced test and a criterion referenced test. The first allows the state to look at how the kids in the state match up to national standards. The second is how kids match up to state standards. It was the norm-referenced test we were interested in. Tennessee students ranked between the 49th and the 59th national percentile rank depending on the grade level and the subject area of the test. To check whether these numbers were inflated, we asked around about other tests that might corroborate these results. This is always a good technique to check the results of any study or statistic. What we found was the National Assessment of Educational Progress or NAEP test. The NAEP test, which is administered by the Department of Education, is given in one subject area every couple of years to a sample of students across the country. In 1998, students were tested in reading and in 1996 and 1992 students were tested in math. The comparison surprised us. Tennessee showed up below the national average in both tests. The discrepancies between the two tests were large. We checked another test. We found Tennessee students scored below the national average in every subject area of the American College Test or ACT given to high school students.\n\nChecking inflation\n\nTo verify the extent of the inflation, The Tennessean built a spreadsheet of all states that used the same test we did, the TerraNova published by CTB/McGraw Hill. We found 17 states that used the TerraNova but only 11 states used it to calculate national norms. We then got the national percentile rank for all of those states. In 80 percent of them, the median student in a particular grade level and subject area was above the national average. We then used the NAEP for comparison and found of those same states, only 50 percent of the median student was above average.\n\nThere were limitations of our methodology (i.e. states tested at different grade levels, they used different forms of the TerraNova, NAEP years didn't correspond to some TerraNova years). To lessen these effects, we aggregated the data and didn't try to make too specific of a comparison.\n\nThe discrepancies didn't surprise testing experts. In fact, they expected them. When we asked why, the answer was uniform: the Lake Wobegon Effect, named after the mythical town in Minnesota Public Radio's \"A Prairie Home Companion\" where \"all the children are above average\".\n\nThe Lake Wobegon Effect is an inflation of the national percentile rank due to discrepancies between the current test scores and an outdated, faulty or limited national average. All our states soon report their students are above average.\n\nMost national norm-referenced standardized tests are normed every seven or eight years. The norming process involved testing a stratified sample of hundreds of thousands of students across the country to derive the national average. The norming process is rigorous, expensive and very time consuming for test publishers.\n\nLimitations\n\nBut the process has some serious limitations.\n\nAge: If test scores rise over time like they have over the last decade, more and more kids suddenly appear above average.\n\nTeaching the test: States that use the same or similar tests over time show a rapid rise in test scores. When those tests change, the scores drop dramatically. Teachers learn what is on the test and may teach types of problems or even specific problems.\n\nEnvironment: Students who help norm a standardized test take the test under a vastly different environment than students who are taking it later as part of an operational assessment program. In a norming test, students are completely unfamiliar with the type of test. Imagine taking the SAT having never seen the types of questions on it. In an operational environment, students may have been taking the same test for years and may have a much higher comfort level. Also, in an operational environment, there is enormous pressure on students and teachers for the scores to be high whereas students in the norming process know that how well they do doesn't matter to them. To test this effect, the University of Iowa re-normed a norm-referenced test in an operational environment and found large differences in what the national average would be.\n\nCheating: It's more common than people think and can skew results upward.\n\nWhile the Lake Wobegon Effect is old, it continues to skew state results, sometimes very seriously. Be aware that every norm-referenced test is a victim of this effect. The result in your state may be an apparent upward trend in scores that just isn't real.\n\nQuestions to ask\n\nHere are some questions you should ask your state assessment officers:\n\nAsk your state's test authorities how they calculate the national average, what test they use and when it was last normed. It's a complicated picture because states may use an off-the-shelf standardized test, part of a test, or just some test questions from a testing company. These questions are often mixed with criterion-referenced test questions.\n\nAsk how often your state changes test forms and how many questions are repeated every year.\n\nAsk about cases of cheating. Often these are called 'anomalies' or some other innocuous term. The state testing office collects the information because they must remove these cases from their statistical analyses. Often it may be a matter of a teacher giving students too much or too little time, but sometimes it is far more egregious.\n\nCompare your schools test results to the NAEP, ACT, SAT or other standardized tests. Also look at trends. If your students are improving on a state test and not on NAEP, you could have a problem.\n\nTalk to state assessment coordinators in other states. They often know what manipulations other states use to make themselves look good.\n\nFind out how your state deals with students who need special accommodations for the test.\n\nDrew Sullivan can be reached by e-mail at drew@tennessean.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-political-census","slug":"2000-march-political-census","url":"https://dwillis.github.io/uplink/article/2000-march-political-census.html","year":2000,"month":"March","monthNum":3,"headline":"Political Census","kicker":"VENEZUELA","author_name":"Carlos Subero","author_title":"","affiliation":"El Universal","summary":"El Universal conducted a 32-question census survey of candidates before Venezuela's National Constituent Assembly elections, enabling readers to understand the elected representatives' views immediately after the July election results were announced.","keywords":["Venezuela","elections","journalism"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["SPSS"],"full_text":"To know how politicians think at a given moment is valuable to readers in a country like Venezuela, which is going through a process of rapid political changes. But how does a news organization inquire into this?\n\nThe solution to this dilemma was an issue raised by the Research department of El Universal, a Caracas-based national journal. We designed a 32-question census addressed to a group of candidates to the National Constituent Assembly with chances of being elected in the last July election. This enabled readers to know the opinions of the Constituent Assembly that they had elected, as of the day after the election.\n\nThe Constituent Census was an unprecedented journalistic experience in the field of elite studies in Venezuela.\n\nThe Constituent Census was an unprecedented journalistic experience in the field of elite studies in Venezuela. It was possible because of the automation of the Venezuelan official ballots by the National Electoral Council. Just as it had been anticipated, the names of the 128-winning candidates were known merely two and a half hours after the balloting centers had closed.\n\nHow it started\n\nThe questionnaire, handed out to the candidates during the two weeks before the elections, contained topics relating to the national politics, some value features and socio-demographic aspects such as age, state of birth, sex and social origin.\n\nThe editor-in-chief's office had expounded to the researchers that its intent was to publish the day after the elections. Therefore, the vital issue that had to be addressed in advance was to determine who would be the winners amongst more than one thousand candidates.\n\nSuch a delicate issue was successfully resolved by resorting to some political reckoning and a little luck.\n\nThe decision on how many and what candidates had to be interviewed was made one month before the elections. It was calculated that two candidates for each of the 128 posts would result in 256 interviews, a reasonable amount to be handled by the team of 11 reporters from the Research department and the Fundación Andrés Mata.\n\nThe open-list, simple majority electoral system was used. To decide who would be the interviewees, the thesis initially adopted was to choose all the candidates identified with President Hugo Chávez, who had a very high popularity ranking. The 128 candidates were chosen from the opponent and independent party factions.\n\nIn Venezuela, candidates are willing to attend to the press on the phone, even to respond to a 32-question query. Consequently, the most difficult step in accomplishing the interviews was to get the phone numbers of the 256 candidates. Only about 10 of the candidates chosen could not be interviewed, whether because they declined to grant the interview or because they could not be contacted.\n\nResults\n\nThe electoral result was surprising. President Chávez supporters won 122 of the 128 posts, and the opposition won only four, two were independent: only one of the candidates elected had not been included on the initial list of interviewees. Out of the winners, we interviewed 92 percent.\n\nTwo reporters downloaded the questionnaires coded using the SPSS processor. Thus, the database was created with 42 different variables, with data on the answers of 256 candidates to the Constituent Assembly.\n\nWithin two hours of the release of partial election results, our final database had already been built in SPSS.\n\nThe first day, the results of four questions were published; these questions allowed us to figure out that 97 percent of the candidates elected was thinking of implementing the presidential re-election.\n\nThe SPSS crossing of the variables on family origin and the socio-educational level of the constituents resulted in one transcendent finding. It has been determined that a half of the constituents appointed, the new dominant political class, meets the condition of being a university professional who comes from the low class or the middle low class. This condition was met only by 37 percent of the Congress elected 8 months ago. None came from the high class.\n\nCertainly, the full potential of SPSS was not used. But some indices were created relating to several questions on the military topic and that of the decentralization policy, which permitted us to determine a high tendency to the political participation of the military, and also towards decentralization. The Constituent Census was the second study on political elites that El Universal conducted during 1999, and one that has been an innovative contribution of precision journalism to the political sciences for consultation at different universities in this Latin American country. A third study is pending with the election of the new Congress, which will provide a more sound knowledge on the quality and thought of the political elite and the change process that is taking place in Venezuela.\n\nCarlos Subero can be reached by e-mail at csubero@callisto.eud.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-making-car-progress","slug":"2000-march-making-car-progress","url":"https://dwillis.github.io/uplink/article/2000-march-making-car-progress.html","year":2000,"month":"March","monthNum":3,"headline":"Making CAR progress","kicker":"DENMARK","author_name":"Nils Mulvad","author_title":"","affiliation":"DICAR Committee","summary":"Danish journalists founded DICAR in 1999 to promote computer-assisted reporting (CAR) methods, establishing training programs, a membership association with 200 members, and online resources, though adoption faces resistance from media managers despite growing interest in spreadsheet and database use.","keywords":["journalism","databases","Denmark"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet","Internet"],"full_text":"It has been four years since three Danish journalists attended bootcamp at NICAR in Columbia, Mo., where we learned how to use databases and spreadsheets in our work.\n\nWe had many discussions of how these brilliant methods could be used in Denmark, where we do not have the same tradition and access to electronic data. Even though the United States and Denmark are far apart, I must say that these methods have survived the emigration.\n\nIn April 1999, we founded DICAR, Danish Institute of Computer Assisted Reporting, which is located at the Danish School of Journalism in Ärhus. Here we share premises with the journalism students making their own Internet paper, Byline.\n\nA lot has happened during the past year. The number of members of the Foreningen for Computerstottet Journalistik (Association of Computer Assisted Reporting), which was founded six months ago, has now reached 200.\n\nHow far we've come:\n\nThis spring we start teaching senior journalism students a ten week CAR-module.\nWe have made an in-service course program in cooperation with DjE, The Danish Institute of Journalistic Supplementary Training. The program includes training in spreadsheets, databases, the use of Internet and other computer-based tools tailored for journalists. This is probably the beginning of a long-term collaboration with DjE, and it has already resulted in the hiring of a part-time course leader.\nNext month, we will gather for the third CAR-day, and we have a longer program than at previous CAR-days. We also arrange conferences and often introductory speakers at other events.\nThe past 18 months, we have been running a web-site, www.dicar.dk, which is continuously updated with new services for members of Foreningen for Computerstottet Journalistik and services which are available to non-members.\nAt the DICAR home page we are building tool boxes for journalists within different beats or subject areas.\nWe are starting a bank of databases for our members.\nWe have hired five students on a part-time basis to solve different assignments for DICAR.\n\nDaily use of CAR\n\nThe number of media using CAR methods is steadily increasing, especially those using spreadsheets and self-made databases. But as always, there is a big step from theoretic lectures to using the methods on a daily basis for stories. We suffer from the fact that the media managers are very reluctant when it comes to new methods. It hurts CAR, but also many of the other good ideas of how the future journalist should work. It really shows in Denmark at the moment, where the journalism craft is affected by the recession in Denmark and the resulting resignations and changes in management. But some editors are slowly recognizing the advantages of CAR.\n\nOur experience is that the authorities actually listen as we get access to still more data, and we seem to have a good dialog with a number of authorities.\n\nIn connection with the CAR work we have published, there have been a number of reports to authorities concerning electronic access to databases.\n\nOur experience is that the authorities actually listen as we get access to still more data, and we seem to have a good dialog with a number of authorities.\n\nWe are still in good contact with the pioneers from United States and with some of our colleagues in other European countries.\n\nWe hope to extend this cooperation, especially with the idea of establishing joint initiatives throughout Europe concerning the right to electronic access to documents. In this connection we have worked out an English-language description of the current status of the right to electronic access of documents in Denmark and a description of the new rules from EU concerning personal data. (If you are interested in the descriptions you can e-mail dicar@dicar.dk.) Last, but not least, we are having a great time together while working cooperatively on DICAR-projects. These days we are beginning to take out re-subscriptions for the first time and of course we are anxious to see how that goes.\nNils Mulvad can be reached by e-mail at nmulvad@po.ia.dk","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-white-collar-crime","slug":"2000-march-white-collar-crime","url":"https://dwillis.github.io/uplink/article/2000-march-white-collar-crime.html","year":2000,"month":"March","monthNum":3,"headline":"White collar crime","kicker":"SWEDEN","author_name":"Helena Bengtsson and Jenny Nordberg","author_title":"","affiliation":"Sveriges Television","summary":"A Swedish financial crime agency created with extensive resources and specialized staff to combat organized crime instead spent its time prosecuting small business owners with bookkeeping problems, as revealed through an investigative journalist's analysis of 804 suspect profiles.","keywords":["fraud","investigation","prosecution"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Access"],"full_text":"\"A stealth plane shooting on doves\" - that's how the most expensive Swedish effort to fight financial crime in years turned out to be. Instead of conducting their own investigations to track down the organized criminals, they sat back and dedicated their time to prosecuting small business owners with bookkeeping trouble.\n\nFor decades, the Swedish justice system has been struggling to come to terms with organized financial crime: tax fraud, inside information crime, offshore decoy companies etc. Through several police and attorney office reorganizations, the white-collar criminals have been operating without much interference. With an estimated cost to society of more than 100-150 billion Swedish Kronor ($120 million) a year, the minister of justice decided to create an all new agency: \"Ekobrottsmyndigheten.\"\n\nWhen the reporter interviewed the responsible officials, they were stunned by our conclusions, and appropriately embarrassed.\n\nFour hundred policemen, 90 specially-trained prosecutors, and a number of tax specialists were installed in new offices in Sweden's three largest cities. With higher salaries than the average police officer and attorney, the most competent staff were recruited to the new \"super-department,\" which got an extensive government budget. After operating for a year and a half, our station's current affairs program \"Striptease,\" decided to examine the results of the costly initiative.\n\nGetting started\n\nThe investigative team got copies of all the complaints from the three agencies throughout Sweden. Only available as paper copies, they created large piles at the researcher's desk. An Access database was created in order to get a picture of the most widely prosecuted crimes and individuals. We wanted to know who they were, and if the Swedish authorities finally had managed to track down the most advanced financial criminals.\n\nThe essential facts of each defendant was typed into the database, including the name, age, occupation and personal id number. When several suspects were to be tried in the same case, they each got a record in the database. Also, the classification of the crime and a short description of events leading up to the prosecution. We made a note of whether the suspect had been arrested or not, which is a determinant on how severe the crime had been considered by the prosecutor.\n\nFinally, we entered the estimated amount of money withheld and how the crime had come to the police's knowledge - by a bankruptcy report or the tax authorities revisions. In a few cases, it was an independent investigation leading up to the complaint. Some also contained a number (1-3), grading the complexity of the crime by the agency themselves. We adapted their system, grading all the cases accordingly.\n\nThe results\n\nEight-hundred-four suspect profiles and 625 complaints later, we started to analyze the results. By asking questions to the database, we summoned up 18 months' work by the most advanced financial crime unit in Sweden.\n\nFirst, we got the average criminal; we were able to see that he (very soon, it was clear that he was a man) was a middle-aged Swedish citizen with a small business, having some trouble keeping his books in order. Moreover, he had managed his business badly and gone bankrupt.\n\nTax authorities discovered some forgotten receipts in a shoebox, and he was taken before the court and accused of tax fraud. Hardly the polished, pinstriped criminal in dark shades taking weekend trips to a tax paradise island to place money in secret bank accounts. Still a definite crime, these were cases that the newly examined attorney at any court could have managed to prosecute.\n\nAs it turned out, only six percent of the cases were the top-notch crimes, the bigger part of the administration were simple bookkeeping incidents. Taxi drivers and cleaning companies trying to escape some of the value tax were dominating our database. When the reporter interviewed the responsible officials, they were stunned by our conclusions, and appropriately embarrassed. The minister of justice, however, would not grant us an interview, hoping that the storm would blow over soon.\n\n\"Striptease\" is Sweden's most influential investigative television program. The title aspires on the fact that power and corruption should be stripped down to the bare truth.\n\nThe program has used CAR for a number of stories recently. One investigated the surprisingly large number of Swedes possessing weapon licenses, who are ex-convicts or suffer from severe mental illnesses. In that case, the weapons register was joined with court statistics and hospital records of people receiving treatment fo\n\nHelena Bengstsson can be reached by e-mail at helena.bengstsson@svt.se","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-recent-car-stories","slug":"2000-march-recent-car-stories","url":"https://dwillis.github.io/uplink/article/2000-march-recent-car-stories.html","year":2000,"month":"March","monthNum":3,"headline":"Recent CAR stories","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Two investigative journalism series examine institutional accountability: one analyzes high school performance using statistical regression and socioeconomic factors, while the other investigates deaths among vulnerable populations by analyzing death records and law enforcement reports.","keywords":["accountability","investigation","vulnerable"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":[],"full_text":"\"Grading Our High Schools\"\nThe Pittsburgh Tribune-Review Feb. 6, 2000\nhttp://triblive.com/frames/grades.html\n\nAccording to the series' author, this investigation \"features a regression analysis using average state assessment test scores and participation in the free and reduced-price lunch program.\"\n\n\"Broken Trust\"\nThe Charlotte Observer Jan. 23-27, 2000\nwww.charlotte.com/special/mentalhealth\nThe Observer identified deaths of the mentally ill and developmentally disabled by searching 370,000 N.C. death records from 1994 through mid-1999, along with hundreds of police and autopsy reports.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-finding-data","slug":"2000-march-finding-data","url":"https://dwillis.github.io/uplink/article/2000-march-finding-data.html","year":2000,"month":"March","monthNum":3,"headline":"Finding data","kicker":"INTERNATIONAL CAR","author_name":"Noemi Ramirez","author_title":"","affiliation":"IRE","summary":"Computer-assisted reporting (CAR) is underdeveloped outside the United States due to lack of electronic databases, but journalists can access valuable U.S. government and institutional resources for international research, including SEC filings, campaign finance records, demographic data, and legal information.","keywords":["databases","journalism","research"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Excel","Internet","World Wide Web","FTP","PDF"],"full_text":"One of the reasons to explain why CAR is underdeveloped in countries other than the United States is the absence of computerized electronic databases. At meetings with reporters from all over the world, I have been asked this question on several occasions: \"I'm green with envy, we don't have those resources in my home country what can we do?\"\n\nMy answer is twofold. You can reenact the work of medieval monks - or nuns by manually creating your own databases, or you can search for information about your country somewhere else outside the national borders. There are a number of databases and online resources kept by American public agencies, companies and institutions that can be helpful when doing international research.\n\nBusiness\n\nSecurities and Exchange Commission Data, Free Edgar: www.freEedgar.com/\n10K Wizard: http://10kwizard.com/\nThese web sites offer a free full-text search of SEC filings as well as other useful tools to background every publicly traded company in the United States, no matter whether the company is American or not. Both Free Edgar and 10K Wizard include information - such as the registration statement or the financial annual reports - that is easily turned into Excel spreadsheets for further analysis. One extremely useful feature is the electronic alert service that will send e-mail notifications when any of the companies in the Watchlist submits an electronic filing to the SEC. For an explanation on using Free Edgar and 10K Wizard, request tipsheet #1138 from the IRE Resource Center.\n\nCampaign finance\n\nCenter for Responsive Politics, www.opensecrets.org/2000elect/lookup/AllCands.htm\n\nThe Center for Responsive Politics defines itself as \"a non-partisan, non-profit research group based in Washington, D.C. that tracks money in politics, and its effect on elections and public policy.\" The donor lookup database retrieves individual and soft money contributions that are above $200.00. To find out how much money employers of foreign companies with offices in the United States are giving to the American presidential candidates, try to enter the name of a company in the Occupation/Employer field.\n\nDemographics\n\nThe International Data Base. Census Bureau's International Programs Center, www.census.gov/ipc/www/idbnew.html\n\nThe International Data Base (IDB) contains statistical tables of demographic and socioeconomic data for 227 countries and areas of the world. For some countries, the data goes back as far as 1950 and as far ahead as 2050. You can download a zip file that contains the files and instructions necessary for installing the actual International Data Base in your computer, or you can import specific tables into most spreadsheet programs.\n\nLaw\n\nPrivate International Law Database. Office of the Assistant Legal Adviser for Private International Law. Department of State, www.state.gov/www/global/legal_affairs/private_intl_law.html\n\nThis Web site provides information on current negotiations and projects covering the private international law of such areas as trade and commerce, finance and banking, trusts and estates, family and children matters, and international judicial assistance.\n\nArms\n\nWorld Military Expenditures and Arms Transfers 1997. U.S. Arms Control and Disarmament Agency, www.state.gov/www/global/arms/wmeat97/wmeat97.html\nThe ACDA collects data on international arm sales, arm treaties, and military expenditures.\n\nThe annual reports include PDF tables by region, organization, country, major supplier and recipient country. For a quick tip on how to import PDF tables, check Tom McGinty's article on the July/August 1999 Uplink issue.\n\nNoemi Ramirez can be reached by e-mail at noemi@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-international-data-sources","slug":"2000-march-international-data-sources","url":"https://dwillis.github.io/uplink/article/2000-march-international-data-sources.html","year":2000,"month":"March","monthNum":3,"headline":"International data sources","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A comprehensive list of international data sources including government databases for securities, census, immigration, and arms transfers, plus scientific research resources.","keywords":["databases","government","international"],"topics":["Data & Databases","Government & Politics","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"International data sources:\n\nSecurities and Exchange Commission Data.\nFree Edgar:\nwww.freEedgar.com/\n10K Wizard: http://10kwizard.com/\nCenter for Responsive Politics:\nwww.opensecrets.org/ 2000elect/lookup/ AllCands.htm\nThe International Data Base. Census Bureau's International Programs Center: www.census.gov ipc/www/idbnew.html\nPrivate International Law Database. Office of the Assistant Legal Adviser for Private International Law. Department of State: www.state.gov/www/globall legal_affairs/private_intl law.html\nWorld Military Expenditures and Arms Transfers 1997. U.S. Arms Control and Disarmament Agency: www.state.gov/www/globall arms/wmeat97/ wmeat97.html\n\nMore international data sources:\n\nINS Annual Statistical Reports www.ins.usdoj.gov/ graphics/aboutins/ statistics/index.htm The Immigration and Naturalization Service provides detailed statistical data about immigration trends in the United States. It includes useful information such as refugee-status applications by geographic area and country.\n\nFederation of American Scientists www.fas.org/ This Web site offers an impressive collection of reports on a variety of issues such as animal health diseases, cyberstrategy, biological and toxin weapons, intelligence and nuclear resources.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-more-international-data-sources","slug":"2000-march-more-international-data-sources","url":"https://dwillis.github.io/uplink/article/2000-march-more-international-data-sources.html","year":2000,"month":"March","monthNum":3,"headline":"More international data sources","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This resource guide highlights two key government and research websites for accessing immigration statistics and scientific policy reports, including INS data on refugee applications and FAS reports on security and health issues.","keywords":["immigration","statistics","research"],"topics":["Data & Databases"],"technologies":["World Wide Web"],"full_text":"More international data sources:\n\nINS Annual Statistical Reports www.ins.usdoj.gov/graphics/aboutins/statistics/index.htm The Immigration and Naturalization Service provides detailed statistical data about immigration trends in the United States. It includes useful information such as refugee-status applications by geographic area and country.\n\nFederation of American Scientists www.fas.org/ This Web site offers an impressive collection of reports on a variety of issues such as animal health diseases, cyberstrategy, biological and toxin weapons, intelligence and nuclear resources.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-going-beyond-oscar","slug":"2000-march-going-beyond-oscar","url":"https://dwillis.github.io/uplink/article/2000-march-going-beyond-oscar.html","year":2000,"month":"March","monthNum":3,"headline":"Going beyond OSCAR","kicker":"NURSING HOMES","author_name":"Dave Gulliver","author_title":"Database Editor","affiliation":"Dayton Daily News","summary":"An investigation into Ohio's nursing home system reveals systemic failures in care quality and regulatory enforcement, particularly affecting elderly residents without adequate financial resources, using inspection data and demographics analysis to expose widespread problems.","keywords":["nursing","elderly","inspection"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["OSCAR","FoxPro","SPSS"],"full_text":"Grace Blankenship was getting some of the best care money could buy, making friends, talking about her favorite players on the Cincinnati Reds. Then her money ran out and everything changed. She was moved from a facility's assisted living wing to a standard nursing home unit. There, a younger, demented male resident attacked her, threw her against a wall and broke three of her ribs. Two weeks later, late one night, she began vomiting and went into shock from severe dehydration. By dawn, she died, with her doctor and family not notified until the last hour of her life. Mrs. Blankenship's story exemplified two main flaws in Ohio's system of caring for its elderly: If you don't have the money, or if it runs out, your only choice is nursing homes that regularly violate standards, with virtually nothing to fear from the state. But you need data, not anecdotes, to prove system-wide problems exist, and even more data to explain why.\n\nStarting out\n\nIn early 1999, the Dayton Daily News decided to look at how Ohio was prepared to handle the booming elderly population. That eventually focused on the available choices for long-term care, and nursing homes are by far the biggest player.\n\nInspections, demographics and enforcement were our workhorses, but several other databases chipped in to the project.\n\nNursing home projects typically use federal OSCAR data, the findings of health and safety inspections, to search for problem nursing homes. For us, that was just the start.\n\nWe used the state's own inspection data, which is used to create OSCAR. We wanted to undercut any \"one-bad-year\" arguments, so we used all the existing data.\n\nInspection data lets you look at types of problems- from dirty silverware to resident abuse-and whether they're widespread or rare, dangerous or mostly harmless. By tallying the citations, you can develop a crude ranking.\n\nBut that alone can be deceptive. For example, a home with one of the highest citations totals was solely for Alzheimer's patients. Advocates argue that such homes should have more staff and better training to ensure better care.\n\nIn the end, you need to know more to make valid comparisons. Demographics data was the answer. We used another Health Department data set, the annual survey of long-term care facilities. That allowed us to link citations totals with other key fields. The most important was a nine-category breakdown of ownership type, such as for-profit partnership or church-related non-profit. We also used variables such as age, race and gender breakdowns, dedicated Alzheimer's units, nursing staff per resident, owner and operator, and so on.\n\nWe linked the tables in FoxPro, then imported the result into SPSS for the serious number-crunching. We filtered the data for \"apples-to-apples\" comparisons of similar types of facilities, and used medians and trimmed means where appropriate to account for skewed data. SPSS's crosstabs and nifty scatterplots and histograms made the job infinitely easier, but (as we learned the hard way on a redlining story years ago) you could get to the same results with database and spreadsheet software.\n\nFollow the process\n\nThe inspections data - the beginning and end for many projects - was just half the picture. The most surprising findings came from the state's enforcement database.\n\nWe found that even when state inspectors found problems, they gave homes little reason to clean up their acts.\n\nThe enforcement data was far more complicated to use than the inspections data. It was drawn from a system designed for quick lookups and case management, not aggregate analysis. That later became apparent when I asked the enforcement chief if the basic numbers were correct: 1,261 enforcement cases, 334 penalties imposed. \"I have no idea,\" he replied. The state couldn't readily compile its results, so it just didn't. (They later did their own analysis and confirmed the numbers.)\n\nBecause of the database design, the enforcement data had even more pitfalls than the inspections data. There were multiple records for each case, one for each type of penalty proposed, and most homes had multiple cases. Typically a few penalties stuck but most were dropped.\n\nThe key was to break the table down and rebuild it several ways. For example, to see how the state handled \"repeat offenders\":\n\nAdd a field to the original table to flag when a penalty stuck.\nGroup the facilities by ID and case number to identify how many cases the state launched against a particular home, and order them by date of case.\nCopy the result to a new table.\nAdd and fill an \"action number\" field. We used a FoxPro SQL program.\nThen join to the original table to add the flag field. Result: How many \"strikes\" a home received before the state would call it \"out.\"\n\nDealing with dirt\n\nLike the enforcement data, most of what we used wasn't meant to be used that way.\n\nThe Health Department assigns each nursing home an ID number, which is supposed to be the standard way of identifying a home. But the inspections data is tailored for inclusion in OSCAR data, so the quality assurance bureau uses the home's Medicare number as the unique-and-only-identifier. Meanwhile, another bureau maintains the demographics table. It uses the department ID field as the identifier. It also has the Medicare ID number, but frequently fills it with a dummy number. We eventually decided to capture as many homes as possible, and to play it safe: We recoded ID numbers by hand.\n\nBut simple dirt in the data wasn't the biggest problem. We had to understand how the system had changed over time. For example, in mid-1995, the feds and the state changed what counts as a serious health or safety violation, \"substandard care\" in health-speak.\n\nIf we didn't account for the differences, our analysis would have been garbage. We wrote another SQL program to recode the citations in different ways - to grade them by their respective rulebooks and to grade them by current and former standards.\n\nUse \"golden\" databases\n\nInspections, demographics and enforcement were our workhorses, but several other databases chipped in to the project.\n\nPerhaps the best was a little in-house Excel worksheet from the Attorney General's office that summed up abuse prosecutions.\n\nThe state's deaths database essentially identified seven cases of nursing home fatalities. Victims' names are redacted from incident reports, but provide dates, locations, and sometimes gender and age - enough to find the death certificate and the name.\n\nWe also drew on the state's complaints hotline database; inspections data for assisted living facilities and group homes; and demographic data for home health agencies. Late in the process, when we'd almost given up, the state provided Medicaid Cost Reports data, which we used to check other findings.\n\nIn short, getting the data was relatively simple - but understanding it well enough to run the numbers took months. By the end, we could talk Medicaid-speak with the best of the bureaucrats.\n\nSo what did we learn?\n\nSearch for data where you don't expect it.\n\nThe project started with a Health Department flak saying they didn't keep an inspections database, but could provide paper summaries (sound familiar, anyone?). We went straight to inspectors to see what really existed. We never expected the Attorney General to have abuse cases in electronic form, but there they were. We didn't expect to get complaints data, but after agreeing to redact names, there it was.\n\nBut know when to stop searching. We probably gathered more than we need. Several databases made only a cameo appearance in the final project. When you have enough to answer your questions, stop there and dig in.\n\nLearn everything about the data. CAR 101: Know the definitions of every field in the database, and every code in the fields. One relatively rare code in the inspections data meant a home's fire code violation actually wasn't a violation. Miss it, and your numbers for total citations would be off.\n\nBut learning the data isn't enough.\n\nFor the numbers to mean anything, we also had to learn federal law on inspections and enforcement, and how it evolved over time. We consumed hundreds of pages of manuals and studies to know what to ask, what data mattered and what had changed.\n\nKeep your data organized.\nWe've lost count of how many data tables we've built, but it's got to be around 1,000. But most were one-time knockoffs to test a particular question. The base tables and the key ones for analysis number about two dozen, all in one folder, and listed on paper and in Excel, with the name, location, and a description of how we built them. All the queries are saved by date and name so we can retrace any steps.\n\nAsk why.\nDon't just settle for a list of the worst homes. See what they have in common. Are they for-profits, cutting corners on quality to save a buck? Are they non-profits that lack trained staff and modern facilities? Do they have a problem population? Why are some problem homes around? And find a new question to ask.\n\nIn Chicago, Mike Berens unearthed the warehousing of the mentally ill in nursing homes. In Tampa, Doug Stanley looked at how the profit motive affects care. We looked at the big picture: What types of care are out there, who has access, and how well the state watches over those in the system.\n\nUse data to narrow your paper chase.\n\nOnce we had identified the problem homes, we requested the detailed, paper reports. That painted pictures on top of the numbers, unearthed more people and stories, and confirmed another tip: Homes routinely appeal inspections, getting some citations dropped from the report-often enough to drop below a penalty threshold.\n\nBut forget the data. It's about people. A bit of an exaggeration, but it will keep you focused on your readers. Even with all our findings, with the directors of Health and Aging saying they'll make changes based on our reporting, what people will remember is Grace Blankenship.\n\nDave Gulliver can be reached by e-mail at dgulliver@coxohio.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-other-stories-in-the-ire-resource-center-on-nursing-homes","slug":"2000-march-other-stories-in-the-ire-resource-center-on-nursing-homes","url":"https://dwillis.github.io/uplink/article/2000-march-other-stories-in-the-ire-resource-center-on-nursing-homes.html","year":2000,"month":"March","monthNum":3,"headline":"Other stories in the IRE Resource Center on nursing homes","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The IRE Resource Center maintains a collection of investigative journalism stories on nursing homes, including reports on the warehousing of mentally ill patients and care quality issues from various news organizations.","keywords":["nursing","investigation","healthcare"],"topics":["Health & Environment","Investigative Methods"],"technologies":[],"full_text":"Other stories in the IRE Resource Center on nursing homes:\n\n\"Warehousing the mentally ill,\" (story #15173), by Michael Berens of the Chicago Tribune, 1998.\n\n\"A crisis in care,\" (story #15500), by Geoff Dutton of The News-Journal in Daytona Beach, Fla., 1998.\n\nWDIV-TV (Detroit) investigation on nursing homes, 1996 (story#13696).\n\nTo order, call the Resource Center at (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-tipsheets-in-the-ire-resource-center","slug":"2000-march-tipsheets-in-the-ire-resource-center","url":"https://dwillis.github.io/uplink/article/2000-march-tipsheets-in-the-ire-resource-center.html","year":2000,"month":"March","monthNum":3,"headline":"Tipsheets in the IRE Resource Center","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE Resource Center offers tipsheets on investigating money flows in healthcare and conducting nursing home investigations, originally prepared for the 1999 IRE Conference by experienced journalists.","keywords":["healthcare","investigation","nursing"],"topics":["Investigative Methods"],"technologies":[],"full_text":"Tipsheets in the IRE Resource Center: \"Money and paper trails of health care,\" (#1011) by Michael Berens of the Chicago Tribune, for the 1999 IRE Conference.\n\n\"Investigating nursing homes,\" (#1009), by Alison Young of the Detroit Free Press for the 1999 IRE Conference.\n\nTo order, call (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-census-workshops","slug":"2000-march-census-workshops","url":"https://dwillis.github.io/uplink/article/2000-march-census-workshops.html","year":2000,"month":"March","monthNum":3,"headline":"Census workshops","kicker":"IRE AND NICAR","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are hosting a series of two-day workshops on working with Census data, with the first scheduled for Arizona in April, followed by sessions in Missouri and Maryland, covering Census preparation, data changes, and story development strategies.","keywords":["Census","workshops","journalism"],"topics":["Demographics & Census","Journalism & Newsroom","Data & Databases"],"technologies":[],"full_text":"IRE and NICAR will hold several two-day workshops on working with Census data, with the first being held in Arizona April 7-9. The Arizona workshop, which will be held at Arizona State University in Tempe, will have Steve Doig, of ASU, and Paul Overberg, of USA Today, as lead instructors. The second workshop will be held at the Missouri School of Journalism, in Columbia, August 4-6. The third workshop will be at the University of Maryland in December, but a specific date had not been set as of publication.\n\nThese Census workshops will be the first in a series and cover getting ready for the Census, what changes there will be from 1990 data, and ideas on the first stories to do and how to do them.\n\nThe initial fee for the workshops will be $75 for two days if you are an IRE member. Membership will be offered at a discounted $40 for a total of $115 if you are not an IRE member. (IRE membership fee will go up to $50 in March.)\n\nMore information is available on the IRE and NICAR Web site, www.ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-recent-car-stories-2","slug":"2000-march-recent-car-stories-2","url":"https://dwillis.github.io/uplink/article/2000-march-recent-car-stories-2.html","year":2000,"month":"March","monthNum":3,"headline":"Recent CAR stories","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This collection includes two investigative journalism series: one analyzing high school performance using regression analysis of state test scores and lunch program participation, and another examining deaths of mentally ill and developmentally disabled individuals in North Carolina through death records and police reports.","keywords":["investigation","schools","deaths"],"topics":["Investigative Methods","Data & Databases","Education"],"technologies":[],"full_text":"\"Grading Our High Schools\"\nThe Pittsburgh Tribune-Review Feb. 6, 2000\nhttp://triblive.com/frames/grades.html\n\nAccording to the series' author, this investigation \"features a regression analysis using average state assessment test scores and participation in the free and reduced-price lunch program.\"\n\n\"Broken Trust\"\nThe Charlotte Observer Jan. 23-27, 2000\nwww.charlotte.com/special/mentalhealth\nThe Observer identified deaths of the mentally ill and developmentally disabled by searching 370,000 N.C. death records from 1994 through mid-1999, along with hundreds of police and autopsy reports.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-the-worlds-not-flat","slug":"2000-march-the-worlds-not-flat","url":"https://dwillis.github.io/uplink/article/2000-march-the-worlds-not-flat.html","year":2000,"month":"March","monthNum":3,"headline":"The world's not flat","kicker":"MAPPING","author_name":"Andy Lehren","author_title":"","affiliation":"Dateline NBC","summary":"The article explains that while the Earth is round, determining its exact shape is complex, involving different geodetic measurements and datums that affect how maps represent locations on our planet.","keywords":["mapping","datum","ellipsoid"],"topics":["Technology","Investigative Methods"],"technologies":["GIS"],"full_text":"The world's not flat\n\nBy Andy Lehren Dateline NBC\n\nIt sounds obvious: the world is round. Aristotle knew that back around 350 B.C. Ptolemy refined it a few hundred years later. Despite fears from the flat earth believers, Columbus made a living off the idea.\n\nThe problem is knowing the exact shape of our sphere.\n\nYou don't have be sailing for a new world to face the question. All you have to do is get a few disparate maps and try to make them work in your GIS program.\n\nThere are two fundamental questions to ask when you get a map from anyone: what's the datum, and what's the projection. There are other details that may be helpful, but these are the most important. This column will try to explain datum, and how to convert it when you have to.\n\nSome more history: Isaac Newton wrote in the late 1600s that, well, the earth's a bit flattened on the top and bottom. The French, after measuring their country around the same time, countered that it's more egg shaped. Newton won the debate. The French ate better food. And we live with the question: just how flat on the top and bottom?\n\nEver since, geodesists have been measuring this. There are hundreds of calculations that have been done figuring out our little ellipsoid. Here are the three that matter:\n\nClarke 1866, developed by Alexander Ross Clarke. He was a busy guy, so the year is important.\nGeodetic Reference System 80 (GRS 80), a group effort by an international geodetic organization.\nWorld Geodetic System 84 (WGS 84), founded by the U.S. military and close to GRS 80, but not exactly the same.\n\nThese different measurements of our spinning ellipsoid are the underpinnings for almost all the maps we look at. You can tell from the history that this is still evolving, and you may well face other datums as you do more mapping.\n\nYou May Not Be Where Your Map Says You Are\n\nYou would think that something like latitude and longitude is absolute. It's not. First, different geodesists came out with different calculations on how elliptical is our planet. If you try to draw three circles by hand, you'll have an idea of the differences.\n\nThese ellipsoid measurements are the underpinnings for datum. You may be most familiar with datum by seeing latitudes and longitudes, though it's more than that. If geodesists draw up different spheres, the datum will be different. It gets murkier. To develop datum, geographers went out and mapped our country. They did not measure every latitude and longitude. They measured a lot, and extrapolated a lot more. They used the ellipsoid calculations as their reference guides for figuring datum. In other words, there's some guess work piled on top of some guess work. Early this century, geographers used the Clarke 1866 ellipsoid, and result was something called North American Datum 1927 (NAD 27), with the center in Meades Ranch, Kansas. It had problems. They measured again later that century. This time they assumed the Earth was shaped more like the GRS 80 ellipsoid, and came out with something called North American Datum 1983 (NAD 83). And now people are running around with GPS units measuring where they are, and assuming the Earth is shaped like the WGS 84 ellipsoid. That's how we've gotten the three major kinds of datum that are the foundations for the maps we use. Three different ellipsoids were used to derive three sets of datum. And that means three sets of latitudes and longitudes. You pull out a GPS, and it can tell you your latitude and longitude. Well, not exactly. It says where you're at based on WGS 84. That's the flavor preferred by most GPS devices. That latitude and longitude may be off by half a meter or more using GRS 80. And it may be up to 300 meters off under Clarke 1866. If you don't know the background of your map, you could use a GPS, look at a map, and realize you may not be exactly where you think you are.\n\nConverting Your Maps\n\nOK, sometimes your maps will be in NAD 27. And sometimes they are in NAD 83. And when you use your GPS, you're in WGS 84.\n\nIn a perfect world, you would try to work in the datum used to make your maps. But that doesn't always happen. You get a map from one agency in one datum. You want to match it against a map in a different datum. Now you need to run a conversion so they line up better.\n\nHere are a couple solutions. These examples are for converting datum only. (Just so you know, the ArcView utility mentioned here can handle projection conversions. There is also a script in older versions of ArcView to switch projections.)\n\n1. ArcView's Projection Utility\nThe new version of ArcView comes with something called the Projection Utility. You can run it outside the program. Just click on the icon on your desktop. Once you're in the utility, point and click your way to the shape file you want to convert. Hit the next button. That ends step one. For step two, first check the box that says \"Show Advanced Options.\" We're assuming the data is not projected, so keep the coordinate system at \"Geographic.\" Choose the datum your map came in, say North American 1927. Hit the next key. For step three, select the datum you want to convert to, like North American 1983. Hit next. Step four is where you navigate to where you want your converted files. Now you can hit the next key and then finish. The utility will create a reference file that keeps track of the datum for your new file.\n\nThe new ArcView program is based on code written by a group of oil and gas companies. In its manuals, ESRI, the maker of ArcView, does not talk about errors and estimations in running conversions. But the program is not perfect. The petroleum consortium whose work is behind the ArcView utility acknowledges that fact. Your end result may be off by a meter or more.\n\n2. Wessex' TransNAD\nIf you have an older version of ArcView, or use MapInfo, here's another solution that's free. Go to www.wessex.com. It's the home page for a company that sells street map data and other things.\n\nFrom the home page, navigate to utilities, and download TransNAD. It's a GUI application that lets you point and click your way quickly to a solution. It only lets you convert between NAD 27 and NAD 83. Unlike the new ArcView utility, this program will not make a new file, but alter your original file. So it's best to work off a copy.\n\nOnce you've installed the program, navigate to the file you want to convert. Choose which conversion you need to run, either from NAD 27 to NAD 83, or visa versa. Hit next to run the conversion. You're done.\n\nAndy Lehren can be reached by e-mail at alehren@nbc.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-nulls-and-averages","slug":"2000-march-nulls-and-averages","url":"https://dwillis.github.io/uplink/article/2000-march-nulls-and-averages.html","year":2000,"month":"March","monthNum":3,"headline":"Nulls and averages","kicker":"STATS AND SUCH","author_name":"Neill A. Borowski","author_title":"","affiliation":"The Philadelphia Inquirer","summary":"Data analysts must carefully handle missing data (nulls) in databases, as calculating summary statistics like averages on incomplete datasets can be misleading and misrepresent the true values. The article warns that missing data can significantly skew results and emphasizes the importance of explicitly checking for and accounting for nulls rather than ignoring them.","keywords":["nulls","missing","averages"],"topics":["Data & Databases","Investigative Methods"],"technologies":["SPSS","Access"],"full_text":"If you've done data analysis for a while, you've probably fallen victim to the nulls. You open a new database and discover several fields that could make your analysis richer and deeper. And then you test for nulls, the signs of missing data that plague many databases, particularly databases used by the government for administrative purposes. Those great fields are missing thousands of records. Can you do anything to salvage them and come up with summary statistics, such as averages and medians? Or, do you have to throw out the field and, with it, the chance for some insightful analysis? Trying to find an average for a variable when one-third of the records are missing can be misleading and dishonest. The tendency might be to report the summary statistics, with a footnote disclosing that these are the averages for cases in which the data were reported.\n\nTrying to find an average for a variable when one-third of the records are missing can be misleading and dishonest.\n\nHowever, any suggestion that the average or median or other summary statistic represents all of the data is misleading. We are working with what should be the universe of the data - all of it. But some is missing. We could try some statistical tests that would draw a random sample of the data we have and shed some light on the probability that the true mean would fall within a range. Yet, the random sample of the incomplete data wouldn't necessarily represent a random sample of all of the data.\n\nPossible effects\n\nConsider a database with 500,000 records. The field of interest is missing 20 percent, or 100,000, of the records. You average the 400,000 records you have and come up with a mean of 5.0. Is this \"good enough\" to use?\n\nThe missing 100,000 records suddenly are found (they slipped under a dusty desk somewhere). The average for the missing 100,000 is 2.0 and, when combined with the 400,000, yields a mean of 4.4. If the average of the missing 100,000 were closer to the average of the 500,000 records - say 4.0 - then the overall average with the missing records replaced would be 4.8 - much closer to 5.0. Does a run of missing data occur often? It has with us. We recently worked on a one-million-plus record database that is a collection of data submitted by hundreds of different entities. The database, which spans two years, has several fields with thousands of records of missing data. Some of the entities didn't file data with the collecting agency for the fields that we found most interesting. Are there sweeping averages or summary statistics (race, age, etc.) that might be calculated from the fields with missing data? Yes. But I'm not comfortable that they would necessarily be representative of the \"whole.\"\n\nChecking for nulls\n\nThis is why it's so important to check for nulls, or missing data. One might run the averages or medians and never discover that some records are missing. SPSS will tell you how many records were included in the calculation and how many were missing. But basic programs such as MS Access will simply calculate the average with no alarm for nulls. The \"Is Not Null\" condition should be used in the average calculation in Access, but this still will not report to you how many nulls exist. Before you calculate the average, count for nulls. And, heaven forbid, the nulls somehow were transformed into \"0's\" in the database. Most programs then will calculate them into the average or the median and - depending on how many there are - come up with a useless number. What can you do if you come up with a lot of missing data? One way is to see if a subset of the database has fewer records missing (or none at all). If the database is for two years, test for nulls in the field in question for each year separately. In the example above, the nulls might have come from a reporting problem with one or two entities in one year. The other year might have 100 percent of the data. Now you can calculate the summary statistics and report them with confidence for the one year you selected.\n\nAnother possible way around the missing data would be to group the data by the entities (towns, police departments, etc.). Are the null fields present for only one or two towns, for example? If the absence of information on this town isn't troubling, calculate the summary statistics for the field and report in the footnote that Smithville isn't included in the data. Of course you can select other filtering methods, depending on the database-type of crime, type of property, etc. The goal is to be able to spot the nulls early on in your analysis. Before you \"give in\" and calculate summary statistics for interesting fields, test the field for nulls. Also review the field for nonsensical data, such as \"O's\" when a zero is impossible.\n\nNeill Borowski may be reached by e-mail at nborowski@phillynews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-some-recent-car-stories","slug":"2000-march-some-recent-car-stories","url":"https://dwillis.github.io/uplink/article/2000-march-some-recent-car-stories.html","year":2000,"month":"March","monthNum":3,"headline":"Some recent CAR stories","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Computer-assisted investigations reveal operational inefficiencies in law enforcement and widespread housing code violations by government entities.","keywords":["staffing","violations","investigation"],"topics":["Investigative Methods","Public Safety"],"technologies":[],"full_text":"Some recent CAR stories: (Stories might also be available in the IRE Resource Center. Check by searching the database at www.ire.org/resourcecenter)\n\n\"Understaffed or Poorly Managed?\" St. Petersburg Times Jan. 2000 www.sptimes.com/Archivel012300/Pasco.shtml The Times' analysis of sheriff's staffing and response times showed the office had deputies available to answer 80 percent of its priority one calls. The average delay at the dispatch center was nearly four minutes.\n\n\"Discarded and Decayed\" The Journal Gazette Dec. 19, 1999-Jan 2, 2000 www.jg.net/jg/landlords/ A six-month investigation of Fort Wayne housing that included computer-assisted research of health, housing code and property found that the county was one of the biggest violators of housing ordinances.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-dangerous-devices","slug":"2000-march-dangerous-devices","url":"https://dwillis.github.io/uplink/article/2000-march-dangerous-devices.html","year":2000,"month":"March","monthNum":3,"headline":"Dangerous devices","kicker":"NICAR DATA LIBRARY","author_name":"Jessica Larson","author_title":"","affiliation":"NICAR","summary":"The MAUDE database tracks medical device failures and patient injuries, replacing the older MDR system, and offers journalists opportunities to investigate defective device manufacturers and safety issues. NICAR has updated the dataset with information from 1996-1999 and provides detailed fields including device problems, patient outcomes, and manufacturer information.","keywords":["devices","safety","reporting"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["MAUDE","MDR"],"full_text":"The Manufacturer and User Facility Device Experience Database (MAUDE), which contains information about medical devices that have caused problems for patients, has a number of interesting facets - particularly for reporters on the health beat.\n\nMAUDE replaced the older MDR (Medical Device Reporting) dataset, which did not require user facilities to report their information about defective medical devices. NICAR recently updated this dataset and now has data from August 1996 through March 1999, which includes voluntary reports, user facility reports, distributor and manufacturer reports.\n\nSome of the fields included in the database:\n\nThe problem with the device.\nThe outcome of the event (patient injury, death, etc.).\nThe brand name and type of device, its age, expiration date and shelf life.\nName and addresses for the device manufacturer and distributor.\nThe location the incident occurred (patient's home, health clinic, etc.).\n\nIn 1992, U.S. News and World Report did a story on dangerous implants (breast, penile, testicular) and other medical devices of concern to the FDA. The MAUDE dataset can provide information for similar stories - although no identifying patient information is included, the frequencies of device failure and resulting injury offer a good starting point. Which manufacturers produce the highest number of defective devices could prove to be another interesting topic. Old or expired devices, particularly when compared with the dates that authorities became aware of the problem, may be worth looking into, as well. Because the format has changed, NICAR is providing the 1996-1998 data and the older information separately. Costs for the 1996-1998 MAUDE is $100 for under 50,000 circulation (50-200 market), $125 for 50-100,000 circulation (25-50 market) and $150 for above 100,000. The older MDR data, 1984-1995, is $75, $100 and $125 respectively. To order this dataset or others available from NICAR, call (573) 884-7711. Jessica Larson can be reached by e-mail at jessica@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-nicar-databases-as-of-feb-1-2000","slug":"2000-march-nicar-databases-as-of-feb-1-2000","url":"https://dwillis.github.io/uplink/article/2000-march-nicar-databases-as-of-feb-1-2000.html","year":2000,"month":"March","monthNum":3,"headline":"NICAR databases (as of Feb. 1, 2000)","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR maintains a comprehensive collection of public databases covering federal enforcement records, accident data, and safety reports across aviation, transportation, and maritime industries from the 1960s through the late 1990s.","keywords":["databases","safety","aviation"],"topics":["Data & Databases","Investigative Methods","Public Safety"],"technologies":[],"full_text":"NICAR databases: (as of Feb. 1, 2000)\n\nNEA grants, 1987-1997\nINS Legal Residency, 1980-1996\nFAA Enforcements, 1967-1999\nFAA Service Difficulty Reports, 1974-March 1999\nAirmen Directory and Aircraft Registry 1998\nFAA Accidents and Incidents, 1973-1999\nNASA Air Safety Reporting system, 1988-1998\nNTSB Investigation of Aircraft accidents, 1985-1998\nDOT Fatal Accidents, 1975-1998\nNHTSA Vehicle Recalls and complaints, 1966-1997\nHazardous Materials, 1971-1998\nRailroad Accidents, 1991-1997\nBoating Accidents, 1969-1998\nBoat Registration, through July 1999\n\nMore information is available at: www.nicar.org/data or by calling (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-nicar-databases-as-of-feb-1-2000-2","slug":"2000-march-nicar-databases-as-of-feb-1-2000-2","url":"https://dwillis.github.io/uplink/article/2000-march-nicar-databases-as-of-feb-1-2000-2.html","year":2000,"month":"March","monthNum":3,"headline":"NICAR databases (as of Feb. 1, 2000)","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR provides access to numerous government databases covering aviation safety, vehicle recalls, transportation accidents, and other regulatory records spanning multiple decades from the 1960s through 1999.","keywords":["databases","aviation","safety"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"NICAR databases: (as of Feb. 1, 2000)\n\nNEA grants, 1987-1997\nINS Legal Residency, 1980-1996\nFAA Enforcements, 1967-1999\nFAA Service Difficulty Reports, 1974-March 1999\nAirmen Directory and Aircraft Registry 1998\nFAA Accidents and Incidents, 1973-1999\nNASA Air Safety Reporting system, 1988-1998\nNTSB Investigation of Aircraft accidents, 1985-1998\nDOT Fatal Accidents, 1975-1998\nNHTSA Vehicle Recalls and complaints, 1966-1997\nHazardous Materials, 1971-1998\nRailroad Accidents, 1991-1997\nBoating Accidents, 1969-1998\nBoat Registration, through July 1999\n\nMore information is available at: www.nicar.org/data or by calling (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-march-bits-bytes-and-barks","slug":"2000-march-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2000-march-bits-bytes-and-barks.html","year":2000,"month":"March","monthNum":3,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The Supreme Court upheld restrictions on drivers' license data access, creating varying challenges for journalists across states, while NICAR announces upcoming bootcamps and updates to its data library with improved documentation for investigative reporters.","keywords":["journalists","bootcamps","data"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"In January, the U.S. Supreme Court upheld the constitutionality of a 1994 law that restricts access to drivers' license data kept by the states. For journalists, this will mean continued difficulty in obtaining the records, but this varies from state to state. Some have established an \"opt-out\" policy that allows the release of the records to the public, except for those individuals who voluntarily choose to have their information withheld. States that don't have such a policy can't release any of the records. The Society of Professional Journalists posted a list on their Web site three years ago, listing each state's policy on the issue. However, this is out-of-date in some cases. The Web site is located at: www.spj.org/foia/drivers/drivside1.htm\n\nUpcoming bootcamps\n\nNow is the time to sign up for bootcamps that will be held this year. Sessions will be held May 7-12, July 16-21 and August 6-11. More information, including registration forms, costs, a typical schedule and travel and lodging options, is available at the NICAR Web site, www.nicar.org. The boot camps are held at the Missouri School of Journalism in Columbia, Mo., with hands-on training from NICAR staff members. There are minority fellowships available for bootcamps. Check out NICAR's Web site for more information. Applications can be obtained from John Green, membership coordinator, jgreen@ire.org, (573) 882-2772.\n\nNICAR data update\n\nSeveral datasets have recently been updated at the NICAR data library, including significant improvements to the documentation that is included with each dataset.\n\nDocumentation now includes references to print and broadcast stories using that data and, in some cases, tipsheets from IRE and NICAR conferences about that topic or beat. All items referenced in the documentation are available in the IRE Resource Center.\n\nOther improvements have been made to make it easier for journalists to work with the data, including better record layouts and codesheets.\n\nHere are the most recent datasets updated:\nHome Mortgage Disclosure Act (HMDA), 1992-1998.\nFederal Aviation Administration Enforcements, 1962 through Sept. 1999.\nFederal Aviation Administration Accidents and Incidents, 1973 through Dec. 1999.\nNTSB Investigations of Aircraft Accidents, 1985-1998.\nNational Inventory of Dams, 1993-1998. There are four states that didn't submit new data: Delaware, Alabama, Iowa and Rhode Island.\nRecreational Boat Accidents, 1969-1998.\nSeveral other dataset updates will be coming soon. Stay tuned to NICAR-L for announcements.\n\nMore information about these datasets and others - including the costs - are available online at www.nicar.org/data or by calling (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-tracing-guns","slug":"2000-april-tracing-guns","url":"https://dwillis.github.io/uplink/article/2000-april-tracing-guns.html","year":2000,"month":"April","monthNum":4,"headline":"Tracing guns","kicker":"CRIME","author_name":"David Olinger","author_title":"Reporter","affiliation":"The Denver Post","summary":"The Denver Post conducted an investigation following the Columbine massacre, focusing on how police guns became crime guns and detailing the 'Gun Pipeline' of firearms linked to various crimes.","keywords":["firearms","Columbine","crime"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"We responded to the Columbine High massacre last April in part by putting a spotlight on the firearms trade, particularly the accessibility of guns to kids and convicted felons. Two results of that effort appeared in The Denver Post in September and December, each as a three-day series. The first described how used police guns had become a primary source of high-capacity magazines, banned from manufacture in 1994 but legal to possess today, and how thousands of old police guns were getting traced as crime guns. The second, called \"The Gun Pipeline,\" detailed how homicides, assaults, the shooting of a federal agent and the pistol taken from a 13-year-old cocaine dealer could all be traced to multiple handgun sales. Both sets of stories relied on databases from the U.S. Bureau of Alcohol, Tobacco and Firearms as primary sources of information. Paying for the databases was the easy part. We got two for $75 - one consisting of guns that ATF was asked to trace from 1994 through 1998, the other a compendium of multiple handgun sales reported to ATF from 1995 through 1997. The hard part was using them to find criminal cases that could be explored as potential news stories. Ultimately we needed help from friendly ATF sources, from police departments small enough to remember a murder or a suicide on a particular date, and from Joseph Vince, a former director of ATF's national tracing center. ATF has since agreed to provide more information in its gun-tracing database, including serial numbers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-april-the-escape-problem","slug":"2000-april-the-escape-problem","url":"https://dwillis.github.io/uplink/article/2000-april-the-escape-problem.html","year":2000,"month":"April","monthNum":4,"headline":"The escape problem","kicker":"CRIMINAL JUSTICE","author_name":"Jack Dolan","author_title":"","affiliation":"Hartford Courant","summary":"An investigation into Connecticut's correctional system revealed systemic issues with prisoners escaping from halfway houses, linking this to failures in monitoring and arrest follow-ups.","keywords":["prisons","escapes","system"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"A throwaway paragraph caught our eye in one of the dozens of stories about the killing of eight-year-old murder witness \"BJ\" Brown, and the arrest of suspect Adrian Peeler. Buried amid the outcries for a new, centralized state-level witness protection program lurked an interesting, and wholly ignored, piece of information: at the time of the killing Peeler was officially listed as escaped from a Department of Correction halfway house. A three-month investigation that started with the DOC inmate database showed that the department routinely allowed prisoners to walk away from halfway houses and home release supervision long before their sentences were finished. By the time we wrote the story, nearly 15 percent of Connecticut's entire inmate population was listed as escaped. Another field in the database, \"deadtime,\" contained the number of days not credited towards a prisoner's sentence because he had taken an unscheduled and unannounced leave of absence from the Department of Correction. Not only did the \"deadtime\" field help us show that almost nobody is ever arrested for escape alone, but it also showed that hundreds of inmates since 1994 had escaped more than once.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-april-advice-from-the-front","slug":"2000-april-advice-from-the-front","url":"https://dwillis.github.io/uplink/article/2000-april-advice-from-the-front.html","year":2000,"month":"April","monthNum":4,"headline":"Advice from the front","kicker":"GUNS","author_name":"Sarah Cohen","author_title":"Database reporter","affiliation":"The Washington Post","summary":"Sarah Cohen from The Washington Post explains how journalists can access federal firearms databases, including the ATF's Firearms Tracing System and multiple sales gun database, which were previously difficult to obtain and can be used to investigate the path of crime guns.","keywords":["guns","database","ATF"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Firearms Tracing System"],"full_text":"Advice from the front\n\nBy Sarah Cohen\nThe Washington Post\n\nJust about this time last year, The Washington Post - like many other news organizations - began requesting data on crime guns from the federal Alcohol, Tobacco and Firearms agency of the Treasury Department.\n\nAfter an eight-month battle with the agency, we published a story linking eight former police weapons in the District of Columbia to homicides. The story, \"Recycled D.C. Police Guns Tied to Crimes,\" described more than 100 crimes tied to weapons once owned by the D.C. police. During this time, the city was paying residents to turn in their guns and was enforcing the strictest gun laws in the nation - a virtual ban on all handguns in the city.\n\nFor years, the ATF had logged guns recovered by local police agencies. It recorded information about the gun, the person caught with the gun and the suspected crime. The database is mainly used to help local police trace the history of guns they recover. ATF claims it is also used to identify rogue dealers who are suspected of breaking federal laws in selling weapons.\n\nThe story idea was simple - how does a good gun turn bad? Where does it go down the road to crime? The answer, we found, was that it sometimes goes bad when it is traded back to manufacturers from police departments, a lucrative deal for cash-strapped police and for the manufacturers.\n\nThe good news is that any news organization can now get the database we - and others - fought so hard to open. There's a strong argument that ATF is still withholding too much information, such as the identity of the dealers who sold the guns and the name of the police agency that requested information about it. But the web of information available can - especially in areas outside the D.C. area - be of use.\n\nThe two primary databases that ATF will provide without a fight are what it considers the public portions of the \"Firearms Tracing System,\" or FTS, and the multiple sales gun database. Since our story ran, other news organizations have simply requested the same database provided to The Post and received it without question for $75.\n\nHere's what you'll get and how you can use it:\n\nMultiple Sales\n\nI'm listing the multiple sales database first because it will often be of more use for breaking stories in certain parts of the country. Under federal law, dealers must record sales of more than one gun in a week to any individual. Those records are turned over to ATF, which lost a lawsuit to keep them secret. Under that ruling, the judge said that dealers have no right to privacy under the law, but left the door open to ATF to appeal it on other grounds. The agency never did and the ruling stands.\n\nThis database was of little use in our area because all of our local states have had laws prohibiting multiple sales for years. But, as The Denver Post's David Olinger points out in this issue, it can be of use to reporters in Western and Southern states with fewer rules.\n\nMultiple sales show the serial number, make and model of the gun, the name of the dealer, its ATF license number and the date of the report to ATF. (The dealer has a month to report it to the federal government.) Less than 2 percent of all guns sold in the U.S. are sold under these transactions, but law enforcement officers and crime experts say they are the source of a disproportionate number of gun crimes. It's worth checking this database whenever you have a serial number for a gun used in a crime.\n\nIn one part of our project, we tried to track guns that had been recovered by police agencies back to multiple sales, essentially tracing guns that were used in crimes then sold by the police. But the dates in the multiple sales database were often garbled. In fact, most large police agencies have stopped reselling recovered weapons, making anything we did find less newsworthy anyway.\n\nTrace data\n\nThe ATF trace database lists the serial numbers of most guns traced by local law enforcement agencies, their make and model, the date of the trace and the date of the recovery, information about where the gun was recovered and the crime it was suspected to be involved in.\n\nOne of the most contentious issues with ATF was the \"completion code.\" The database lists about 100 different codes, for example, \"S4,\" to indicate whether the gun was successfully traced. ATF argued that disclosing these codes would be disclosing a national secret. They eventually did identify the codes.\n\nWe knew that \"S5\" meant that the gun was traced to a law enforcement agency, not a private dealer or individual. In fact, we were encouraged even before Columbine that we had a story, since D.C. was the source of many S5's. It turned out to be wildly inaccurate and is a lesson in what flaws the trace database has.\n\nAnother point of contention is the list of serial numbers redacted. The ATF removed any serial numbers that are included in the multiple sales database for four years - the amount of time that they've been keeping multiple sales.\n\nAnd the ATF made it clear that it would fight as much as it could to keep dealers' identities secret, despite the fact that these dealers' names were already in the public eye thanks to a study requested by an anti-gun congressman.\n\nData Flaws\n\nThere are two big flaws in the database, both related to how police agencies trace guns.\n\nIn most police agencies that do 100 percent tracing - that is, they trace every gun they recover - the trace request is part of the protocol in the property room. That means there are two big problems with the trace. First, the \"crime\" listed by the police may or may not end up being the crime committed with the gun. In fact, ATF defines ANY gun recovered by police as a crime gun. Its argument is that leaving a gun out on the street is illegal anywhere. So even found guns are associated with crimes.\n\nFor our story, this meant tracking down the ballistics reports and the victims to make sure that the guns we say were used to commit homicides were actually the guns listed in the trace data. (Everywhere else, we described the guns as \"connected with\" or \"linked to\" crimes.)\n\nFor our story, the other problem was how manufacturers trace the guns. In fact, almost all of the \"S5\" guns in D.C. were, in fact, service weapons either shot at suspects or discharged accidentally. But guns that had been traded by the D.C. police didn't show up as government weapons - the manufacturers gave ATF the most recent buyer, not the original buyer. This is understandable, but meant that the figures in the ATF database for some areas will be wildly erroneous. (This depends on how the guns were traded back to manufacturers. If they went through dealers, the figures for your area will probably be reasonably accurate.)\n\nOther databases:\n\nLike the Denver reporters, we found a state database of as much or more use than the ATF's. In Virginia, every gun recovered by police in some counties had been typed into a mirror system. This included the names and dates of birth of the people the firearm was taken from, along with a description of what happened. Unlike the federal data, it also listed the name of the police agency that recovered the weapon, making the reporting in that state much easier.\n\nSarah Cohen can be reached by e-mail at cohensh@washpost.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-other-stories","slug":"2000-april-other-stories","url":"https://dwillis.github.io/uplink/article/2000-april-other-stories.html","year":2000,"month":"April","monthNum":4,"headline":"Other stories","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"These journalism investigations examine how illegal guns circulate through various channels including street distribution, law enforcement resales, and unlicensed online dealers.","keywords":["guns","illegal","dealers"],"topics":["Crime & Justice","Investigative Methods"],"technologies":["Internet"],"full_text":"Other stories:\n\n\"The life of a gun,\" by The Seattle Times (#15060) focused on just one gun to explore how and why illegal guns find their way onto the streets.\"\n\n\"Cops Selling Guns,\" by ABC News 20/20 (#16289) reported on the little-known practice of law enforcement agencies reselling guns they seize from criminals.\n\n\"Gun Show Not Only Place Felons Can Shop,\" by The Tribune (Tampa, Fla.), found guns are being bought and sold on the Internet by unlicensed dealers. (#15941)\n\nCall (573) 882-3364 to order.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-hunting-felons","slug":"2000-april-hunting-felons","url":"https://dwillis.github.io/uplink/article/2000-april-hunting-felons.html","year":2000,"month":"April","monthNum":4,"headline":"Hunting felons","kicker":"GUNS","author_name":"Janet Roberts","author_title":"","affiliation":"St. Paul Pioneer Press","summary":"The St. Paul Pioneer Press investigated convicted felons who were illegally hunting in Minnesota and found 236 violent felons who had received hunting permits despite being prohibited from possessing firearms, raising questions about state enforcement.","keywords":["felons","hunting","weapons"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["Access"],"full_text":"The story had been done before, but it was ripe for another visit: Convicted felons, who aren't supposed to carry firearms, were hunting in the Minnesota woods with guns.\n\nThe St. Paul Pioneer Press had to look no further than Donald Blom, a suspect in Minnesota's most intensely covered crime in 1999. Charged in the May 1999 abduction and killing of a 19-year-old convenience store clerk, Blom had a pistol, two shotguns and a rifle hidden in his cabin.\n\nBecause of those guns, Blom - convicted sex offender - was found guilty last summer of being a felon in possession of firearms. Among the evidence presented at his federal court trial: Blom's Minnesota hunting license and the registration he filed after bagging a deer.\n\nWe figured Blom wasn't the only felon roaming the woods in flame-orange clothing. Two databases helped us confirm that: one listing people who received special hunting permits from the state Department of Natural Resources, the other a Bureau of Criminal Apprehension database listing convicted felons. We matched names and birth dates to find people listed in both files.\n\nNarrowing list\n\nFrom there, we narrowed the list. State law says people convicted of certain violent crimes are forbidden from possessing a firearm within 10 years of their release from supervision. Because we were unable to determine release dates from the conviction data, we cast a narrower net, winnowing our list to felons convicted of violent crimes within the past 10 years.\n\nWe had to build our own lookup table of violent crimes. Only certain convictions result in the loss of gun privileges. We built an Access table of those crimes, then matched them against our hunters' convictions.\n\nWithin an hour, we found 236 felons who were not supposed to possess firearms but had received hunting permits from the DNR. Some had been convicted of multiple offenses. Our cast of characters included rapists, a murderer and others convicted of violent crimes such as aggravated assault, stalking, armed robbery and possessing a machine gun.\n\nThe actual number of violent felons with hunting permits is surely much higher. Because the state keeps computer records only of special hunting permits - those for hunting bear, turkey, moose and female deer - the newspaper was unable to run criminal checks on the hundreds of thousands of other hunters whose licenses are not computerized.\n\nAlso, our analysis was conservative. We matched first, middle and last names. Felons who didn't list their middle names on their hunting permit applications didn't make our list. And because we narrowed the list by the felon's conviction date rather than release date, we likely missed many other hunters who are not yet eligible to carry a gun.\n\nState response\n\nReporter Amy Mayron's story pointed out that if the newspaper, in an hour's time, could come up with more than 200 likely violators of state and federal weapons laws, the state could do the same. The state's response? They don't have the time or manpower to do the checks.\n\nTheir response had been the same two years earlier, when Twin Cities television station WCCO used the same databases to report that convicted felons were getting hunting licenses and taking game with firearms. The station's report featured a state lawmaker who was outraged by the findings and vowed to pass legislation to force state officials to perform the checks.\n\nOur reporter found that lawmaker equally outraged when she told him that two years later, nothing had changed. A couple of days after our story ran, it was easy to see why. The chairman of the Senate Environment and Natural Resources Committee said he would block any bill that would use hunting permits to check for convicted felons with guns. Lawmakers in Minnesota, where hunting is a rite of passage, are not about to enforce gun laws by snooping on hunters.\n\nA couple of lessons from this story: Don't be dissuaded from delving into a database just because it's incomplete. Even though the licenses of hundreds of thousands of Minnesota hunters never get entered into a computer, those for more than 200,000 others do. Readers are smart enough to understand that if we could delve into an incomplete database and find 200 felons registered to hunt, there's likely a whole lot more hunting felons out there.\n\nAnd don't be afraid to revisit a story that someone else has already done. Sometimes, these stories are just as good the second time around.\n\nJanet Roberts may be reached by e-mail at jroberts@pioneerpress.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-young-hunters","slug":"2000-april-young-hunters","url":"https://dwillis.github.io/uplink/article/2000-april-young-hunters.html","year":2000,"month":"April","monthNum":4,"headline":"Young hunters","kicker":"HUNTING ACCIDENTS","author_name":"James E. Wilkerson and David Washburn","author_title":"","affiliation":"McClatchy (The Morning Call/San Diego Union-Tribune)","summary":"Pennsylvania journalists analyzed hunting accident data from the state Game Commission to debunk the myth that younger hunters are safer, discovering that hunters 17 and under are actually more likely to be involved in accidents despite recent safety instruction.","keywords":["hunting","accidents","data"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":[],"full_text":"In terms of popularity, hunting in Pennsylvania ranks just behind a cold Rolling Rock and just ahead of NASCAR, so getting our hands on hunting accident statistics seemed like a natural.\n\nThe Pennsylvania Game Commission had a database of hunting accidents going back to 1983. Getting the data from the commission was quite easy and it was relatively clean. However, it was lacking some basic information, meaning a couple of our story ideas were killed right off the bat. For example, it didn't contain information on alcohol involvement in accidents. Nor did it contain information about where on the body a person was shot or any information that identified the individuals involved.\n\nWe saw a chance to clear up a sometimes-fatal misconception, but to make it work we needed a human story.\n\nBut the table did contain a wealth of other information concerning every reported shooting accident in the state going back to 1984. We had information about what they were doing at the time of the shooting, the cause, the topography of the area, whether they were wearing required blaze orange and their ages, among many other items.\n\nBackground\n\nHunting accident analysis has been well covered by other papers, so James ran some queries based on information we'd seen featured in stories archived by IRE and NICAR. He started with the types of guns involved, peak periods, what types of animals the hunters were going after, etc. Because neither of us are hunters, Dave spent some time attending the hunter education classes required for licensing.\n\nAfter a couple of weeks we compared notes, and one item stood out: Although Dave's contacts insisted that younger hunters (17 and under) were the safest because they'd more recently received instruction, the data clearly showed that younger hunters were much more likely to be involved in accidents, on both ends of the gun.\n\nAfter consulting with our outdoors editor and confirming that top state officials, as well as instructors, believed the myth about young hunters, we had the premise of our story. It appeared that even the state's top game commission officials had bought into the myth without checking their own records.\n\nWe saw a chance to clear up a sometimes-fatal misconception, but to make it work we needed a human story.\n\nBeyond records\n\nWe matched dates and locations of the incidents to our paper's computerized library and homed in on some half-dozen incidents involving young hunters. The first person we contacted, a widow in Columbia County, turned out to be the interview we needed to make the story work. Her husband had been accidentally shot and killed by their 12-year-old son during their first hunting trip a few years before.\n\nThat boy's experiences, just weeks after he'd taken the state-mandated hunting safety course, provided a springboard for us to examine hunting-safety instruction in Pennsylvania and compare it to other states where hunting is just as popular. The story showed that our state was far behind many others when it came to exposing students to gun-safety programs.\n\nJames Wilkerson can be reached by e-mail at james.wilkerson@mcall.com. David Washburn, who is now with the San Diego Union-Tribune, can be reached by e-mail at david.washburn@uniontrib.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-gun-issue-sources","slug":"2000-april-gun-issue-sources","url":"https://dwillis.github.io/uplink/article/2000-april-gun-issue-sources.html","year":2000,"month":"April","monthNum":4,"headline":"Gun issue sources","kicker":"","author_name":"","author_title":"","affiliation":"Media General News","summary":"A tipsheet providing contact information and resources for journalists covering gun policy issues, listing major organizations representing both pro-gun and gun-control perspectives.","keywords":["guns","sources","contacts"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Gun issue sources:\n(An excerpt from a tipsheet by Mark Johnson, Media General News, tipsheet #1140)\nNational Rifle Association: www.nra.org or Bill Powers, director of public affairs, (703) 267-3820\nHandgun Control Inc./Center to Prevent Handgun Violence: www.handguncontrol.org or Naomi Paiss, director of communications, (202) 898-0792\nViolence Policy Center: www.vpc.org or Bill McGeveran, communications director, (202) 822-8200\nSecond Amendment Foundation: www.saf.org or Dave LaCourse, public affairs, (425) 454-7012","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-more-gun-issue-sources","slug":"2000-april-more-gun-issue-sources","url":"https://dwillis.github.io/uplink/article/2000-april-more-gun-issue-sources.html","year":2000,"month":"April","monthNum":4,"headline":"More gun issue sources","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A compiled list of key sources and contacts for reporting on gun-related issues, including government agencies, industry organizations, and advocacy groups.","keywords":["guns","sources","contacts"],"topics":["Public Safety"],"technologies":[],"full_text":"More gun issue sources:\nU.S. Bureau of Alcohol, Tobacco and Firearms: (202) 927-8500, public affairs\nNational Shooting Sports Foundation/Sports Arms and Ammunition Manufacturer's Institute, Jim Chambers, executive director, (703) 242-1690 or saami@ntpix.net\nCoalition to Stop Gun Violence, Desmond Riley, press contact, (202) 534-0340\nJohns Hopkins Center for Gun Policy and Research, Susan DeFrancesco, coordinator, (410) 955-3995 or sdefranc@jhsph.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-user-friendly-forms","slug":"2000-april-user-friendly-forms","url":"https://dwillis.github.io/uplink/article/2000-april-user-friendly-forms.html","year":2000,"month":"April","monthNum":4,"headline":"User-friendly forms","kicker":"TECH TIP","author_name":"David Herzog","author_title":"CAR specialist","affiliation":"Providence Journal","summary":"A Providence Journal investigative reporter explains how to use Microsoft Access forms as building blocks for creating stand-alone programs that allow journalists to query databases without programming knowledge, using a chronology table as an example.","keywords":["Access","database","forms"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Access"],"full_text":"One of the things I've learned in my few years on the Providence Journal investigative team is that for such a small state, Rhode Island breeds some big corruption cases. As the CAR specialist on the paper's I-team, one of my roles is to gather the data to help uncover wrongdoing and make sure other reporters can get to it easily.\n\nNot everyone working on these investigations wants to learn how to query databases, so I've used Microsoft Access to create stand-alone programs that allow reporters to point and click their way to useful information.\n\nAccess provides all the tools to create these programs and enough power to accomplish everything you need without resorting to programming.\n\nBefore getting into the nuts and bolts of creating a program, it's necessary to understand how you can design forms to create screens for entering and querying data. That's what this article is about. Once you know how to create a form, it's easy to learn how to create a program.\n\nForms are the building blocks of the programs you create. You use the forms to design screens for your program, enter data into tables and query tables. Another issue of Uplink will carry the details of making forms work with other objects to create the program.\n\nNearly every investigation relies on a chronology to organize the story, so let's use one to get started.\n\nFor this lesson, I'm using an Access database file called apriltech.mdb, which you can download from the NICAR web site http://www.nicar.org. After downloading the file, open it in Access. (I'm using Access 97). You can open the Chronology table in design view to look at the fields and data types.\n\nThe simplest way to create a form is to let the Form Wizard do the work, then tweak the form manually. For this form we want to make these three modifications: (1) Create a button to close the form. (2) Create a \"pick list\" of story topics. (3) Create a toolbar with buttons to allow easy querying.\n\nCreate form\n\nCreate the form by clicking on the Forms tab, then New. Pick Form Wizard, then select the Chronology table from the pick list below. Click Ok. As you click the Next buttons, the Form Wizard will walk you through the steps of creating a form. Select all the fields, except for Edate, which Access will fill in automatically. Then, pick Columnar layout and pick a style. Finish and the wizard displays the form. Now you're ready to modify the form.\n\nClose button\n\nLet's start with the close button. First go into the form's design view by clicking on the Design View button at the upper left (hint: it looks like a square and pencil). Make room for the button by pulling the Form Footer rule down. Then click on the command button in the tool box (if you put your cursor over each button, Access will tell you the name of each one) and click on the form.\n\nThe Command Button Wizard appears. In the Categories box pick Form Operations. In the Actions box, pick Close Form. Click on next and choose a button style. Click Next, then Finish. Go back to the form view and save the changes to your form. Click on the close button to make sure it works (you have to be in Form view).\n\nPick List\n\nNext, we'll add a pick list that contains pre-defined story topics. Return to the form's design view, select the topic field and delete. Click on the combo box button in the toolbar and then click on the form, where the topic field was. The Combo Box Wizard starts.\n\nSelect \"I will type in the values that I want\" and click on Next. Under Coll type the story topics: Bribery, Sedition and Mutiny, using the Tab key to create a new line for each word. Then click Next.\n\nIn the next step, click Store that value in this field and select Topic from the pick list and click on Next. Now type a label for the pick list: \"Topic\" and click Finish. Switch to the form view and you see that the Topic combo box has a pick list with your story topics.\n\nEntering data\n\nNow try entering data into your form and use the Tab key to move through the fields. (If the cursor skips over the Topic field: Go back into Design View, right click on the form background and select Tab Order from the pop-up menu. Click Auto Order and Access creates a tab order based on the placement of the fields on the form. Click OK and go back to Form View.)\n\nIf you need to adjust any of the field widths or placement of field labels, you can do that in Design View. Finish adding records so you have a dozen or so for querying.\n\nCreating toolbar\n\nTo make things easy for users, we'll create a toolbar that will step users through queries using this form. Go back to Design View and select View-Toolbars-Customize from the menu.\n\nIn the Customize box click New and call your toolbar E-Z Query. A blank toolbar appears. Now we need to add buttons that will run the queries. In the Customize box, click the commands tab to access the available buttons. In the Categories box select Records. Then, drag and drop these three command buttons from the right onto the blank toolbar: Filter by form, Apply Filter and Remove Filter/Sort.\n\nSince these buttons won't make sense to your users, we will create text buttons. Right click on the first button on the toolbar (while still in Design View) and in the text box that appears in the pop-up menu, type (1) Define query. Then click Text Only (Always) in the pop-up menu. Change the next two buttons to (2) Run query and (3) Show all.\n\nNow we will make sure the toolbar appears with the form. Go back to the Customize box and click the Toolbars tab. Select E-Z Query and click the Properties button. Under Toolbar type, pick Toolbar. Under Docking, pick Allow any. With these settings, you'll be able to move the toolbar under the toolbars at the top of the Access window and make it look like a menu bar. Attach the toolbar to the form by modifying the form's properties. To open the properties box, left click the form select box (it's the square to the immediate left of the ruler that runs from left to right) then right click to get the pop-up menu. Select Properties, then the Other tab. In the Toolbar field, pick E-Z Query, then close the Properties box.\n\nReady to go\n\nSave your database and get ready to query using the 1-2-3 method. Click the (1) Define Query button and you'll get a blank to enter query parameters. Let's look at records having to do with the Bribery story. Go to the Topic pick list and select bribery. Then click on the (2) Run query button to select just those records. At the bottom of the form, you'll see how many records were selected. After looking at your records, pick (3) Show all to remove the filter. You can run queries on any of the fields and use Access' wildcards.\n\nThat's all you have to do to make data mining easy for other reporters. In the next article, we'll put together a stand-alone program that will make it even easier for others to get information out of your data.\nDavid Herzog can be reached by e-mail at david_herzog@projo.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-recent-car-stories-online","slug":"2000-april-recent-car-stories-online","url":"https://dwillis.github.io/uplink/article/2000-april-recent-car-stories-online.html","year":2000,"month":"April","monthNum":4,"headline":"Recent CAR stories online","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Pioneer Press analyzed nonprofit arts organizations' finances in the Twin Cities, finding they experienced significant financial growth during the 1990s. Newsday's investigation revealed disparities in property tax burdens, with the heaviest taxes falling on homeowners in predominantly Black and Hispanic communities on Long Island.","keywords":["arts","finances","taxes"],"topics":["Investigative Methods","Data & Databases","Demographics & Census"],"technologies":[],"full_text":"\"Awash in money, arts groups seek help\" St. Paul Pioneer Press March 5-6, 2000 www.justgo.com/twincities/ The Pioneer Press looked into the finances of nonprofit arts and cultural groups in the Twin Cities. The newspaper's analysis of I-990 forms from 1993 through 1997 found that these groups have enjoyed unprecedented prosperity in the 1990s.\n\n\"An Uneven Burden\" Newsday March 5, 2000 www.newsday.com/news/ncovsun.htm A computer-assisted analysis of home sales from Jan. 1998 to Sep. 1999 found the heaviest tax burdens in Long Island's Nassau County are among homeowners in communities where blacks and Hispanics make up the majority of the population.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-recent-car-stories","slug":"2000-april-recent-car-stories","url":"https://dwillis.github.io/uplink/article/2000-april-recent-car-stories.html","year":2000,"month":"April","monthNum":4,"headline":"Recent CAR stories","kicker":"","author_name":"","author_title":"","affiliation":"The Courier Houma, The Daily Comet, The Opelousas Daily World, The Philadelphia Inquirer","summary":"Two major investigative journalism projects: Louisiana newspapers examine environmental regulatory oversight, while Philadelphia Inquirer documents systematic crime classification manipulation by police.","keywords":["investigation","police","environment"],"topics":["Investigative Methods","Health & Environment","Crime & Justice"],"technologies":[],"full_text":"Recent CAR stories\n\n\"Louisiana: Polluter's Paradise?\"\nwww.dailycomet.com/pollute.html\nThe Courier Houma, The Daily Comet and The Opelousas Daily World\nFeb. 24-Feb. 28, 2000\nNew York Times Regional newspapers in Louisiana have teamed up to investigate how well the Louisiana Department of Environmental Quality is meeting its obligations.\n\n\"Crime in Philadelphia\"\nThe Philadelphia Inquirer\nwww.philly.com/packages/crime/\n\nApril 18, 1997 to Feb. 4, 2000\n\nIn a stream of series over two years, the Inquirer has documented how Philadelphia police have been systematically \"downgrading\" crimes for years, booking them as lesser offenses that don't appear in the FBI's national listing of crime rates.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-playing-the-lottery","slug":"2000-april-playing-the-lottery","url":"https://dwillis.github.io/uplink/article/2000-april-playing-the-lottery.html","year":2000,"month":"April","monthNum":4,"headline":"Playing the lottery","kicker":"FIRST VENTURE","author_name":"Mark Houser","author_title":"","affiliation":"Pittsburgh Tribune-Review","summary":"A Pittsburgh Tribune-Review investigation uses computer-assisted reporting to analyze Pennsylvania lottery sales data by zip code and demographic information to determine whether poor communities spend more on lottery tickets than wealthy ones, ultimately choosing the sales method over winner analysis due to methodological challenges.","keywords":["lottery","analysis","investigation"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["Excel","Claritas"],"full_text":"\"Confronting the Numbers,\" a three-month Pittsburgh Tribune-Review investigation of the Pennsylvania lottery, began by confronting the kind of statistical conundrum that makes journalism such a fun job. The main point of my first computer-assisted reporting project was to see if the conventional wisdom - that the poor gamble on the lottery more than the rich - was true in our metro area. Not an easy question, it turned out. There are essentially two ways to use state data to determine who buys lottery tickets. One relies on sales information. The Pennsylvania Lottery will provide a spreadsheet of annual lottery sales by retailer. The spreadsheet includes the retailer's address, so a CAR reporter can sum the sales data by zip codes.\n\nBut numbers don't tell the whole story. Eventually, I had to put the mouse down and pick up the phone. I even had to (gasp!) leave the office.\n\nUsing demographic statistics, such as Claritas data, it's easy to figure per capita sales for a zip code, and compare that to median incomes in the same neighborhood.\n\nThe problem with this method is that it can't determine whether most, all, or none of a store's sales are to people who live in the neighborhood. Maybe the store is selling to commuters on their lunch hour or on the way home from work.\n\nShow a lottery official your sales-based analysis, and he immediately will try to shoot holes in it for that reason.\n\nAlternate method\n\nThe other method focuses on winners. Lotteries have to report anyone who wins more than $600 in a game to the IRS, and those records are public.\n\nBecause winners are a random sample of all players (if not, you've probably got a bigger story on your hands), a CAR reporter can extrapolate what share of sales comes from a given neighborhood based on what share of winners lives there.\n\nThe problem with this method is that $600 threshold. In Pennsylvania, the prize for hitting the three-digit Daily Number on a $1 straight bet is $500. Those winners aren't reported to the IRS. Numbers games are extremely popular in Pittsburgh, especially with its large elderly population. They are the game of choice among the heaviest players, the ones who collect lists of lucky numbers they notice on license plates or passing buses.\n\nA Pennsylvania lottery official confirmed to me that if savvy numbers players want to bet $10 on 7-9-2 straight, they buy 10 $1 tickets. That way, if they hit they get 10 $500 payments instead of a single pot of $5,000 that will be reported to the IRS.\n\nGiven that choice, I opted to use the sales method. It is more straightforward and easier for readers to understand.\n\nI eliminated from my analysis any ticket sales in the downtown area or in the airport, where sales were most likely to be to people who didn't live nearby.\n\nThough Pennsylvania is notoriously secretive, they didn't balk at mailing the sales data on two disks of Excel spreadsheets. Because the data were statewide, I was able to give customized stats to sister papers in neighboring counties, allowing them to do similar stories.\n\nThe results\n\nSurprise! Sales were highest in working class neighborhoods, and lowest in the wealthier suburbs.\n\nNow comes the part most dangerous for CAR geeks - getting buried in spreadsheets. I made top ten lists of this, figured per capita of that, made pie charts and line graphs and all sorts of neat stuff. I became a data weasel. But numbers don't tell the whole story.\n\nEventually, I had to put the mouse down and pick up the phone. I even had to (gasp!) leave the office.\n\nI visited the library and read what the experts and other pundits had to say about lotteries. I called the professors, the industry experts, and the state budget gurus. Several long interviews and hundreds of inches of notes yielded no quotes at all, but helped me understand the subject.\n\nI found out about a 1987 legislative resolution to study how much the lottery causes problem gambling - which the state has since ignored.\n\nMonths of poking around caused some leaks to spring. We got a tip that 15 employees got custom Pennsylvania Lottery sportcoats, purchased from the lottery director's tailor, costing the state $500 each.\n\nI visited a Gamblers Anonymous meeting in a church basement in the neighborhood where sales are highest. There I met several self-professed lottery addicts who were surprised and angry to hear the state director and revenue secretary both claimed such people did not exist.\n\nAt the region's top-selling store, our photographer met a poor family that spends $9 a day on tickets - two bucks for Mom and Dad, and one for each of their seven children. Their story sold the package and generated the most response from our readers.\n\nI waited until late in the game to go to the capital and interview the bigwigs. By that time, I had a lot of pointed questions ready, and was able to anticipate likely evasions and press for further explanations.\n\nThe state still has a lottery; I don't think any revelations are likely to change that. But as a result of our story, the state legislature held hearings to review the lottery's financial health, which our package called into question. Soon after, the auditor general announced his office would launch a formal audit.\n\nAnd the lottery director caught a tremendous amount of flak for those jackets. Three months after the series ran, he resigned, citing a need to spend more time with his family.\n\nMark Houser can be reached by e-mail at mhouser@tribweb.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-text-to-date","slug":"2000-april-text-to-date","url":"https://dwillis.github.io/uplink/article/2000-april-text-to-date.html","year":2000,"month":"April","monthNum":4,"headline":"Text to date","kicker":"BEGINNER TIPS","author_name":"","author_title":"","affiliation":"","summary":"Instructions for converting character date fields (like '01021999') to formatted date fields (like '01/02/1999') using either Access or FoxPro database software.","keywords":["database","formatting","conversion"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["Access","FoxPro","SQL"],"full_text":"Here are versions in Access and FoxPro for converting character fields, such as \"01021999\" to a true date field, \"01/02/ 1999.\"\n\nNo matter what program you're using, start by inserting a new field in your table. In this example, the original date field is called DOB and the newly created field is called BIRTHDT. The date appears as mmddyyyy.\n\nIn Access:\n\nIn the grid view, put the name of the new field, BIRTHDT, in the field grid. In the criteria line: Left([DOB],2) & \"/\" & Mid([DOB],3,2 & \"/\" & Mid([DOB],5,4)\n\nThe SQL version looks like this: UPDATE [TABLENAME] SET BIRTHDT= Left([DOB],2) & \"/\" & Mid([DOB],3,2)\n\n& \"/\" & Mid([DOB],5,4)\n\nThis tells the computer, start at the left side of the DOB field and take two digits, then insert a forward slash (/), then go to the middle of the DOB field and start at the third position and take two digits, then insert another forward slash and finally go to the middle of the DOB field and start at the fifth position, taking four digits.\n\nIf your date has only two digits for the year, replace the final 4 with a 2.\n\nMake sure to use the ampersand (&) and not the plus sign.\n\nIn FoxPro:\n\nHere's the SQL (notice slightly different syntax): Replace all BIRTHDT with; CTOD(substr(D0B,1,2) + \"/\" + substr(DOB,3,2) + \"/\" substr(D0B,5,4))","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-trimming-spaces","slug":"2000-april-trimming-spaces","url":"https://dwillis.github.io/uplink/article/2000-april-trimming-spaces.html","year":2000,"month":"April","monthNum":4,"headline":"Trimming spaces","kicker":"QUICK TIP","author_name":"Tom McGinty","author_title":"Training Director","affiliation":"NICAR","summary":"A tutorial on creating and using a Microsoft Access module that removes leading, trailing, and extra spaces between words in database fields.","keywords":["Access","module","spaces"],"topics":["Technology","Data & Databases"],"technologies":["Access","Visual Basic"],"full_text":"This module will trim leading and trailing spaces in a field, as well as extra spaces between words. For example, it will turn \"(space) Tom (space)(space)(space)McGinty(space).\" into \"Tom(space)McGinty\".\n\nTo find the Microsoft article about this module, go to www.microsoft.com/directory and click on \"Searchable Knowledge Database.\" Search for story number Q148396. From there you can copy and paste this module directly into Access.\n\nHere's how the module works: Go to the module tab on your database window and click on the \"new\" button. The window that pops up should have the following two lines at the top: Option Compare Database Option Explicit If it does not, add them. Next, type the following syntax below those two lines:\n\nFunction StripExtraChars (PassedStr, RemoveExtraChar$) On Local Error GoTo stripExtraChars_Em Dim i As Integer, GotChar As Integer Dim HoldStr As String, HoldChar As String , Exit if passed value is Null. If IsNull (PassedStr) Then Exit Function Trim extra characters from passed , string. PassedStr = Trim$(PassedStr) , Cycle through string and remove , extra string characters specified in , the RemoveExtraChar value. For i = 1 To Len(PassedStr) HoldChar = Mid$(PassedStr, i, 1) If HoldChar = RemoveExtraChar\n\nThen If GotChar = False Then GotChar = True HoldStr = HoldStr & HoldChar End If Else GotChar = False End If If Not GotChar Then HoldStr = HoldStr & HoldChar End If Next i StripExtraChars HoldStr StripExtraChars_End: Exit Function StripExtraChars_Err MsgBox Error$ Resume StripExtraChars_End End Function\n\nNext, under the \"Debug\" menu, choose \"Compile and save all modules.\" It will prompt you to name the module. Name it \"TrimSpaces\" and then close the module.\n\nNow, let's say you have a field called NAME that has extra spaces between first and last names, and unwanted spaces at the beginning and end of some of the names First, create a new field in your table called \"NEWNAME\"(NEVER change your original data).\n\nThen start a new query, add your table to the query and then, under the \"Query\" menu, choose \"Update Query.\" Pull the \"NEWNAME\" field down into the \"Field\" row of the update query. Then type this in the \"Update To:\" row of the query:\n\nStripExtraChars([NAME],Chr$(32))\n\nThat applies the new module you just created to the NAME field and puts the results into the NEWNAME field. The second argument of the function, Chr$(32)), is the ASCII code for spaces. The SQL for the same update query would look like this: update TABLENAME set NEWNAME = StripExtraChars([NAME],Chr$(32)) Tom McGinty can be reached by e-mail at tmcginty@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-uplink-story-ideas","slug":"2000-april-uplink-story-ideas","url":"https://dwillis.github.io/uplink/article/2000-april-uplink-story-ideas.html","year":2000,"month":"April","monthNum":4,"headline":"Uplink story ideas","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR's Uplink newsletter invites journalists to submit story ideas about computer-assisted reporting (CAR) projects, technical solutions, and uncovered beats or issues for potential publication.","keywords":["journalism","reporting","data"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Uplink story ideas: Have you or one of your colleagues recently published a story using CAR that has not been done before or involved particularly difficult data work?\n\nDo you know of a technical problem (or its solution) that others may like to hear about? Or is there some issue or beat that we haven't covered?\n\nIf you have a story idea, we'd like to hear from you. Please contact managing editor Maryjo Sylwester either by e-mail at maryjo@nicar.org or by phone at (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-impact-of-choice","slug":"2000-april-impact-of-choice","url":"https://dwillis.github.io/uplink/article/2000-april-impact-of-choice.html","year":2000,"month":"April","monthNum":4,"headline":"Impact of choice","kicker":"EDUCATION","author_name":"Griff Palmer","author_title":"","affiliation":"The Daily Oklahoman","summary":"An investigation into Oklahoma City's choice school movement reveals that these schools disproportionately serve white, wealthy students from educated neighborhoods, while neighborhood schools become increasingly concentrated with minority and poor students, coinciding with declining test scores.","keywords":["choice","schools","segregation"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["FoxPro","Excel","SPSS","Atlas GIS"],"full_text":"Impact of choice\n\nIn the past five years, the Oklahoma City school district has, with growing enthusiasm, embraced the \"choice school movement.\" The growth of the movement has coincided with the dismantling of the district's court-ordered busing system and a return to neighborhood schools, many of them with student bodies that are 90 percent or more black.\n\nOklahoma City's choice schools are widely viewed as unqualified successes. But dissenting voices have argued that the establishment of choice schools, catering to elite students, imposes a cost on other students.\n\nEducation reporter Bobby Ross and I decided to put a yardstick to the debate: Are choice schools having an impact on other schools in Oklahoma City? How much of an impact? What are the implications? Our measurement tools included FoxPro, Excel, SPSS and Atlas GIS.\n\nWe found that students attending Oklahoma City's choice schools are disproportionately white, and are drawn disproportionately from the city's wealthiest, best-educated neighborhoods. We also found that neighborhood schools have greater concentrations of minority students and poor students than live within those schools' attendance boundaries. This bleeding off of talent from the wealthiest neighborhoods has coincided with a marked decline in standardized test scores among students in neighborhood schools.\n\nData used\n\nAt the heart of our analysis was a data file containing the home addresses and school assignments of Oklahoma City's 40,000 students.\n\nSeveral fellow journalists expressed surprise that we were able to get the school district to give us this data. There is ample room, though, under the Family Educational Rights and Privacy Act of 1974, for a school district to release student directory information.\n\nThe school district also provided us boundary files for neighborhood school attendance zones. We geocoded students' addresses and then used Atlas GIS to assign geocoded addresses to corresponding school attendance boundaries.\n\nThen we were able to measure how many students were leaving their neighborhood school zones to go to choice schools. But we also wanted to know something about the economic status of those leaving and those staying.\n\nEconomic status\n\nWe first tried to link student addresses to county property tax data, intending to use property values as proxy measures of student economic status. We discovered problems with this approach, though. We felt that we would have to develop separate measurement scales for those who lived in rental property and those who lived in their own homes. However, the tax rolls didn't identify rental properties or provide rent cost data.\n\nWe abandoned this approach, swallowed hard, and turned to our marketing department for a piece from its collection of Claritas data: estimated 1999 median household income and 1990 educational attainment by census block group. (See the Jan./Feb. 2000 issue of Uplink for a more detailed discussion on measuring student economic status).\n\nWe used Atlas GIS to assign each student's address to the census block group within which it fell. Then, we linked the student address data to the Claritas income and education data, joining on the block group field.\n\nBlock group-level income and education data served as rough indicators of individual students' socioeconomic status. This method allowed us to make observations based on the socioeconomic status of the block groups in which students lived, but not on individual students.\n\nWe used SPSS to calculate income quartiles for all middle and high school students, based on the block group in which they lived.\n\nWe assigned an income quartile ranking to each student in the data set. Then, we used FoxPro to create three tables:\n\n1.) One grouping the data by middle school/high school income quartile within each school.\n2.) One grouping by middle school/high school income quartile within each attendance zone.\n3.) One grouping by income quartile among students living in each attendance zone but enrolling in a choice school.\n\nWe joined the three tables on school name. The result was a table that contrasted the number of students in each quartile living within a school attendance zone with the number of students actually attending the school and the number going to choice schools.\n\nOpening the table into Excel, we calculated percentage of each school's enrollment, attendance zone and choice school out-migration falling into each quartile.\n\nWe found that in 11 of Oklahoma City's 13 neighborhood middle and high school attendance zones, a disproportionately low number of students from the lowest income quartile, and disproportionately high number of students from the highest income quartile, had left neighborhood schools for choice schools.\n\nSchool lunch data reinforced this observation. We found that the opposite was happening in neighborhood schools.\n\nOther measures\n\nVirtually all of the income disproportionality between neighborhood schools and their attendance zones was attributable to the out-migration of students to choice schools. A tiny portion was attributable to out-migration to other neighborhood schools.\n\nHowever, we were unable with our data to measure how many students were leaving neighborhood school attendance zones for suburban schools and private schools. While we weren't able to distinguish between students recaptured from private schools and suburban public schools, we were able to identify those students enrolling in Oklahoma City schools from addresses outside the district. (A disproportionate number of high-income students from out of district were getting into choice schools.)\n\nWe used similar methods to analyze household education levels within neighborhood schools' attendance zones, and to contrast them with the household education levels of the block groups in which out-migrating students lived.\n\nTo get an idea of racial proportionality, we compared the racial makeup of each choice school with the racial makeup of the nearest neighborhood school serving the same grades. In virtually every instance, we found that the choice schools, while having diverse populations, had disproportionately more white students.\n\nWe also analyzed district admission statistics and found that a greater percentage of minority applicants than of white applicants were being denied admission to those choice schools that used selective admission criteria.\n\nThese measurements provided a framework on which Bobby hung anecdotal evidence gathered through countless hours spent inside choice schools and neighborhood schools, and through innumerable interviews and focus-group sessions.\n\nDid we prove that neighborhood-school students are being harmed by the choice school system?\n\nNo. But we did create a replicable set of measurements against which to discuss studies showing that students in schools with greater concentrations of poverty tend to perform more poorly on standardized tests.\n\nGriff Palmer can be reached by e-mail at gpalmer@oklahoman.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-april-bits-bytes-and-barks","slug":"2000-april-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2000-april-bits-bytes-and-barks.html","year":2000,"month":"April","monthNum":4,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering Census 2000 workshops in three locations with expert instructors, while also updating several federal databases including the Consolidated Federal Funds Report and announcing upcoming conferences in New York and Kentucky.","keywords":["census","workshops","databases"],"topics":["Journalism & Newsroom","Data & Databases","Technology"],"technologies":["Internet","World Wide Web","FTP"],"full_text":"Census Workshops\n\nA third date for the IRE and NICAR Census 2000 seminars has been set for Dec. 8-10 in College Park, Md. Others will be held April 7-9 in Tempe, Ariz., and Aug 4-6 in Columbia, Mo.\n\nThese intensive two-day sessions will be a fast-track lesson on what you need to know to start producing stories when the data begins to emerge at the end of this year.\n\nInstructors will be Paul Overberg, a database editor at USA Today, and Steve Doig, the former Miami Herald research editor who now teaches computer-assisted reporting at Arizona State University.\n\nMore information, including registration forms and a schedule, is available at www.ire.org/training/censusworkshoptempe.html\n\nAlso check out Doig's new Web site on using data from the 2000 Census. The site is at http://cronkite.pp.asu.edu/census (note: make sure \"frames\" are enabled on your browser).\n\nThe project, supported by a grant from the Russell Sage Foundation, features a variety of pages that will be useful for journalists.\n\nNICAR data update\n\nNICAR has updated the Consolidated Federal Funds Report (CFFR) data through September 30, 1998, the National Endowment of the Arts grants database through fiscal year 1998 and the Recreational Boating Accidents database through the end of 1998. The CFFR data includes all federal money paid to states, counties and local agencies. That includes social security payments, grants and direct loans. A sample slice and record layout are available at www.nicar.org/data/cffr/ The recreational boats database contains data on accidents that resulted in death or injury requiring more than first aid. NICAR has data going back to 1969. More information and sample slices of the tables are available online at www.ire.org/datalibrary/databases/BoatAcc/ NEA database includes grants awarded since 1987 to arts organizations throughout the United States. More information and a sample data slice are available online at www.ire.org/datalibrary/databases/NEA/ Contact the data library to order these or any other datasets by calling (573) 884-7711.\n\nConferences\n\nA registration form for the IRE National Conference, June 1-4 in New York City, is available in this issue on page 15. The first day of the conference will be an optional CAR day with panels planned on topics such as: housing issues such as slumlords and land flips, environmental databases, responding to disasters, census, campaign finance, deadline data work, newsroom training and mapping. More information is available online at www.ire.org/training/ny00/\n\nSchedule information and registration forms for the National CAR Conference, Sept. 14-17 in Lexington, Ky., should be available soon on the IRE Web site. More information is available by calling (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-may-the-daily-grind","slug":"2000-may-the-daily-grind","url":"https://dwillis.github.io/uplink/article/2000-may-the-daily-grind.html","year":2000,"month":"May","monthNum":5,"headline":"The daily grind","kicker":"CAR ON DEADLINE","author_name":"Neil Reisner","author_title":"Reporter","affiliation":"The Miami Herald","summary":"This article discusses the importance of preparation and 'thinking in rows and columns' to successfully integrate computer-assisted reporting into daily deadline workflows.","keywords":["CAR","deadline reporting","data analysis"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Access","World Wide Web"],"full_text":"Driving CAR on deadline is not hard. It's kind of like the old saw about the tourist in New York who asks an old man on the street: \"How do you get to Carnegie Hall?\" The answer: \"Practice, practice, practice.\" Except, in the case of computer-assisted reporting, it's more like: \"Prepare, prepare, prepare.\" Now, this may seem like a contradiction in terms – How can we prepare for the unexpected? But it's really no different than any other deadline reporting we do. We all carry around a mental list of sources to use on breaking stories, whom to call when there's a major crime, an unexpected political battle or a big education story that demands coverage for tomorrow's paper or tonight's broadcast. It's no different with CAR. In CAR's case, though, it's a matter of knowing what data are available, knowing how to analyze it, whom to call to verify or interpret it. It's a matter of what can be called \"thinking in rows and columns\" – thinking always about how data can supplement a story. Now, we're not talking about big projects here. We're talking about daily or two-day stories; sometimes we're not talking about stories at all but \"computer-assisted paragraphs,\" those little bits of context that make a good story even better. Here are some examples from recent history at the Broward County edition of The Miami Herald: Tobacco and kids. A couple of months ago, we sent a FOIA to the FDA requesting the database they used to build their \"Children & Tobacco Compliance Checker\" on their Web site. It contained information on how states and various retailers were doing in the effort to enforce laws barring sale of tobacco products to young people. The response to our letter was typical: \"We don't have any way of doing that, but we've asked our outside contractor for a program and they're working on it.\" A few weeks after that, they called: \"We've got the data and we've put it up on the Web site. You can download it.\" Great. The FDA in its wisdom had decided to make the data \"easy\" to get by releasing it to the world instead of to the handful of reporters who'd asked for it. Now what had been a leisurely, spare-time kind of story became competitive. (The Herald's Broward edition competes fiercely with the Sun-Sentinel, the Chicago Tribune daily located across town.) We downloaded the data, saw it was in a beautifully simple, comma-delimited format and imported it into Access. Browsing through it, there were a few undefined codes, so we put a call in to the FDA for interpretation. We also saw that while cities were in the data, counties weren't. That meant quickly creating a look-up table to group South Florida cities into the counties in which The Herald was interested. Finally, the analysis: How was Florida doing compared to the rest of the country? How was South Florida doing? How were our local retailers doing? The result: We were doing better on all counts. Florida's push – and it's a big one – to get kids off tobacco was bearing fruit. Though this was not scientifically gathered data – it was not random but showed only what happened at the retailers FDA checked, a fact made clear in the story. So what we had was a nice, incremental story, a pat on the back for Florida, maybe not destined for 1A but for sure good on 1B. Total time: Two days. And it would have been one had the FDA not taken its sweet time calling back. Elections. Local elections are an example of how CAR can add depth and context to stories we already do. There are basically two questions we ask after an election: Who won? How did they do it? That often means staring at a table: Voting districts or precincts down the side, candidates across the top. Reporters often eyeball the results in certain \"key\" precincts, call a couple of pundits and throw a story in the paper or on the air. We do it a little differently. We've arranged with our county election supervisor to download precinct-by-precinct results at the end of election evening or, at worst, early the next morning. We import that into an Access database in which we've already prepared several lookup tables showing voter registration, precinct or district demographics, etc. With that, it's easy to pull out areas or demographics in which we're interested to show, for example, how predominantly black districts voted or to look at the \"condo canyons\" full of retired, New Deal Democrats. Add a mapping program such as ArcView, and you can draw a precinct-by-precinct picture of exactly how the vote went. And that lets us ask the candidates and the pundits exactly why they think various demographics voted the way they did instead of asking who made the difference. Another example: Every week, we survey 34 gas stations around the county and print the price of regular gasoline at each. Since the beginning of the year, we've also keyed the results into an Access database. Because we'd taken the time to prepare, we were in an ideal position to add local context – county and neighborhood averages, week-by-week price fluctuations and the like – to our stories when gas prices jumped dramatically a few months ago. Conclusion. What these examples have in common is this: We think in rows and columns, that is, we look for stories in which data might be used. We take our opportunities when we see them. We anticipate stories we know we're going to need to do and prepare data appropriately. And we do a couple of other things: We prowl constantly for useful data and maintain a data warehouse, available to all reporters at their desks. We use these databases to background people, find neighbors, check salaries, etc.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-may-whos-teaching-our-kids","slug":"2000-may-whos-teaching-our-kids","url":"https://dwillis.github.io/uplink/article/2000-may-whos-teaching-our-kids.html","year":2000,"month":"May","monthNum":5,"headline":"Who's teaching our kids?","kicker":"EDUCATION","author_name":"Chris Cantergiani","author_title":"Reporter","affiliation":"WSB-TV Channel 2/Atlanta","summary":"Reporters investigated the criminal backgrounds of public school employees by cross-referencing employment records with state prison and parole databases.","keywords":["education","criminal records","background checks"],"topics":["Education","Crime & Justice","Investigative Methods"],"technologies":[],"full_text":"We hear from viewers and consultants that crime and education are two big topics of interest. Usually those are two separate story ideas, but last fall we decided to work on an investigation that touched on both.\n\nOur premise was simple: Find out how many of the state's educators have criminal records.\n\nThe strength of our state's Open Records Act meant we could do the background checks ourselves. All we needed to obtain was the employment records of all current public school employees and the state's prison and parole databases.\n\nWe teamed up with The Atlanta Journal-Constitution in August to do the investigation and collaborated our efforts to break the story on air and in print Nov. 4-5.\n\nOur investigation found no one had ever checked the criminal backgrounds of","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-may-railroad-safety","slug":"2000-may-railroad-safety","url":"https://dwillis.github.io/uplink/article/2000-may-railroad-safety.html","year":2000,"month":"May","monthNum":5,"headline":"Railroad safety","kicker":"CAR ON DEADLINE","author_name":"Trebor Banstetter","author_title":"Reporter","affiliation":"The Palm Beach Post","summary":"Following a fatal train accident, a reporter used Federal Railroad Administration databases to reveal that Palm Beach County had the highest rate of railroad crossing crashes in Florida.","keywords":["railroad safety","FRA","public records"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["Access","Excel"],"full_text":"A horrific crash at a West Palm Beach railroad crossing shocked residents in March 1999: A car sitting at the crossing gates was pushed into the path of a speeding Amtrak liner after being rear-ended by another car, killing a 19-year-old. The crash was the latest in a series of car-train smashups in South Florida. As a reporter covering transportation for The Palm Beach Post, the crashes made me wonder just how safe railroad crossings really are. I was fortunate to have some CAR training from NICAR and Cox computer journalism guru Elliot Jaspin. I had already spent some time poking around railroad safety databases, and saw the West Palm Beach crash as an opportunity to put railroad-crossing safety in a broader perspective. Fortunately for CAR reporters, the Federal Railroad Administration has an extensive national database of railroad crossing collisions online going back to 1975, at http://safetydata.fra.dot.gov/officeofsafety. The Web site has a searchable front-end, but it allows you to download the entire database, which is more useful for CAR work. This file has all reported cases of an impact between on-track equipment (usually a train) and anything on a public or private highway-railroad intersection, including a wealth of other information. Getting the data. First, I got the record layout in PDF format, which is very detailed and includes all codes used in the data. Then I went to the \"Download Database Files\" page. Several databases are available, but I wanted the \"Grade Crossing Accidents\" file. One downside is that you cannot download every year at once. I wanted to look at five year's worth of data, so I downloaded each year and merged them into a single Access table. The data downloads in a self-extracting file that becomes a fixed-length text file. The record layout shows where the fields start and stop so you can import it into a spreadsheet or database software. After importing the data into Microsoft Access, I ran several queries to gauge the situation in Palm Beach County. The findings. The findings were significant. In the past five years, the county had more crashes than any other in Florida — in fact, it was among the worst in the nation. The crashes here also had higher injury and fatality rates than elsewhere in the state. I used two more FRA databases available on the same site (Public and Private Crossings) to compare the number of crashes with the total number of railroad crossings in the county. Again, Palm Beach was at the top of the state's list of crashes per crossing. It took a flurry of traditional reporting to discover the reasons behind the county's poor safety rate. Problems included a combination of the area's high growth rate, increasing traffic congestion and heavy freight and passenger railroad traffic. I also sat alongside a train engineer for a ride from West Palm Beach to Miami to see the problems up close. I was astounded at the number of near misses the train encountered on a single trip. The weekend after that fatal West Palm Beach crash, The Post published the story. It took only a few days to turn around, but it put the incident into a broader context and gave our readers something to think about the next time they approach railroad crossings.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-may-intranet-start-up","slug":"2000-may-intranet-start-up","url":"https://dwillis.github.io/uplink/article/2000-may-intranet-start-up.html","year":2000,"month":"May","monthNum":5,"headline":"Intranet start-up","kicker":"CAR ON DEADLINE","author_name":"Robert Gebeloff","author_title":"Database Editor","affiliation":"The Record","summary":"This article details how 'The Record' established a newsroom intranet to democratize data access, allowing reporters to quickly background subjects and find demographic statistics on deadline.","keywords":["intranet","newsroom","data management"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Intranet","Access","Microsoft Back Office","Pentium","Unix","Internet"],"full_text":"When word spread in Norwood, N.J., that authorities had granted a variance for a Korean congregation to build a large church, some residents began pressuring town leaders to reverse the decision. The primary argument against the new building: It would draw hundreds of \"outsiders\" into town and disrupt the peaceful suburban atmosphere. But before he even set foot in Norwood, reporter Adam Geller knew more about the community than some of its residents. At his desk in The Record's Hackensack newsroom, Geller made a few clicks on our CAR Lab intranet and was able to compile various demographic facts about the town. By throwing a few simple stats up against the perception of some Norwood residents, Geller was able to illustrate a major component of the conflict better than he could have with a thousand words of explanation. And he didn't have to touch a spreadsheet or database manager. Examples like this are exactly what we hoped to see when we began planning our newsroom intranet in 1998. While we use CAR in projects and short-term enterprise, we also subscribe to the philosophy that CAR should be for everyday use. And two years after we laid down the first line of intranet code, we can attest that there's no better way to have an impact on the daily news report than to \"democratize\" your data — making it widely available to all journalists in the shop, regardless of computer skill level. Our staff of about 100 reporters and editors is now averaging 80 queries a day through our intranet. Not every single query gets in the paper, of course, but it shows that our staff is using this tool as a regular part of the reporting process. I won't mislead you — building and maintaining an intranet does not produce the same high-profile results as hardcore CAR enterprise work. But if you plan it right, and pick the proper tools, you can minimize the workload and lay the groundwork for newsroom-wide CAR. Before delving into an overview of intranet planning and development, a few comments are in order about the advantages of an intranet over other systems for building database front-ends for general staff use. Many of you use Microsoft Access, which has fantastic tools for building search forms that allow users with no Access experience to query databases. The problem with Access front ends, however, is that they're not as efficient over a network as a \"client-server\" system. If you build front ends into Access databases and put them in a common folder on the network, vast quantities of data have to move back and forth between the user's computer and the computer where the database resides. This can become a major drain on network bandwidth. A client-server intranet, on the other hand, keeps network traffic to a minimum. The user types input into the client machine – a Web page on their desktop – and the input is passed on to the server. The query is run on the server, and just the results are sent back to the client. The database itself stays put and bandwidth use is minimal. So if you're hoping to make data available to a lot of users, an intranet is the way to go. But before you begin, you'll need to plan thoroughly in three areas: The Back End. This is the guts of your intranet, the stuff behind the scenes that makes it all work. Your back end has to include some kind of Web server and a scheme for handling database queries. There are different types of servers, and I'm not going to advocate any specific model. For specs, just get as powerful a machine as you can. I can happily report that our Web server is a low-end Pentium and still does an adequate job. The question of server type is thornier. There are a LOT of ways to build an intranet, and a lot of different opinions about the best way. If your Web server is fueled by the Microsoft Back Office suite, there is one set of tools for building the front-end user interface. If you have a Unix box, then the set of tools is completely different. My advice, which admittedly will irk the purists of specific technologies, is to use the tools that are most accessible. If your company already has a corporate intranet on a Windows server, then see if you can get space on it. If your IT department is filled with Unix geeks, then tap into their knowledge. As for data, we now have our tables stored on an SQL Server. But when we started, we used Access and achieved acceptable results. Our Web server knows where to find our data because we use \"open database connectivity,\" or ODBC. This is a simple setting in the Control Panel of our Web server. You give each of your databases a name, and then give the Web server actual location of the data on the network. Then, when you're building a Web page and want to tap into a database, you just insert the ODBC name and your Web server knows where to get the data. Whatever scheme you use, the key to efficiency is to make sure all your tables are indexed on the fields that your users will query. The Front End. Once you know what's running your server, you'll need to decide what to use for building Web pages. We use Microsoft Front Page for our static Web pages – our Internet guide, CAR newsletter, etc. – and Active Server Pages for our databases. There are many other options, however. ColdFusion is used in some newsrooms. I've seen people on the NICAR list talking about MySQL too. I don't want to talk about the pros and cons of each, but rather about some general design concepts. First of all, the whole point of doing this is not to give users the power to do every conceivable trick with a database that you can do with Access. Rather, the goal should be to pick out the queries that will be most useful to the staff as a whole. In many databases, all your staff really will want are simple name and town look-ups. We try to add value to the results. If a reporter is looking at campaign contributions by the late local oil baron Leon Hess, they can click on the results and see what other employees of Amerada Hess contributed. If the reporter is running a name through our crime database, they can click on the results and see what happened to the co-defendants. We also have learned not to cram every field into the results. With our criminal background search, the user types in a name, and if there's a hit, the page displays the county and case number. One more click yields about 80 fields of information. Front-end software such as ColdFusion and FrontPage offer wizards that will quickly build simple look-ups, which is a good way to get started. But we found ourselves wanting to do more than the wizards provided, so we learned how to build Active Server Pages with Visual Basic Script. Knowing how to code gives you absolute freedom to build whatever front-ends your reporters will need, and it's not the most difficult skill to pick up. I learned by studying a few sample scripts donated by Tom Torok when he was at The Philadelphia Inquirer, and then by picking up the wonderfully simple Active Server Pages for Dummies. Operation/Maintenance. Like a lot of things, you'll get out of your intranet what you put into it. I'm fortunate enough to have CAR specialists, plus a part-time data entry person to help me out. That has allowed us to expand the site to keep more than two dozen databases up to date. Still, we have a dozen more that I haven't found time for yet. Keeping a good intranet is a job, but the question is who does the job. At our place, the library is heavily involved in maintaining the corporate intranet, and the CAR intranet is our baby. But at your paper, the library might be interested in helping out, and the more help you can get the better. I should caution, however, that you should resist the temptation to turn your intranet over to non-journalists. Sure, the technical skills and labor can be handled by anyone, but the real gold in doing this is in the design. It takes a database editor to decide what databases to post and to design an interface that will best serve reporters. Most of us dread the notion of becoming a \"service bureau.\" And rightly so. As journalists, our ambition should not be to become the source of fascinating facts for stories by other reporters. At the same time, however, I feel that it's a disservice to our news organizations if we simply acquire and horde databases. I can cite countless stories in The Record that were made better because of information or ideas a reporter developed from using our intranet. It's ultimately a compromise. The downside is it takes some work to build and maintain. But the upside is that you're helping reporters help themselves, and that, in turn, gives you the freedom to pursue your own work. Using newsroom intranets: Reporters at The Record use the intranet in a variety of ways. Political reporter Maia Davis uses voter registration records all the time to learn the home address of a person in the news. On deadline, Scott Fallon pulled the salary and title of a teacher accused of sexual assault. Reporter John Chadwick regularly finds unlisted phone numbers in the voter registration records. Alex Nussbaum discovered that the mayor in one of his towns was giving out incorrect information about property assessments.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-may-using-newsroom-intranets","slug":"2000-may-using-newsroom-intranets","url":"https://dwillis.github.io/uplink/article/2000-may-using-newsroom-intranets.html","year":2000,"month":"May","monthNum":5,"headline":"Using newsroom intranets","kicker":"","author_name":"","author_title":"","affiliation":"The Record","summary":"Reporters at The Record leverage voter registration records and intranet resources to access public information for investigative journalism, including voter data, salary records, contact information, and property assessment details.","keywords":["records","reporters","information"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Intranet"],"full_text":"Reporters at The Record use the intranet in a variety of ways.\n\n• Political reporter Maia Davis uses voter registration records all the time, not just for finding the names of party members in certain towns, but also to learn the home address of a person in the news.\n\n• On deadline, Scott Fallon pulled the salary and title of a teacher accused of sexual assault.\n\n• Reporter John Chadwick regularly finds unlisted phone numbers in the voter registration records.\n\n• Alex Nussbaum discovered that the mayor in one of his towns was giving out incorrect information about property assessments.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-may-intranet-essentials","slug":"2000-may-intranet-essentials","url":"https://dwillis.github.io/uplink/article/2000-may-intranet-essentials.html","year":2000,"month":"May","monthNum":5,"headline":"Intranet essentials","kicker":"CAR ON DEADLINE","author_name":"Noemi Ramirez","author_title":"","affiliation":"IRE and NICAR","summary":"This article explores the diverse ways newsrooms use intranets as research hubs, from tracking public records to storing training materials and commercial database links.","keywords":["intranet","research tools","public records"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Intranet","Internet","World Wide Web","CD-ROM"],"full_text":"And you thought of your company's intranet as a mere sophisticated version of the traditional office bulletin board? Not quite. An informal survey among Newslibbers – subscribers of the NewsLib discussion list who have had experience with developing an intranet at a newspaper or TV station – shows that intranets play an important and sometimes essential role in promoting CAR in the newsroom. Elisabeth Donovan, Research Editor at The Miami Herald, reports that the Herald's intranet, one of the first ones in the country together with the NeRD page of the News & Observer, \"started in 1995 as a collection of Web links on a file server.\" This static compilation of online resources, however, evolved into a more complex structure in January 1998. According to Donovan: \"That year we added several searchable databases, including voter registrations, city/county employee databases, etc.\" Having these data sets easily available and, even more importantly, sharing the input forms so that \"reporters on the project could track the people they were investigating\" was crucial in developing the Miami vote fraud project, which in 1999 won The Miami Herald its sixteenth Pulitzer Prize. Intranets, however, prove to be most useful for daily reporting, says Russell Clemings, a CAR reporter who has maintained the intranet at the Fresno Bee since 1997. Clemings, who highlights the importance of having a budget for keeping the intranet updated, says his newspaper's intranet helps reporters \"track down and background news subjects. The local contribution data, which goes back to 1992, is especially useful in covering development.\" Breaking news is a challenging situation for reporters, editors and researchers. Once again, a well set up intranet designed to meet the culture and information needs of its organization can make all the difference on deadline. Tom Boyer, research editor at The Seattle Times, recalls that when white supremacist Buford O'Neal Furrow Jr. opened fire at a Los Angeles day-care center on Aug. 10, 1999, his paper was a full day ahead of the rest of the country because of their people-tracking resources. \"We were able to identify the guy, found his family, friends and associates and found obscure court records on him in the first 12 hours,\" Boyer says. The Seattle Times' intranet, which started as a \"virtual Rolodex\" of staff phone numbers in 1996, has since then added dozens of public record databases including real estate, business licenses, drivers, voters, state licensing and campaign contributions. Boyer, whose plans to improve the intranet are focused on refining the interfaces to allow for simultaneous searches in multiple databases, concludes that because of their intranet \"we basically are going to have a significant home-court advantage on any story that involves public records in Washington.\" Newspapers are not the only medium that has taken advantage of the intranet technology. C.B. Hayden, director of research services at ABC News, says, \"The intranet has been the focus of our efforts to provide the highest level of research service to a large international news organization.\" Like many others, ABC's intranet grew out of a network of CD-ROM databases in 1998. Two years later, offering more than 50 different fee-based services and receiving 100,000 hits per month, the research staff uses the intranet in a sophisticated way to customize content from database providers. Dedication, purpose and enthusiasm are keys for the success of the intranet project at ABC. \"Every day we are making decisions on how to develop it,\" Hayden says. \"We find that there seems to be no end to the creative ways we could use it.\" What does an intranet need to become the foremost research tool in your newsroom? Here you have a few suggestions offered by members of the NewsLib listserv: In-house archives: Text archives, live and archived photo systems, internal databases like celebrity or experts contact information, link collections of online resources organized by beat and category. Training materials: Instructional cheat sheets on how to use public records, online databases, or file a FOIA request. Databases of public records: General databases like death records, vital statistics, voter registration, campaign contributions, cars, driving licenses, business licenses, property records, real estate. Beat-specific databases like workers' compensation, city/county/state employees and salaries, inmates, aircraft, government aerospace data. Commercial resources: Lexis-Nexis Universe, Dialogselect, Wilson Biographies, Gale Biography Resource Center, Facts on File, Keesing Records of World Events, InfoUSA, Bell & Howell Proquest, Encyclopedia Britannica, Hoover's, Merlin, CD-Law Legal Research, Autotrack, Public Records Online. Other research resources: Percentage calculators, weights and measures converter, zip-code directory, phone and reverse phone directories, online searchable AP stylebook. Company information: Daily announcements, research request forms, expense forms, job requisition and evaluation forms, letterheads, phone and e-mail staff directory, staff schedules, conference room scheduler, job postings, bulletin board for staffers to post things for sale and, just for fun, softball league team rosters! Feedback: Design critique, best headlines of the week, tips, help documentation. That's not bad for a start, is it? So next time you need to turn to the Internet to get some data, think twice. Maybe the answer is right under your nose or, better said, under the firewall of your Intranet.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-may-realistic-goals","slug":"2000-may-realistic-goals","url":"https://dwillis.github.io/uplink/article/2000-may-realistic-goals.html","year":2000,"month":"May","monthNum":5,"headline":"Realistic goals","kicker":"CAR ON DEADLINE","author_name":"Heather Newman","author_title":"Reporter","affiliation":"Detroit Free Press","summary":"This article provides practical advice for using computer-assisted reporting on tight deadlines, emphasizing the use of familiar databases and expert networks.","keywords":["CAR","deadline","research"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Excel","Nexis","Internet","World Wide Web","Google","Deja","MapQuest"],"full_text":"You're sitting at your desk one afternoon, when the story hits the fan. You've only got a few hours, but you want to make the most of your computer skills. What do you do? Be realistic. Computer-assisted reporting can put you leagues ahead of your PC-challenged colleagues when it comes to responding to breaking stories, but only if you use it wisely. The last thing you want is to get caught up in technical problems and do a mediocre job on the story. So what can you realistically accomplish with CAR on deadline? Numbers and background. Use the Net to flesh out that list of who to call for the story. There is no better way to quickly find a public relations person for a particular company than to visit the firm's Web site and read through past press releases. As a bonus, those releases may have information that's pertinent to your story now. Is the president resigning? Chances are, you can find the last shareholder proxy that told how much he makes, or records of his insider stock trades, as part of the company's investor relations section. Get experts. A Lexis/Nexis search on your topic may turn up some great experts. So might a search of the Web, though be careful to double-check sites and attributions to make sure you're speaking with someone legitimate. ProfNet's Expert Database, posted at www.profnet.com, may have someone who can speak knowledgeably for you. If you're a beat reporter, it might be worth compiling the names and e-mail addresses of experts you speak with often, then use them in a (rare) mass mailing if you're desperately searching for someone who can be quoted in your story. Two tips: First, tell those experts you occasionally do this, and ask them if it's okay to contact them. Second, use this resource as little as possible, and you won't burn out a good deadline source. Crunch some quick numbers. Don't be afraid to fire up Excel or some other spreadsheet to help you if you need to crunch numbers, whether it is the number of beds in your local hospitals, changes in the stock prices for local public companies or budgets for different city departments. If you've got more than a dozen numbers to calculate, Excel will be faster even if you have to type in all the raw data yourself. Analyze familiar databases. If you're reporting on a recall of listeria-tainted meat (as the Free Press did for much of last year) and you have a database of USDA recalls that you've already compiled and used before, it's a snap to pull it open to calculate grand total poundage of meat recalled for listeria or other problems. Because you've used the database before, you know where to find it, what it contains, how accurate it is and how it works. Those are the tools you need to do a snap analysis on deadline – one that will make you look extremely smart when compared to your competition. Similar problems. Using a good Web search engine such as Google (www.google.com) or a good newsgroup search engine such as Deja (www.deja.com), you may be able to find someone who has suffered from whatever problem you're reporting on. The catch: It's darned hard to find local sources on deadline. If you can get away with someone in another state, however, it's worth the attempt. What's NOT reasonable to expect from CAR on deadline? New databases. First, you may have troubles getting the data you need at all, if it's not posted already on the Web. Second, even if you have the data on the Web or on disk, if you've never used it before, you're bound to run into glitches. Even if the data doesn't cause you. Get directions: Don't overlook the Net's ability to help you with the most basic of tasks — figuring out where you have to drive to report the story. MapQuest (at www.mapquest.com) is just one of dozens of mapping sites that can give you directions to that explosion.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-may-get-directions","slug":"2000-may-get-directions","url":"https://dwillis.github.io/uplink/article/2000-may-get-directions.html","year":2000,"month":"May","monthNum":5,"headline":"Get directions","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article highlights how online mapping services like MapQuest can assist journalists in finding directions to news story locations quickly and efficiently.","keywords":["mapping","directions","journalists"],"topics":["Technology","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"Don't overlook the Net's ability to help you with the most basic of tasks — figuring out where you have to drive to report the story.\n\nMapQuest (at www.mapquest.com) is just one of dozens of mapping sites that can give you directions to that explosion.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-may-great-scouts-cyberguides-for-subject-searching-on-the-web","slug":"2000-may-great-scouts-cyberguides-for-subject-searching-on-the-web","url":"https://dwillis.github.io/uplink/article/2000-may-great-scouts-cyberguides-for-subject-searching-on-the-web.html","year":2000,"month":"May","monthNum":5,"headline":"Great Scouts!: CyberGuides for Subject Searching on the Web","kicker":"","author_name":"Nora Paul, Margot Williams, Paula Hane","author_title":"editor (Paula Hane)","affiliation":"Information Today","summary":"A guide to using CyberGuides for effective subject searching on the web, reviewed for the IRE Journal's May issue.","keywords":["searching","web","guides"],"topics":["Journalism & Newsroom","Technology"],"technologies":["World Wide Web","Internet"],"full_text":"\"Great Scouts!: CyberGuides for Subject Searching on the Web\" by Nora Paul, Margot Williams, Paula Hane (editor). Cyber-Age Books/ Information Today (www.infotoday.com). A review of this book will appear in the May issue of the IRE Journal.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-may-tipsheets","slug":"2000-may-tipsheets","url":"https://dwillis.github.io/uplink/article/2000-may-tipsheets.html","year":2000,"month":"May","monthNum":5,"headline":"Tipsheets","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Three tipsheets offer guidance on using computer-assisted reporting (CAR) and backgrounding techniques to enhance journalism under deadline pressure, with resources and database recommendations.","keywords":["reporting","deadline","databases"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"\"Backgrounding the individual on deadline,\" (tipsheet #1083) by Don Ray, of Exclusive News Group, offers ten steps to backgrounding on deadline and a comprehensive list of resources that are essential to a functioning newsroom.\n\n\"CAR on Deadline,\" by Neil Reisner, of The Miami Herald, (tipsheet #908) provides a list of databases that could be turned into interesting stories.\n\n\"Quick CAR,\" by Janet Roberts of The St. Paul Pioneer Press, (tipsheet #909) provides some keys to success for using CAR to enhance deadline stories. To order, call the IRE Resource Center at (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-may-tipsheets-2","slug":"2000-may-tipsheets-2","url":"https://dwillis.github.io/uplink/article/2000-may-tipsheets-2.html","year":2000,"month":"May","monthNum":5,"headline":"Tipsheets","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Three tipsheets from journalism resources offer guidance on deadline reporting techniques, including backgrounding sources, using computer-assisted reporting (CAR) databases for story ideas, and tips for quickly incorporating CAR into breaking news coverage.","keywords":["deadline","reporting","databases"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"\"Backgrounding the individual on deadline,\" (tipsheet #1083) by Don Ray, of Exclusive News Group, offers ten steps to backgrounding on deadline and a comprehensive list of resources that are essential to a functioning newsroom.\n\n\"CAR on Deadline,\" by Neil Reisner, of The Miami Herald, (tipsheet #908) provides a list of databases that could be turned into interesting stories.\n\n\"Quick CAR,\" by Janet Roberts of The St. Paul Pioneer Press, (tipsheet #909) provides some keys to success for using CAR to enhance deadline stories. To order, call the IRE Resource Center at (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-may-tipsheets-3","slug":"2000-may-tipsheets-3","url":"https://dwillis.github.io/uplink/article/2000-may-tipsheets-3.html","year":2000,"month":"May","monthNum":5,"headline":"Tipsheets","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This article describes three journalism tipsheets that provide guidance on using computer-assisted reporting (CAR) and backgrounding techniques to enhance deadline journalism, including resources and databases available to newsrooms.","keywords":["journalism","reporting","deadline"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"\"Backgrounding the individual on deadline,\" (tipsheet #1083) by Don Ray, of Exclusive News Group, offers ten steps to backgrounding on deadline and a comprehensive list of resources that are essential to a functioning newsroom.\n\n\"CAR on Deadline,\" by Neil Reisner, of The Miami Herald, (tipsheet #908) provides a list of databases that could be turned into interesting stories.\n\n\"Quick CAR,\" by Janet Roberts of The St. Paul Pioneer Press, (tipsheet #909) provides some keys to success for using CAR to enhance deadline stories. To order, call the IRE Resource Center at (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-may-following-the-money","slug":"2000-may-following-the-money","url":"https://dwillis.github.io/uplink/article/2000-may-following-the-money.html","year":2000,"month":"May","monthNum":5,"headline":"Following the money","kicker":"CAMPAIGN FINANCE","author_name":"Darrel Rowland","author_title":"","affiliation":"The Columbus Dispatch","summary":"A short item pointing readers to another Uplink story or resource on tracking money.","keywords":["money","tipsheet","resources"],"topics":["Investigative Methods"],"technologies":[],"full_text":"Want to read another","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-a-taxing-issue","slug":"2000-june-a-taxing-issue","url":"https://dwillis.github.io/uplink/article/2000-june-a-taxing-issue.html","year":2000,"month":"June","monthNum":6,"headline":"A taxing issue","kicker":"TAXES","author_name":"Richard Dalton","author_title":"","affiliation":"Newsday","summary":"Fifteen years after an investigation into racial disparities in Long Island's property taxes, Newsday revisits the topic, finding persistent systemic issues. A shift to market value assessments is spurred by fiscal pressures.","keywords":["property taxes","racial disparity","assessment"],"topics":["Business & Economy","Government & Politics","Demographics & Census"],"technologies":[],"full_text":"Nearly 15 years after a Newsday series revealed that minorities on Long Island pay disproportionately higher property taxes, the area's tax assessment system persists. So the newspaper decided to take another look at the issue earlier this year. What we found was that despite hundreds of millions of dollars in payments to settle claims by overtaxed residents over the last decade and despite lawsuits by organizations including the New York Civil Liberties Union, Nassau County continued to defend a decades-old tax system that defied standard practices. Shortly after our story appeared, the county legislature voted to reassess based on market value. While some legislators who voted for the change cited Newsday's findings, observers generally said the main reason for the about-face was the county's ever-worsening fiscal position, caused in large part by the multimillion-dollar cost of settling lawsuits stemming from incorrect assessments. Across the country, most government entities base property taxes on the market value of a home. But Nassau County has been calculating property taxes based on what it would have cost to construct the home in 1938, when the system was implemented. That effectively means that minority and low-income areas pay a higher rate of property taxes. Here's why: A home in a rich, white neighborhood may have had the same construction costs in 1938 as a home in a poor, minority neighborhood. So the two homeowners would pay the same taxes. Meanwhile, the home in the wealthy neighborhood may have soared in value over the decades, so the taxes would be a significantly smaller percentage of home value compared to the home in the poor neighborhood, where property values often have risen far less.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-june-starting-with-modules","slug":"2000-june-starting-with-modules","url":"https://dwillis.github.io/uplink/article/2000-june-starting-with-modules.html","year":2000,"month":"June","monthNum":6,"headline":"Starting with modules","kicker":"TECH TIP","author_name":"Tim Henderson","author_title":"","affiliation":"NICAR","summary":"This article introduces Access Modules, which allow programmers to extend Access database functionality beyond built-in query tools and import features, starting with a basic 'Hello World' program example.","keywords":["modules","programming","database"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["Access","FoxPro","Perl","SAS","Visual Basic"],"full_text":"If you're new to programming, Access Modules are going to take you to the next level of database work. If you're an old hand at coding in FoxPro, PERL or SAS, you probably know that modules are your ticket to getting under the hood of Access into the functionality you're used to getting from those programs.\n\nModules mean you're no longer limited to the query functions Access provides for queries, and you're no longer forced to depend on the Import Wizard to dive into text files.\n\nIn this article we'll see a module that takes a nightmare text file with no delimiters and data spread over multiple lines and suck it into a database for analysis. You can't do THIS with the Import Wizard!\n\nBut first we'll warm up with an introduction to the concept of modules for those who haven't looked into them before.\n\nA quick 'Hello'\n\nLaunch Access and open the exercise file (Henderson.mdb) or open any Access database. The exercise file can be downloaded from the NICAR Web site, www.nicar.org. Click on the Modules tab and hit the 'New' button. You may see a couple of lines of text like this:\n\nOption Compare Database\nOption Explicit\n\nDon't worry about what they mean for now. UNDER those lines, type this:\n\nSub SayHi()\n\nAnd press 'Enter.' Notice that as you move off the line, Access colorcodes the keyword 'Sub' (for subroutine--a kind of program), adds a line setting it off, and automatically adds another line:\n\nEnd Sub\n\nThese are the starting and ending points of our program. All our program code will go between them, so type these lines there:\n\nDim theName As String\ntheName = InputBox(\"What's your name?\")\nMsgBox (\"Hi there, \" & theName & \"!\")\n\nModules are your ticket to getting under the hood of Access\n\nIf you have trouble, make sure you didn't add spaces to 'theName', 'InputBox' or 'MsgBox' - these must be single words. Notice the way Access follows your typing and nudges you in the right direction at every turn. If you mistyped or moved off a line before it was finished, Access complained. Menus appeared and guided you at various points.\n\nNow, to run your program hit F5 (or use Run to/Continue, or the arrow on your toolbar).\n\nIt may not be the most useful program in the world, but you've already declared a variable, assigned it a value based on user input, and given feedback to the user. You'll use these basic building blocks time and time again in any programming environment. If you're intrigued and want to get more familiar with Visual Basic for Applications (VBA) before continuing, I suggest the introductions and tutorials at www.athree.com/modules/index.html\n\nSubroutines and functions\n\nLet's talk quickly about subroutines. As in the example above, they run independently and leave no trace behind when they're finished. We're going to stick to subroutines here because they're easier to explain. But you should know that you can also create functions to supplement the Access built-in functions, like Trim() and Substring(). The coding procedure is very similar, and once created in a module, the functions become available in all your Access queries, just like the built-in functions. For examples of Functions check the Neatcode link from Microsoft (see greybox on the side) which includes many helpful and instructive functions you can import into your Access files. Add your own code, and you'll have the foundation of a library of custom functions available whenever you query your data.\n\nNewsroom database pros often build such a library to solve recurring problems like formatting text files for publication in a particular front-end system, a dependable way of calculating age from date of birth, or extending the Proper() function to do a better job of normalizing capitalization.\n\nImporting tricky data\n\nNow let's get into a meatier project, where we'll learn to use code to scan through a text document for fields to import to a database. This is the kind of thing you would have done in a FoxPro .prg file way back in 1995 when I joined NICAR.\n\nMake sure pacer.txt (the text file available as part of the download from the NICAR Web site) is in the same directory as Henderson.mdb and open the Access file, if you haven't done that already. Open pacer.txt in a text editor like Notepad, make sure Word Wrap is off and take a look. What a mess, huh? We need information on cases in the table called Bankruptcy. The saving grace for the text file is that at least we can predict where the information, such as debtors' names, will fall within each line so we can pull them out with the Access MID() function.\n\nHit the Modules tab, select Bankruptcy and hit the 'Design' button to see the code. Examine the ImportBankruptcy() subroutine and run it with F5 to see that it does in fact fill the Bankruptcy table with values from the file.\n\nA good place to start a job like this is not programming at all but documentation. Notice that there are lines in green that serve only as reading material for someone looking at the code - to do this, you precede the line with a single quote mark. It's called 'commenting' and it's what we all do best. It's a good idea, to get started on a complicated coding job, to just go ahead and write what you want to do, step by step, in comments. For this job, it would look like this:\n\n'Declare variables we need for all the\n'jobs below\n'Open the text file we want to import\n'Read each line into a text variable and\n'note the line number\n'When we get to a line with fields for\n'our table, pull them out into variables\n'Put together an SQL statement to\n'update the table\n'Run the SQL statement.\n\nNow you can start coding each task separately between the comments, and you've broken the job down so it doesn't seem quite so intimidating.\n\nInside and out\n\nThe cool thing about modules is that they let you automate tasks you could do anyway in the point-and-click part of Access, but also let you delve into areas beyond that.\n\nFor instance, part of what you're doing in this module is creating and running an SQL query, an Append query, that you could have produced in the query grid or in SQL View.\n\nYou do that with these lines of code:\n\nDim db As Database\nSet db = CurrentDb\n\n(code making an SQL statement and assigning it to the variable 'theSQL')\n\ndb.Execute theSQL\n\nBut at the same time modules give you the power to create and manipulate text files, and otherwise interact with Windows in ways you can't with interactive Access. We open our import file and read it in line by line with this code:\n\nDim myFile As Integer\nDim myFileName As String\n\nmyFileName = \"pacerfile.txt\"\nmyFile = FreeFile\n\nOpen myFileName For Input As #myFile\n\nTo open a file, you need to give it a numeric 'File Handle' and the FreeFile function assigns it a free handle automatically so you don't need to worry about it. If you wish, you could accomplish the same thing this way:\n\nOpen \"c:/data/uplink/pacerfile.txt\" For Input as #1\n\n\"For Input\" just means that you're going to read stuff from the file into your program. See Help on the Open statement for more options.\n\nOnce we have the file open, VBA doesn't give us a straightforward way (that I know of) to move to a particular line in the file. But the Line Input function will read a new line into a variable every time we call it, so we deduce the line we're on and keep track of it in the 'linecount' variable.\n\nFor efficiency you'd probably want to stop the Line Input statement after it reaches Line 8 - since we don't need to read any more of the file at that point - but I left it as is to make the code more portable for your own purposes.\n\nOnce we've gathered the data we need into variables, we parse those variables into an SQL statement and run it. I've included just one file, with fictitious names, but you could easily loop this code through all the files in a directory to populate a database.\n\nBugging out\n\nThe MsgBox function from our first exercise is a great way to handle the inevitable bugs that pop up when you write any kind of code. Here we've commented out a section that shows us our SQL in a message box, rather than executing it, so you can be sure it's OK before you run the program.\n\nThat's how I figured out I need to surround the text values with single quotes, and the dates with pound signs, in order for the SQL to successfully transfer the values into the table.\n\nIf something's going wrong with your program, a message box here and there can report the value of various variables that might be causing you trouble and will save you hours of agonizing.\n\nMoving along\n\nNow that you've been titillated by the possibilities of modules and VBA, try following some of the links with this article and check out some of the sample modules supplied by Microsoft in Neatcode, free for the downloading. Take a look at some of the online tutorials and take advantage of your Help files to learn about specific functions available only in VBA. And the NICAR-L crowd includes masters of VBA ready to respond when you get stuck. So go ahead, code yourself silly.\n\nTim Henderson can be reached by e-mail at thenderson@herald.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-death-and-dollars","slug":"2000-june-death-and-dollars","url":"https://dwillis.github.io/uplink/article/2000-june-death-and-dollars.html","year":2000,"month":"June","monthNum":6,"headline":"Death and dollars","kicker":"MENTAL HEALTH","author_name":"Debbie Cenziper","author_title":"Reporter","affiliation":"The Charlotte Observer","summary":"The Charlotte Observer investigated deaths in North Carolina mental health facilities by analyzing death certificate data, uncovering a lack of oversight that allowed dangerous trends and negligent caregiving to go undetected in both public and private psychiatric institutions.","keywords":["deaths","investigation","oversight"],"topics":["Data & Databases","Health & Environment","Investigative Methods"],"technologies":["Access"],"full_text":"It seemed like a simple question at the time: How many people are dying under questionable circumstances in the North Carolina mental health facilities they turned to for care? In July 1999, The Charlotte Observer decided to find out. Unfortunately, no regulatory agency, advocacy group or mental health organization tracked deaths of the mentally ill and developmentally disabled in North Carolina, despite the more than $1.4 billion the state spends annually on its public health system and $500 million on private facilities. With no one watching, dangerous trends went unnoticed, and negligent or abusive caregivers went unpunished. I turned to the state's center for health statistics for a database that offered information pulled from death certificates. The database had 370,000 deaths - every death in North Carolina since 1994. With assistance from Observer database editor Ted Mellnik, I used Access to filter the database for deaths in state-run psychiatric hospitals. Then, we filtered that list by cause of death, and came up with a list of questionable deaths, deaths by such things as choking, drowning and suicide.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-june-the-right-projection","slug":"2000-june-the-right-projection","url":"https://dwillis.github.io/uplink/article/2000-june-the-right-projection.html","year":2000,"month":"June","monthNum":6,"headline":"The right projection","kicker":"MAPPING","author_name":"Andy Lehren","author_title":"","affiliation":"Dateline NBC","summary":"Maps cannot simultaneously preserve all properties of the Earth's spherical surface when flattened, so cartographers must choose which attributes (area, distance, direction, or shape) to prioritize through different projection methods. Understanding map projections and their trade-offs is essential for accurate geographic interpretation and measurement.","keywords":["projections","distortion","cartography"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["GPS","GIS"],"full_text":"You may remember those big plastic maps that hung on your elementary school wall - the ones where Greenland was as big as South America. you've been messing around with mapping software and notice it puts New York about 600 miles further than it really is to Los Angeles. What's going on? Was your elementary school actually right? Is your computer telling you that L.A. is further out than you thought? This shows how you can get tripped up with maps. The short version of the problem: The world is round. Maps are flat. When map makers try to convert our sphere onto a flat sheet of paper - or computer screen - something has to give. Maps are always going to get something wrong. The trick is getting them to give you the right answer to your question.\n\nDon't hesitate to befriend a local geographer, perhaps at a university or planning agency, to make sure you are using the right projection.\n\nThe things that maps can get right or wrong is the shape of a region, its area, the distance from one place to another, and the direction.\n\nThe solution is picking the right projection. Every projection gets something right. And every projection gets something distorted. No projection can do all things perfectly all the time.\n\nThe mapping column in the March issue of Uplink showed how different maps use different calculations for the shape of the Earth. This is called datum. A latitude and longitude from one popular datum could be several hundred feet off from the datum used by your hand-held GPS unit. If you didn't know that, you could mistakenly report, say, that the scene of a flood was outside a town's official flood plain. That column outlined how to convert a map's datum.\n\nProjections\n\nProjections are different than datum, though they are often mentioned in the same breath. Datum are used for knowing where any measurement sits on the planet. Projections try to convert our planet into a flat map. Every electronic map has underlying datum. But not every map is projected. And projections - which are essentially a lot of math - are always built on top of some underlying datum.\n\nThere are hundreds of projections. And the things they try to get right, as they flatten our planet onto our computer screen, are: area, distance, direction and shape.\n\nThe map on your elementary school wall, with Greenland the size of South America, was not doing a good job of representing area. Put a coin on part of Greenland, and it might cover eight times the amount of land in real life, than if you put that same coin on part of South America. The computerized map that calculated New York much further than reality to Los Angeles was distorting distance.\n\nYet these same maps may do an excellent job of preserving shape and direction. Sailors used these kinds of maps to navigate. They could follow a straight line on their map and stay on course as they curved around the world.\n\nIf you've negotiated for a map, and a government agency tells you they keep all their maps in a State Plane coordinate system, that's a cue that the agency's maps are already projected.\n\nFor instance, a North Carolina map using a state plane coordinate system uses a projection called Lambert Conformal Conic. It's a projection that works well for elongated states. It does a decent job for measuring distances, and has minimal distortion of shape and area in small settings. But the direction from the northwest corner of the state down to the southeast corner is distorted; it's an arc, curving around the planet, rather than a straight line.\n\nTo make matters more confusing, not every State Plane uses the same projection, so you would have trouble edge matching a series of State Plane maps if you don't run conversions.\n\nIf you've heard that a map's projection is \"Geographic\" - like many U.S. Census Bureau geographic files - that means they are not projected. You'll need to project them depending on the analysis you want to do.\n\nIn any case, be sure to check out a map's metadata - the detailed file describing any electronic map - to learn about its projection before any conversion.\n\nConverting maps\n\nDon't convert your map if you don't need to. Think about the analysis you need to undertake. The reason is that every conversion involves estimation, and the more you convert a map from its original projection, the more locations could be imprecise.\n\nSometimes the name of the projection tells you what it does best - \"cylindrical equidistant\" or \"Albers equal area.\" Direction tends to be best preserved with azimuthal maps. Shape is represented well with conformal maps.\n\nKeep in mind that there is always some amount of estimation involved in using any projection, so it's important to be familiar with the projection you are using.\n\nIf you are only analyzing a small area, you may be able to get by using just your local Universal Transverse Mercator zone or State Plane coordinates. Keep in mind that there is always some amount of estimation involved in using any projection, so it's important to be familiar with the projection you are using. Also, to spare you a shock: a given projection will no longer refer to a point in its latitude and longitude, but in its own coordinate system.\n\nSome handy reference books are listed on the margins of this column. Also, don't hesitate to befriend a local geographer, perhaps at a university or planning agency, to make sure you are using the right projection.\n\nAnalysis\n\nWhen it comes to analyzing data, you'll probably be most concerned with preserving distance and area. Distance would be a concern when, say, you want to buffer to determine what's within some miles of a hazardous chemical spill. Area would be an issue when, for instance, you want to calculate the population density within that buffer.\n\nAs discussed in the March column, the new version of ArcView comes with something called the Projection Utility. You can run it separately from ArcView. Just click on the icon on your desktop.\n\nOnce you're in the utility, the first step is to point and click your way to the shape file you want to convert. Hit the next button. For step two, first check the box that says \"Show Advanced Options.\"\n\nLet's assume the data is not projected, so keep the coordinate system at \"Geographic.\" Click the datum tab and highlight the correct datum, say North American 1927. Hit the next key. For step three, select the projection you want to convert to, like a particular UTM zone.\n\nThe selection should automatically fill in the correct datum, but it's still a good idea to check and be sure. Hit next. Step four is simply to navigate to where you want to store your converted files. Now you can hit the next key and then finish. The utility will create a reference file that keeps track of the projection and datum for your new file.\n\nThe conversion utility is not perfect. Those who wrote the code for this utility noted it may be off by a meter or more.\n\nAndy Lehren can be reached by e-mail at alehren@nbc.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-track-federal-funds","slug":"2000-june-track-federal-funds","url":"https://dwillis.github.io/uplink/article/2000-june-track-federal-funds.html","year":2000,"month":"June","monthNum":6,"headline":"Track federal funds","kicker":"THE MONEY TRAIL","author_name":"John Wilkerson","author_title":"","affiliation":"University of Missouri","summary":"The Consolidated Federal Funds Report (CFFR) data, compiled by the Bureau of the Census, provides detailed information about federal government expenditures across states, counties, and municipalities, enabling journalists to track funding patterns and add depth to local reporting investigations.","keywords":["funding","federal","data"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["CFFR","FAADS","Federal Procurement Data System"],"full_text":"Following the money is one of the mainstays in investigative journalism. When following money through the government, federal funding is often the spring that feeds the flow of money through all levels of government. Analyzing the Consolidated Federal Funds Report data (CFFR) can help put state, county and city government funding into perspective and give local reporting added depth.\n\nThe Bureau of the Census collects data from other local and state agencies and compiles the data into the CFFR. The data covers the Federal Government's expenditures and obligations at the state, county and subcounty (municipality, township) level.\n\nEach geographic level is identified by name and a Federal Information Processing Standard (FIPS) code. County and Subcounty FIPS codes are only unique on the state level. For example, there are 50 states with the FIPS county code \"001,\" so if you are running a query on a county with the code \"001\" you will be running a query for all 50 counties in the US with that code if you do not SELECT both fields. The same is true for the FIPS subcounty code field.\n\nAnalyzing the Consolidated Federal Funds Report data can help put state, county and city government funding into perspective and give local reporting added depth.\n\nCFFR data also includes:\n\nCongressional district codes that identify where money is spent.\nObject codes, which are broad classifications of what money is spent on, such as grants, procurement contracts and insurance. There are nine object codes, and they generally correspond with the classifications used in the annual federal budget.\nProgram codes that identify the various programs money is spent on.\nAgency codes that identify what agency received money.\nPopulation of the geographic area that received money.\n\nThe data does not tell you who or what company receives money. This kind of information can be found in the Federal Assistance Awards Data System (FAADS) and the Federal Procurement Data System. NICAR carries the procurement data, and FAADS can also be downloaded at www.census.gov/govs/wwwlfaads.html\n\nNICAR has CFFR data available from 1986. The data has changed over the years, but 1986 was the first year that the federal funds report accounted for most grants on the local level.\n\nJerry Zremski of The Buffalo News analyzed all 13 years of the data to get an historical, geographic understanding of the data. The Buffalo News series showed that showed Buffalo is part of a swath from the Northeast to the Midwest that has been receiving less and less money. At the same time, the federal government has been feeding more money into the Sunbelt.\n\nA small number of the programs in the data are recorded on the state level even though the money for these programs is allocated on a county and local level. You can find these programs listed in a book put out by the General Service Administration.\n\nIn other instances, where the money ends up can be deceiving. Frank Kummer of the New Jersey Courier-Post was looking for towns that had received the most federal money, and how it was spent. He found that in some cases, federal money is recorded as going to a town or municipality but doesn't get used there. Kummer found that Paulsboro, N.J., a run-down town, accounted for a big part of federal spending. This was because a government contractor happened to be located in Paulsboro. The money passed through the company but was dispersed throughout the area.\n\nWhat you should know about CFFR data:\nProgram codes change. For program additions/deletions/changes, go to: www.cfda.gov/public/cat-changes.htm\nThe population field is not accurate, especially on the county and subcounty level. This is because the population figures come from different years. If you want to calculate per capita information, use the population figures at: www.census.gov/population/wwwlestimates/co_99_1.html\n\nFor any questions concerning the CFFR data, there is an in-depth record layout at: www.census.gov/govs/www/cffr99.html\n\nKummer also points out that CFFR data tells us only where the federal government spends money, not where it gets money John Wilkerson can be reached by e-mail at gnilrits2@yahoo.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-covering-tax-assessments","slug":"2000-june-covering-tax-assessments","url":"https://dwillis.github.io/uplink/article/2000-june-covering-tax-assessments.html","year":2000,"month":"June","monthNum":6,"headline":"Covering tax assessments:","kicker":"","author_name":"","author_title":"","affiliation":"Newsday","summary":"Newsday provides resources for exploring property tax assessment disparities, including investigative stories about uneven tax burdens and links to national assessment data and professional organizations.","keywords":["taxes","assessment","property"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":["World Wide Web"],"full_text":"Newsday's property-tax assessment stories are at www.newsday.com/speciall assess/taxmain.htm. For this story, click on \"Uneven Tax Burden\" on that page.\n\nFind sources for assessment around the country at www.orps.state.ny.us/reflusamap/index.htm The site also has links to professional organizations, such as the International Association of Assessing Officers.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-exposing-flaws","slug":"2000-june-exposing-flaws","url":"https://dwillis.github.io/uplink/article/2000-june-exposing-flaws.html","year":2000,"month":"June","monthNum":6,"headline":"Exposing flaws","kicker":"TAXES","author_name":"David Slade","author_title":"","affiliation":"The Morning Call","summary":"Journalists can use county tax assessment databases to expose unfair and inaccurate property valuations that create disparities in taxation across similar properties, as demonstrated by investigations in Pennsylvania and Ohio counties.","keywords":["assessment","databases","taxation"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Access","Excel","9-track tape"],"full_text":"Newspapers routinely devote ink to stories about property tax increases and property tax reform, but sometimes overlook the question of whether properties are correctly and fairly valued for taxation. Often they are not, and you can prove it using the databases of tax assessments maintained by county governments in any state. In Pennsylvania, counties aren't required to update property values, and they routinely go decades without a reassessment. As a result, properties carry tax assessments bearing little relation to their relative value. (See Richard Dalton's story on page one) Even in states like Ohio that do require regular reassessments of property values, there can be wide disparities, as The Columbus Dispatch reported in two-day series in March. In the two counties I have examined in Pennsylvania, people who owned similar homes purchased for similar amounts often had substantially different assessments, meaning they were effectively paying different tax rates.\n\nThe database gave us the facts, but also gave us the names of property owners, whose comments helped tell the story.\n\nI was part of a reporting team at The Scranton Times that analyzed tax assessments in Lackawanna County in 1997. We found so many examples of assessments that were clearly uneven and unfair that it was hard to decide which ones to use.\n\nWaiting to be told\n\nIn county after county throughout the United States, the same stories are waiting to be told. The databases can expose serious flaws that go to the heart of local systems of taxation, and even when they don't, they are tremendous resources.\n\nThe databases typically show who owns every property in a county, what they pay in taxes, when they bought the property, and what they paid for it, the mailing address of the property owner, information on tax-exempt properties, and more.\n\nAt The Scranton Times we used Microsoft Access and Excel to search and sort Lackawanna County's database. Most of our useful information came from comparing purchase prices of recently sold properties to tax assessments that were in many cases set in the 1960s, and from researching properties with unusually high or low assessments. The database gave us the facts, but also gave us the names of property owners, whose comments helped tell the story. Most of the people interviewed had no idea their properties were over- or under assessed, due to their confusion about assessments, mileage rates and taxes. We also used the database to check out tips about people whose homes weren't taxed at all, like the brother of a local tax collector. And we found countless properties that were over- or under assessed due to mistakes or deliberate policies of the county assessor's office. I'm now with The Morning Call, where I'm part of a team covering Carbon County's first tax reassessment in 31 years. Among other things, we'll be using assessment databases from before, during and after the reassessment to track the winners and losers as the property tax burden shifts.\n\nWhat to expect\n\nHere are some things you might encounter if you decide to do a tax assessment database project: Counties to overcharge for the data.\n\nWe'll also look at the flaws in the previous system, which are similar to the problems in Lackawanna County, but we won't be focusing on them because more than 600 property owners already demonstrated that the system was beyond repair and convinced a judge to order the reassessment.\n\nThe assessment series that ran in The Scranton Times can be found at www.nepanews.com/stories/Tax%20assesment/taxseries.htm\n\nCoverage of Carbon County's reassessment can be found on The Morning Call's Web site, www.mcall.com.\n\nThe Scranton Times bought Lackawanna County's database on nine-track tape for $200. The Morning Call, after initially being asked to pay about $1,500 for a 46,000-record database in Carbon County, paid $250 after threatening to sue over the cost.\n\nDirty data. In counties that haven't reassessed for 30 years, much of the information in the database will have been typed in from decades-old paper files. In the assessment databases I've worked with the name and street address fields were incomplete and full of misspellings, and other fields, such as purchase price and property size, were often blank.\n\nRegardless of appeals, it's worth checking the assessments of local big businesses and politicians.\n\nThe good news is that properties are typically identified by a unique number, like a social security number for properties, that ties each record to a map location and property record.\n\nThe need to check paper files. Plan to check the county's paper files on each individual property you plan to write about. Between the time you request the data and the time you report on it, assessments and property owners can change. Also, the paper records often contained more information than the electronic ones. Lackawanna County's paper records included handwritten notes. My favorite paper record in Lackawanna County was an assessor's card noting that a house had a low, reduced assessment because it had burned down. A more recent note on the same card showed that the house, which we found had been renovated, had just sold for more than $100,000.\n\nLearn the local rules.\n\nCounties often have quirky assessment rules, often dating back decades. Knowing them before you start searching the data will save you time, and some of the rules might be worth a story.\n\nFor example, in Lackawanna County corner lots are assessed at 25 percent more than non-corner lots. And church rectories are assessed at 10 percent of true value.\n\nQuick tips\n\nIn Pennsylvania, talk to the State Tax Equalization Board to get the big picture and learn about the common-level ratio, a formula used to level out old assessments with new ones. Also check with the Pennsylvania Economy League, which keeps track of property tax reassessments. Similar organizations may exist in your state.\nCheck the courthouse for lawsuits against the county assessor's office. Large companies often appeal tax assessments.\nSort your database by municipality, or by school district, to see how property taxes fall in different parts of the same county. Because assessments don't change when a property is sold in Pennsylvania, areas that have prospered since the last reassessment tend to have relatively lower assessments than areas that have not.\nCheck out the top businesses and politicians.\n\nIn Pennsylvania, tax assessments can be reduced by appeals boards filled with political appointees, so it's worth checking to see who has received a break.\n\nRegardless of appeals, it's worth checking the assessments of local big businesses and politicians. In Lackawanna County, I discovered through the database that two huge landfills were each paying less in taxes than a typical large retailer would. One landfill, worth an estimated $600 million, was assessed at a lower value than a local hospital's parking garage because the county assessor's office considered landfills to be unimproved land - a position that assessors elsewhere scoffed at.\n\nDavid Slade can be reached by e-mail at david.slade@mcall.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-the-perl-y-gates","slug":"2000-june-the-perl-y-gates","url":"https://dwillis.github.io/uplink/article/2000-june-the-perl-y-gates.html","year":2000,"month":"June","monthNum":6,"headline":"The Perl-y gates","kicker":"PROGRAMMING","author_name":"Matthew Ericson","author_title":"Reporter","affiliation":"The Philadelphia Inquirer","summary":"A programming competition demonstrates Perl's superior speed and efficiency for data parsing compared to VBScript, highlighting why Perl has become an essential tool for computer-assisted reporting and web development.","keywords":["Perl","programming","parsing"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Perl","Visual Basic","SQL Server","ODBC","ADO","Unix"],"full_text":"Ok, it wasn't as thrilling as the chariot race in Ben Hur, but we found it exciting anyway. Tom Torok, then The Inquirer's Visual Basic guru, and I, a Perl fanatic, squared off to see whose programming language could most quickly parse a report into a tab-delimited format. The results, as reported by Torok to the NICAR mailing list: \"Matt Ericson and I just had a Perl vs. VBScript race on parsing a 130 meg report-form file into a tab-delimited file suitable for import into SQL Server.\n\nThe outcome: Matt (and Perl) beat the pants off Torok (and VBScript). His Perl script parsed nearly 750,000 records in less than half the time (with less than half the code.)\n\nProgrammers discovered Perl was a great language to use when creating dynamic web pages - it's also a great tool for cleaning up data in computer-assisted reporting.\n\nI'm not only sold, but I also have to dress like a chicken for a week.\"\n\nPerl - \"Practical Extraction and Report Language\" or \"Pathologically Eclectic Rubbish Lister\" - is a programming language foreign to most users of Microsoft Windows. It started as a utility to read text files and produce reports on Unix-based computer systems. With the growth of the World Wide Web, programmers discovered it was a great language to use when creating dynamic Web pages.\n\nHere's a few reasons it works so well:\n\nIt's also a great tool for cleaning up data in the computer-assisted reporting world.\n\nQuick, free and competent\n\nPerl has a wide range of built-in functions for dealing with data in many formats-fixed-width text files, delimited text files, queries from databases - and then dealing efficiently with the data. You don't have to worry about whether you've defined your variables as the right types - all variables are the same type in Perl - so you can use the same variable once as a number and later as a string. Perl doesn't care - it converts it transparently. A string that grows longer than 255 characters? Perl will expand it on the fly. The same goes for arrays - they can grow as big as needed. And while it's not as fast a language as C, it more than holds its own with Visual Basic. And, unlike C, you can write useful, short programs quickly and easily. Plus, it's free. How can you beat that? (Download it from www.perl.com.)\n\nData in many ways\n\nPerl has myriad ways to read data in text files. By default, it reads line-by-line. But if a record spans multiple lines, you can change the end-of-record delimiter. Or, read the whole file in one fell swoop.\n\nIs the data tab-delimited? Then use Perl's split function to break a record into its respective fields.\n\nDelimited by something other than tabs? The split will handle that, too. Just tell it what character (or characters) is the delimiter.\n\nIs the data fixed-width? Use the unpack function to pull it apart.\n\nPerl has drivers to access most types of databases. There are drivers for Microsoft SQL Server, Oracle, Sybase and others.\n\nPlus, if you're using Perl on Microsoft Windows, you can use ODBC and ADO just like you can in Visual Basic (with a few slight syntax changes). And, with SQL Server, Perl is an option for writing Data Transformation Services scripts.\n\nPowerful search and replace\n\nPerl's strongest feature is its use of \"regular expressions,\" a fancy Unix term for an extremely powerful search-and-replace mechanism.\n\nUnlike most programs, where search-and-replace means searching for a word and replacing with another word, regular expressions let you search for patterns of characters.\n\nIt can, for example, find a pattern of 2 digits followed by a letter and then use pieces of that search in the replacement - taking that letter and putting it before the 2 digits, for example.\n\nPowerful associative arrays\n\nAnother unusual Perl feature is its associative arrays - arrays that are indexed off of a string, rather than a number.\n\nMost programming languages let you store data in arrays: you put one piece of data in the first element of the array, another piece of data in the second element, and so on. Associative arrays, also called \"hashes,\" are arrays that are indexed not on integers, but on arbitrary string values.\n\nAssociative arrays: For example, the array here is what you might store in an array named \"color:\" $color{\"apple\"} = \"Red\"; $color{\"banana\"} = \"Yellow\"; $color{\"grape\"} = \"Purple\"; If you are accessing a database of fruit, you can find the color of the fruit by looking in the associative array. If the current record in the database is for an apple, and has a field named name_of_fruit, then $color{$name_of_fruit} will return \"Red.\"\n\nAssociative arrays let you build lookup tables on the fly, and you can also use associative arrays to count things. If you are parsing a text file containing street addresses, you can keep track of how often a street appears in the file by adding 1 to the array for each street name:\n\n($count{$street} = $count{$street} + 1) When you're done with the file, to find out how often Broad Street appeared, print out $count{'BROAD STREET'}. Sure, it's not going to replace Group By queries in SQL, but for quick-and-dirty counting, it can't be beat.\n\nIt flies on the Web\n\nMany of the same functions that make Perl superb for data manipulation also make it a great language for generating dynamic Web pages. On Microsoft Windows computers, Perl can be embedded in Active Server Pages just like VBScript. Plus, you get access to all of the ASP objects - the Request objects to let you process data from an HTML form, the Server objects to open connections to ODBC and ADO databases, and so forth.\n\nThe bad news\n\nWhat's the catch? The most intimidating thing to Perl newcomers is its cryptic syntax. If you've grown up with C and Unix utilities, Perl will seem right at home. However, if you're coming from a DOS and Visual Basic background, the abundance of strange characters - all those tildes, curly braces, carets, slashes, backslashes and so forth be downright befuddling. You end up with commands such as: foreach $a @data ) { $a s/^ +//; This makes perfect sense to Perl programmers, but resembles nothing you ever saw in Visual Basic. (FYI: that command removes any extra spaces from the start of each element in an array.) It's worth it, though. Take the time to learn Perl and soon you'll find yourself cringing when someone suggests you write a script or Web page in Visual Basic. (Especially when you find that one-line Perl script can sometimes do more than 20 lines of VB.)\n\nMore resources:\n\nFor beginners using Microsoft Windows systems, Learning Perl on Win32 Systems, by Randal L. Schwartz, offers a good place to start. It's published by O'Reilly and Associates.\nThe bible of the language, Programming Perl, was written by Larry Wall, the language's creator, and is probably the most complete guide to the language. Also published by O'Reilly, it's commonly known as \"the camel book\" because of the picture of the camel on the cover. Compared to Learning Perl, it's less of a tutorial and more of a reference manual. While it assumes some Unix familiarity, it's still excellent for users who haven't used Unix.\n\nMatthew Ericson can be reached by e-mail at mericson@phillynews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-fronts-for-forms","slug":"2000-june-fronts-for-forms","url":"https://dwillis.github.io/uplink/article/2000-june-fronts-for-forms.html","year":2000,"month":"June","monthNum":6,"headline":"Fronts for forms","kicker":"TECH TIP","author_name":"David Herzog","author_title":"","affiliation":"NICAR","summary":"This tutorial teaches journalists how to create a startup screen in Microsoft Access by building a Welcome form and using macros to launch it automatically when the database opens.","keywords":["Access","forms","macros"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Access","Visual Basic"],"full_text":"Once you've learned how to use Microsoft Access to build forms for entering and querying data (See Uplink, April 2000), you can link them to create a program for sharing data with other reporters.\n\nIn this exercise, we'll build upon the chronology we created as part of the April Tech Tip by linking it to a startup screen. If you need a copy of the database we used last month, you can download it from the NICAR web site (www.nicar.org).\n\nBefore getting into the how-to, it's important to understand the structure of an Access database. An Access database is a container for objects. Tables, forms, queries and reports are the most commonly used objects. Access also includes macros (sequences of defined actions) and modules (Visual Basic code) to help you control programs. We'll use macros, which are powerful enough for our program.\n\nCreate the startup screen\n\nBegin by opening the program and creating a new form. Once you've opened the database, click on the Forms tab, then New. Make sure Design View is selected, then click OK and a blank form appears. This blank form is your canvas for creating a startup screen. By setting the form's properties, you can change the color. By clicking buttons on the toolbar you can add text and graphics.\n\nAn Access database is a container for objects. Tables, forms, queries and reports are the most commonly used objects.\n\nFirst, let's make sure that this startup screen launches whenever we start the program. Close the form and save it; call it Welcome. Now we will use a macro to make it the startup screen. Click on the Macros tab, then pick New. The Macro design grid opens.\n\nIn the Action column we well specify what we want the macro to do.\n\nClick on the drop down arrow and scroll down the list until you find OpenForm. Select it. Then, in the Action Arguments section on the bottom part of the screen, click in the Form Name box and select Welcome from the list. Keep all other settings as they are. Close the macro design grid and name it AutoExec. When Access opens a database, it looks to see the whether there is an AutoExec macro. If there is, Access runs it. Test your macro by closing your database, then opening it again.\n\nOpen the Welcome form again by clicking on the Forms tab, highlighting Welcome and clicking the Design button. Let's modify the form's appearance.\n\nChange the color by clicking once anywhere on the form with the right mouse button to get the pop-up menu. Select Properties. Click the Format tab to get a list of the formatting properties you can set. Click on Back Color, then the ellipsis button to get a color palette. Select a color.\n\nNow we will set the properties of the form to remove some of the frills. Click once on the form select box, the small square located at the intersection of the rulers at the top left corner. Click the right mouse button to get the pop-up menu and select Properties. Click on the Format tab to set the following properties:\n\nViews allowed: Form\nScroll bars: Neither\nRecord selectors: No\nNavigation buttons: No\nControl Box: No\nClose the form properties box.\n\nGive the form a title by clicking on the Label button on the toolbox, then clicking on the form where you would like the title to go. Type \"Digging for data\" without the quotation marks, then hit Enter. Look at the text on your form. Obviously, this could look better. We can change the text properties by right-clicking the text, then clicking Properties in the pop-up menu. Use the Fore Color, Font Name, Font Size, Font Weight, Font Italic, Font Underline and Text Align properties to modify your label's appearance.\n\nNow add a picture to your form. I grabbed the logo from NICAR's web site. Here's how I added it to the form: First I clicked on the Image button on the toolbox. Then, using the mouse, I drew a frame for the logo by left clicking on the form, dragging diagonally and releasing the button. I added the logo by finding the graphic file on my computer.\n\nButton up\n\nSo far, all we've done is work on the window dressing. It's time to put the form to work with some buttons.\n\nThe first button will shut Access down. Click on the Command Button item on the toolbox, then use the cross to draw the shape of your button, just as you did to add a frame for your graphic. When you release the mouse button, the Command Button Wizard starts. The wizard makes creating command buttons a snap. In the Categories box, pick Application. In the Actions box, pick Quit Application. Then click on the Next button. In the next step, select the type of button you want, then click Next. At the next screen, click Finish.\n\nTest your command button by switching to form view, then clicking it to see what happens. Start the database again and open the Welcome form in design view, where we'll add a button that minimizes the startup screen and opens the Chronology form. We will link the button to a new macro that will manipulate the forms.\n\nClick on the Macros tab and select New. In the first Action cell, click Minimize. This minimizes the active window (the Welcome form, in our case). In the second Action cell, select Open Form. In the Action Arguments section at the bottom of the screen, select Chronology in the Form Name option. Close the macro design view and give it a logical name, such as open_chrono.\n\nGo back to the Welcome form in design view. Add a command button using the Command Button item on the toolbox. Draw a button on the form using your mouse and release the button to launch the Command Button Wizard. In the Categories box, select Miscellaneous, and in the Actions box select Run Macro and click the Next button. Select \"open_chrono\" as the macro to run and click the Next button. Design the button by clicking on Text and typing \"Chronology\" without the quotation marks. Click Next, then Finish.\n\nTest your work by switching to the Welcome form's form view. Click on the Chronology button and see what happens. The Welcome form should shrink and the Chronology form should open. Now click on the Close button at the bottom of the Chronology form.\n\nThe Chronology form closes and the Welcome form remains minimized. We will add a macro and a button to make sure the Chronology form also re-opens the Welcome form. Create a new macro by clicking on the Macros tab, then clicking New. In the first Actions cell, select Close. In Action Arguments section, select Form as the Object Type and Chronology as the Object Name. In the second Actions cell, select Open Form. In the Action Arguments section, pick Welcome as the form name. Close the macro design view and save this as close chrono.\n\nNow we need to link the macro to a new button on the Chronology form. Open the Chronology form in design view. Click on its close button and then hit the delete key to get rid of it. Create a new button by clicking on the Command Button item on the toolbox and drawing the outline of the button using the mouse. When you release the mouse button, the Command Button Wizard starts. Pick Miscellaneous from Categories and Run Macro from Actions and click Next. Then select \"close_chrono\" from the list of macros and click Next. Select the style of button you want and click Next, then Finish. Close your form and save it.\n\nIt's time to see how everything works together. Close Access and open the database again. Use the command buttons to navigate between the Welcome form and Chronology form.\n\nThere's more to know about creating programs to share data (such as how to secure the program on a network). But using the principles in this Tech Tip and April's Tech Tip, you can add all the tables you need and build more forms for adding and querying data.\n\nDavid Herzog can be reached by e-mail at david_herzog@projo.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-bits-bytes-and-barks","slug":"2000-june-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2000-june-bits-bytes-and-barks.html","year":2000,"month":"June","monthNum":6,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR announce an upcoming training calendar featuring computer-assisted reporting bootcamps, census seminars, mapping workshops, and a national CAR conference across multiple locations through December.","keywords":["training","reporting","conference"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Bits, Bytes and Barks\n\nIRE and NICAR training calendar\n\nHere's a glimpse at some of the training opportunities IRE and NICAR will be offering in the coming months. Check out the IRE Web site, www.ire.org/training, for more events, details and registration information.\n\nAug. 4-5: Census Seminar, Columbia, Mo.\n\nAug. 6-11: CAR Bootcamp, Columbia, Mo.\n\nSept. 14-17: National Computer Assisted-Reporting Conference, Lexington, Ky. Registration forms and more information are available on the IRE Web site training page.\n\nSept. 29-Oct. 1: CAR workshop, jointly sponsored with the University of Florida, Gainesville, Fl.\n\nOct. 20-22: Mapping Bootcamp, Columbia, Mo.\n\nNov. 12-17: CAR Bootcamp, jointly sponsored with The Poynter Institute, St. Petersburg, Fl.\n\nDec. 8-10: Census Seminar, College Park, Md.\n\nMore information is also available by calling (573) 882-0684.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-data-update","slug":"2000-june-data-update","url":"https://dwillis.github.io/uplink/article/2000-june-data-update.html","year":2000,"month":"June","monthNum":6,"headline":"Data update","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"The NICAR database library has updated the National Bridge Inventory System database with bridge inspection records from 1998 and 1999, including maintenance data collected by the Federal Highway Administration. The dataset is available for purchase at various price points depending on circulation size and market rank.","keywords":["bridges","database","inspection"],"topics":["Data & Databases"],"technologies":[],"full_text":"The NICAR database library has updated the National Bridge Inventory System database to include all of the 1998 inspections as well some 1999 inspections through December 1999.\n\nThis dataset includes maintenance records collected by the Federal Highway Administration for assessing the nation's bridges. It includes information about the bridge's structure, repair and last inspection.\n\nData for the entire U.S. costs $100 (under 50,000 circulation or 50-200 market), $125 (50-100,000 circulation. or 25-50 market), or $150 (100,000-plus or top-25 market). You can also order per state for $50 (Under 50,000 circ. or 50-200 market), $75 (50-100,000 circulation. or 25-50 market) or $100 (100,000-plus or top-25 market).\n\nContact the data library to order this or any other data set by calling (573) 884-7332. More information is also available at www.nicar.org/data/bridgesi","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-june-nicar-on-the-web","slug":"2000-june-nicar-on-the-web","url":"https://dwillis.github.io/uplink/article/2000-june-nicar-on-the-web.html","year":2000,"month":"June","monthNum":6,"headline":"NICAR on the Web","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR has updated its data library website with improved information about available databases for journalists, including descriptions, sample data, record layouts, pricing, and references to related stories and resources.","keywords":["databases","journalists","resources"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"The NICAR data library has recently finished updating the information on its Web site about the databases available to journalists. Check out the improved site at www.nicar.org/datalibrary\n\nFor each database, there is a detailed description of the data, sample slices and record layouts for each table and information about how much it will cost.\n\nWe've also added useful information about stories others have done on this topic or using this particular data. It includes references to IRE Resource Center stories and/or tipsheets that might be useful when getting started on a similar story.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-driving-the-big-car","slug":"2000-julyaugust-driving-the-big-car","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-driving-the-big-car.html","year":2000,"month":"July/August","monthNum":7,"headline":"Driving the big CAR","kicker":"SMALL NEWS ORGANIZATIONS","author_name":"Sean P. Johnson","author_title":"City Editor","affiliation":"Oshkosh Northwestern","summary":"The article discusses the importance of introducing Computer-Assisted Reporting (CAR) in small newspapers, emphasizing that CAR is not only for big papers but can be beneficial for smaller newsrooms as well.","keywords":["CAR","small newspapers","reporting"],"topics":["Investigative Methods","Government & Politics","Journalism & Newsroom"],"technologies":["Excel","Access"],"full_text":"Sit down. Relax. Take a deep breath. Now, search the recesses of your mind for that perennial favorite children's story about the little blue engine. Remember what he said: \"I think I can. I think I can\" all the way up the hill and \"I thought I could. I thought I could\" all the way down. For small newspapers everywhere, this must be your mantra for introducing CAR to your newsrooms. Immediately banish from anyone's thinking the notion that CAR is strictly for big papers or big projects. It is a tool that every reporter and editor in every newsroom can use every day if they approach it just like the little blue engine: You have to think you can do it and look for the opportunity, no matter how small it may seem. That has been our approach at the Oshkosh Northwestern, a 24,000 circulation daily in Wisconsin, and it is an approach that has worked exceptionally well. Our reporters use CAR skills on an almost daily basis, often without knowing they are doing it. Most of the time, the application is small, but it prepares our reporters for tackling the big project when it comes along. They are comfortable with using CAR and believe that they can. Here's how we have done it.\n\nStarting simple\n\nIt started out really as an effort to make life a little easier and more organized for everyone, but also provided a good primer on basic CAR skills. We encourage our reporters to do enterprise stories that cross beats. However, tracking down sources could sometimes be a pain if the reporter who normally covered a particular beat was not around to give you the information. So we came up with a solution to create a database of all sources used by all reporters on their beats. We had the reporters type their information into spreadsheets giving names, numbers and other pertinent information. We took the largest Excel file and converted it to an Access database, then appended the files to that. We then created a front-end using the form function of Access, then Presto! A database reporters can use on a daily basis to search for sources, numbers, and other information. The database can also be updated and edited through the front end.\n\nElections\n\nOne of the great strengths of CAR is the ability to quickly do math, organize information and show trends. One of the most helpful little projects taken on at the Northwestern is our election template, which is nothing more than a simple spreadsheet that does instant election math and analysis for us. I came up with the idea for the template shortly after my first election night as city editor. Oshkosh is divided into some 40 election wards, which are grouped together into 12 voting districts. Now, the problem reporters were dealing with was the way the wards and districts were organized. As the city had grown, wards had been added. While they may have been geographically contiguous, they were not numerically sequential. It all added up to chaos on election night, as reporters tried to add the right wards together to get district totals for maps, graphics, and analysis. Essentially, they were not able to do it on election night for the next morning’s paper, and even the day after, mistakes were often made. To fix the problem, I spent a couple of hours one afternoon setting up a spreadsheet that would automatically add the ward totals and report them out for the corresponding districts. I also created portions of the spreadsheet that would calculate percentage of the vote and percentage turnout. And it all began to happen as the reporter typed in the information. The result: instant election night analysis that allowed our reporters to write better election stories on deadline. We moved past reporting the basics of who won and how many votes to telling you where those votes came from and what that meant. In the April 1999 elections, the spreadsheets proved their worth when we were the only source covering a hotly contested judges race to report that not only was the race decided by fewer than 100 votes, but that the losing candidate had lost the election in his home ward. That same year, our coverage of Common Council elections was the only reporting that showed how local college kids had taken to the polls and ousted an incumbent and helped a newly formed neighborhood group put two of its candidates on the council. That's powerful stuff for readers, and it's all because of a simple spreadsheet.\n\nOn to bigger things\n\nOK. So we can do the simple stuff. But what about \"major projects?\" Well, it's that simple stuff that gave our reporters the ability to do just that. In 1998, the newspaper began working on a story, the effects of which are still being felt today. For several months, the Northwestern had questioned the city's handling of rental housing in this college town. Its regulation seemed ineffective. The potential for a disaster loomed. There was not an existing database of complaints to work with - only the paper inspection and notice logs kept by the city. However, because of the skills our reporters had gathered from creating the source database and the election spreadsheet and other smaller projects, they were confident enough to build their databases of those paper records. They first typed the information into spreadsheets, then converted it into database files. Then, they started looking for trends. What they found shocked a lot of people. Property owners with political connections or financial resources were simply able to skirt the rules. But they didn't stop there. Using census and budget information, they were able to tie the addresses of chronic problem properties to poorer areas of the city and show how the declining property values there were driving up the taxes of property owners everywhere. Now, that's a pretty good story right there. But the data work was only the beginning. Our reporters spent two months interviewing all the parties involved, using that data to confront those who denied there was a problem and to ask probing questions of all. The end result was a three-day series that no one challenged as containing any false information. The city, realizing it was not capable of policing the problem on its own, forged an agreement with the local prosecutor and the state to improve enforcement and criminally prosecute problem cases. Some landlords got out of the business, and those who remained began meeting with city inspectors regularly to review housing code issues. It was a big project accomplished by a small newspaper because reporters believed they could do it and had developed the skills to make it happen.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-julyaugust-case-dismissed","slug":"2000-julyaugust-case-dismissed","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-case-dismissed.html","year":2000,"month":"July/August","monthNum":7,"headline":"Case Dismissed","kicker":"AWARD WINNER","author_name":"Gerry Lanogosa","author_title":"Investigative Producer","affiliation":"WTHR-TV, Indianapolis","summary":"An investigative team explores if local law enforcement keeps its promise to enforce DUI laws by examining case dismissals when police fail to appear in court.","keywords":["DUI","law enforcement","court dismissals"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Access","Excel"],"full_text":"Local law enforcement agencies spend hundreds of thousands of dollars on public service announcements warning that drinking and driving will not be tolerated. It's pretty tough talk: \"We've been telling you, drive under the influence and you will lose your license and you will go to jail,\" said one spot that ran on local stations in Indianapolis. But do the police always keep that promise? Our team (Reporter Jeremy Rogalski, Photojournalist Bill Ditton, Investigative Producer Kathleen Johnston, and I) decided to find out. A tipster had told us a well-connected local lawyer was getting a large percentage of his DUI cases dismissed because police failed to show up in court. Sure enough, although we weren't able to prove any collusion, the lawyer turned out to be one of the top beneficiaries of police missing court dates. And our application of some very basic computer-assisted reporting principles allowed us to broaden our story by demonstrating a system-wide pattern of cases thrown out because of police no-shows.\n\nUsing a database obtained from the Marion County Justice Agency (our city's criminal justice information service), we found nearly one in 10 DUI cases dismissed in that manner. From the time we got our data, it took about three months to get our story on the air. Our dataset covered all alcohol-related cases (we separated DUI cases out later) from 1993 through 1998, including case ID numbers; charge codes and descriptions; case disposition codes; judges, attorneys and officers involved with each case, and personal data on the defendants. It took the county three months to pull it together. The law in our state allows agencies to charge for \"enhanced access\" requests like this. The local rate is a hard-to-swallow $50/hour for programming or processing. They didn't charge us for CPU time, but we ended up paying $650 for programming.\n\nThe records came to us via e-mail in three dozen Excel files containing more than a million records (about 151 MB). We imported them into an Access database for our analysis. After the initial clean-up, we had about 30,000 closed DUI cases. A few simple queries later, we determined that the number of dismissals because of police no-shows was about 25 percent. It was a tantalizing figure - and it was wrong.\n\nWe've done enough of these projects - and heard enough horror stories at IRE conferences - to know that you never go with the first set of numbers. After some sober reflection and some painful integrity checks - a process that took several weeks and numerous consultations with the county's data guru - we came up with a final number of about 10 percent. That's still a significant finding. Those integrity checks also involved going early and often to the paper files - that old-fashioned practice a dazzling database often tempts reporters to forget. By the time it was all over, we had pulled around three hundred hard-copy case files to make sure the data meant what it said. And we found that often it did not, which brings us to an overarching point that can't be overemphasized: Relentlessly question your data, and then be willing to accept the logical conclusions. Sometimes, that means throwing out cherished premises.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-julyaugust-kentucky-fried-car","slug":"2000-julyaugust-kentucky-fried-car","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-kentucky-fried-car.html","year":2000,"month":"July/August","monthNum":7,"headline":"Kentucky Fried CAR","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE's Kentucky Fried CAR conference will be held in Lexington from September 14-17, offering over 50 panels and 40 training sessions on computer-assisted reporting techniques for journalists of all levels. The event features expert-led sessions on topics ranging from Census data analysis and mapping software to investigating crime, government, and public records.","keywords":["conference","journalism","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Data & Databases"],"technologies":["Access","Excel","SPSS","SAS","ArcView","SQL","Perl"],"full_text":"More than 50 panels and another 40 hands-on training sessions will be offered at IRE's national computer-assisted reporting conference in Lexington, Kentucky, Sept. 14 through Sept. 17.\n\nThe Thursday through Sunday conference, known as Kentucky Fried CAR is co-hosted by the Lexington Herald Leader. The conference will feature the leading practitioners and have sessions for the beginner and for the expert; for the reporter and for the editor; and for the print journalist, the broadcast journalist, and the online journalist. Conference sponsors include the Gannett Foundation, the Scripps-Howard Foundation, Knight-Ridder, and WKYT-TV. The conference will take place at the Hyatt Regency in Lexington where rooms start at only $104 a night. We plan to have breakout sessions for people interested in special topics and lots of opportunities for small workshops in the computer demonstration room. And of course there will be opportunities to appreciate the attractions of Kentucky, from horses to bluegrass to distilleries. To register for the conference, which costs only $150 plus $50 for non-members, you can go to www.ire.org/training/lexington or you can call IRE at 573-882-2042. The conference will include a special batch of sessions on the upcoming Census data. The sessions will be led by experts Steve Doig of Arizona State University and Paul Overberg of USA Today. There will also be a track of using mapping software and a close look at the best and safest use of statistics.\n\nOther topics\n\nRacial profiling\nGetting ready for the hurricane season and other natural disasters\nThe gun trade\nDoing your first CAR story\nDeciphering property records\nFinding the bad doctors\nUsing CAR effectively at small newspapers\nEditing CAR stories\nWhen the plane crashes: From bad rudders to runway incursions\nTraining the newsroom\nThe latest, greatest CAR stories\nUnlocking the secrets of the prison industry\nCool stuff. The latest in practical software\nTeaching CAR at the university\nThe recipe for illness: Covering the food industry\nDriving CAR on the highway beat\nCampaign finance: What to do after the election\n20 shortcuts to finding what you need on the Web\nCAR and broadcast: Getting past the time machine\nChildren at risk: Daycare and foster care\nDecrypting death records: Ingenious uses\nCrime and violence: What data to use\nBeyond the city limit: International data for local stories\nTracking venture capital\nMeasuring school test scores: More ways to an A\nCAR for the statehouse\nPlundering environmental data\nPublic employees: Absenteeism, benies and other snafus\nElderly care: Understanding the nursing home data\nLocal government: Budgets, bids, purchases and change orders\nThe Military Records Maze\nBuilding the Intranet for Your Newsroom\nUnveiling the hidden work of lobbyists\nSports CAR: Stadiums, teams and profits\nThe property scam: Landlords, land-flipping, and blight\nTracking an individual through on-line resources\nThe issues and ethics of public access and the need for privacy\n\nHands-on training\n\nYou can take up to three one-and-a-half-hour classes, which will include:\n\nDatabase managers: Beginning, Intermediate and Advanced (Taught in Access)\nSpreadsheets, Beginning, Intermediate and Advanced (Taught in Microsoft Excel)\nStatistics (Taught in SPSS with special sessions in SAS)\nMapping: Beginning, Intermediate, Advanced (Taught in ArcView)\nBasic primer in Structured Query Language\nUsing SQL Server\nA Primer in Perl\nData Cleaning: String functions and more","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-car-specialists","slug":"2000-julyaugust-car-specialists","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-car-specialists.html","year":2000,"month":"July/August","monthNum":7,"headline":"CAR specialists","kicker":"SMALL NEWS ORGANIZATIONS","author_name":"Craig Lincoln","author_title":"computer-assisted reporting specialist","affiliation":"Duluth News-Tribune","summary":"A computer-assisted reporting specialist argues that newspapers of all sizes need dedicated CAR staff to provide authoritative data-driven coverage, sharing lessons from his success at the Duluth News-Tribune.","keywords":["reporting","newspapers","data"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Photoshop","Internet"],"full_text":"Let us not dance around the issue any more: All newspapers, no matter how small, need a computer-assisted reporting specialist. To say otherwise is to say that newspapers don't need copy editors who know how to operate pagination software, graphics artists who know how to make a bar chart, or even photographers who can shoot straight and can work in Photoshop.\n\nThis is my conclusion after a decade of trying to talk editors at two newspapers into making CAR a big part of our reporting. I've tried doing it on the sly with the computer sitting on the desk of my publisher's secretary after she went home, on my home computer, and as a sideshow to busy beats. None of that worked well.\n\nWe're becoming the authoritative source on a lot of topics, such as the economy, that we simply quoted experts on in the past.\n\nI believe what we are doing at the Duluth News-Tribune (circulation in the 50,000 range) is working well enough to say it is the best way to go. My first job is taking care of computer-assisted reporting issues. My second is filling in on general assignment tasks when needed, just like any other reporter at a paper this size.\n\nAs a result, we're making CAR a regular part of the paper. We're giving our readers something that nobody else has in this market. We're becoming the authoritative source on a lot of topics, such as the economy, that we simply quoted experts on in the past.\n\nSo, I'll preach to the choir here. I hope I give you some ideas how to get this job if you are at a smaller newspaper and need ammunition. I'll tell you how we've succeeded here and how we've failed.\n\nIf you despair when thinking about the money required to get mapping and statistical software, take heart: We're succeeding without being anywhere near the cutting edge.\n\nDaily routine\n\nI'm sure my days are quite similar to CAR specialists at bigger newspapers, although I also sense that the daily demands have a lot more variety. Reporters will come to me for help converting files or press releases into a usable format. They'll ask a lot of questions about basic math, or basic file management. Most weeks, I'll search our warehoused data for someone's phone number, address or criminal history at least a few times.\n\nMostly, however, I do three main tasks: Generate and write stories, do computer work and analysis for other reporters, and help editors plan ahead well enough so we can include data analysis in our stories.\n\nWriting stories is the best part. For one thing, they can be fun. Each year, Duluth throws a big party that starts with a marathon. We've written about runners who have cut hours off their times, the demographics behind marathons' growth, and why our local marathon is financially successful. (It has more to do with the beer and bands after the marathon than the entry fees).\n\nFor another thing, I'm the reporter who gets to needle politicians about touchy subjects, too. For some reason, some of our officials think our region's population and economy are growing. The Census and our other reporting help us remind people (especially our mayor) that we aren't.\n\nOn the beat\n\nWe've had a lot of success adding computer work to our beat coverage. The state of Minnesota releases its test scores on the Internet, on the same day at the same time for everyone. We'd look like country bumpkins if we weren't able to download the data and crunch it into something meaningful.\n\nOur business team helped craft a package of economic indicators to track our region's economy. Now, every six months, we put out the most comprehensive and regular report on our economy that people have ever seen up here. It pays lots of dividends beyond helping our readers understand what's happening to their pocketbooks. We even have gained respect from our local university economists.\n\nFinally, each week, I swallow hard and spend an hour sequestered in a room with editors so I can find out what's coming up over the next several weeks. That way, I can try to make sure we plan well enough to do all the computer work we should.\n\nWe have used computers on a few big projects. One, in particular, won a Headliner Award and was a finalist in the IRE contest (see \"Death Beds,\" in the IRE Resource Center, story #15068). But I don't believe that's how we have made this beat an entrenched part of our culture.\n\nInstead, my beat evolved into something I never expected and wouldn't have wished for, except that it has been successful so far. I have developed a few rules to keep it successful.\n\nMuch of the success comes from a web of relationships that would be hard to dismantle.\n\nI thought my job would be judged mostly by the number of cool CAR stories I turned out. Nope. It's judged, it seems, by how much I spread around the help among the editors' teams.\n\nI work with the school reporters on test scores, the city reporters on property values, anybody who covers an election on campaign finances, the business reporters on a bunch of topics, the crime reporters looking for scofflaws, and - most important here in the north woods - the outdoor reporters wondering where to find big fish. Much of the success comes from a web of relationships that would be hard to dismantle.\n\nMy rules\n\nSo, my first rule: Don't be a Lone Ranger. I also spend a lot of time on small, simple tasks. We did a quick analysis of property values around bed and breakfasts, for example, and it's common for me to help other reporters find the right demographic information for an info box. Those good feelings go a long way, hence my second lesson: Think small. And I also spend a lot of time turning my hard-earned data analysis over to reporters, who make some phone calls and write the story and get their name first on the byline. When I'm cranky, it seems they have fun and get top billing, too. But again, they should remember that help in the future and hopefully editors will want more of it. So, my third lesson: Swallow your pride. It also would be nice to have an editor that has the time and expertise to dig for money and technology, but I don't. Instead, I spend far too much time politicking and trying to find out what happened to budget requests. And \"far too much time\" isn't much, as busy as I am.\n\nHence, my final lesson: Learn the art of the well-timed memo.\n\nWhich brings me to my final piece of advice, whether it's useful or not. As I mentioned early in this article, I tried every way I could think of to get computer-assisted reporting in my newspaper but none of them worked. I even tried to talk editors into making it my specialty for years.\n\nI finally succeeded with a well-timed memo. The message of the memo was simple.\n\nI made two points: We said we wanted sophisticated reporting in our newspaper. CAR, of course, is the best way to go, but we were failing because data analysis always was second priority to beat responsibilities.\n\nWe also faced new competitive pressures from a string of community newspapers surrounding us. We had added a lot of community coverage to the News-Tribune, but CAR was the way to give our readers something they couldn't get anywhere else. That would make it more likely we would prevail in the newspaper war.\n\nIn other words, I said we were failing to get enough sophisticated work into our newspaper. And that making me a CAR specialist would help us win the competitive battle, i.e., make more money. (OK. That's a marketing argument. But remember my third rule: Swallow your pride).\n\nIt worked.\n\nI also was lucky. I had some good editors at the time, good enough to understand where newspapers were headed. They didn't just want enough quotes to fill the white space. They actually wanted a newspaper.\n\nWe're far from being as far along as we should be. I haven't persuaded editors to make CAR training a required part of a reporter's experience here. We have been slow keeping up with technology over the past few years, and my pleas for mapping software haven't been successful yet.\n\nBut then the Census is coming up and I can't justify trying to report it without mapping software. I'm getting busier and busier and adding mapping to my duties means other people are going to have to start doing the simpler CAR work.\n\nI'm just waiting for the right time for my next memo.\n\nCraig Lincoln can be reached by e-mail at clincoln@duluthnews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-one-man-show","slug":"2000-julyaugust-one-man-show","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-one-man-show.html","year":2000,"month":"July/August","monthNum":7,"headline":"One-man show","kicker":"FIRST VENTURE","author_name":"Joe Kolman","author_title":"Reporter","affiliation":"The Billings Gazette","summary":"A Montana journalist describes how computer-assisted reporting (CAR) enabled him to investigate felons with guns and discover databases unknown to officials, positioning small-market papers to compete with larger news organizations despite challenges with data access and government cooperation.","keywords":["reporting","database","investigation"],"topics":["Investigative Methods","Crime & Justice","Technology"],"technologies":["Access","Excel"],"full_text":"One-man show\n\nBy Joe Kolman The Billings Gazette\n\nThe words of the Montana Department of Corrections spokesman made it all worthwhile: the intense week of NICAR Boot Camp, the hours of frustration inflicted on me by Access and Excel and the constant reassuring I had to give my editors that, yes, the story on felons with guns would be coming soon.\n\nI was checking details of a story I was about to publish that said as many as 100 of Montana's most dangerous ex-cons - including murderers, rapists and kidnappers bought hunting licenses last year, meaning they were in possession of rifles, shotguns or at the very least, archery equipment.\n\nState officials acknowledged that no one had ever done such a check before. But the Corrections spokesman said they would start doing so.\n\n\"We thought your idea was pretty darn good,\" the spokesman said.\n\nCAR puts me in a position to do stories that before were the domain of much larger papers with more time, reporters and resources.\n\nOf course, it wasn't my idea. The week I attended Boot Camp, the St. Paul Pioneer Press did a similar story. Full of inspiration upon my return, I knew that such a story would work in Montana, a vast state where there is hunting galore little computerized cooperation between agencies.\n\nThere is also very little computer-assisted reporting, which has been both good and bad.\n\nOpen field\n\nThe obvious benefit is that the field is wide open. The Gazette is the largest paper in the state with a circulation of just 55,000. None of the papers have CAR reporters. There has been little training offered.\n\nBut even in my limited experience, I realized that much meaningful work can be done here with CAR. I obtained Y2K reports from local, state and federal agencies that listed scads of databases no one knew existed.\n\nAs I had hoped would happen, CAR has opened up a whole new world of reporting for me. I've always loved rooting through documents and reports. But I frequently found myself wishing I had access to more information.\n\nThe opportunities presented by CAR make me want to stay at the office long into the evening. For me -- a one-man bureau almost 200 miles from the home puts me in a position to do stories that before were the domain of much larger papers with more time, reporters and resources.\n\nI found that I could work on data cleanup in between phone calls and daily stories. I've also found tons of help on the NICAR listserv. It is truly amazing to be sitting here cussing Bill Gates one moment and the next receiving instant help from CAR experts all over the country.\n\nOn the flip side\n\nDownsides include dealing with government officials who have never fielded electronic information requests. The responses ranged from flat refusals to asking me to sign a pledge that I would not use the information for mailing lists.\n\nOne state official was very concerned that I would make a mailing list out of his database of municipal sewer systems, as if moonlighted as a parts salesman for treatment facilities.\n\nTo be fair, other officials simply e-mailed information, no questions asked.\n\nFacing the editors\n\nOn the home front, it was difficult to explain to editors just what I was doing and why it was taking so long. It is hard to explain over the phone the process of my data negotiations and the hours it sometimes takes to clean up information once I receive it. Then there are the hours of reporting and writing not exactly what daily editors like to hear.\n\nAnd perhaps I erred by enticing editors with flashy stories like felon hunters and then, while I was working on that story, giving them small CAR stories that were not earth shattering. There seemed to be little interest in the less sexy stories such as which town had the most hunting and fishing licenses per capita or my finding of previously missing child support debtors by matching them up with hunting licenses.\n\nBut I was more than inspired enough and received enough encouragement to plow ahead. Heeding advice from the Boot Camp, I requested loads of information my first two weeks back. I continued to make at least one request a week so that I always had new records to examine.\n\nStory examples\n\nOther stories I've done in the last five months include:\n\nA study of the financial impacts of nonprofit groups in Montana.\nAn examination of the low fees the Forest Service charges for such uses in the region as cabins on public land.\nRevealing that the number of concealed weapon permits have doubled over the last five years and are held by scores of state employees, including the chief justice of the Supreme Court and three lawmakers.\nFinding that Montana leads the country in the number of ice jams on rivers over the last century and also in the number of deaths related to ice jams.\n\nOne of the most fun CAR stories I've done came not from a spreadsheet or database but from negotiation. The state refused to even give me a sample of driver's history records. Instead, they insisted I buy them for $4 each from my local bureau. So I bought my record, the record of a well-known traffic violator and the record of Gov. Marc Racicot. Guess which one of us had an expired license and had been driving illegally for six months? Some of the governor's aides were sure that I'd been tipped. I simply told them that if they had turned over the information when I first requested it, it's likely I would have never found out. Unfortunately, they still refuse to provide the entire database.\n\nIt's hard for me to believe that it has taken me this long to catch the CAR bug. Being able to do this work has been rewarding both personally and professionally - I found that even with my limited experience my value in the job market increased almost immediately after I started using CAR for my stories.\n\nMy enthusiasm for CAR work is not shared by all the reporters I've talked to about it. The very idea of spreadsheets and databases makes their eyes glaze over. But for those who see the benefits, the rewards are great.\n\nJoe Kolman, now a reporter with the Omaha World-Herald, can be reached by e-mail at joekolman@hotmail.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-sampling-for-answers","slug":"2000-julyaugust-sampling-for-answers","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-sampling-for-answers.html","year":2000,"month":"July/August","monthNum":7,"headline":"Sampling for answers","kicker":"THE ENVIRONMENT","author_name":"Charles Lussier","author_title":"investigative journalist","affiliation":"The Courier, Daily Comet, Daily World","summary":"A collaborative investigation by three small Louisiana newspapers revealed that the state's Department of Environmental Quality failed to resolve long-standing pollution cases and hold violators accountable, hampered by poorly maintained and fragmented databases containing numerous errors and incomplete information.","keywords":["investigation","environmental","database"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["Excel","spreadsheet"],"full_text":"Sampling for answers\n\nIt seemed easy. Go to the state capitol in Baton Rouge for two or three months and see how well the Louisiana Department of Environmental Quality is protecting the earth, air and water.\n\nThe agency has long been a lightning rod for controversy, some of its own doing and some because of the sensitivity of its duties. Of particular interest was a 1998 study by the Tulane Environmental Law Clinic suggesting the DEQ was rarely penalizing polluters for environmental violations. The study, however, raised as many questions as it answered.\n\nLouisiana's three New York Times Regional newspapers - The Courier, the Daily Comet in Thibodaux and the Daily World in Opelousas - decided to look closer. It was not an easy decision; the three newspapers have fewer than 25,000 readers each. But the editors decided the story was important, so we pooled our resources and tried to tell the story as best we could. I was picked to lead the investigation.\n\nThe difficulty in finding clear answers to simple questions, however, quickly turned three months into seven.\n\nIn the end, the resulting five-day series, published Feb. 21-25, showed that the state's top environmental agencies failed on many occasions to resolve long-standing pollution cases and hold the wrongdoers accountable. It also showed that the agency lacked the kind of critical information that could allow ordinary citizens to make their own assessment of the agency and how well it protects the state's environment.\n\nData surprises\n\nFrom the beginning, the plan was to obtain DEQ's electronic data on enforcement and let that tell the story.\n\nThat was easier said than done. As it turns out, DEQ has lots of data. But little of it covers more than any one branch of the agency, and most of it is full of holes and riddled with errors. In fact, the agency had more than 250 databases from which to choose. We eventually submitted public records requests for about half a dozen databases.\n\nTo help us extract the truth from this mess, upon the advice of a couple of IRE members, we sought outside help. I had had some limited experience dabbling with spreadsheets, but I felt I needed someone with more expertise to come along for the ride. Unfortunately, Louisiana is not blessed with many journalists familiar with databases and other computer-assisted reporting techniques. So, we placed want ads in a couple of newspapers. We ended up reaching outside the profession and hired Ashley McManus, a computer programmer living in Baton Rouge.\n\nFor close to two months, Ashley and I wrestled with this information, but were largely defeated by its many inherent flaws. The data had no common denominator. Typos were common. Many seemingly important data fields were started, but often unfilled or contained infrequent entries. In fact, only a few fields were even close to complete.\n\nIt was a classic \"dirty data\" situation. We tried to compare the numbers with official totals, but we found to our surprise that the agency collected no reliable official totals of its own which we could use to check our results. We were able to use this information only sparingly.\n\nTo get a more complete picture, I sought advice from Steve Doig, a professor at Arizona State University, an IRE member, and an unofficial adviser to this project. He recalled a crime study he had performed at The Miami Herald years ago in the days before desktop computers. Instead of obtaining a database, he pulled an old-fashioned statistical sample of a couple of hundred paper files. It was not of the caliber of a Gallup Poll, but the sample was helpful for discovering overwhelming trends.\n\nSampling\n\nWe decided to pull a random sample of our own. It consisted of 200 DEQ cases selected at random from more than 11,000 cases contained in the agency's main enforcement database. On two laptop computers, we created Excel spreadsheets and filled the headings with an ambitious list of items we wanted to check. And then for another month, we visited DEQ public file rooms and pored over documents.\n\nBut once again, poor record-keeping caused problems. We were unable to locate a third of the cases. So one of the stories became the poor quality of the agency's data. We were also able to show that a significant proportion of the cases were either still open or had a status that could not be determined. This confirmed the impressions of many citizens that the agency was slow to act decisively and would tolerate pollution for years without bringing the cases to a conclusion.\n\nThe search also yielded specific stories. Three of the cases uncovered had local significance to the readers of our three newspapers. They were cases of pollution which DEQ proved ineffective at stopping. Reporters at each of the papers then pursued these stories in greater depth.\n\nOne warning\n\nBe careful about borderline trends. Our error ratio was 6.6 percent, which would be too high for a professional statistician, though it was fine for our purposes. If we had declared that a slim majority, or 54 percent, of the case files were missing, we would have been wrong. The real answer would have been that between 47.4 and 60.6 percent of the cases were missing. That's probably a majority, but not necessarily. We would have handed our critics ammunition.\n\nObviously the sample wasn't the end of our investigation. We still had a lot of work to do, interviewing sources and amassing various government documents. We also managed to use some of the more recent DEQ electronic data for some limited comparisons.\n\nThe sample, though, was crucial. Besides the authority it lent to our work, it also provided a worthwhile framework with which to examine paper files. Those days reading case files proved the most illuminating of the investigation.\n\nIt's ironic that an old-fashioned sample was more fruitful than modern-day database crunching. Obviously, I would have preferred to have the data gift-wrapped for me to analyze, but with poorly run government agencies that often doesn't happen. So, if you have the time, or even if you're just fitting in this kind of work while doing other things, a study of the sort we undertook can pack a lot of punch and shine a light on worthy information that you would not have otherwise obtained.\n\nCharles Lussier can be reached by e-mail at charles.lussier@houmatoday.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-state-level-data","slug":"2000-julyaugust-state-level-data","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-state-level-data.html","year":2000,"month":"July/August","monthNum":7,"headline":"State-level data","kicker":"ENVIRONMENT","author_name":"MaryJo Sylwester","author_title":"","affiliation":"IRE and NICAR","summary":"Environmental reporters should access state-level databases for more current information than federal sources, prepare for lengthy FOIA requests, and thoroughly study relevant regulations and technical terminology before analyzing environmental data, which is often incomplete or cryptic.","keywords":["environmental","databases","FOIA"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"There are literally hundreds of databases on environmental issues, both at the state and federal level, and many have had little scrutiny from the media. Federal data are typically 2 years old by the time it is released, while you can usually get more current information from the state level.\n\nBudget extra time to get the information. It could take months from your first request until you actually get the data. Environmental agencies at the state level seem to be even less experienced than other state agencies in handling FOIA requests.\n\nBeware that environmental laws can vary from state-to-state, which will change not only the record keeping but also the interpretation of the data. This will also make it difficult to draw comparisons between states.\n\nStudy the topic thoroughly before diving into the data - particularly the regulations for the topic and state/region you're looking at. Covering the environment has enough stumbling blocks because of the technical jargon, but computerized environmental data can be even more cryptic.\n\nThe usual warnings about dirty data are even truer when it comes to environmental data. Be prepared to look through as many paper records as possible.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-cheap-and-easy-data","slug":"2000-julyaugust-cheap-and-easy-data","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-cheap-and-easy-data.html","year":2000,"month":"July/August","monthNum":7,"headline":"Cheap & easy data","kicker":"SMALL NEWS ORGANIZATIONS","author_name":"Craig Lincoln","author_title":"","affiliation":"Duluth News-Tribune","summary":"This article outlines cost-effective data journalism strategies for reporters, including using census data, school test scores, employee salaries, and economic indicators, while suggesting data-sharing arrangements with other news organizations to reduce costs.","keywords":["data","journalism","economics"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"Here are a few ideas for CAR stories and for getting data without having much money or time:\nDemographics data are cheap. The key is, of course: www.census.gov. We have used the IRS county-to-county migration databases (about $10 a state) and a variety of small-area population estimates from the Census and Minnesota's State Demographer.\nPutting big charts of school tests scores in your newspaper can make you look lawfully smart to your editor and your readers. If your state is like Minnesota, there will be a lot of demographic information in the databases about each school.\nEmployee salaries are handy to have, even if it's just for the inevitable standoff just before the strike by a public employees' union. It's better just to get the data each year, rather than make last-minute requests.\nYou can make your newspaper the expert on the local economy. We have used employment data, house sales data (work with real estate agents), car sales (work with your advertising department), spending data and other readily available economic indicators to write a report on the regional economy.\nFinally, this is how we cheat. We get national data from a central Knight Ridder purchasing pool, and a lot of Minnesota data from the St. Paul Pioneer Press.\nIf I weren't in that network, I would find a non-competing newspaper with similar interests and try to set up a similar bartering arrangement - especially for databases that are used more on a regular, look-up basis.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-repeat-offenders","slug":"2000-julyaugust-repeat-offenders","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-repeat-offenders.html","year":2000,"month":"July/August","monthNum":7,"headline":"Repeat offenders","kicker":"CRIMINAL JUSTICE","author_name":"Jeff Porter","author_title":"","affiliation":"Arkansas Democrat-Gazette","summary":"An Arkansas Democrat-Gazette investigation using court databases and case files reveals that repeat offenders often receive multiple convictions before facing prison time, highlighting systemic issues like overcrowded prisons and overwhelmed courts.","keywords":["convictions","offenders","database"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["FoxPro"],"full_text":"Convict me once. Convict me twice. Convict me once again. And I still might not get locked up. That's the key finding in an Arkansas Democrat-Gazette story looking at the state's criminal justice system and repeat offenders. We reported: Some repeat offenders get second, third and fourth chances before going to prison. They're out there, writing bad checks or stealing cars or breaking into houses. Some are doing worse. Such is the case in many states, not just Arkansas. The reasons are complex and overlapping - crowded prisons, overworked courts, law enforcement's emphasis on drug crimes. While the system tries to sort out its problems, a few people take advantage.\n\nStarting point\n\nThe primary starting point was a database from the Arkansas Administrative Office of the Courts. The file contains more than 400,000 circuit court records. In Arkansas, circuit court judges preside over felony cases. Data fields include offender names, dates of birth, charges and outcomes of the cases.\n\nAlthough we set out to write a people story, numbers had a place. We used the courts database to create most of the statistics used in the story.\n\nOne field missing: any unique identifier for the defendants. We semi-overcame that when we began our first step, finding out which offenders had the most convictions. We simply grouped by first and last names, plus dates of birth, to identify each unique offender. Using database software Visual FoxPro, we counted each conviction of each unique offender.\n\nWe also weeded out records we didn't want. At this point, we included only felony convictions. While circuit courts deal with felony cases, they also deal with misdemeanor appeals, and some felony cases are bargained down to misdemeanors. And we decided to not count, at this point, those charged with violating their probation. We did eventually fold back into the story the misdemeanor plea bargains, cases that didn't result in convictions and probation violations.\n\nPaper files\n\nNext step: We chose a handful of repeat offenders and obtained both court and prosecutor files - a time-consuming process that involved trips from one side of the state to the other. But the case files generated details that the electronic information didn't include, plus a long list of source names: criminals, victims, witnesses, police, prosecutors, lawyers, judges. That turned the story into a story. We went on to profile four repeat offenders, choosing them to represent the most common types of crimes and circumstances. One of our four was a violent criminal, but, typically, most repeat offenders commit property or drug-use crimes. One such offender committed at least 10 felonies before a judge sentenced her to prison. Wrote a prosecutor to a judge, before her prison sentence: \"In my seven years as a prosecutor, I have never witnessed a nonviolent offender who has attempted to manipulate the criminal justice system more.\" The story pointed out that since her release, she'd faced a series of hot-check warrants. That's not a big crime for most, but her history was different. In 1993, during a stay in Milwaukee, she bounced an $8,500 check at a car dealership and drove away with a Cadillac Fleetwood.\n\nA few days after our story ran, her parole officer picked her up. She was later released again.\n\nThe place for numbers\n\nAlthough we set out to write a people story, numbers had a place. We used the courts database to create most of the statistics used in the story.\n\nAlmost one of every four people convicted of a felony in Arkansas will be found guilty of a second crime.\n\nFor example:\n\nThey may not go to prison for a first offense, particularly if it is a nonviolent crime, such as burglary or credit-card fraud. Fewer than half of first-time offenders do in Arkansas.\n\nThey may not go to prison the second time, either. Or even the third or fourth. About three in 10 repeat offenders will simply be put on probation.\n\nAnd if they violate the rules of their probation, as about one of three felons is at least accused of doing, they still may not do time. Arkansas' circuit judges send just six of every 10 violators to prison or jail. The rest get only a longer probation or a bigger fine or, sometimes, only a scolding. We were fairly confident of our numbers, but we did bounce the statistics off several experts, including a set of professors at a local university. None were quoted in the main story, but they confirmed key statistics and made a series of helpful suggestions.\n\nJeff Porter can be reached by e-mail at jporter@ardemgaz.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-eliciting-change","slug":"2000-julyaugust-eliciting-change","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-eliciting-change.html","year":2000,"month":"July/August","monthNum":7,"headline":"Eliciting change","kicker":"CRIMINAL JUSTICE","author_name":"James Walsh","author_title":"","affiliation":"Star Tribune","summary":"The Star Tribune overcame significant legal and technical obstacles to build a criminal justice database linking probation and jail records across Minnesota counties, resulting in the 1999 investigative series 'Catch & Release Justice' that exposed how chronic offenders repeatedly commit crimes with minimal consequences.","keywords":["probation","database","offenders"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":[],"full_text":"After all the delays, after the legal fights and technological obstacles, we were finally ready to proceed with the stories we'd planned to do all along. Months earlier, the Star Tribune had requested jail booking records from Minnesota's 87 counties and probation records from the seven largest counties in the Twin Cities metropolitan area. We wanted to quantify the anecdotes we'd heard for years: Offenders on probation were repeatedly committing new crimes without receiving the threatened sentence from their previous convictions.\n\nThe stories went beyond data analysis to find real offenders and real victims. It also helped elicit real change.\n\nThe original idea was to build a database of the Twin Cities offenders on probation over the past five years, link it to another database of all offenders booked into jail over the same period to find the people who committed new offenses while on probation for old ones.\n\nFrom the beginning, however, there were problems.\n\nNo statewide database\n\nMinnesota has no statewide criminal justice system database. Each county not only keeps its booking and probation records in different formats and in different systems, there are no unique identifiers to ensure that one John Smith isn't another John Smith. Those differences not only made it difficult for counties to share records with other counties, it made it nearly impossible for one county's probation agency to share data with the same county's sheriff's department.\n\nOK. We'd create our own way to link the databases. Converting first name, middle name, last name and date of birth into a character string, we could create keyfields that would allow us to pull a thread through nearly 100 different databases of offenders.\n\nOne problem: The state's largest sheriff's department and several of the largest probation agencies insisted that the dates of birth of offenders were not public. Without those dates, how could we be certain that we were indeed finding the same offenders with multiple records in multiple counties? We filed several lawsuits.\n\nWe had dates of birth from all the counties but Hennepin, the state's largest. Using data sets from the state Sentencing Guidelines Commission, state Supreme Court information system and other counties, we found birth dates for more than 28,000 of the 89,000 people booked in Hennepin County from 1994 through 1998. More importantly, we tracked down the dates of birth for the county's 1,200 most frequent offenders.\n\nThe result of our work was the July 1999 publication of \"Catch & Release Justice,\" a series of stories that showed how thousands of chronic offenders continue to cycle through jails over and over again with little serious consequence. The series also chronicled the pain and damage they cause by repeatedly creating new victims. But we weren't finished yet.\n\nDates of birth\n\nThe lawsuits still were out there. And soon after publication, Star Tribune editor Tim McGuire took local officials to task in a column for withholding offenders' dates of birth. A week later, the newspaper reached settlement agreements with probation and sheriffs' officials and we finally began receiving the dates of birth.\n\nMcGuire urged us to write the stories that we had originally proposed - find the repeat offenders on probation at the time they committed new crimes. We moved forward.\n\nAgain, we faced problems. Ramsey County was unable to provide offenders' conditions of probation in a computerized format, making impossible a cross-county analysis of how well criminals complied with those conditions. And Hennepin County, which supervises about 30 percent of the state's probationers, provided incomplete data. The probation department gave us data on 64,461 cases over five years but most were missing dates on which the probation was to end. They also seemed not to know how many people were actually given probation. When we turned to the Hennepin County District Court, which provided us with data on all sentenced offenders over the same five years, we found that there were really 102,000 probation cases. When we asked probation officials about this, they simply instructed us to \"go with the higher number.\"\n\nUsing nearly 220,000 records, CAR Editor Dan Browning tracked overlapping probations - offenders sentenced and given new probation while still on the books for an older offense. We also examined whether officials knew when the offenders they supervised were booked into jail on suspicion of committing new crimes.\n\nResults\n\nThe stories went beyond data analysis to find real offenders and real victims. It also helped elicit real change. State lawmakers earlier this year approved development of a new statewide criminal history information system - signed into law by Gov. Jesse Ventura - that will eventually allow officials across the state to know when offenders are booked, charged and sentenced anywhere in Minnesota. In addition, the series sparked action by the state's business community. By lending their expertise and financial muscle, business leaders are helping speed the development of a statewide database of all offenders placed on probation. Two counties - Hennepin and Anoka - now download the names, offense information and conditions of all offenders with probation into a state computer system each night. Other counties will be added over the next couple years. Charlie Weaver, Minnesota's commissioner of public safety, has credited the newspaper for providing the spark to change what had been a critical - but largely invisible - problem. \"Everyone is responsible for how we got to this place. And the offenders know how to play the system,\" Weaver said. \"If there aren't consequences for the repeat offenders, they're going to continue to reoffend.\"\n\nJames Walsh can be reached by e-mail at jwalsh@startribune.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-sql-server-pays-off","slug":"2000-julyaugust-sql-server-pays-off","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-sql-server-pays-off.html","year":2000,"month":"July/August","monthNum":7,"headline":"SQL Server pays off","kicker":"TECH","author_name":"David Heath","author_title":"","affiliation":"The Seattle Times","summary":"SQL Server is a powerful database solution for journalists and computer-assisted reporters that can handle large datasets and networked data access far better than Access, eliminating the need for tedious data transfers and enabling efficient remote querying.","keywords":["database","reporting","analysis"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["SQL Server","Access","FoxPro","SPSS","CD-ROM","ODBC","Uniquery","ColdFusion"],"full_text":"SQL Server pays off\n\nHere's a data nightmare. You're on deadline and you need to do an analysis of on-the-job deaths. You've got OSHA's workplace safety data - so that's good - but the data's kept on four CD's. It takes you an hour just to copy the data to your hard drive. Then you discover that if you combine all the data into a single two-gigabyte table, Access chokes. You have no choice but to break it up. What you thought would be a simple question - how many deaths has OSHA investigated in the past 25 years? - takes you all day to answer. And that's just the start. There's a simple way to avoid situations like this: Use SQL Server.\n\nSQL Server is high-powered database software from Microsoft that, as the name suggests, works on a server over a network. You can use the software without a server, but you lose many of the benefits that way. While I don't have personal experience with this, I assume that you could get the same benefits from other server-client software such as Oracle or Informix.\n\nSQL Server has gained a reputation in newsrooms as a great tool for putting databases on an Intranet. In fact, it's also great for the kind of slicing and dicing of data that CAR folks do all the time. I have hardly touched Access or FoxPro in the last three years. There are many reasons for this, the chief one being that I haven't needed to.\n\nHere's why:\n\nSQL Server can handle a terabyte of data, so I've never come close to seeing it choke. Access cannot handle anything exceeding a gigabyte, which is not a lot these days and can be quite frustrating.\n\nYour data is always warm. By keeping it on a server, you no longer have to waste time copying from a CD or restoring from a backup tape. As I was writing this, a reporter asked me to search for a name in several different databases. I was able to do it in less than five minutes because the data was ready to go.\n\nBecause the server is doing most of the work, my computer isn't tied up when I'm doing queries. In fact, I can run several queries at once and still be free to do other things on my computer while I'm waiting for the results.\n\nSQL Server doesn't have as many point-and-click features for doing queries, which means you have to master SQL. But one advantage is that you can easily connect to a SQL Server database with Access or FoxPro and use their mouse-friendly querying tools. You can also easily import data directly into SPSS through what's called an ODBC connection. They are easy to create.\n\nThat means that you can set up any reporter on the network to do queries directly from SQL Server in Access. And you can do analysis from any PC on the network. You don't have to worry about copying data from one machine to another.\n\nIf your server is on the Internet, you can do queries from any computer on the Internet. You could go home, log onto America Online, open up Access and, with an ODBC connection, do queries on data kept in the office. The data is password protected, so it's secure.\n\nIntranets\n\nOn top of all that, SQL Server is in fact a great tool for putting databases on the Intranet. Here's my routine:\n\nWhen I get any data, I automatically \"upsize\" it to SQL Server, which is no more difficult than importing the same data into Access. As always, I index the fields I'm going to be querying. Then I'm ready to go.\n\nIf the data have people's names in it or may be of interest to others, I create a view and put it on our Intranet. The view simply creates a facade that converts field names and codes into plain English. To create search pages on the Web, I've used Uniquery and Cold Fusion. Both work great because they require no programming. Who has time to become a Web developer?\n\n(This could be the subject of a separate article, but the big advantage of Uniquery is that it allows you to search all databases at once or individually. The big advantage of Cold Fusion is that you can easily customize the search page.)\n\nIf you are especially ambitious, a la Tom Torok at The New York Times, you can use SQL Server 7.0's advanced features, such as English query. This allows users to ask questions in plain English, such as \"How many firefighters were paid overtime last year?\" SQL Server converts that to an SQL statement, such as \"select count(occupation) from payroll where overtime > 0 and year = '1999' and occupation = 'Firefighter'\".\n\nSQL Server 7.0 also allows for full-text indexing, which means that you can search for a word or words in a field of text. That's much faster than using a simple LIKE query.\n\nWhat you need\n\nSQL Server is insatiable when it comes to memory and space. You also need to learn how to create an ODBC connection. With your mouse, click Start Settings Control Panel ODBC Data Sources Add SQL Server. Then type in a name, type in the name of the server, click on Next, choose either NT authentication or a password, click on Next, change the default database, click on Next and then on Finish.\n\nWhat do you need to make SQL Server work? You can install it on your personal computer. But to get all the benefits I've mentioned, you really need a nice Windows NT server - one with at least 500 megabytes of memory and 96 gigabytes of disk space.\n\nThe biggest downside to using SQL Server is that to get full benefit from it, you really need to master SQL. The good news is that there's not a steep learning curve. Once you've gotten SELECT, UPDATE, CREATE TABLE, and INSERT down, you will be able to do almost any query with it. Also, you can use the SQL you learn in Access or FoxPro. The challenge of learning SQL Server pays off, because you can use it for any database need you have. You can slice and dice. You can query data that's ready to go. You can connect data savvy reporters with Access. And you can make the data available to everyone via the Intranet. The software is specifically designed to work efficiently with multiple users. So why bother with anything else?\n\nDavid Heath can be reached by e-mail at dheath@seattletimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-hidden-costs","slug":"2000-julyaugust-hidden-costs","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-hidden-costs.html","year":2000,"month":"July/August","monthNum":7,"headline":"Hidden costs","kicker":"LOCAL GOVERNMENT","author_name":"Robert Gebeloff","author_title":"","affiliation":"The Record","summary":"New Jersey public employees have accumulated over $200 million in unused sick and vacation time benefits that are often hidden from public view, with a team of reporters uncovering this financial obligation by analyzing state audit documents across dozens of municipalities and school districts.","keywords":["buyouts","benefits","transparency"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["spreadsheet"],"full_text":"Hidden costs\n\nBy Robert Gebeloff The Record\n\nA veteran police chief retires. The town council goes into closed session. And before you know it, the chief is relaxing on the beach with a cool drink in one hand and a six-figure lump sum payment in his bank account - far away from the angry mob at the next council meeting who just read about the deal in your paper, two weeks after the fact.\n\nSound familiar?\n\nIn New Jersey, lucrative buyouts are fairly common for long-time public officials, and the primary justification is unused sick and vacation time. While most private sector employers do not allow workers to carry over or accumulate these benefits, public employees enjoy a different standard.\n\nRecently, we decided to take a stab at measuring the overall impact of such benefits in a way that went beyond the \"outrage of the week\" type story.\n\nWhile the lucrative payments being handed to outgoing police chiefs and school superintendents made for good copy, we wondered what was going on at lower levels, when long-time police sergeants and teachers retired.\n\nWhat reporters Jan Barry, Peter Sampson and Don Stancavish discovered was staggering - that local public employees had banked more than $200 million worth of unused sick and retirement time.\n\nMoreover, they found, this cost was pretty much hidden from public view. In addition to settling the big cases in closed session, most government agencies failed to itemize the cost of paying retirees in budget documents, instead rolling the figures inside line items for salaries or miscellaneous costs.\n\nThe back door\n\nFortunately, the reporters found a back door. State regulators had, since the mid-1990s, urged local officials to upgrade accounting practices to meet national standards. And as part of this effort, auditors for most local agencies had begun calculating the estimated payoff cost for unused sick and vacation time as part of the annual audit process.\n\nThe audits were a godsend. Instead of recapping the five most notorious buyouts, our reporters could tell readers in Ho-Ho-Kus how much they'd have to eventually pay retiring town employees and readers in Wanaque how much they'd have to pay retiring public school workers - with official figures generated by auditors working for the government agencies themselves.\n\nBut as the reporters began plowing through hundreds of audit documents cover 93 towns, 103 school districts, and a couple dozen county agencies quickly encountered a journalistic obstacle.\n\nContext. It would have been very easy to just stick all the big numbers we were finding in these audits into the newspaper, but the truth is that the numbers alone would mean very little to readers. A $100,000 obligation in a small school district is not the same as a $100,000 obligation in a large county agency. And what does $100,000, $500,000 and $1 million mean to readers anyway - those are just abstract large numbers. Clearly, we needed to do some analysis before we could present this information in the paper.\n\nPutting it in context\n\nWe decided to crunch numbers on two levels. First, in an effort to see the story through the eyes of the reader, we decided to express our findings on a town-by-town basis. This meant combining the municipal, school and county obligations into one total figure for each town. It wasn't as simple as adding up three numbers. Many of our school districts serve more than one community. And all of our county agencies cover multiple towns. So we developed a method for distributing the obligations among communities. For county agencies, we made a spreadsheet of assessed property value and calculated the percentage of the county total in each town. Then we assigned that percentage to the county obligation. In other words, if Hackensack contains 2.2 percent of the county's assessed value, then Hackensack taxpayers are ultimately responsible for paying 2.2 percent of the accumulated unpaid sick and vacation days of county employees. We created similar worksheets for regional school districts. Only, we based the breakdowns on enrollment. In general, towns support regional districts based on the proportion of enrollment generated by each participating town.\n\nSo we split regional district figures along the same lines.\n\nOnce we finished this round of crunching, we were still left with a fairly abstract large number for each community. To bring this figure home, we decided to perform a property tax analysis.\n\nWe set up a spreadsheet laying out the property tax characteristics for each town: the tax rate, the total levy, the value of a tax point, etc. We then wrote formulas that calculated how much property taxes would go up if the obligation for unused sick and vacation days were added to the tax levy.\n\nWhile no town would actually have to bear such a burden all at once, we explained to readers that this was a real cost and that it was a cost they would have to pay eventually in the coming years.\n\nMeanwhile, I'm happy to say that our reporters did an excellent job of NOT bogging down the narrative of their stories with these details. The data gave the story real backbone, but the actual writing was filled with anecdotes and discussion of the issues.\n\nFor the paper, we printed a town-by-town chart that contained only the summary data, plus an explanation of our methods. We then put several more charts online that contained the nuts-and-bolts data.\n\nRobert Gebeloff can be reached by e-mail at gebeloff@bergen.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-keep-an-eye-on-ferrett","slug":"2000-julyaugust-keep-an-eye-on-ferrett","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-keep-an-eye-on-ferrett.html","year":2000,"month":"July/August","monthNum":7,"headline":"Keep an eye on Ferrett","kicker":"CENSUS DATA","author_name":"Paul Overberg","author_title":"","affiliation":"USA Today","summary":"Ferrett, a redesigned federal data analysis tool built in Java, enables journalists and researchers to easily query large government datasets like the Current Population Survey without specialized technical knowledge, though it currently has some performance issues in its beta phase.","keywords":["database","government","analysis"],"topics":["Data & Databases","Technology","Investigative Methods"],"technologies":["Ferrett","Java","Excel"],"full_text":"Keep an eye on Ferrett\n\nBy Paul Overberg USA Today\n\nA new tool for digging into big federal data sets shows promise, and it's useful now, even in its buggy beta phase. It's the new version of Ferret (Federal Electronic Research and Extraction Tool), renamed Ferrett (http://ferret.bls.census.gov/java/netscape.htm). Built in Java and available free to anyone with a Web browser, it lets users tap microdata-individual records-federal economic, demographic and health surveys. Let's say you want to know whether registration and voting rates for 60- to 65-year-olds are different from those of 65- to 69-year-olds. Or how the age breakdown of union members compares with that of workers covered by union contracts.\n\n\"Our goal is to develop an expert-system-type approach about the data,\" opening it to new audiences, said Cavan Capps, Census Bureau Ferrett project leader.\n\nOr the workforce distribution of your state's foreign-born population.\n\nYou can get all this from the Current Population Survey, a monthly 45,000-household sample that yields a wealth of annual demographic data snapshots in addition to its main job, producing the monthly unemployment rate. But only a few people know how to wade through the survey's hierarchical file structure, various weightings for household, family and person records, top-coded variables and other subtleties.\n\nIn its infancy\n\nBefore the Web, using CPS required buying computer tapes and mastering arcane concepts with codebook and phone help. Ferret 1.0, released three years ago, simplified the task tremendously. It offered a Web interface to build queries, and also allowed users to slice out just the variables they wanted to download. It carried pretty good online documentation, but it still required subject familiarity.\n\nThe newest version is better. It's been renamed Ferrett to avoid trademark entanglements. It walks a user through the process of picking a survey and variables. You can also check off which values you want from each variable. Finally, Ferrett allows on-screen tabulation or download.\n\nOnce you've chosen the survey and variables, arranging them is much like using Excel's pivot table wizard. An animated graphic of a ferret wearing a mortarboard demonstrates.\n\nIt's not without problems. It's slow. A memory leak causes it to occasionally forget where you were in building a query. Some of the point-and-click steps require a second or third try before they're executed. Windows sometimes take three tries to close. The navigation is pretty good, but not great.\n\nBut even with all that, Ferrett's power makes it worthwhile if you need to slice up the CPS for answers that go beyond published tables.\n\nFuture potential\n\nFerrett's real potential lies in its promise to open up other government datasets over an effort called the Data Web. The National Center for Health Statistics has helped with Data Ferrett's development, and some of its surveys should be available soon. Among them:\n\nNational Health and Nutrition Examination Survey (HANES)\nNational Health Interview Survey (NHIS)\nNational Ambulatory Medical Care Survey (NAMCS)\n\nOther Census-data sets that may soon be added include annual population estimates, annual County Business Patterns and the 1997 Economic Census.\n\nEven state and local data sets may become available, said Cavan Capps, the Census Bureau project leader. Already, Ferrett taps 350 gigabytes of data on computers scattered around the country. \"A Web browser is designed to be a document browser, but then you happen on data,\" Capps said. \"Ferrett is designed to be a data browser.\"\n\nExperts to audience\n\nCapps said that only a handful of labor economists and demographers understand CPS concepts, variables and weights and how to link them in time series to measure change. Building the rules that experts know into Ferrett, Capps said his team hopes to make the data self-aware, and useful to new audiences.\n\nOne example is state-level CPS data. Each March, the CPS panel is asked a host of questions on household members and their income sources the previous year.\n\nThe panel typically doesn't include enough people in each state to produce sound results. But two years of March data does, and that's how Census experts use it to publish reports. Capps hopes Ferrett will be able to understand such rules and make them transparent.\n\n\"Our goal is to develop an expert-system-type approach about the data,\" opening it to new audiences, he said.\n\nIn addition to the March demographic and income questions, the CPS each October carries school enrollment questions, and every other November surveys voting and registration in the just-past national election. Periodic supplements address computer usage, fertility, marital history, displaced and contingent workers and job tenure.\n\nAn example\n\nA simple example might help illustrate. I entered Ferrett and logged in with my e-mail address. (If you can't trust the Census Bureau's promise to never disclose it, who can you trust?) On the introductory screen, only one option's available. Clicking on that button opens Ferrett's tabbed palette. A Windows-style left pane allows browsing and selection of the survey you need.\n\nI chose the March 1999 CPS and searched for \"age\" and \"union.\" Ten variables came back. I selected two and clicked on the \"Browse variable\" button at top right.\n\nThat gave me a screen where the variable's details were laid out, and check boxes allow the option of filtering by certain values. Selecting a variable there pushes it into the Data Basket.\n\nMoving on to the \"Get Data\" tab, the option is to \"Extract\" (download) or \"Tabulate.\"\n\nClicking on the latter opens a screen where the selected variables can be dragged and dropped as row and column variables, just like Excel's pivot table wizard.\n\nClicking the green \"Go\" button launches the process. Going back and choosing \"Extract\" produces a download option.\n\nFerrett automatically weighted the tabulation so it got national totals, not just survey respondent totals. It also added a MARSUPWT variable to the download.\n\nThis is the weighting by which I'd multiply each record by to produce the national total. I didn't have to specify this, Ferrett just knew.\n\nThis tool isn't ready for quick answers or deadline work. A query can take several tries, Capps admits. Ferrett should come out of beta this fall.\n\nPaul Overberg can be reached by e-mail at poverberg@usatoday.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-more-on-perl","slug":"2000-julyaugust-more-on-perl","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-more-on-perl.html","year":2000,"month":"July/August","monthNum":7,"headline":"More on Perl","kicker":"TECH TIP","author_name":"Matthew Ericson","author_title":"","affiliation":"The Inquirer","summary":"The Inquirer used Perl regular expressions to standardize 700,000 crime records with inconsistently formatted Philadelphia addresses, cleaning up about 70,000 malformed entries by identifying patterns and rearranging address components into a standard format.","keywords":["addresses","programming","data"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Perl","SQL Server"],"full_text":"One of the best examples of how regular expressions can be used to clean up data came when The Inquirer obtained a 700,000-record database of every reported crime in Philadelphia since the early 1990s. Unfortunately, police had been sloppy in entering addresses in a standard format. Here's just a few of the ways that a crime at 400 N. Broad St could have been recorded:\n\nBROAD 400 N\nN BROAD 400\n400 N BROAD\nBROAD 400N\nN400 BROAD\n\nAfter looking at the data, we found that about 90 percent of the addresses fell into the first category - \"BROAD 400 N.\" However, with a data set of 700,000 records, that still left about 70,000 records with bad addresses. After a little more playing with the data, we found about a dozen different styles for addresses.\n\nThe data gave the story real backbone, but the actual writing was filled with anecdotes and discussion of the issues.\n\nTo standardize the addresses, I wrote a Perl program that went through the database record-by-record-as part of a SQL Server DTS script - and used regular expressions to pick out the various parts of the street address.\n\nFor each record, the program checked to see if the address field matched this pattern: a word, a space, a number, another space, and a direction. The syntax Perl uses to express this is:\n\n(\\w+) (\\d+) ([NSEW])\n\nHow this worked with an address of BROAD 400 N:\n\nThe \\w indicates an alphanumeric character. The + says look for one or more of them. So \\w+ means look for one or more alphanumeric characters in a row. So BROAD meets this condition. It's five alphanumeric characters in a row.\n\nThe \\d+ tells Perl to look for one or more digits in a row; 400 meets this condition.\n\nThe brackets tell Perl to look for a single character. The letters in the brackets are the acceptable values. In this case, a street direction of N, S, E or W.\n\nBut where Perl outshines most other search-and-replace utilities is in taking pieces of the search term and rearranging them into the replacement. By placing parentheses around groups in the search term, the script remembered what it found. The value matched by the first set of parentheses was saved in the variable $1. The value matched by the second set was saved in the variable $2, and so on. So to turn BROAD 400 N into 400 N BROAD, the script looked for (\\w+) (\\d+) ([NSEW]) and replaced the address with $2 $3 $1.\n\nSince the block number needed to be first, then the street direction, and then the street name, the program took the block numbers from the second set of parentheses - the (\\d+), stored in $2 - and put it back as the first item in the replacement.\n\nThe street direction - stored in $3 - was the next item, and the street name - the first item searched for - became the last item in the replacement.\n\nWhat if a particular record didn't match the pattern (\\w+) (\\d+) ([NSEW])? The program moved on to the next possible pattern - out of the dozen or so we were trying - and checked to see if it matched. If it did, it took the pieces, rearranged them in the proper order, and moved to the next record.\n\nIf it didn't, it tried yet another pattern. And if it exhausted all possible patterns, it flagged that record as an error and moved on to the next record.\n\nMatthew Ericson can be reached by e-mail at mericson@phillynews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-newsroom-training","slug":"2000-julyaugust-newsroom-training","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-newsroom-training.html","year":2000,"month":"July/August","monthNum":7,"headline":"Newsroom training","kicker":"TIPSHEET OF THE MONTH","author_name":"Jennifer LaFleur","author_title":"CAR editor","affiliation":"San Jose Mercury News","summary":"The article outlines different computer-assisted reporting (CAR) training models for newsrooms, including universal training, specialist fellowships, and outside vendors, with practical dos and don'ts for effective implementation.","keywords":["training","journalism","skills"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"From veteran nerds to those just learning the basics of computer-assisted reporting, we are all in the process of developing better ways to spread these skills throughout our newsrooms. Many models are developing, no single model is the answer. That's why sharing experiences at gatherings such as this is so important.\n\nHere's an outline of some training models and the benefits and drawbacks of each.\n\nThrow it at everyone and see where it sticks\n\nAlmost five years ago, when the San Jose Mercury News first put computers on desktops in the newsroom, we did intensive training in all the basics (Internet, spreadsheets, databases) for everyone.\n\nBenefits:\n\nEveryone gets exposed\nMost basic skills are widespread\nGet a lot of buy-in from the newsroom\n\nDrawbacks:\nA lot of skills spread very thinly\nVery time-consuming\nLess pay-off, training a lot of people to create a few users\n\nThe fellowship/the specialist\n\nThe Post-Dispatch plan requires reporters to apply for a six-week specialist program. They must have a story idea, data and be pulled off their beat to work with the CAR editor. Additionally at the P-D: they are then required to go back and be the CAR coordinator for their team and are part of a larger virtual CAR team.\n\nBenefits:\n\nGreater pay-off\nOther people serve as mentors for their team\nSkills level is more in-depth\n\nDrawbacks:\nDifficult to get team editors to commit\nFewer skills throughout the newsroom\nOthers may feel left out\n\nOutside training\n\nBenefits:\n\n\"Prophet in his own land\" - outsiders are smarter\nBetter commitment from participants\nSomeone else is doing the work\n\nDrawbacks:\nCost\nFollow-up\nSpecialization to your needs - for example, corporate spreadsheet training won't focus on journalism.\n\nDos and Don'ts for training:\n\nTailor the training to the reporters who will be using it. Use data that are relevant to them.\nUse real data that can be used to do real stories.\nDon't train more than 8 people at one time, unless you have a coach.\nDon't train reporters unless they will have access to the same software very soon afterward on which to practice their new skills.\nDon't hold a \"gee whiz\" CAR session if the reporters will not have hands-on training shortly afterward.\nConduct a skills evaluation BEFORE designing the training program.\nDon't train folks with very different skills in the same room.\nDon't train reporters how to use resources that you don't plan to have available.\nStart CAR with some small projects with realistic goals.\nDon't expect too much immediately - everyone needs practice.\nIf you use a commercial vendor, have them model your training around journalism.\nProvide lots of examples of what others have done.\nEncourage lots of questions.\nProvide follow-up with someone to turn to as they set out on their first projects.\nDon't just explain how, explain why and show them the process. Save shortcuts for later.\nUse workbooks they can practice with later.\nEvangelize and find experts.\nDon't make it seem too easy. Don't make it seem too hard.\n\nJennifer LaFleur can be reached by e-mail at jlafleur@postnet.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-julyaugust-bits-bytes-and-barks","slug":"2000-julyaugust-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2000-julyaugust-bits-bytes-and-barks.html","year":2000,"month":"July/August","monthNum":7,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The FBI's 1998 Uniform Crime Reports database is now available through NICAR's data library, containing crime statistics reported by law enforcement agencies, alongside other databases including IRS tax-exempt organizations and ATF firearms licensees. The article also provides guidance on identifying unknown file extensions and highlights new tipsheets added to the IRE Resource Center from the 2000 National Conference.","keywords":["crime","database","nonprofit"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","Adobe Acrobat"],"full_text":"FBI Uniform Crime Reports: The 1998 version of UCR is now available from the NICAR data library. It is comprised of six databases and includes crime information reported to the FBI by law enforcement agencies around the country. Most of the data consist of the \"index\" crimes: murder, non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny-theft, motor-vehicle theft and arson. Data from earlier years are also available.\n\nA discussion on the NICAR-L listserv recently yielded several ways to hunt up the answer.\n\nIRS Tax-Exempt Organizations: This database is a listing of the organizations filing an IRS 990 form - the form required for nonprofit groups in lieu of an income tax return. The data includes the organization's name, address, contact person, total annual income, total assets, and codes describing their activities generically. The records in this dataset include filings for tax periods ranging from 1974 through the first two months of 2000.\n\nATF Firearms Licensees: This dataset was recently updated to reflect those individuals, organizations or businesses licensed to sell guns, as of March 2000. The database includes stores where guns are sold, rifle clubs, museums and pistol ranges who want to own guns that are off limits to them without the permit.\n\nMore information is available at www.ire.org/datalibrary/databases.\n\nFile extensions\n\nYou just received data in a file with some obscure extension that you've never seen before. How do you figure out what it is?\n\nProperties: In the Explorer window, right click on the file and select \"properties.\" Although this is not comprehensive, it will often show the type of file.\n\nGlossary of File Extensions: This Web site lists thousands of file extensions for PC's. It does not have descriptions, however. www.ace.net.nz/tech/TechFileFormat.html\n\nEvery file format in the world: This Web site has thousands listed alphabetically. Most only list the name, but a few have more detailed descriptions. www.whatis.com/ff.htm\n\nAnother Web site: Similar to the ones listed above, this also lists file extensions used by various programs. wwww.delanet.coml-pparishlfiletype.htn\n\nIRE Resource Center\n\nDozens of new tipsheets have been added to the IRE Resource Center's tipsheet database from a variety of panels that were held at the 2000 IRE National Conference in New York.\n\nTipsheet topics include Census 2000, day care, housing issues, environment, education, juvenile crime and justice, military, business and others.\n\nThe database can be searched at www.ire.org/resourcecenter. Many of the tipsheets can be downloaded or viewed in Adobe Acrobat. You will need to sign in as \"guest\" with \"guest\" as the password.\n\nMore information is available from the Resource Center at (573) 882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-census-and-gis","slug":"2000-september-census-and-gis","url":"https://dwillis.github.io/uplink/article/2000-september-census-and-gis.html","year":2000,"month":"September","monthNum":9,"headline":"Census and GIS","kicker":"MAPPING","author_name":"Steve Doig","author_title":"CAR specialist","affiliation":"Arizona State University","summary":"The article discusses the importance of using GIS software to map U.S. Census data, highlighting the advantages GIS provides for creating detailed demographic maps effortlessly as compared to traditional graphic design methods.","keywords":["GIS","Census","Mapping"],"topics":["Demographics & Census","Technology","Data & Databases"],"technologies":["GIS","MapInfo","Maptitude","ArcView","ArcInfo","Excel","TIGER","shapefile"],"full_text":"Still trying to persuade the newsroom bean counters that they should pay for a good geographical information system mapping program? Tell them the U.S. Constitution requires it. Well, okay, the Constitution doesn't actually mention computer mapping. But it does mandate a national census every 10 years. And a newsroom that tries to do census stories without a GIS will be about as effective as one that covers breaking news without cell phones and digital cameras. It can be done, but it's a lot more difficult. The census, in fact, is a perfect story for mapping. The reason is that every scrap of census information is connected to some particular piece of geography. Moreover, a key part of most census stories is explaining how the demographics of various places compare to each other. Which do you think readers, viewers and editors would rather see: A page full of numbers, or a nicely shaded map? Of course, journalists were printing and broadcasting maps long before GIS programs came along. But even relatively simple maps can take a graphic artist hours to complete using a drawing program. And census maps are likely to be complex. For instance, a census tract map of a typical metro area will contain hundreds of polygons to be drawn and shaded, while creating and shading a map of the nation's 3,156 counties could mean days of work. A GIS, however, can make such maps in seconds. Okay, you're convinced. So start shopping for a good GIS program. There are a variety of them available, including MapInfo and Maptitude. But I and many other CAR specialists would recommend ESRI's ArcView 3.2. ArcView is pricey - expect to pay about $1,200 for a copy. But ArcView's advantages are many. It works well on both PC and Macintosh computers. ArcView maps are easily exported into files that can be opened and enhanced by your graphic artists. The widespread use in the CAR community means you can get quick help from journalism mapping experts via the NICAR-L and JAGIS-L (Journalism and GIS) e-mail lists. You can get excellent training at NICAR's mapping boot camp scheduled for Oct. 20-22. And ESRI's 'Virtual Campus' has online courses in all sorts of GIS topics. Even better, ArcView and its big brother, ArcInfo, are widely used by federal, state and local government agencies. This means a big step forward in meeting your second major need: Acquiring map files. A GIS program like ArcView works by merging data tables, such as you might see in an Excel spreadsheet, with map files which contain the information necessary to draw and label political boundaries, streets, bodies of water and other geographic features. Thus, you'll want a collection of map files (which ArcView calls 'shapefiles') to cover all the kinds of maps you're likely to need for stories: Census tracts, streets, ZIP codes, election precincts, police patrol zones, etc. Chances are good, though, that most such shapefiles can be had from your local government GIS specialist via a public records request. Shapefiles useful for working with older census data, such as 1990 tract maps, also can be downloaded for free from the Census Bureau at www.census.gov/geo/www/cob or from ESRI's ArcData Online site at www.esri.com/data/online. However, the shapefiles for the 2000 census geography are not yet available. Worse, the Census Bureau indicates it may not be until next summer before those shapefiles can be downloaded from the Census site. That's too late for anyone wanting to work with the first wave of data, which will hit in March. Happily, there is another alternative - The Census Bureau's TIGER/Line files, the national map database used by the bureau to create all its maps. The updated 2000 version of TIGER, the one that will be used to create the official boundaries for all the 2000 census geographic units, is expected to be ready just before that first wave of data is released. TIGER doesn't come in the shapefile format that is used by ArcView. But ArcView does have a so-called extension - an add-on script that gives the program additional capabilities - that will translate TIGER files into shapefiles. If you have ArcView, you can practice with the 'TIGER Reader' extension now by downloading the 1998 TIGER files for your state from www.census.gov/geo/tigerline/tl_1998.html. One other 2000 Census mapping problem also awaits resolution. It's the question of how different your 2000 census geography is from the 1990 version, particularly if you want to look at demographic change during the decade. State and county boundaries should be the same, but some cities will have annexed territory during the '90s. Even more likely are changes to census tracts. While most of them will be unchanged, tracts that gained or lost a lot of population may be split or combined. The Census Bureau promises to release 'Relationship Files' comparing 1990 and 2000 blocks and tracts - but probably not until next summer. Rather than wait, you'll want to make your own comparison table for your area once the 2000 geography is set. And that's another good task for a GIS. The next IRE and NICAR Census workshop will be held in College Park, Md., Dec. 9-10. More information, including registration forms and hotel details, is available at www.ire.org/training/censusworkshops.html Steve Doig can be reached by e-mail at steve.doig@asu.edu","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-september-rampage-killers","slug":"2000-september-rampage-killers","url":"https://dwillis.github.io/uplink/article/2000-september-rampage-killers.html","year":2000,"month":"September","monthNum":9,"headline":"Rampage killers","kicker":"CRIME","author_name":"Ford Fessenden","author_title":"database journalist","affiliation":"The New York Times","summary":"The New York Times published a four-part investigative series analyzing 100 mass murder cases over 50 years, finding greater links to mental illness than violent video games and demonstrating the power of precision journalism techniques and computer tools to manage large datasets.","keywords":["investigation","violence","journalism"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"In March, The New York Times published \"Rampage Killers,\" a four-part series that revealed the results of a database project we undertook of 100 cases of explosive public murder over the past 50 years. These killings are not new, we said, although they may have increased in the 90s. We found more evidence of serious mental illness than of fascination with violent video games, and pointed out some important ways these criminals differed from regular murderers.\n\nThe series showed what you can do with the techniques of precision journalism. It showed how you can speak with authority when you let computer tools help you manage and distill large amounts of information.\n\nThe seeds were sowed at The New York Times in 1999, when Thom Shanker, an editor in the Washington bureau, suggested a few days after the Kosovo pogrom","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-a-beginners-lessons","slug":"2000-september-a-beginners-lessons","url":"https://dwillis.github.io/uplink/article/2000-september-a-beginners-lessons.html","year":2000,"month":"September","monthNum":9,"headline":"A beginner's lessons","kicker":"MAPPING","author_name":"Jeff Thomas","author_title":"","affiliation":"Colorado Springs Gazette","summary":"A journalism team used mapping software and GIS tools to analyze voting patterns in a school bond election by comparing precinct-level data across years and demographics, discovering that while the technology enhanced their reporting, it required significant effort to master and acquire necessary data files.","keywords":["mapping","precinct","voting"],"topics":["Investigative Methods","Technology","Government & Politics"],"technologies":["ArcExplorer","ArcView","spreadsheet","TIGER","shapefile"],"full_text":"The first moral of this story is: New CAR tools don't always make life easier. They can help you tell a smarter story – as our first attempt at using mapping software did for us in one case – but you're still likely to break a sweat getting up the learning curve.\n\nShortly after I returned from NICAR's three-day mapping seminar in Columbia, Mo., last October, voters in our largest school district rejected a bond issue. Three years earlier, they had approved an even larger bond. Naturally, we asked: Why the turnaround? Finding the neighborhoods where the drop was most dramatic would steer us to the kind of people we needed to talk to. Mapping software could help us do this. It also could help us see if the swing precincts had older or younger people, more or fewer kids, higher or lower income, etc.\n\nWe had used spreadsheets in the past to unearth voting trends in the precincts, but the chore usually wasn't complete until we had broken out the colored pencils and a large precinct map. Mapping software gave us the ability to automate the colored-pencil part, giving us much more lead time to report the story. It also would show us the underlying demographics of the precincts where the vote swing was most pronounced.\n\nGetting started\n\nI dusted off a copy of ESRI's ArcExplorer CD, which I had picked up at NICAR's Boston conference several months earlier. This is a stripped-down version of ArcView, the standard in most GIS-equipped newsrooms. As I found out, it did the job, barely. For example, ArcExplorer lacks the extensions necessary to export a map in a format the art department can work with. Publishing a map would have been nice, but more-informed reporting was our first goal, and the tight turnaround on the story precluded building a map from scratch on the Macs, so we went without the map.\n\nWe poured the precincts' \"yes\" and \"no\" votes on the bond issue into two columns of a spreadsheet, and poured the votes from the 1996 election into two other columns. For each precinct in each year, we calculated a support ratio (yes votes/no votes). Then, for each precinct, we calculated the percentage change in the ratio from 1996 to 1999. Precincts with the most-negative change numbers were those where support had dropped most dramatically. We added a column indicating voter turnout in each precinct. This took just a few minutes.\n\nThe bigger job was to assemble the needed digitized maps, which ESRI calls shapefiles: voting precincts (to plot the change in voting behavior), Census tracts (to plot demographics), and streets (to help the eye navigate the map).\n\nI asked the county's GIS department for a digital map of precinct boundaries. No problem, they said: That will be $500. The GIS manager, however, was on my side and sneaked me onto the next day's county commission agenda, where I made my plea for a fee waiver, which was granted. I left the meeting with the CD.\n\nI downloaded a street shapefile from the Census Bureau's TIGER service (http://tiger.census.gov). A Census tract shapefile was downloaded from CEISIN (http://sedac.ciesin.org).\n\nFiles in hand, I opened ArcExplorer, launched a new project, and used the \"add theme\" function to bring the three files in. The first beads of sweat began to form when I tried to view all three themes at once. When viewing the street and Census-tract themes, I could not add the precinct theme. Or, when viewing the precinct theme, I could not add streets and tracts to the view.\n\nGlitches\n\nWhich brings me to the second moral of the story: Always check the projections of your digitized maps. After wailing to the NICAR listserv, I was instructed to check the metadata of each map to make sure the projections matched. Sure enough, the precinct map was in a different projection than the street and tract maps. This is a potential pitfall that Andy Lehren spent a fair amount of time hammering into our heads at the NICAR seminar, but I guess it took a real-life example to drive the point home.\n\nIn any case, ArcView can convert projections, but ArcExplorer cannot (and won't let you add the necessary conversion extensions), so the county converted the precinct map projection and shipped me the new shapefile. All three themes now could be seen at once: precinct boundaries, census tracts, and beneath them both, streets.\n\nThe next step was to pour the demographic into the Census-tract map. This required joining a table of demographics to the table underlying the tract shapefile in ArcExplorer. I bummed a MapInfo CD of 1996 Census-tract demographics from our marketing department and moved the selected records to an Excel sheet. To make the join, I had to modify the short tract ID numbers (e.g., 2.01) in Excel to match the longer ID contained in ArcExplorer tract shapefile (e.g., 080410002.01).\n\nI saved the table as a .dbf file, closed it and used the \"add table\" function in ArcExplorer to open it. More problems. For reasons still unknown, the lengthened tract ID numbers on the Excel sheet did not survive the transition to a table within ArcExplorer. Again I ran to the NICAR list, where I was advised to move the Excel data to Access, export the table from Access as a .dbf file, then pick up the table with ArcExplorer. This worked. I used the same routine to move the precinct voting numbers into ArcExplorer.\n\nIn ArcExplorer, I joined the voting-numbers table to the table underlying the precinct shapefile, and the demographic table to the table underlying the Census-tract shapefile. This was the point where ArcExplorer could start doing its job.\n\nResults\n\nNow that the demographic numbers were a part of each census tract's record in the shapefile, we could identify the tracts with the greatest concentration of, say, school-age kids. The ones with the highest concentrations were indicated by, say, vertical lines.\n\nAnd now that the vote-swing numbers were part of each precinct's record in the shapefile, the precincts with the biggest swings against the bond issue could be indicated with a color. Precincts where the swing was less dramatic appeared in successively lighter shades of the color. This is quick, point-and-click stuff.\n\nLooking at both themes – demographics (lines) on top of vote swing by precinct (color), – we were able to produce this story nut: \"A Gazette computer-assisted analysis shows that [the] steep drop in support was spread across the district. Perhaps surprisingly, the reversal was evident in precincts on the northeast and northwest sides, where simple demographics would seem to indicate strong support for schools. Those neighborhoods are filled with young families. Median household income is higher – by several thousand dollars – than it is countywide. Voter turnout tends to be high.\"\n\nWe never published a map, but the reporter was able to zero in on the critical precincts, ask better questions, and force the school district to confront the reality that it has serious credibility issues to deal with in neighborhoods it had counted on.\n\nWhich brings me to the final moral of this story: The effort is worth it.\n\nJeff Thomas can be reached by e-mail at jeff@gazette.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-mapping-bootcamp","slug":"2000-september-mapping-bootcamp","url":"https://dwillis.github.io/uplink/article/2000-september-mapping-bootcamp.html","year":2000,"month":"September","monthNum":9,"headline":"Mapping Bootcamp:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A three-day GIS bootcamp will be held in Columbia, Missouri from October 20-22, featuring instruction from Dateline NBC and St. Louis Post-Dispatch journalists on mapping, geocoding, and spatial analysis using ArcView GIS.","keywords":["mapping","bootcamp","GIS"],"topics":["Technology","Journalism & Newsroom","Investigative Methods"],"technologies":["ArcView"],"full_text":"This three-day bootcamp will be held in Columbia, Mo., Oct. 20-22.\n\nAndy Lehren of Dateline NBC and Jennifer LaFleur of the St. Louis Post-Dispatch will teach the basics of mapping, geocoding and spatial analysis using ArcView GIS.\n\nMore information, including registration and hotel details, is available at www.ire.org/training/oct20.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-flood-dangers","slug":"2000-september-flood-dangers","url":"https://dwillis.github.io/uplink/article/2000-september-flood-dangers.html","year":2000,"month":"September","monthNum":9,"headline":"Flood dangers","kicker":"MAPPING","author_name":"Ron Nixon","author_title":"","affiliation":"The Roanoke Times","summary":"A Roanoke journalist used GIS mapping and building permit data to reveal that over a thousand structures, including a new biomedical center, were being built in flood-prone areas despite the city's flood control project, prompting officials to reconsider development policies.","keywords":["flooding","mapping","development"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["GIS"],"full_text":"Flooding is a way of life in the city of Roanoke. The city has had major floods in 1940, 1972, 1977 and 1985. The last flood was a hundred-year flood causing millions of dollars in damage and costing several lives. In the late 1990's the city decided to enlist the U.S. Army Corps of Engineers to relieve the flooding problems.\n\nThe Corps proposed channeling the Roanoke River, which runs through the city and several surrounding localities. The Corps and the city said the project would reduce the amount of flooding by 50 percent. But other localities pulled out of the project saying it would provide little benefit in reducing flooding.\n\nThey were right.\n\nUsing mapping, flooding data and interviews with flooding experts we were able to show that the city was actually helping to create the very problems that the flood control was trying to fix. Our analysis showed that there were over a thousand properties in the flood plains in the city.\n\nAnd despite the numerous floods that have occurred in the last 20 years, more structures continued to be built. Even a new multi-million dollar bio-medical center is slated to be built in the flood plains.\n\nIt's a recipe for disaster that we were only able to show using GIS. Even the city had no idea of just how many structures were in the flood plain.\n\nThe stories prompted dozens of letters to the paper, and local, state and federal officials are beginning to take a closer look at building in the flood plains in Roanoke. Many residents are pushing the city to remove more structures from the flood plains. Officials from FEMA are also looking at the flood plain development in the city.\n\nThis project showed me the power of GIS, but it also showed me that you have to be careful when using the powerful tool.\n\nI began looking into the flood dangers when three nearby localities pulled out of the flood control project and started moving homes and business out of the flood plains. I asked the city why it wasn't doing the same and how many structures were in the flood plains. They said they did keep a database of buildings in the flood plains, so we began to try and find out for ourselves.\n\nFor a couple hundred dollars, I purchased the building permit database of structures built since the last great flood, and bought digital maps from Federal Emergency Management Agency.\n\nWith help form Andy Lehren at Dateline NBC, I culled the flood plains from the digital maps. I then overlaid the locations of buildings with maps of the city and the local river.\n\nI did several interviews and thought I had my story done until I noticed something in the building permit database. It was a structure owned by The Roanoke Times, my employer, but no one had ever heard of it. My analysis showed that it was in the flood plain and had been built fairly recently. I called former employees to see if they knew about it. No one did. Finally, with maps in hand, I drove to the address. I found an empty lot.\n\nPanicked, I called the city back to find out what was wrong with the data. It took several days, but they finally got back to me and said the address showing a Roanoke Times' owned building in the flood plain was actually a temporary structure that had been put up for a local festival. There were dozens of other addresses in the building permit database like that. The city had no way of showing permanent structures in it database; using it, you couldn't tell a sign from a house.\n\nThere goes the story I thought. Then I remembered that a few weeks earlier, I had gotten the city's real estate database. The database had all permanent structures in it, the year built, type of building, everything I needed to complete the project. The impact of this project is still on-going.\n\nRon Nixon can be reached by e-mail at ronn@roanoke.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-tipsheets","slug":"2000-september-tipsheets","url":"https://dwillis.github.io/uplink/article/2000-september-tipsheets.html","year":2000,"month":"September","monthNum":9,"headline":"Tipsheets:","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"A tipsheet from the 2000 IRE national conference introducing mapping as a computer-assisted reporting (CAR) tool, including suggestions for data visualization and guidance on accessing resources through the IRE website.","keywords":["mapping","reporting","data"],"topics":["Investigative Methods","Technology"],"technologies":[],"full_text":"\"An introduction to mapping as a CAR tool,\" tipsheet #1159 from the 2000 IRE national conference.\n\nThis tipsheet explains the uses and importance of mapping, and it includes some suggestions for data to map.\n\nThe tipsheet can be downloaded from the IRE Web site at www.ire.org/resourcecenter.\n\nSearch the tipsheet database, then follow the directions below the abstract for downloading a PDF file.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-reporting-census-2000-and-related-resources","slug":"2000-september-reporting-census-2000-and-related-resources","url":"https://dwillis.github.io/uplink/article/2000-september-reporting-census-2000-and-related-resources.html","year":2000,"month":"September","monthNum":9,"headline":"\"Reporting Census 2000\" and related resources","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Guide for journalists on accessing and using Census 2000 data through shapefiles and TIGER files, with resources for downloading geographic data from the Census Bureau and ESRI.","keywords":["Census","shapefiles","TIGER"],"topics":["Demographics & Census","Technology","Data & Databases"],"technologies":["shapefile","TIGER","ArcView"],"full_text":"\"Reporting Census 2000\": http://cronkite.pp.asu.edu/census\n\nShapefiles can be downloaded from the Census Bureau at www.census.gov/geo/www/cob or from ESRI's ArcData Online site at www.esri.com/dataonline.\n\nYou can practice with the \"TIGER Reader\" extension now by downloading the 1998 TIGER files for your state from www.census.gov/geo/tigerline/tl_1998.html.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-no-boundaries","slug":"2000-september-no-boundaries","url":"https://dwillis.github.io/uplink/article/2000-september-no-boundaries.html","year":2000,"month":"September","monthNum":9,"headline":"No boundaries","kicker":"MAPPING","author_name":"Jennifer LaFleur","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"Journalists across Europe, particularly in Denmark and the Netherlands, are increasingly using GIS mapping software to analyze and visualize data for investigative stories, enabling faster production and deeper analysis than traditional methods.","keywords":["mapping","journalists","data"],"topics":["Technology","Investigative Methods","Journalism & Newsroom"],"technologies":["GIS","MapInfo","Excel","ArcView"],"full_text":"As mapping skills spread, journalists are plotting data beyond U.S. boundary files. Reporters in Denmark, the Netherlands, Sweden and other European countries use GIS (geographic information systems) software for showing crime incidents, business data or demographic trends.\n\nAt Morgenavisen Jyllands-Posten, Denmark's largest newspaper, journalists Nils Mulvad and Flemming Svith do mapping regularly using a GIS program called MAPS by Denmark-based FSS. Currently, according to Svith, about five people are using mapping in the newsroom.\n\nRecently, Svith has mapped the wages of municipal employees compared to costs and wages in the private market. His story showed that 1.4 billion Kroner (about $188 million) were being put in the wrong pockets and that about 100,000 employees earned too little. \"I could only have done this story using a map,\" Svith said.\n\nGIS also speeds up production. \"Before, graphics did the maps and you couldn't do any analysis and it took a whole day to create. Now, I can do a map and send it to graphics. We have a lot more maps in the paper now.\"\n\nNetherlands\n\nDick van Eijk a reporter and editor with NRC Handelsblad in the Netherlands also uses mapping in reporting. Some are quicker mapping projects. \"A growing number of maps is not the result of extensive data analysis, but not a copy of a map provided by others either. In these cases we receive a data file, e.g. from a government agency, from Eurostat, from the OECD or another organization. This file contains data for every municipality, region of country in a particular area. From these data we generate our own thematic maps using mapping software. Some simple calculations may precede the creation of the map, e.g. dividing by number of inhabitants of the area. Sometimes this is done in a spreadsheet (Microsoft Excel), sometimes in the mapping program itself (MapInfo, both on the Mac and on PC).\"\n\nAt NRC Handelsblad there are three people familiar enough with MapInfo and Excel to do mapping – two reporters and one person in the graphics department.\n\nVan Eijk says they also do bigger CAR projects using mapping. In April, Van Eijk did a project on royal decorations awarded on Queen's Day.\n\n\"Typically some 2,000 people a year receive such an award on Queen's Day, and some 2,000 more on other occasions. It turned out that the geographical distribution of these decorations was very uneven. That was remarkable. I had a hunch that this was a structural phenomenon, but I could not prove that using one year of data. So this year I analyzed all the royal decorations awarded on Queen's Day since the new law on these decorations in 1996.\" The new law requires uniform rules in every part of the country for determining who gets awards.\n\n\"Using five years of data I could show not only that the geographical distribution of royal decorations was very uneven, but I could also prove that this uneven distribution was consistent over the years. In some municipalities the number of decorations per head was more than twenty times as high as in others.\"\n\nNew journalists are also learning mapping. Peter Verwey, director of the new media department at the School of Journalism in Utrecht, the Netherlands, incorporates ArcView training into digital journalism coursework. Verwey's students used maps to study the relationship between the crime rate in a police district and the money spent on the police force in that district. They also have used maps to look at restaurant inspections in Utrecht.\n\nAs CAR has become a more common tool in newsrooms in Europe so have geographical information systems. Will records access issues slow European journalists' ability to do mapping?\n\nNot in Denmark, says Mulvad of Morgenavisen Jyllands-Posten and the chair of the Danish Institute for Computer-Assisted Reporting.\n\n\"The country is divided geographically in lots of ways – postal codes, local authorities, and so on. It's very easy to divide political thinking, income, police solving crimes, prostitutes, foreigners and so on geographically and put it in a map in the paper,\" he said. \"Actually, because of that and because of the easy way to use maps, we think mapping perhaps will be used before using database managers for a lot of journalists.\"\n\nJennifer LaFleur can be reached by e-mail at jlafleur@postnet.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-for-more-information","slug":"2000-september-for-more-information","url":"https://dwillis.github.io/uplink/article/2000-september-for-more-information.html","year":2000,"month":"September","monthNum":9,"headline":"For more information:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A reference guide providing resources for geographic data visualization and mapping tools, including links to mapping software, GIS platforms, and computer-assisted reporting institutions, with an example map showing medal distribution across municipalities.","keywords":["mapping","GIS","data"],"topics":["Technology"],"technologies":["ArcView","MapInfo"],"full_text":"For more information:\n\nMAPS: www.mapping.dk/\n\nArcView: www.esri.com\n\nMapinfo: www.mapinfo.com\n\nDanish Institute for Computer-assisted Reporting (DICAR) www.dicar.org\n\nThe map to the right, created by Dick van Eijk, shows the absolute number of higher medals per municipality.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-school-test-scores","slug":"2000-september-school-test-scores","url":"https://dwillis.github.io/uplink/article/2000-september-school-test-scores.html","year":2000,"month":"September","monthNum":9,"headline":"School test scores","kicker":"MAPPING","author_name":"Seth Hemmelgarn","author_title":"","affiliation":"Mercury News","summary":"A California journalist mapped school performance rankings across a county using GIS tools, discovering that highest-ranked schools clustered in wealthy areas while lowest-ranked schools appeared in lower-income neighborhoods, a pattern made clearer by overlaying median income data from Census tracts.","keywords":["mapping","schools","income"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["dBase","ArcView"],"full_text":"In California, school funding is tied to schools' performance on standardized tests. Starting with the 1999 scores, each school is given a 1 to 10 ranking. Schools with a 1 are in the bottom 10 percent of the state, and schools with a 10 are in the top 10 percent.\n\nThe state released the data in January. We wanted to map where schools with the highest and lowest ranks were in our county. The only problem was, we had the score for each school, but we didn't have the address. So we got the addresses from the state, and created a dbaseIV file with all of the information we needed.\n\nAfter I geocoded the file in ArcView, I used the query wizard to pinpoint the schools with ones and tens. Not surprisingly, most of the highest-ranked schools formed a blob running along the county's western foothills, where millionaires are commonplace. The lowest-ranked schools showed up in a patch on the city's eastside, where incomes are generally lower.\n\nThis was my first official mapping assignment after I attended NICAR's incredibly worthwhile mapping bootcamp, so I wanted to have somebody else review my work. That's where Carl Neiburger, mapping guru and Mercury News cohort, came in.\n\nI'd used two different queries to show the highest and lowest ranking schools, which resulted in two different shape (shp) files. Carl suggested selecting all of the ranks – from 1 to 10 – into one shp file. Then, I just eliminated the ranks we didn't need. This meant one less shp file to keep track of.\n\nCarl also enlisted the help of Nadine Selden, from the paper's marketing research department. Nadine pulled the median income for each of our county's Census tracts from Claritas, and Carl included this in the map. The result was a map that not only showed where the high and low-scoring schools were, but what the income ranges in those areas were.\n\nSeth Hemmelgarn can be reached by e-mail at SHemmelgarn@sjmercury.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-mapping-listservs","slug":"2000-september-mapping-listservs","url":"https://dwillis.github.io/uplink/article/2000-september-mapping-listservs.html","year":2000,"month":"September","monthNum":9,"headline":"Mapping Listservs:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Guide to journalism mailing lists including NICAR-L for computer-assisted reporting, JAGIS-L for geographic information systems, and CENSUS-L for census-related discussions, with subscription instructions for each.","keywords":["journalism","mailing","lists"],"topics":["Journalism & Newsroom","Technology","Demographics & Census"],"technologies":["Internet","World Wide Web","modem/BBS"],"full_text":"NICAR-L is a busy list (expect 10-20 messages a day) hosted by the National Institute for Computer-Assisted Reporting.\n\nIt's frequented by hundreds of working journalists whose CAR skills range from expert to beginner. To join the list, send this one-line message: SUB NICAR-L firstname lastname to listproc@lists.missouri.edu.\n\nJAGIS-L: Tom Johnson's JAGIS homepage is at http://online.sfsu.edu/~jagis/. There's a sign-up link there.\n\nCENSUS-L was started early this year as an offshoot of NICAR-L. Discussion is primarily about the Census, but there may be talk about mapping as well.\n\nTo join, send this one-line message: SUB CENSUS-L to majordomo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-forming-a-users-group","slug":"2000-september-forming-a-users-group","url":"https://dwillis.github.io/uplink/article/2000-september-forming-a-users-group.html","year":2000,"month":"September","monthNum":9,"headline":"Forming a users group","kicker":"MAPPING","author_name":"Doug Haddix","author_title":"","affiliation":"The Columbus Dispatch","summary":"A Columbus-based media company created an internal GIS users group across its newsroom, marketing, and direct mail departments to share mapping expertise, data resources, and software costs. The collaborative effort has proven beneficial for data-driven journalism projects and newsroom efficiency.","keywords":["mapping","journalism","collaboration"],"topics":["Technology","Investigative Methods"],"technologies":["ArcView"],"full_text":"It all started with the self-proclaimed \"BAG lady.\"\n\nMedia analyst Gwen Brown maps data extensively to promote advertising through direct mail and in \"The BAG,\" a collection of circulars distributed weekly to 700,000 households in central Ohio.\n\nLast summer, Gwen suggested the creation of a mapping users group open to all of the local companies under the umbrella of Wolfe Enterprises Inc. In addition to On Target Marketing, the direct marketing arm of the Dispatch Printing Co., the family-owned company's holdings include The Columbus Dispatch and WBNS-TV, the local CBS affiliate.\n\nThe idea for a GIS users group emerged during a three-day course on using ArcView mapping software. The class included 14 employees from the newsroom, the newspaper's marketing department and the direct marking group.\n\nThe class diehards formed the nucleus of the GIS users group, whose mission is to share data resources, technical expertise, mapping-related news and ideas about creative uses. Members meet quarterly, rotating among the company sites, and correspond regularly between sessions.\n\nSeveral of the members have used mapping software for years, so they bring a wealth of experience and technical support to those of us in the newsroom who ventured into mapping just a year ago.\n\nMany newsrooms could benefit from becoming part of a GIS group, whether it involves marketing and advertising or other departments at their media company, a more far-flung corporate group such as ours, or a community group of users in local government and business.\n\nAlready, we've seen numerous benefits:\n\n• Substantial discounts on software and licenses bought at the corporate level with multiple users.\n\n• Professional bonds with mapping experts outside the newsroom who can help trouble-shoot technical problems and assist on enterprise work. Gwen spent several hours in the newsroom looking over data for a project on the Columbus School District and offering ideas on how to map it effectively. Cherie Smith of the marketing department took the same data and ran a regression analysis on it to find the most influential demographic factors in student proficiency test scores.\n\n• The sharing of data that already has been cleaned up and standardized. For example, our marketing department provided a database table of all drug stores in central Ohio with latitudes and longitudes. That enabled us to produce a quick map to run with a business story about an escalating retail war in the Columbus area among the big drug chains.\n\n• Creative brainstorming on tackling upcoming news stories. Our group includes several people who have worked extensively in analyzing census data; their expertise will be pivotal in helping The Dispatch prepare for the mapping mother lode of demographic data from the 2000 census.\n\n• The sharing of local shape files.\n\n• Connections with other mapping users. Newsroom representatives have attended the quarterly meetings of a local government GIS users group and reported key developments to the Wolfe Enterprises group.\n\n• The sharing of training tips and handouts from conferences, such as the mapping boot camp last fall by NICAR and the annual IRE convention.\n\nInvolvement in the GIS users group has kept the newsroom on a steady course as we continue to explore the practical uses of mapping software. Think of the group as a local version of the IRE and NICAR listserves – a source of inspiration, story ideas, trouble-shooting and emotional support.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-september-past-projects-and-investigations-of-the-columbus-dispatch","slug":"2000-september-past-projects-and-investigations-of-the-columbus-dispatch","url":"https://dwillis.github.io/uplink/article/2000-september-past-projects-and-investigations-of-the-columbus-dispatch.html","year":2000,"month":"September","monthNum":9,"headline":"Past projects and investigations of The Columbus Dispatch","kicker":"","author_name":"","author_title":"","affiliation":"The Columbus Dispatch","summary":"The Columbus Dispatch highlights major investigations into educational inequality in Columbus schools and ineffective spending by the Appalachian Regional Commission, demonstrating the publication's commitment to investigative journalism on systemic disparities.","keywords":["investigations","inequality","spending"],"topics":["Investigative Methods","Education","Government & Politics"],"technologies":[],"full_text":"Past projects and investigations of The Columbus Dispatch are available online at www.dispatch.com/news/special/\n\nThe stories include:\n\n\"Dividing Lines\": The Dispatch investigated the division of Columbus public elementary schools along lines of race, poverty, student achievement, teacher experience, building conditions and educational resources. Published June 25-28, 2000.\n\n\"Appalachia: Hollow Promises\": In this five-day series, The Dispatch investigated the Appalachian Regional Commission's spending to find out where the money from the commission was going and why it often bypassed some of the neediest counties in the region. Published Sept. 26 - 30, 1999.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-private-foundations","slug":"2000-october-private-foundations","url":"https://dwillis.github.io/uplink/article/2000-october-private-foundations.html","year":2000,"month":"October","monthNum":10,"headline":"Private foundations","kicker":"PHILANTHROPY","author_name":"Jeff Porter","author_title":"","affiliation":"Arkansas Democrat-Gazette","summary":"An exploration of the significant economic impact of private foundations in Arkansas, revealing the vast assets they control and their contributions to societal needs.","keywords":["philanthropy","foundations","economy"],"topics":["Business & Economy","Investigative Methods"],"technologies":[],"full_text":"Outside of the grant-seeking or grant-making industries, the term \"private foundation\" means little. But for colleges and universities, hospitals and libraries, museums and zoos, the term means this: money. We assign scores of reporters to cover government. An entire section is devoted to business. But when it comes to philanthropy - billions and billions of dollars - we write routine stories about big gifts and seldom look below the surface. James M. Naughton, president of nonprofit The Poynter Institute, would tell you that's wrong. In a speech last year for the Knight Program in Ethics in the Department of Journalism and Mass Communications at Washington and Lee University, he said: \"Nonprofit businesses had 7 million employees. They represented 6 percent of the nation's total economic output. Since 1970 they were growing at a rate four times as fast as the rest of the economy. You would think that they - and the implications of their growth - would be at least Topic C, if not Topic A or B, in the news.\" Taking Naughton's advice, the Arkansas Democrat-Gazette last fall embarked on a detailed look at private foundations. What we found was astounding. Arkansas' 273 private foundations controlled about $1.5 billion in assets and handed out $116 million during the last year reported. And that's for a state that lags behind the rest of the nation. Without the foundations, the impoverished Mississippi River Delta would be even more desperate. Fewer students could afford college tuition. The arts would have a much smaller canvas. To match just one 1998 grant from the Walton Family Foundation in Bentonville, Ark., the Girl Scouts would have to sell 110,000 boxes of cookies. It became clear to us that, directly or indirectly, the life of every Arkansan is touched. Naughton repeated an old journalism adage: \"Follow the money.\" That turned out to be easy. Most nonprofit organizations of any size are required to file tax returns - called 990 forms - with the Internal Revenue Service each year. The returns are open to the public. The nonprofits themselves are required to provide copies for reasonable fees.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-october-money-talks-in-russia","slug":"2000-october-money-talks-in-russia","url":"https://dwillis.github.io/uplink/article/2000-october-money-talks-in-russia.html","year":2000,"month":"October","monthNum":10,"headline":"Money talks in Russia","kicker":"INTERNATIONAL CAR","author_name":"Natalya Shulyakovskaya","author_title":"","affiliation":"The Orange County Register","summary":"A journalist explores Russia's surprisingly accessible data markets, discovering that personal information can be purchased for relatively small amounts of money, starting with an anecdote about a 1998 train ticket purchase that revealed how easily personal data is collected and potentially sold.","keywords":["databases","privacy","Russia"],"topics":["Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Five hundred dollars speaks volumes: $1,500 can do miracles. And you might not want to know what $5,000 worth of Russian databases is ready to tell you. (Well, I didn't have enough cash to properly test it.)\n\nRussia is a wonderland for a CAR junkie.\n\nMy ears first perked up when I bought a train ticket from St. Petersburg to Moscow in early 1998. A hefty woman selling tickets at the train station demanded my ID and dutifully typed my name and passport number into her computer. When a printer spewed out my yellow ticket, my name was on it.\n\nLater, I asked a friend who runs IT services for a new Moscow magnate if there was a way to get a copy of a database listing all train tickets. He said: \"Sure, I have","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-boot-camp-lessons","slug":"2000-october-boot-camp-lessons","url":"https://dwillis.github.io/uplink/article/2000-october-boot-camp-lessons.html","year":2000,"month":"October","monthNum":10,"headline":"Boot camp lessons","kicker":"FIRST VENTURE","author_name":"Edward L. Carter","author_title":"Reporter","affiliation":"Deseret News","summary":"A journalist describes lessons learned from a NICAR computer-assisted reporting boot camp while analyzing city employee salary data for a pay equity story, emphasizing the importance of understanding open records laws, requesting complete datasets, and starting with a manageable minimum story approach.","keywords":["salary","data","journalism"],"topics":["Investigative Methods","Data & Databases","Demographics & Census"],"technologies":["Excel"],"full_text":"Analyzing city employee salaries for a story on pay equity reinforced three lessons I learned at NICAR's computer-assisted reporting boot camp:\n\n- Do your homework\n- Get everything\n- Guarantee the \"minimum story\"\n\nMy initial attempt at applying what I learned at a March boot camp was the well-traveled \"pet name\" story, which seemed to make an impression on readers and editors of the Deseret News, a 70,000-circulation daily in Salt Lake City.\n\nWith that small success under my belt, I decided to research employee salaries for two cities on my beat. Editors asked me to expand it regionally. The result was a story about the challenges facing female employees in Utah's patriarchal culture.\n\nMy first lesson\n\nThe first lesson I learned was the importance of doing your homework, especially on FOI issues. When I approached the cities' personnel directors, I was told I could have ranges of salaries but not the exact earnings of employees. However, I knew from researching my state's open records laws that salaries had to be disclosed. I submitted formal requests and waited.\n\nAfter the obligatory stops on the attorneys' desks, my requests were approved. One city told me I would have to pay a programmer's hourly costs to run my reports. I recalled IRE and NICAR Executive Director Brant Houston's advice (\"Your negotiating position should begin at zero,\" he told us) and I told the personnel director I wouldn't pay unless they first told me how much per hour and how many hours it would require. Two days later, I got a call saying a floppy disk was ready for me. Money was never discussed again.\n\nWhen I attempted to convert the disk's text file into a Microsoft Excel document, I ran into a snag. Only about half the data came in, and I couldn't find the rest. Bypassing administrators, I called the programmer back and asked him to re-send the salary data in Excel format.\n\nGetting everything\n\nMy second lesson was to follow Brant's advice about getting everything. I recall him saying at boot camp that it's easier to remove unnecessary data than it is to fight for information a second time. Unfortunately, I learned the hard way. I had only asked for employee names, salaries and job titles. Once I began running some pivot table reports, I realized I also needed tenure and the city department in which the employee worked (it wasn't always obvious from the job title). I ended up going back to the cities to get the departments, but I never got the years of service. I also had to backtrack and get employees' gender since I couldn't tell from some of the names.\n\nAdditionally, I neglected to push for the record layouts and code sheets, which was stressed at boot camp. In this case, I was able to work through the problem.\n\nMinimum story\n\nThird, I learned the value of the \"minimum story.\" Being a computer-assisted reporting novice, I didn't want to get in over my head. After spending several days perusing and sorting the data, I had several ideas about where the story could go. But the most obvious thing — and the easiest story to do — was that female employees were very scarce at the top management levels. In one city, only two women graced the 100 top-paid employees. I also realized that women were concentrated in jobs as secretaries, police dispatchers and librarians — not a revelation, to be sure, but the data allowed me to quantify it.\n\nI pursued that angle, and used a few simple Web searches to gather information from a couple of national organizations about pay gaps between male-dominated jobs and female-dominated jobs. One of the most interesting results came from pivot table reports I ran to compare men and women who held the same job titles. That showed that women who got the same jobs as men earned comparable salaries, but the hang-up was that few women had crossed the line from the traditional female jobs to the traditional male jobs. Readers responded well to the story, a topic of interest not frequently investigated in this state.\n\nEdward L. Carter can be reached by e-mail at carter@desnews.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-doctors-and-damages","slug":"2000-october-doctors-and-damages","url":"https://dwillis.github.io/uplink/article/2000-october-doctors-and-damages.html","year":2000,"month":"October","monthNum":10,"headline":"Doctors and damages","kicker":"MEDICINE","author_name":"Jack Dolan","author_title":"","affiliation":"The Hartford Courant","summary":"Journalists used data analysis and investigative legwork to de-anonymize the National Practitioner Data Bank by cross-referencing malpractice records with public state disciplinary actions to identify problematic doctors for an investigative series.","keywords":["database","investigation","doctors"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["National Practitioner Data Bank"],"full_text":"Cracking the \"public use\" version of the National Practitioner Data Bank — the secret national repository for records of adverse actions against doctors since 1990 — took ten percent CAR skills and 90 percent shoe leather.\n\nThe data file has the barest of identifying information attached to more than 200,000 records including malpractice payments made on behalf of medical practitioners, state license actions and disciplinary measures taken by hospitals. The practitioner's names are replaced by random ID numbers generated by Department of Health and Human Services computers. Ages and medical school graduation dates are given to the nearest decade.\n\nSo, before you start trying to attach names to the anonymous records of 100,000 doctors in the databank, make some careful choices about whom you think is worth tracking down.\n\nFor our \"White Coats, Dark Secrets\" series, The Hartford Courant focused on doctors with at least 25 entries overall, at least $3 million in total malpractice payments and at least one disciplinary action taken by a state licensing board. We ignored dentists and podiatrists since their screw-ups, although a great inconvenience for the injured patient, aren't likely to be life threatening.\n\nBe prepared for strong resistance from the medical community to almost any criteria you select. They will argue that the number of malpractice suits a doctor faces depends more on his specialty than his skill. They'll also say certain areas of the country are simply more litigious than others. The truth is, they are right on both counts.\n\nOur criteria were so broad, however, that nobody from the American Medical Association or on the staff of the NPDB seriously argued that we were unfairly skewering good doctors who had caught a few bad breaks.\n\nOnce you've picked your targets, the real work begins. State disciplinary actions are almost always public, but that doesn't make them easy to find. Most states have Web sites with some information about regulatory actions. Some, such as Massachusetts, New York and Florida, let you look up physicians by name. When working with the NPDB, however, you don't have the name.\n\nInstead, we found ourselves working through a tedious process of elimination. For example, if one of the anonymous doctors on our list had a six-month suspension in New York in 1996 and a reprimand in Massachusetts in 1998, we'd go to the records from those states and compare the lists. Most of the time that left us with a shorter list of only a handful of doctors who fit both criteria. Then we checked each doctor's date of birth and year of graduation from medical school (both conveniently provided by the AMA's Web site in most cases) to see which matched the decades listed in the NPDB.\n\nWhen we needed records from a state that didn't have a decent Web site, we turned to Public Citizen's \"Questionable Doctors\" series. This hefty, three-volume paperback catalogue lists license surrenders, revocations, suspensions and licenses put on probation from fifty states and the District of Columbia. But it's not cheap. The 2000 version, with actions from 1990 through 1999, costs $400.\n\nAnother drawback to the books, aside from the expense and the paper, is that the doctors are listed alphabetically by state and name. That makes a lot of sense if you are looking up your family practitioner. But when you are going from the NPDB you'll be searching for a type of action taken in a given year. We nearly went blind doing this in the Public Citizen books.\n\nOnce you have a couple of possible names, however, the Public Citizen books are a great resource. Instead of going from Web site to Web site to make sure all of the actions against a doctor match in each state, you have each doctor's full regulatory history at your fingertips. That means you can quickly pick the right doctor from a lineup of three or four possibilities. If you exhaust the leads provided by state disciplinary actions, and","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-programs-for-parsing","slug":"2000-october-programs-for-parsing","url":"https://dwillis.github.io/uplink/article/2000-october-programs-for-parsing.html","year":2000,"month":"October","monthNum":10,"headline":"Programs for parsing","kicker":"TECH TIP","author_name":"David Heath","author_title":"","affiliation":"Seattle Times","summary":"The article explains how to handle difficult-to-import data files using programming, specifically Visual Basic, when traditional tools like Excel's import wizard fall short. It outlines five key programming concepts needed to parse loosely structured data and recommends learning resources for getting started.","keywords":["programming","data","parsing"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Visual Basic","FoxPro","Perl","Excel","Access","Word","Monarch","QBasic","HTML"],"full_text":"If you work with data much, at some point you're going to deal with files that defy easy import.\n\nKnowing a few tricks can sometimes save you lots of work. For example, if a text file is missing carriage returns at the end of each line — a common dilemma — you can open it in Word and save it into a new text file. Voilà. You now have carriage returns.\n\nYou can also do a lot with search and replace tools, Excel's import wizard and Monarch, a terrific software package that converts reports back into databases. But there are times when even Monarch falls short. That's especially true if columns don't line up or the structure of each record is not consistent.\n\nWhen the going gets rough, the tough write a program.\n\nWriting your own program is the only solution I know if you're trying to parse data that's loosely structured. Say that you want to take a collection of Web pages and put all the links into a database. The links aren't hard to find; they always begin with \"<a href=\" But otherwise, the pages may lack any consistent format.\n\nThere is a way to do this, but you have to learn a smidgen of programming. If you already write simple programs to clean data, you probably already have most of the skills needed. If you are a novice or have never tried programming, the good news is that you don't have to learn that much. You'll be working with a limited set of functions and statements.\n\nI'm going to show you how to deal with problems using Visual Basic. You could do the same type of thing with FoxPro or, for those fearless of its steep learning curve, Perl. The latter is especially powerful at pattern matching, making it an ideal tool for finding HTML tags.\n\nVB's chief advantages are that it's already built in to Access, Excel and Word — so you don't have to buy any new software — and it was especially created to be an easy-to-learn programming language. (Although you've already got a version of VB, if you buy the full version you'll get a lot of debugging tools that can be a life saver.)\n\nTo manipulate text in VB, you need to learn five things:\n\n1. How to declare variables, which simply means defining them as either characters or numbers.\n\n2. How to open and close files.\n\n3. How to read from files and write to files.\n\n4. How to use text functions, such as MID, INSTR, LEFT, TRIM and LEN\n\n5. How to create loops.\n\nWhen you're first learning Visual Basic, it's easy to get distracted making pretty pictures: forms, buttons, labels, etc. But your program doesn't have to look good; it just needs to get the job done. In fact, all the tools you need can be found in the old QBasic, that free program that comes with DOS. A great book for learning Basic programming is \"QBasic by Example,\" written by Greg M. Perry.\n\nThere's no way to teach you everything you'll need to start programming in a brief article, so I highly recommend getting a book. However, I can walk you through a fairly simple program that parses data, and that should give you a pretty good idea of what to expect.\n\nLet's take the U.S. Census home page, at www.census.gov, and parse all the links into a comma-delimited file that we can easily import into a database. Save the page to your hard drive and note the location, because you'll need it for later. The file is tricky because the page is very loosely structured and the HTML tags are not always consistent. Sometimes they are followed by quotation marks and sometimes they aren't. So we have to include some conditional statements in our programs.\n\nOf course, it's not efficient to write a program to parse a single page, but this program could parse scores of Web pages all at once. You'll find plenty of situations where such a program can do wonders.\n\nTo run this, go to Tools | Macro | Visual Basic Editor in Excel and type in the following code. Then simply click on the Run button.\n\nPrivate Sub Command1_Click()\n\nStart by declaring the data type of all the variables you'll use.\n\nI've just made up the variable names on the left, starting some with \"str\" to signify string, int to signify integer or lng to signify Long Integer.\n\nDim strLine As String\nDim intRead, intWrite As Integer\nDim lngFindAHREF, lngEndHere, lngFindAlt As Long\nDim lngStartHere, lngFindEndTag, lngFindName As Long\nDim lngFindGreaterThan, lngFindLessThan As Long\nDim strHref, strAlt, strName As String\nDim strNextLine As String\n\nFreeFile() is a useful function that assigns an unused number to a file. Basic requires files to be numbered for opening, closing, reading and writing.\nintRead = FreeFile()\n\nIn this case, intRead is now equal to 1. We can now open our Web page to read from as file No. 1.\nOpen \"c:\\census\\www_census_gov.html\" For Input As #intRead\n\nNext we want to create a second file, No. 2, that we are going to write the data to.\nintWrite = FreeFile()\nOpen \"c:\\census\\filetoimport.txt\" For Output As #intWrite\n\nSo all we've done is opened the Web page, which was previously saved to your computer. Your path may differ from that above. Then we created a file called \"filetoimport.txt\" that we will write to.\n\nWe're now ready to look through the Web page for patterns. The statement for that is\nDo while NOT EOF(intRead)\nMeaning read until you get to the End Of File #1, followed by\nLine Input #intRead, strLine,\nmeaning\nGo to the next line and put it into my variable \"strLine.\"\nDo While Not EOF(intRead)\nLine Input #intRead, strLine\n\nTime to start looking for the pattern \"a href\". The function InStr gives the number of the characters from the left that a pattern appears.\n\nvbTextCompare _\nignores upper and lower cases.\nlngFindAHREF = InStr(1, strLine, \"A HREF\", vbTextCompare)\n\nIf we find \"a href\" in a line, we want to grab that line eight characters to the right of the start of the tag.\nIf lngFindAHREF > 0 Then strLine = Mid(strLine, lngFindAHREF + 8)\n\nSome of the lines wrap, so we have to make them one long line using a loop.\nDo While Not InStr(1, strLine, \"</A\", vbTextCompare) > 0\nLine Input #intRead, strNextLine\nstrLine = LTrim(strLine) + \" \" + LTrim(strNextLine)\nLoop\n\nWe're looking for the end of the first tag and will put the text in between into a variable called strHref.\nlngEndHere = InStr(1, strLine, \">\", vbTextCompare)\nstrHref = Mid(strLine, 1, lngEndHere - 1)\n\nNow we're looking for the ALT tag, which is what shows up if an image can't be displayed on the Web page. If we find it, we'll put the text in that tag into the variable \"strAlt.\" To help find the end of the tag, we're searching for quotation marks. I've used the Chr() function because we can't put quotations within quotations.\nlngFindAlt = InStr(1, strLine, \"ALT\", vbTextCompare)\nIf lngFindAlt > 0 Then\nlngStartHere = InStr(lngFindAlt, strLine, Chr(34)) + 1\nlngEndHere = InStr(lngStartHere, strLine, Chr(34))\nstrAlt = Mid(strLine, lngStartHere, lngEndHere - lngStartHere)\nEnd If\n\nFinally, we're looking for the last tag, which sometimes will include text that you see on the Web page as a hyperlink.\nlngFindEndTag = InStr(1, strLine, \"</a\", vbTextCompare)\nstrLine = Mid(strLine, 1, lngFindEndTag 1)\n\nlngFindGreaterThan = InStr(1, strLine, \">\")\nlngFindLessThan = InStr(1, strLine, \"<\")\nIf lngFindLessThan > 0 Then\nstrLine = Mid(strLine, lngFindGreaterThan + 1)\nEnd If\nlngFindName = InStr(strLine, Chr(62))\nstrName = Mid(strLine, lngFindName + 1)\n\nWe have a little bit of clean up work to do. There are extraneous quotation marks to remove and pieces of unwanted text. Also, some of the links begin with \"http://\" and some don't.\nIf Left(strHref, 1) = Chr(34) Then strHref = Mid(strHref, 2)\nIf InStr(1, strHref, \" \") > 0 Then strHref = Mid(strHref, 1, InStr(1, strHref, \" \") - 2)\nSelect Case Left(strHref, 1)\nCase \"h\"\nstrHref = strHref\nCase \"/\"\nstrHref = \"http://www.census.gov\" + strHref\nCase Else\nstrHref = \"http://www.census.gov/\" + strHref\nEnd Select\nIf Right(strHref, 1) = Chr(34) Then strHref =\nMid(strHref, 1, Len(strHref) - 1)\n\nWe're combing the ALT tags with the hyperlinks because they do essentially the same thing.\nIf strAlt = \"\" Then strAlt = strName\n\nNow we'll write the variables to a comma-delimited file.\nWrite #intWrite, strHref, strAlt\n\nIt's time to clean out the variables and run through the loop again.\nstrHref = \"\"\nstrAlt = \"\"\nstrName = \"\"\nEnd If\nLoop\n\nYou always have to close files that you've opened. The command for that is simple.\nClose\nEnd\nEnd Sub\n\nDavid Heath can be reached by e-mail at dheath@seattletimes.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-vbasic-resources","slug":"2000-october-vbasic-resources","url":"https://dwillis.github.io/uplink/article/2000-october-vbasic-resources.html","year":2000,"month":"October","monthNum":10,"headline":"VBasic resources","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A curated list of web resources for learning and exploring Visual Basic programming, including Microsoft's official documentation and community-driven developer sites.","keywords":["Visual","Basic","Resources"],"topics":["Technology"],"technologies":["Visual Basic"],"full_text":"Try these Web sites for more on Visual Basic:\n\nhttp://msdn.microsoft.com/vbasic\n\nwww.vbexplorer.com\n\nwww.programmersheaven.com/zone1/index.htm\n\nwww.vb-helper.com\n\nwww.vbsquare.com/beginning","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-uplink-story-ideas","slug":"2000-october-uplink-story-ideas","url":"https://dwillis.github.io/uplink/article/2000-october-uplink-story-ideas.html","year":2000,"month":"October","monthNum":10,"headline":"Uplink story ideas","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"A journalism newsletter invites readers to submit stories about computer-assisted reporting projects, technical solutions, or story ideas they'd like covered, with contact information for the associate editor.","keywords":["journalism","reporting","data"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Have you or one of your colleagues recently published a story using CAR that has not been done before or involved particularly difficult data work?\n\nDo you know of a technical problem (or its solution) that others may like to hear about?\n\nIs there some issue or beat we haven't looked at you would like to see covered?\n\nWe'd like to hear from you. Please contact associate editor Paul Monies either by e-mail at paulm@nicar.org or by phone at (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-fund-raising-expenses","slug":"2000-october-fund-raising-expenses","url":"https://dwillis.github.io/uplink/article/2000-october-fund-raising-expenses.html","year":2000,"month":"October","monthNum":10,"headline":"Fund-raising expenses","kicker":"PHILANTHROPY","author_name":"Harvy Lipman","author_title":"Reporter","affiliation":"The Chronicle of Philanthropy","summary":"An investigation by The Chronicle of Philanthropy reveals that many large nonprofits are failing to accurately report fund-raising expenses on their IRS Form 990 tax returns, despite the IRS having broad definitions of what must be reported as fund-raising costs. Researchers found that over 25% of nonprofits reporting $500,000 or more in gifts claimed zero fund-raising expenses, though audited financial statements often showed these costs were actually incurred.","keywords":["nonprofits","transparency","reporting"],"topics":["Business & Economy","Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"When the Internal Revenue Service last year issued rules forcing nonprofit organizations to provide copies of their tax returns to the public, journalists who cover charities suddenly found themselves with access to a wealth of data that previously was about as easy to extract as an impacted wisdom tooth.\n\nBut as The Chronicle of Philanthropy has discovered after years of rummaging through those IRS Forms 990, reporters should be cautious about relying on the information nonprofits provide. A case in point might be called \"The Mystery of the Disappearing Fund-raising Expenses.\"\n\nActing on a tip from a source who teaches nonprofit accounting at a major university — who told us that one of her research papers discovered that many nonprofits with sizeable incomes were reporting spending nothing on fund-raising — we decided to do a computer analysis of the fund-raising expenses reported on the 990s filed by the largest nonprofits in 1996 (the most recent year for which computerized data were available). We reasoned that any charitable organization that raised $500,000 or more in gifts during the year must have spent something to raise those funds.\n\nKeep in mind that the IRS definition of what must be reported as fund raising is pretty broad. It includes, of course, the cost of hiring outside soliciting firms and the salary paid any full-time fund-raisers on staff. But it also covers the portion of any paid staff time devoted to fund-raising (including writing grants seeking money from foundations), an allocated portion of the cost of producing a newsletter if that publication includes requests for funds, the cost of publicizing fund-raising events, and the production of any fund-raising manuals or other materials.\n\nThe data we needed was obtainable from the Urban Institute's National Center for Charitable Statistics in Washington, which has CD-ROM sets of basic data filed by the nation's nonprofits. We found nearly 5,000 organizations which reported at least $500,000 in gifts for that year — of which more than 25 percent had reported zero fund-raising expenses. In many cases — approximately half — there were good explanations; the most common being that the charity reporting no fund-raising costs was affiliated with another nonprofit organization that handled all the fund-raising. Another common reason covered the foundations formed by state universities: The schools themselves — which as government entities don't file 990s — do the fund-raising, while the foundations distribute the money from the gifts, so the foundations legitimately are reporting zero in the fund-raising column.\n\nBut a large number simply didn't bother to report what they'd spent on fund-raising. In almost every case, the organizations' audited financial statements did list the expenses, which is an important lesson for anyone covering nonprofits: Don't settle for the 990, ask for the audited financial statements, and get real huffy if they don't want to give them to you (ask them what they have to hide).\n\nWe also obtained databases from three states — California, New York and Ohio — that require registration from all professional fund-raisers and charities that use solicitors. We found dozens of examples of charities that had hired solicitors and still told the IRS they'd spent nothing on fund-raising.\n\nThe Zoological Society of Cincinnati, for instance, spent at least $500,000 on fundraising, including the hiring of a telemarketing company that conducted a multi-year soliciting campaign. Trinity University in San Antonio spent $2.5 million on fund-raising over three years. Neither reported any to the IRS.\n\nSome of the explanations were priceless: One group, in a classic case of Orwellian Newspeak, told us that \"to put fund-raising in the column called fund-raising would be misleading.\" Its argument was that since all the organization does is raise money to distribute to other charities, its fund-raising amounted to its official program (program services, the money spent on a charity's stated function, is reported on a different line of Form 990). The IRS was not amused, pointing out that the organization's program is to distribute money, not raise it, and that everything it spent on fund-raising should be listed as fund-raising expenses.\n\nThe Arizona Community Foundation insisted it does no fund-raising, and then sent us a copy of its newsletter which included a half-page solicitation of bequests.\n\nA large number of charities pointed the finger at its outside accounting firms, saying they filled out the tax forms. Most of the major firms refused to comment, and those that did insisted their policies require precise reporting of all fund-raising costs.\n\nWhy would nonprofits go to such lengths to hide their fund-raising costs? Two reasons: One, most of the watchdog groups in this area rate charities based on the percentage of their revenue they spend on their programs, and if you can make it seem like as little as possible is going toward fund-raising, you come out looking better to potential donors. Two, there are virtually no penalties for hiding fund-raising costs, because the IRS doesn't even look at them.\n\nThe results of our investigation should be an object lesson for journalists and the general public. Examine the tax forms of nonprofit organizations with care, and apply common sense.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-the-story","slug":"2000-october-the-story","url":"https://dwillis.github.io/uplink/article/2000-october-the-story.html","year":2000,"month":"October","monthNum":10,"headline":"The story","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This article provides resources and references for investigating nonprofit fundraising expenses and charitable giving practices, including access to relevant databases and organizations that track philanthropic data.","keywords":["fundraising","nonprofits","philanthropy"],"topics":["Business & Economy"],"technologies":["Internet","World Wide Web"],"full_text":"The Chronicle of Philanthropy story on fundraising expenses is available from the IRE Resource Center, Story #16587. Call (573) 882-3364 for more information.\n\nOn the Web:\nNational Center for Charitable Statistics, http://nccs.urban.org.\n\nIndependent Sector, a coalition of foundations, nonprofits and corporations, www.indepsec.org.\n\nThe Foundation Center, http://fdncenter.org.\n\nNational Council of Nonprofit Associations, www.ncna.org.\n\nQuality 990, a project to improve the quality of IRS form 990 reporting, www.qual990.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-tipsheets","slug":"2000-october-tipsheets","url":"https://dwillis.github.io/uplink/article/2000-october-tipsheets.html","year":2000,"month":"October","monthNum":10,"headline":"Tipsheets","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"These IRE tipsheets provide guidance on investigating tax-exempt organizations and nonprofits, including where to find disclosure information, government contacts, and state regulators.","keywords":["nonprofits","investigation","disclosure"],"topics":["Business & Economy","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"Tipsheet #1235 from 2000 IRE national conference. This tipsheet for covering tax-exempt organizations shows where to find information for particular organizations and how to compare them as a sector of the economy. It also outlines disclosure procedures.\n\nTipsheet #1022 from 1999 IRE national conference. This provides a list of government offices to contact for information regarding charities that solicit within your state as well as a charity and nonprofit organization source list.\n\nTipsheet #1021, also from 1999, provides an index of charity regulators in every state as well as a sample 990 form filled out from a nonprofit in Hawaii.\n\nThe tipsheets can be downloaded from the IRE and NICAR Web site at www.ire.org/resourcecenter. Search the tipsheet database, and then follow the directions below the abstract for downloading a PDF file.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-october-disciplinary-records","slug":"2000-october-disciplinary-records","url":"https://dwillis.github.io/uplink/article/2000-october-disciplinary-records.html","year":2000,"month":"October","monthNum":10,"headline":"Disciplinary records","kicker":"EDUCATION","author_name":"Meggen Lindsay","author_title":"Editor","affiliation":"The Miami Student","summary":"Student journalists at Miami University fought a legal battle to access college disciplinary records, ultimately losing when the U.S. Department of Education intervened to enforce FERPA privacy protections, though the decision is being appealed.","keywords":["disciplinary","records","privacy"],"topics":["Investigative Methods","Education","Data & Databases"],"technologies":[],"full_text":"The pursuit of open disciplinary records at colleges and universities has become a constant and often embittered battle between student press and school administrators.\n\nI experienced this conflict first-hand last year when I worked on a three-month project to create and analyze a database of Miami University's disciplinary records.\n\nCollege disciplinary records are considered \"academic\" records and are protected under the Family Educational Rights to Privacy Act. The Miami Student, Miami University's student paper, argued in 1996 that the records are not academic and should not be covered by FERPA. The Student sued the school for access to them under the Ohio Public Records law.\n\nThe Ohio courts sided with the students, and although the university appealed to the U.S. Supreme Court, which refused to hear the case, full disciplinary records were to be released. But when the university began releasing records in 1998, the U.S. Department of Education asserted that Miami was in violation of FERPA, and sued the school in federal court to prevent the records' release.\n\nThe Department won its case last April, and the decision is being appealed by The Chronicle of Higher Education, one of the papers that initially requested Miami's records.\n\nAs editor of The Student last year, I became interested not only in the court cases, but also in the records themselves. There were close to 300 records in a box on my office floor — throughout the lawsuit, no one had even looked at the actual records.\n\nBecause The Students' request failed, the disciplinary records the paper received (from the university) were devoid of all personally identifiable information such as the students' names, ages and Social Security Numbers.\n\nThey consisted of two letters sent to the charged student: one initial citation informing the student of what he or she was being charged with, and one final letter explaining the case's outcome. A description of the incident, the violation with which the student was","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-sticker-shocked","slug":"2000-november-sticker-shocked","url":"https://dwillis.github.io/uplink/article/2000-november-sticker-shocked.html","year":2000,"month":"November","monthNum":11,"headline":"Sticker shocked","kicker":"HOUSING","author_name":"David Heath and Justin Mayo","author_title":"","affiliation":"The Seattle Times","summary":"An analysis of Seattle's housing market reveals that despite rapid price increases in desirable neighborhoods, homes are more affordable now compared to the last 15 years due to rising incomes and lower interest rates.","keywords":["Seattle","housing","affordability"],"topics":["Data & Databases","Business & Economy","Investigative Methods"],"technologies":["SQL Server","ArcView","SPSS","Excel"],"full_text":"Talk around the office espresso machine in Seattle often includes gee-whiz accounts of a neighbor's house that sold at some astounding price. Tales of properties going for $100,000 over list price are common. One 3,000-square-foot house in trendy Washington Park overlooking Lake Washington recently sold for $1 million over the $1.6 million asking price. However, a detailed analysis by The Seattle Times showed that houses are more affordable today than at almost any time in the past 15 years. Rising incomes and falling interest rates during that period have actually overcome soaring home prices. The thrust of the eight-part series, called 'Sticker Shocked,' was to break down housing trends by neighborhood. The newspaper obtained 15 years' worth of sales data from assessor's offices in three counties, and some GIS maps, that served as the backbone to the analysis. We used SQL Server for the primary database work, ArcView for the mapping, and SPSS, a statistical software, to calculate medians. Excel proved extremely useful for computing financial information, such as interest rates, payment over time and annuities. In Bill Gates' neighborhood of Medina, the average house sold for $811,000 last year, while the median price was $469,000. Depending on which number you use, prices rose either 55 percent or 15 percent in a year. The latter is far more accurate. Using a simple formula borrowed from county assessors and real-estate appraisers: comparing homes based on price per square foot. We'd calculate the median price per square foot by neighborhood and do the math from there. This gave us a large number of sales for each neighborhood in each year, making it easy to show annual changes in appreciation.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-november-inmate-imports","slug":"2000-november-inmate-imports","url":"https://dwillis.github.io/uplink/article/2000-november-inmate-imports.html","year":2000,"month":"November","monthNum":11,"headline":"Inmate imports","kicker":"CRIMINAL JUSTICE","author_name":"Greg Paraham","author_title":"Researcher","affiliation":"The Commercial Appeal","summary":"A reporter and researcher collaborate to analyze prison data by matching Wisconsin inmate records with out-of-state prison populations to investigate whether privatized Tennessee prisons disproportionately house violent offenders. The project required extensive data preparation, including converting disparate data sources into linked FileMaker databases for analysis.","keywords":["data","prison","investigation"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Word","Access","FileMaker Pro"],"full_text":"Good training.\n\nThat phrase from military jargon, which means learning something valuable from a difficult task, best describes what happened when an investigative reporter asked if I could help translate some prison data he was expecting.\n\nI said yes without hesitation. As a researcher, I had worked with the reporter on other successful projects involving electronic public records.\n\nThen I discovered the condition of the data. Some of it was zipped across several disks. Some of it was on printouts and had to be scanned and then edited. All of it would need translation to a database manager because of the expected large number of records.\n\nThe next few weeks promised to be interesting.\n\nProject description\n\nThe reporter's task was to identify prison inmates from the Wisconsin prison system that had been exported to privatized prisons in our state. He suspected that two for-profit Tennessee prisons were receiving more murderers and sex offenders than other states that accepted out-of-state inmates.\n\nHe sought to verify this by using a master list of prisoners provided by the Wisconsin Department of Corrections along with the inmate population lists from prisons in Oklahoma, Minnesota, Tennessee and Texas. Matching the lists would pinpoint the identities and locations of out-of-state inmates as well as their offenses.\n\nData preparation\n\nMy task was to get the data in shape for the analysis, which turned out to be the easy part. After unzipping the disks, I used Microsoft Word and discovered that I had a 69-megabyte, field-length delimited blob of text that represented the entire inmate population in the Wisconsin prison system.\n\nI used Microsoft Access and the field-length explanation sheet sent by the Wisconsin prison official with the data to create a tab-delimited text file.\n\nI decided to use FileMaker Pro for Windows to analyze the data, because I am more familiar with its field calculation, database linking, layout and scripting (programming) capabilities.\n\nImporting the data created a 72,610 record database with 53 fields of information for each inmate. This number reflected current and past prisoners. We identified inmates who had been released to exclude them from the data set, reducing the data to 20,673 records.\n\nI created field calculations for text strings and dates and indexed necessary fields to aid in querying and sorting the records later.\n\nMeanwhile, the reporter scanned the inmate lists he received from the five prisons being examined. This data was later proofread, formatted and uploaded to another FileMaker database.\n\nThe databases were linked by the inmate's Wisconsin Department of Corrections identification number, which identified the inmates housed in the five out-of-state prisons.\n\nAfter we checked for the correct inmate name and date of birth for records that matched, we had 4,268 records that were needed for the analysis.\n\nAn immediate concern arose when we found conflicting locations for some inmates. The master list did not always reflect recent relocations. The reporter ran these down and the correct locations were added to our databases.\n\nWe were faced also with several numeric codes for prison names, sentences, offenses, race, etc. Two code types, prison name and offense identification, were used in creating lookup files that were linked to the main database. When codes matched, text was programmed to appear along with the code making it easier to identify locations of inmates and their offenses.\n\nThe offense identification codes were contained in as many as 16 fields within a record. This was confusing because some codes were duplicated within the record. We couldn't determine if any code was primary. Searching the database would be cumbersome if each field had to be included in a query.\n\nFurther examination of the data provided a solution, however. Another set of coded fields with numeric indicators for assaults, sex offenses, drug offenses, escape attempts and life sentences (indicating murderers) could be searched instead of the offense codes.\n\nTesting the data\n\nThe reporter and I spent a few hours over a couple of days testing the databases to ensure that the results of queries made sense. We checked that the number of inmates in the out-of-state prisons matched the hard-copy printouts received from prison officials. We double-checked that the prison names and codes matched because the name data was keyed and copied into data fields. We searched the names of inmates from the various prisons to verify their locations, offenses and sentences.\n\nData analysis\n\nI decided at this point to use FileMaker's scripting and layout capabilities to create the queries and reports based on the reporter's data requests. Querying the data was reduced to using the pull-down menu and selecting the appropriate search.\n\nWe used eight pieces of information from the records to create a profile of the type of prisoner being sent to our state:\n\nLife sentences for murder convictions\nSex offense indicators\nAssault indicators\nDrug offense indicators\nMaximum security rating of inmates\nLarge sentences\nPrior felony convictions\nEscape indicators\n\nQuerying the data and then sorting by prison identification code and inmate counts verified the reporter's suspicions. Private prisons were indeed making Tennessee a repository for violent criminals exported from crowded public prisons in Wisconsin.\n\nOf the five facilities under examination, one Tennessee prison was receiving the majority of Wisconsin's most offensive exiled offenders.\n\nOnce the reporter had the results of the data analysis, he was able to interview prison officials and present our findings for confirmation and comment.\n\nThe data gave us the names of some killers who were in prison for multiple charges in addition to murder. I searched the archives of newspapers in Wisconsin and the Associated Press to get background. We also used the names to find (in public records databases) relatives of inmates who might want to talk about an exported loved one.\n\nGood training indeed.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-neighborhood-blight","slug":"2000-november-neighborhood-blight","url":"https://dwillis.github.io/uplink/article/2000-november-neighborhood-blight.html","year":2000,"month":"November","monthNum":11,"headline":"Neighborhood blight","kicker":"HOUSING","author_name":"Janet Roberts","author_title":"","affiliation":"St. Paul Pioneer Press","summary":"An investigation into the effectiveness of housing inspections in St. Paul, revealing systemic issues in addressing repeat offenders of housing code violations.","keywords":["blight","inspections","St. Paul"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["GIS"],"full_text":"Anyone who's spent time answering the newsroom phone has heard from them - the people who live next door to the house with the falling-down garage or the giant trash pile or the rodent den. One at a time, the sad tales don't amount to much - maybe a news feature on a slow day. But start digging around in your city's housing inspections data, then plot what you find on a map, and the story gets better. At the St. Paul Pioneer Press, one of our city hall reporters, Charles Laszewski, wanted to investigate complaints that the housing inspections chief wasn't tough enough on owners of run-down properties. So we asked for the city's housing inspections database. We defined a problem property as one with repeated violations spanning two years or more. Then we turned to our mapping software. Our inspections data listed the street address for every property inspectors visited. We wanted a map showing the rate of problem properties per 1,000 housing units. Our efforts resulted in uncovering neighborhoods where one of every three houses had chronic housing code violations. The series brought about some results: The City Council added money to the budget to hire more housing inspectors. The mayor announced a police sweep to arrest property owners with outstanding housing court warrants. And the county is spending more money to maintain tax-forfeited properties.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-november-big-apple-rents","slug":"2000-november-big-apple-rents","url":"https://dwillis.github.io/uplink/article/2000-november-big-apple-rents.html","year":2000,"month":"November","monthNum":11,"headline":"Big Apple rents","kicker":"HOUSING","author_name":"Kevin McCoy","author_title":"","affiliation":"New York Daily News","summary":"A comprehensive analysis of New York City's rental market reveals a significant increase in median rents during the 1990s, although growth in income has kept pace.","keywords":["New York","rental","market"],"topics":["Data & Databases","Business & Economy"],"technologies":[],"full_text":"Never mind about the Knicks and the Yanks, the Mets and the Giants. New York City's true sport is real estate. All over the city, everyone talks about the ups and downs of the perpetually tight and expensive apartment market. In August, the New York Daily News cut through the hype of real estate brokers who portray in-house market surveys as the real state of the market and published the most accurate look possible at the overall state of the city rental market. The Census Bureau posted the 1999 vacancy survey online in mid-summer. The data itself was in very good condition and required little cleaning. The surveys provided a first-ever look at exactly how much New York City rents rose during the 1990s and identified the neighborhoods hit with the biggest increases. The data showed that median household rents rose 36 percent citywide during the last decade. In at least one Manhattan neighborhood, the increase was 67 percent. The paper brought the numbers to life by interviewing residents of neighborhoods where rents rose sharply and finding out what the increases meant to their lives. Although the Census Bureau doesn't conduct vacancy surveys for other cities and towns, the 2000 Census data will enable any reporter to conduct a similar analysis of local housing costs.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-november-web-links","slug":"2000-november-web-links","url":"https://dwillis.github.io/uplink/article/2000-november-web-links.html","year":2000,"month":"November","monthNum":11,"headline":"Web links","kicker":"","author_name":"","author_title":"","affiliation":"Daily News","summary":"A Daily News article provides links to a story and accompanying maps and tables, with the full content also available in the IRE Resource Center.","keywords":["links","archive","maps"],"topics":["Investigative Methods"],"technologies":["World Wide Web"],"full_text":"Web links\n\nThe Daily News story can be found at the following site:\n\nwww.nydailynews.com/2000-08-06/News_and_Views/City_Beat/a-75651.asp.\n\nThe maps and tables that ran with the story can be found here:\n\nwww.nydailynews.com/2000-08-06/News_and_Views/City_Beat/a-75663.asp.\n\nThe story is also available in the IRE Resource Center, story #16917.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-discarded-and-decayed","slug":"2000-november-discarded-and-decayed","url":"https://dwillis.github.io/uplink/article/2000-november-discarded-and-decayed.html","year":2000,"month":"November","monthNum":11,"headline":"Discarded and decayed","kicker":"HOUSING","author_name":"Ron Shawgo","author_title":"Reporter","affiliation":"The Journal Gazette","summary":"The Journal Gazette conducted an ambitious computer-assisted reporting project to identify major landlords and property managers in Fort Wayne's rental housing market, revealing that the county was a major violator of housing ordinances by failing to manage tax-delinquent properties. The project involved obtaining and merging six databases from multiple agencies to link housing violations with property owners.","keywords":["landlords","database","violations"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Access","Excel"],"full_text":"Landlords affect thousands of lives and have a major impact on the urban landscape. With that in mind, The Journal Gazette set out in April 1999 to identify the top players in Fort Wayne's rental housing industry. Although The Journal Gazette has several computer-assisted stories under its belt, this was our most ambitious CAR project. Eight months after drudging through lengthy data negotiations, importing hassles and mind-numbing data cleaning, reporter Lisa Shidler and I had a three-day package, \"Discarded and decayed.\"\n\nOur stories revealed that by failing to take over tax-delinquent properties, the county was one of the biggest violators of housing ordinances. We also talked to some typical slumlords and the city's largest landlords, who view housing violations as a cost of doing business.\n\nThe data hurdle\n\nOf course, getting data was our first hurdle. We figured that if we had a database of Fort Wayne housing violations we could link those addresses with property tax records to find the owners.\n\nWe knew our main task would be identifying property managers, the landlords who rent houses owned by other people. Because their names would not be listed as titleholders, we worked on the assumption that an address that receives numerous tax bills could be a property manager's business address. The assumption largely held true.\n\nWe contacted the county treasurer, the county building department, the city-county health department and the city office that inspects buildings for code violations. The six databases we ultimately acquired from those agencies contained county property tax records, health code violations, rodent inspections, building code violations, building demolitions and a list of properties the county could claim for back taxes.\n\nWhile some agencies gave us free data, the county wanted nearly $3,000 for the property records. After we informed officials that their 2-cents-per-record fee violated state law, the price was greatly reduced.\n\nOfficials I dealt with ran the gamut. One health official was the office's main computer authority and e-mailed files to me for free. Another county official knew little about copying data, so I helped guide him through his archaic software to save the data on disk in ASCII format.\n\nData slicing\n\nWe primarily used Microsoft Access and Excel to sort the data. (Although the project would have benefited from mapping software, it was done in our pre-mapping days.)\n\nThere was a lot of data slicing to do. For example, the rodent inspection office, a division of the health department, had its data on 10 files, one for each year since 1990. Taking into account that new fields were added to the files from one year to the next, I spliced them together.\n\nWe then added the rodent database to a much larger one listing health code violations, such as improperly disposed trash, discarded tires and appliances, and child neglect. That database also came in several files with slight differences and had to be spliced.\n\nOur two largest databases came from the county treasurer. One contained information on the property being taxed, and the other contained tax payment details, including addresses where tax bills are sent. Although the files contained a massive amount of information, we needed them only for current and former owners and the billing addresses.\n\nThe tax files caused our only data importing headaches. As our information systems people worked out the kinks, we began cleaning the data we had in hand.\n\nBecause we wanted to link all of our tables by their address fields, making addresses uniform was important. Input errors, omissions, misspellings and a general lack of uniformity in those addresses made the task time consuming and nearly overwhelming.\n\nAddress problems\n\nSome addresses had no directional prefix (N, S, E or W); others had no street type (road, drive, circle). Data cleaning took several months and required combining address fields and eliminating leading spaces, a couple of new Access tricks I learned with the help of NICAR-L subscribers. We also got a list of county and city streets with their numerical address ranges. Knowing the ranges - addresses where a street starts and where it ends - was essential in handling streets with common names and placing a violation in the proper locale. Although our focus was Fort Wayne, health code violations covered the entire county, and the city or town was not always listed. A violation on Main Street could be in Fort Wayne or any of eight other towns with a Main Street in the county. Having address ranges, which often vary from town to town, helped us narrow our options.\n\nBecause our violations databases seldom listed homeowners, we linked addresses in them to the treasurer's database, which contained current and past owners and dates the property transferred to their names.\n\nThe ranges also helped us correctly place violations on Fort Wayne streets with similar names. For example, if a violation was written as 1234 Reed, we had to verify if it was Reed Road or Reed Street, both of which are in the city. If an address could not be verified, the violation was discarded.\n\nWe also used the Census Tract Street Locator (http://tier2.census.gov/ctsWctsl.htm) to verify some addresses. The locator allows you to select a county and search for street address ranges by entering a partial street name. It is good for all but the newest streets.\n\nBecause our violations databases seldom listed homeowners, we linked addresses in them to the treasurer's database, which contained current and past owners and dates the property transferred to their names. Those transfer dates were important, because they identified who owned a house when a violation was committed. Using violation and transfer dates as sorting criteria, we ran Access queries to link owners with offenses. But that was only partially helpful with our database of building code inspections, which covered things such as peeling paint, rotting eaves and crumbling foundations. The date on that database referred only to when the house was first inspected for violations. Subsequent inspection dates were not listed.\n\nThe database also gave no details on the number of violations a house received over the years under various owners, making it impossible to rank problem landlords. Although it gave us a starting point in our search, we had to delve into paper records for details.\n\nCounty titleholders\n\nIt was clear from the time we started compiling the data that the county would be part of the story. Although not landlords, the county commissioners, who have the authority to take title to properties for back taxes, were listed as the titleholder of many properties with violations.\n\nConcerned about maintenance and liability issues, the county in the mid-1980s decided to leave tax-delinquent property in the owner's name with the option of claiming it at any time. After discovering the county connection, we asked for and received a database of about 1,000 properties \"certified\" to the county.\n\nMany of the houses are vacant. Some properties eventually are turned over to nonprofit agencies. But many are left to decay.\n\nAs a result of our series, the county has initiated ways to eliminate some of the complications for giving unwanted properties to nonprofit agencies. The county also formed a committee to address the housing problem and decided to have an additional tax sale in an attempt to get the properties back on the tax rolls.\n\nRon Shawgo can be reached by e-mail at rshawgo@jg.net.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-discarded-and-decayed-2","slug":"2000-november-discarded-and-decayed-2","url":"https://dwillis.github.io/uplink/article/2000-november-discarded-and-decayed-2.html","year":2000,"month":"November","monthNum":11,"headline":"Discarded and decayed","kicker":"","author_name":"Ron Shawgo","author_title":"Reporter","affiliation":"The Journal Gazette","summary":"The Journal Gazette conducted an ambitious computer-assisted reporting project in 1999 to identify major players in Fort Wayne's rental housing industry, revealing that the county and numerous landlords were major violators of housing ordinances. The investigation required extensive data acquisition and cleaning from multiple agencies, combining property tax records with housing violations databases to identify problematic property owners and managers.","keywords":["landlords","data","violations"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Access","Excel"],"full_text":"Landlords affect thousands of lives and have a major impact on the urban landscape. With that in mind, The Journal Gazette set out in April 1999 to identify the top players in Fort Wayne's rental housing industry. Although The Journal Gazette has several computer-assisted stories under its belt, this was our most ambitious CAR project. Eight months after drudging through lengthy data negotiations, importing hassles and mind-numbing data cleaning, reporter Lisa Shidler and I had a three-day package, \"Discarded and decayed.\"\n\nOur stories revealed that by failing to take over tax-delinquent properties, the county was one of the biggest violators of housing ordinances. We also talked to some typical slumlords and the city's largest landlords, who view housing violations as a cost of doing business.\n\nThe data hurdle\n\nOf course, getting data was our first hurdle. We figured that if we had a database of Fort Wayne housing violations we could link those addresses with property tax records to find the owners.\n\nWe knew our main task would be identifying property managers, the landlords who rent houses owned by other people. Because their names would not be listed as titleholders, we worked on the assumption that an address that receives numerous tax bills could be a property manager's business address. The assumption largely held true.\n\nWe contacted the county treasurer, the county building department, the city-county health department and the city office that inspects buildings for code violations. The six databases we ultimately acquired from those agencies contained county property tax records, health code violations, rodent inspections, building code violations, building demolitions and a list of properties the county could claim for back taxes.\n\nWhile some agencies gave us free data, the county wanted nearly $3,000 for the property records. After we informed officials that their 2-cents-per-record fee violated state law, the price was greatly reduced.\n\nOfficials I dealt with ran the gamut. One health official was the office's main computer authority and e-mailed files to me for free. Another county official knew little about copying data, so I helped guide him through his archaic software to save the data on disk in ASCII format.\n\nData slicing\n\nWe primarily used Microsoft Access and Excel to sort the data. (Although the project would have benefited from mapping software, it was done in our pre-mapping days.)\n\nThere was a lot of data slicing to do. For example, the rodent inspection office, a division of the health department, had its data on 10 files, one for each year since 1990. Taking into account that new fields were added to the files from one year to the next, I spliced them together.\n\nWe then added the rodent database to a much larger one listing health code violations, such as improperly disposed trash, discarded tires and appliances, and child neglect. That database also came in several files with slight differences and had to be spliced.\n\nOur two largest databases came from the county treasurer. One contained information on the property being taxed, and the other contained tax payment details, including addresses where tax bills are sent. Although the files contained a massive amount of information, we needed them only for current and former owners and the billing addresses.\n\nThe tax files caused our only data importing headaches. As our information systems people worked out the kinks, we began cleaning the data we had in hand.\n\nBecause we wanted to link all of our tables by their address fields, making addresses uniform was important. Input errors, omissions, misspellings and a general lack of uniformity in those addresses made the task time consuming and nearly overwhelming.\n\nAddress problems\n\nSome addresses had no directional prefix (N, S, E or W); others had no street type (road, drive, circle). Data cleaning took several months and required combining address fields and eliminating leading spaces, a couple of new Access tricks I learned with the help of NICAR-L subscribers. We also got a list of county and city streets with their numerical address ranges. Knowing the ranges - addresses where a street starts and where it ends - was essential in handling streets with common names and placing a violation in the proper locale. Although our focus was Fort Wayne, health code violations covered the entire county, and the city or town was not always listed. A violation on Main Street could be in Fort Wayne or any of eight other towns with a Main Street in the county. Having address ranges, which often vary from town to town, helped us narrow our options.\n\nBecause our violations databases seldom listed homeowners, we linked addresses in them to the treasurer's database, which contained current and past owners and dates the property transferred to their names.\n\nThe ranges also helped us correctly place violations on Fort Wayne streets with similar names. For example, if a violation was written as 1234 Reed, we had to verify if it was Reed Road or Reed Street, both of which are in the city. If an address could not be verified, the violation was discarded.\n\nWe also used the Census Tract Street Locator (http://tier2.census.gov/ctsWctsl.htm) to verify some addresses. The locator allows you to select a county and search for street address ranges by entering a partial street name. It is good for all but the newest streets.\n\nBecause our violations databases seldom listed homeowners, we linked addresses in them to the treasurer's database, which contained current and past owners and dates the property transferred to their names. Those transfer dates were important, because they identified who owned a house when a violation was committed. Using violation and transfer dates as sorting criteria, we ran Access queries to link owners with offenses. But that was only partially helpful with our database of building code inspections, which covered things such as peeling paint, rotting eaves and crumbling foundations. The date on that database referred only to when the house was first inspected for violations. Subsequent inspection dates were not listed.\n\nThe database also gave no details on the number of violations a house received over the years under various owners, making it impossible to rank problem landlords. Although it gave us a starting point in our search, we had to delve into paper records for details.\n\nCounty titleholders\n\nIt was clear from the time we started compiling the data that the county would be part of the story. Although not landlords, the county commissioners, who have the authority to take title to properties for back taxes, were listed as the titleholder of many properties with violations.\n\nConcerned about maintenance and liability issues, the county in the mid-1980s decided to leave tax-delinquent property in the owner's name with the option of claiming it at any time. After discovering the county connection, we asked for and received a database of about 1,000 properties \"certified\" to the county.\n\nMany of the houses are vacant. Some properties eventually are turned over to nonprofit agencies. But many are left to decay.\n\nAs a result of our series, the county has initiated ways to eliminate some of the complications for giving unwanted properties to nonprofit agencies. The county also formed a committee to address the housing problem and decided to have an additional tax sale in an attempt to get the properties back on the tax rolls.\n\nRon Shawgo can be reached by e-mail at rshawgo@jg.net.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-for-similar-stories-on-housing","slug":"2000-november-for-similar-stories-on-housing","url":"https://dwillis.github.io/uplink/article/2000-november-for-similar-stories-on-housing.html","year":2000,"month":"November","monthNum":11,"headline":"For similar stories on housing","kicker":"","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"A guide to investigative journalism resources on housing issues, including stories examining substandard conditions, HUD programs, and government subsidy distribution practices.","keywords":["housing","investigation","subsidies"],"topics":["Business & Economy","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"For similar stories on housing, check out the following files at the IRE Resource Center, (573) 882-3364:\n\nStory #16811. The Lexington Herald-Leader examines substandard housing conditions. Reporters used database and shoe-leather reporting to find wealthy landlords and the tenants who live in squalor. The paper also finds that city inspectors are not doing their jobs.\n\nStory 13662. The San Francisco Bay Guardian looks into the Housing and Urban Development program that allows housing authorities to demolish and rebuild their worst housing stock.\n\nStory #16435. Florida Trend magazine examines allegations that the Florida Housing Finance Corporation, which distributes millions in government subsidies, is handing out money to benefit a favored few.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-second-chances","slug":"2000-november-second-chances","url":"https://dwillis.github.io/uplink/article/2000-november-second-chances.html","year":2000,"month":"November","monthNum":11,"headline":"Second chances","kicker":"CRIME","author_name":"David A. Milliron and Chris Cantergiani","author_title":"","affiliation":"The Atlanta Journal-Constitution and WSB-TV Channel 2","summary":"An investigation by The Atlanta Journal-Constitution and WSB-TV found that over 4,900 people in Georgia have received multiple second chances under the First Offender Act since 1985, exploiting loopholes and poor record-keeping to repeatedly seal felony convictions and hide them from employers.","keywords":["loopholes","convictions","records"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["Access","Excel"],"full_text":"Georgia courts have given Barry Williams, a low-level drug offender, a second chance - five times. Williams is one of more than 4,900 individuals found to be given a second bite at the apple multiple times since 1985 under a law intended to give criminals with no previous record a second chance. But an analysis by The Atlanta Journal-Constitution and WSB-TV Channel 2 of computer records kept by the Georgia Bureau of Investigation and the state Department of Corrections found those who already had a felony conviction - or had used the second chance offered by the legal system managed to take advantage of loopholes and bad record-keeping to get another crack at first offender status.\n\nAnd why not? Those sentenced under Georgia's First Offender Act are immediately eligible for parole, and their convictions are removed from their record if they stay out of trouble during probation.\n\nA potential employer, for example, would not see the conviction of a job applicant who had successfully completed the terms of the agreement. The permanent record of the conviction would be seen only by law enforcement agencies.\n\nSealed records\n\nThe idea for the investigation came out of our analysis of felony conviction records while working to expose thousands of Georgia public school teachers who failed to disclose arrests on teaching applications and with the state Department of Professional Standards, which licenses educators. For the prior investigation, we acquired databases from the Department of Corrections listing all convictions on record since the late 1970s. But the first batch of data didn't include Social Security Numbers, which are public record in Georgia, so we had to go back and request they be included.\n\nWhen we got the second batch of data, however, we quickly noticed thousands upon thousands of records that were populated with S's in the name, Social Security Number and date of birth fields. A quick call to the state Department of Corrections introduced us to the First Offender Act, which they said required them to seal the records.\n\nAt a judge's discretion, someone who pleads guilty or nolo contendere to a felony in Georgia may be sentenced under the First Offender Act unless the crime is murder, armed robbery, kidnapping, rape, aggravated child molestation, aggravated sodomy or aggravated sexual battery. The court is required to review a person's criminal record for past first-offender sentencings or felony convictions. Once a first-offender sentence is complete, a person is not considered to have had a criminal conviction for that offense.\n\nThat's when we stumbled upon the current investigation. While running queries on the teacher-felon project, we found a teacher who had a prior arrest - yet a subsequent arrest had been sealed. Oops!\n\nYes, the first batch of data had not been redacted; it just excluded Social Security Numbers. And since both sets contained a unique identifier much like a Social Security Number for each offender, we simply performed an inner join in Microsoft Access to pluck an individual's name and date of birth from the unredacted database.\n\nHeavy lifting\n\nNow it was time for the heavy data lifting. All we had to do was apply the law to the data. Barry Williams, our low-level drug offender, was one of the more than 4,900 individuals who matched one of our criteria. On Williams' fifth second chance, he still got a break even though his felony record went back 14 years. To identify people like Williams who were improperly afforded first offender status, we carved the master database into separate tables that we later exported to Microsoft Excel. Anyone who had never been afforded first offender status was immediately eliminated from the pool. Those with a single first offender status were placed in one database along with all their arrest history, and those with multiple first offender status in another.\n\nCoding offenders\n\nSince neither of us are big fans of writing code unless we absolutely have to, we decided to code all the first offenders with an \"X\" in the state-provided \"FOS\" field. We then sorted each spreadsheet by name and conviction date in ascending order.\n\nNow, this may be too simplistic a solution, but it worked fine for us. Since our spreadsheet was properly sorted, all we had to do is create the \"evaluate\" field and autonumber the individual convictions.\n\nWe wrote a simple formula to populate the \"evaluate\" field that would tell us which conviction was number 1, which was number 2, etc. Our formula assumes the column to be checked is the \"unique identifier\" field in Column A, and that we wanted the result to appear in the \"evaluate\" field, or Column E. Here's the one-line formula:\n\n=IF(A2~A1,1,E1+1)\n\nWe then just copied the formula down as many rows as we had data, and this did all of the counts for us. When the contents of the \"unique identifier\" field changed, then contents of the \"evaluate\" column were reset to 1, otherwise \"evaluate\" was incremented by 1. For those of you who prefer coding, we could have done it as such:\n\nset mg = Range(Range(\"A2\"),Range(\"A2\").End(xldown)) Range(\"E1\").value=1 mg.offset(0,1).formula = \"=if(A2=A1,E1+1,1)\"\n\nOK, now here's where it got real simple. We copied the contents of the \"evaluate\" field and did an Edit Paste Special | Values command so that we could preserve the auto increments as we further analyzed the data. We then did a quick filter of the \"FOS\" field for the X's and the \"evaluate\" field for records with a \"1\" in them. Those records were deleted. We then again filtered the \"FOS\" field, but this time for blank cells, which were deleted.\n\nSeveral second chances\n\nAltogether, 2,410 individuals since 1986 have been sentenced as first offenders even though they had prior felony convictions. Another 2,504 have received first offender status more than once, according to GBI records.\n\nAmong the reasons:\n\nNot all court systems register their convictions with the state in a timely manner.\nThe transition from paper to computer can be slow in many counties, creating gaps in record keeping.\nThe criminal histories in the Georgia Bureau of Investigation's \"Georgia Crime Information Center\" are not accurate or up to date.\n\nSince our investigation, the state has unveiled a new automated system it is piloting in metro Atlanta that officials claim will allow law enforcement and justice officials to access an individual's criminal history in seconds, rather than what court officials said often took months. The state's attorney general has also said the state legislature needs to work to eliminate the loopholes in the system.\n\nDavid Milliron can be reached by e-mail at dmilliron@ajc.com. Chris Cantergiani can be reached by e-mail at chris.cantergiani@wsbtv.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-the-following-stories-also-address-jails-and-prison-problems","slug":"2000-november-the-following-stories-also-address-jails-and-prison-problems","url":"https://dwillis.github.io/uplink/article/2000-november-the-following-stories-also-address-jails-and-prison-problems.html","year":2000,"month":"November","monthNum":11,"headline":"The following stories also address jails and prison problems","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article highlights multiple journalism investigations into prison and jail system problems, including issues with overcrowding, recidivism, and the private prison industry's accountability.","keywords":["prisons","overcrowding","accountability"],"topics":["Crime & Justice"],"technologies":["World Wide Web"],"full_text":"The following stories also address jails and prison problems:\n\nThe Atlanta Journal-Constitution and WSB-TV joint project will soon be available in the IRE Resource Center. Search the story database at www.ire.org/resourcecenter\n\nStory #16665. The Arkansas Democrat-Gazette finds that many criminals convicted of felonies are receiving second, third and fourth chances because of crowded prisons and overworked courts.\n\nStory #16545. Mother Jones takes a look at a private prison in Youngstown, Ohio, run by Corrections Corporation of America.\n\nStory #15666. This National Journal story takes a look at the private prison industry, including questions on inmate complaints and accountability.\n\nCheck out IRE's Web site at www.ire.org/resourcecenterl for more information.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-tipsheets","slug":"2000-november-tipsheets","url":"https://dwillis.github.io/uplink/article/2000-november-tipsheets.html","year":2000,"month":"November","monthNum":11,"headline":"Tipsheets","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"A collection of journalism tipsheets providing guidance on investigating housing issues, including inspector misconduct, rental scams, and industry data collection.","keywords":["housing","investigation","journalism"],"topics":["Investigative Methods","Business & Economy"],"technologies":[],"full_text":"Tipsheet 183. This tipsheet contains tips and includes a chart to help reporters interested in housing issues (New York, 2000).\n\nTipsheet #1182. Discusses how the St. Paul Pioneer Press investigated housing inspectors (New York, 2000).\n\nTipsheet 204. Explores various housing-related scams including land flipping and slumlords (New York, 2000).\n\nTipsheet #1029. This tipsheet includes a timeline of an investigation into housing problems in Baltimore (Kansas City, 1999).\n\nTipsheet #1311. Includes tips on how to collect information from the rental housing industry (Lexington, 2000).\n\nTipsheets are available from IRE's Web site, www.ire.org/resourcecenter","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-raving-the-wrong-way","slug":"2000-november-raving-the-wrong-way","url":"https://dwillis.github.io/uplink/article/2000-november-raving-the-wrong-way.html","year":2000,"month":"November","monthNum":11,"headline":"Raving the wrong way","kicker":"DRUGS","author_name":"Hanque Curtis","author_title":"Reporter","affiliation":"Orlando Sentinel","summary":"A reporter used computer-assisted reporting to uncover flawed data in Florida's designer drug death study, revealing that state officials had improperly counted 80 cases unrelated to drug abuse, including terminal cancer patients and nursing home residents.","keywords":["databases","investigation","Florida"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Access"],"full_text":"Raving the wrong way\n\nBy Hanque Curtis\nOrlando Sentinel\n\nAnything can happen in Florida, including counting great-grandmothers as rave victims in a state drug abuse study. That's what a few clicks of the computer showed. Now, months later, facts forced Gov. Jeb Bush's drug czar to remove 80 names from a list cited as evidence of an epidemic of designer drug-related deaths. Those cases included terminal cancer patients who shot themselves. Nursing home patients who fell. And a four-year-old boy treated for spinal meningitis. None had any ties to illegal drug abuse.\n\nThis came to light thanks to a two-day IRE workshop on computer-assisted reporting. When it ended, I started looking for a story to put to use what I had learned.\n\nFirst ideas\n\nThe first couple of ideas seemed too complex.\n\nThen, I heard that state officials had asked medical examiners to cull their files from 1997 through 1999 for any cases that tested positive for any of 20 \"designer\" drugs associated with raves and all-night dancing. Having reported on heroin abuse in Central Florida since 1996, I was looking for ways to expand my coverage. So I submitted a public records request for the confirmed cases, hoping to identify and track trends in designer drug abuse.\n\nThe request seemed routine, but a spokesman for the year-old Office of Drug Control claimed it would be necessary to contact each law enforcement agency that might have investigated the deaths to make sure the cases were public.\n\nTo prevent being delayed for months, I suggested blacking out the victims' names but leaving such vital statistics as age, sex, race, date of death, the medical examiner's district and the drugs detected. I asked for this knowing it could be used to identify the victims and obtain their autopsy reports, if needed.\n\nRecord requests\n\nWhile waiting for the statewide information, I submitted public records requests to the medical examiners in the six counties of Central Florida for copies of the records they provided for the designer-drug study. This made certain I would have complete information on the cases in The Sentinel's coverage area.\n\nThe local cases arrived within days. While waiting for the statewide information, I created an Access table for 60 local deaths submitted to the Office of Drug Control for review. The table had 13 fields of information for each case. Besides recording vital statistics, the fields included day of the week of each death, the presence of alcohol, if the victim met the typical age range of designer drug users, cause of death and circumstances of death.\n\nTracking this information was wonderfully easy.\n\nIt was nothing compared to what I used to do. Back in 1997, another reporter and I copied all information from more than 3,800 traffic tickets issued by a drug squad patrolling Florida's Turnpike to write a story about racial profiling. The information filled a small pile of legal pads. Then, we had to add column after column of numbers. Finally, we had to double-check each other's math.\n\nNow, all I have to do is remember to use the correct commands.\n\nData arrives\n\nWhen the statewide information on designer-drug deaths arrived three weeks later, it quickly became obvious that the study was riddled with mistakes.\n\nIn case after case, the victims appeared too young or too old to be designer-drug users. Eighty-four of the 254 victims ranged in age from 38 to 84 years old. The youngest was six months old. And records showed that it was unlikely that many of the cases had been scrutinized before being included in the study. Out of 33 cases submitted by one medical examiner for review, the state listed all 33 as confirmed cases of designer drug abuse. These included an 80-year-old Alzheimer's patient and a 75-year-old woman who died from asthma. Many others had tested positive for common painkillers or methamphetamines. The latter has been a commonly abused drug for more than 50 years, and its abuse in Florida is associated with farm workers, truck drivers and bikers. It did not seem likely that 40- and 50-year-old men who died in truck crashes had much to do with the state's rave culture. Reporting this involved cherry picking the most questionable cases in Central Florida, my area of coverage, as well as some from Miami and St. Petersburg to give the story statewide context. This involved contacting the three medical examiners' offices responsible for those areas and, in some cases, local law enforcement agencies to obtain reports on the deaths.\n\nAt the same time, I requested all written communications between the state Medical Examiners Commission and the Office of Drug Control about the creation of the study.\n\nThese 150 pages of records provided me with a timeline showing the study had been conducted in less than three months. It was done within that relatively tight window for a first-of-its-kind-study in Florida so the state's new drug czar, Jim McDonough, could present it at a drug summit of state legislators in February.\n\nMcDonough told the summit that his findings on designer drug abuse came from personally reading death reports and \"a very thorough, autopsy-by-autopsy review.\"\n\nThe Access tables and queries I created showed that the state's top drug abuse expert was mistaken. The state list of 254 deaths, now reduced to 174, remains under review. Meanwhile, I am able to track designer drug-related deaths in Florida's 67 counties simply by logging into my own database.\n\nHanque Curtis can be reached by e-mail at hcurtis@orlandosentinel.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-ires-resource-center-has-the-following-stories-on-drugs","slug":"2000-november-ires-resource-center-has-the-following-stories-on-drugs","url":"https://dwillis.github.io/uplink/article/2000-november-ires-resource-center-has-the-following-stories-on-drugs.html","year":2000,"month":"November","monthNum":11,"headline":"IRE's Resource Center has the following stories on drugs","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"The IRE Resource Center has compiled stories on emerging drug trends including designer drugs, ecstasy culture in major cities, and the growing use of GHB as a date-rape drug, featuring coverage from the Orlando Sentinel, New York magazine, and the Detroit News.","keywords":["drugs","ecstasy","GHB"],"topics":["Investigative Methods","Crime & Justice"],"technologies":[],"full_text":"IRE's Resource Center has the following stories on drugs:\n\nThe Orlando Sentinel's story on designer drug deaths will soon be available in the IRE Resource Center. Search the story database at www.ire.org/resourcecenter\n\nStory #16742. New York magazine takes an in-depth look at the growing ecstasy culture popularized by widespread use of the drug in major cities. The story also looks at how the medical and law enforcement communities are reacting to it.\n\nStory #16621. The Detroit News investigates the \"date-rape\" drug, GHB, and finds it is growing in popularity with casual users in Michigan and across the country.\n\nCall (573) 882-3364 for more information on how to order.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-filemaker-intranet","slug":"2000-november-filemaker-intranet","url":"https://dwillis.github.io/uplink/article/2000-november-filemaker-intranet.html","year":2000,"month":"November","monthNum":11,"headline":"FileMaker Intranet","kicker":"TECH TIP","author_name":"Paul D'Ambrosio","author_title":"","affiliation":"Asbury Park Press","summary":"FileMaker Pro Unlimited 5 offers a quick and resource-efficient solution for setting up searchable intranet databases without programming, allowing newsrooms to deploy working databases within minutes by importing files and enabling web-sharing features.","keywords":["database","intranet","FileMaker"],"topics":["Technology","Data & Databases","Journalism & Newsroom"],"technologies":["FileMaker Pro","DBF","Excel","FileMaker Home Page 3.0","FrontPage 2000","Pentium","Internet","World Wide Web"],"full_text":"If you need to set up a full-service Intranet with a powerful database server, but you don't have a lot of time or resources, you may want to look at FileMaker Pro Unlimited 5. With its built in Web server and data manager, you can essentially turn any PC on your newsroom or company network into a file server. This may sound like a product pitch (trust me, I get nothing from the FileMaker Pro people), but you can have a searchable database up and running in a matter of minutes without any programming. You just import a DBF or XLS file, turn on the Web-sharing feature and away you go.\n\nFMU's server software allows any machine on the Intranet to access the databases you want your users to see and search. We even have a source page that reporters can add or change records to on the fly.\n\nLarger databases will take longer to set up since you will need to index all the key search fields, a slow process.\n\nFMU's server software allows any machine on the Intranet to access the databases you want your users to see and search. We even have a source page that reporters can add or change records to on the fly.\n\nYou can use FMU's default search page, which allows you to search any field in the database, or design your own with FileMaker Home Page 3.0, a separate product. If you shop around, you will likely find you can buy both products for less than $1,000.\n\nWe have more than a dozen databases running on a Pentium Pro 200 dual chip server with 256 megs of RAM. The databases range from 500 records to more than 600,000. Once indexed, searches are lightning fast for both the large and small data sets.\n\nHow it works\n\nOur system runs this way: the newsroom's Intranet page was designed using FrontPage 2000 with links to our databases. Each database search page was designed with FileMaker Home Page since FrontPage can't read the proprietary HTML that FMU uses.\n\nSince we are part of a larger Intranet network, the home page resides on a different server, but FMU and the search pages have their own machine. I estimate the FMU server gets up to two-dozen hits at a time with more than 100 PCs attached to it. I have not noticed any degradation during searches, even at peak hours.\n\nOne downside to FMU is a lack of clear documentation. We found integrating the Home Page files into the overall Intranet frustrating until Robert Weston from AP's The Wire Web site, who has used FMP for a number of months, cleared up the confusion. If you run into this problem, the best suggestion is to use the \"Consolidate\" option in Home Page and copy the search folder to one level above where your databases reside in FMU.\n\nIt is also best to use flat files for speed. Joining files may save space, but the learning curve on that is steeper.\n\nMore information\n\nMore information about FMU and Home Page can be found at www.filemaker.com. You can also download demos of both products at www.filemaker.com/downloads/index.html. Be careful not to confuse the Unlimited version with the standard FileMaker Pro. Only the Unlimited can publish databases to the Internet or Intranet.\n\nIf you set the software up properly but other machines cannot see it, you may want to consult with your IS folks. They may have to tinker with the firewall or change your TCP/IP settings.\n\nWeb links\n\nPaul D'Ambrosio can be reached by e-mail at pmd@app.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-web-links-2","slug":"2000-november-web-links-2","url":"https://dwillis.github.io/uplink/article/2000-november-web-links-2.html","year":2000,"month":"November","monthNum":11,"headline":"Web links","kicker":"","author_name":"Paul D'Ambrosio","author_title":"","affiliation":"app.com","summary":"A list of recommended websites for FileMaker information and tips, including the official FileMaker site and several community resources.","keywords":["FileMaker","resources","tips"],"topics":["Technology"],"technologies":["FileMaker Pro"],"full_text":"For general information and tips on FileMaker, check out these sites:\n\nwww.filemaker.com\n\nwww.fmfiles.com/newtips.html\n\nwww.filemakertoday.com\n\nwww.fmpro.com\n\nwww.fileville.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-the-following-stories-on-housing-issues-are-available-in-ires-resource-center","slug":"2000-november-the-following-stories-on-housing-issues-are-available-in-ires-resource-center","url":"https://dwillis.github.io/uplink/article/2000-november-the-following-stories-on-housing-issues-are-available-in-ires-resource-center.html","year":2000,"month":"November","monthNum":11,"headline":"The following stories on housing issues are available in IRE's Resource Center","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE's Resource Center offers several investigative journalism pieces on housing issues, including The Seattle Times' housing affordability series and investigations into corruption and inadequate oversight in public and private housing systems.","keywords":["housing","investigation","corruption"],"topics":["Investigative Methods","Business & Economy","Government & Politics"],"technologies":["Internet","World Wide Web"],"full_text":"The following stories on housing issues are available in IRE's Resource Center:\n\nThe Seattle Times' series, \"Sticker Shocked,\" can be seen online at http://seattletimes.nwsource.com/homes/ or it is available in the IRE Resource Center, story #16511.\n\nStory #15321. WWOR-TV investigates the New York City Housing Authority, the country's largest landlord, and finds that tax-funded maintenance money lined the pockets of a few housing inspectors.\n\nStory #14992. Northwestern uncovers a housing inspection system where landlords with the right resources could bypass city housing codes.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-tipsheets-2","slug":"2000-november-tipsheets-2","url":"https://dwillis.github.io/uplink/article/2000-november-tipsheets-2.html","year":2000,"month":"November","monthNum":11,"headline":"Tipsheets","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"A collection of IRE tipsheets providing guidance for reporters investigating housing issues, including inspection fraud, scams, rental industry practices, and related investigations across multiple cities.","keywords":["housing","investigation","reporting"],"topics":["Investigative Methods","Business & Economy"],"technologies":[],"full_text":"Tipsheet 183. This tipsheet contains tips and includes a chart to help reporters interested in housing issues (New York, 2000).\n\nTipsheet #1182. Discusses how the St. Paul Pioneer Press investigated housing inspectors (New York, 2000).\n\nTipsheet 204. Explores various housing-related scams including land flipping and slumlords (New York, 2000).\n\nTipsheet #1029. This tipsheet includes a timeline of an investigation into housing problems in Baltimore (Kansas City, 1999).\n\nTipsheet #1311. Includes tips on how to collect information from the rental housing industry (Lexington, 2000).\n\nTipsheets are available from IRE's Web site, www.ire.org/resourcecenter","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-web-links-people-and-records","slug":"2000-november-web-links-people-and-records","url":"https://dwillis.github.io/uplink/article/2000-november-web-links-people-and-records.html","year":2000,"month":"November","monthNum":11,"headline":"WEB LINKS People and records","kicker":"","author_name":"Liz Donovan","author_title":"Reporter","affiliation":"The Miami Herald","summary":"A tipsheet from The Miami Herald's Liz Donovan provides journalists with a comprehensive list of online public records databases and people-finder tools useful for investigative reporting, including resources for accessing government records, court documents, property information, and locating individuals.","keywords":["databases","records","investigation"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"The following is adapted from a tipsheet by The Miami Herald's Liz Donovan at NICAR's 2000 conference in Lexington, Ky, tipsheet # 1327.\n\nPublic records\n\nPublic records databases, www.pac-info.com\nState and local government databases, http://gwis2.cir.gur.edul-gpricelstate.htm\nProperty appraiser databases, www.people.virginia.edul-dev-pros/Realestate.html\nLocating U.S. corporations, www.internetprospector.org/sesstate.html\nInmate locator, www.corrections.comllinks/inmate.html\nGuidestar charity search, www.guidestar.org\nPilots and planes, www drive.net\nNational Center for State Courts, wwww.ncsc.dni.us/COURT/SITES/courts.htm\nPACER U.S. Courts national index, http://pacer.uspci.uscourts.gov/\n\nPeople finders\n\nFind person, http://person.langenberg.com\nInfoBel world phone directory, www.infobel.b\nWeb site owners, www.networksolutions.com\nSources for prospect research, www.people.virginia.edul-dev-pros/Webresources.html\nInvestigative links, www.inil.com/usersIdgus/wgator.htm\nNavy http://members.aol.com/navyspiesl qstart.htm\nSocial Security Number lookup, www.informus.com/ssnukup.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-beat-books","slug":"2000-november-beat-books","url":"https://dwillis.github.io/uplink/article/2000-november-beat-books.html","year":2000,"month":"November","monthNum":11,"headline":"Beat Books","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE is now offering beat-specific books on home mortgage lending disparities and aviation safety investigations, priced at $15 for members and $20 for nonmembers.","keywords":["books","investigations","reporting"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"The following beat books are now available from IRE:\n\nBooks are $15 each ($20 for nonmembers). Order online at www.ire.org/store/books or call (573) 882-2042.\n\n\"Home Mortgage Lending: How to Detect Disparities\", by Jo Craven McGinty.\n\n\"Covering Aviation Safety: An Investigator's Guide\", by Marie Tessier.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-using-subqueries","slug":"2000-november-using-subqueries","url":"https://dwillis.github.io/uplink/article/2000-november-using-subqueries.html","year":2000,"month":"November","monthNum":11,"headline":"Using subqueries","kicker":"TECH TIP","author_name":"MaryJo Sylwester","author_title":"Staff","affiliation":"NICAR","summary":"This article explains how to use subqueries in database management to solve complex data problems, such as finding the most recent inspection for each facility or determining an officer's rank at the time of discipline.","keywords":["subqueries","database","SQL"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Access","dBase","FoxPro","SQL"],"full_text":"Using subqueries\n\nFollow along:\n\nThere are fake data tables that accompany this article available to download off the NICAR Web site. Go to www.nicar.org and click on \"Uplink Tech Tip Downloads.\" Inside there you'll find a self-extracting file called \"mjtricks\" that can be downloaded. Inside are a Microsoft Access database (1997 version) and Dbase IV files of the same data in case you are using FoxPro.\n\nWhen working in a database manager, subqueries can prove immensely useful once you master the slightly complex syntax. Essentially subqueries allow you to come up with an answer based on another query that is imbedded within your main query.\n\nThat probably sounds confusing, but keep reading. It makes more sense when you see them in action. So I've got a couple examples of how you could use a subquery for you to follow along.\n\nThe first is fairly straightforward: I've got a list of inspections, where there could be multiple records for the same facility. But I want to find the last (or most recent) inspection for each facility. So my result would have one record for each facility with the date of the most recent inspection.\n\nThe second example has a couple levels to get to the final answer: I've got a list of discipline actions (in this case against police officers) and I also have a list of promotion dates that includes the person's rank. I want to find out what rank each officer was at the time they were disciplined. My result would have one record for each discipline action and include the date of the action and their rank at the time.\n\nI've created some fake data tables for this demonstration that can be downloaded off the NICAR Web site. Go to www.nicar.org and click on \"Uplink Tech Tip Downloads.\" Inside there you'll find a self-extracting file called \"mjtricks\" that can be downloaded. Inside are a Microsoft Access database (1997 version) and Dbase IV files of the same data in case you are using FoxPro.\n\nAs you will quickly see, these are very basic tables with bare-bones information just to show how this works. If you were working with real data, it would likely have many more fields and other details you might want to include.\n\nInspections\n\nLet's start with the inspections data. The table is called \"inspect.\" There you will find the site number (unique ID for each facility), the date of inspection and type of inspection. Notice there are multiple records for some facilities, but not all.\n\nIn this case we want to find the most recent inspection date for each facility. To do this, we'll use the MAX() function imbedded within a subquery in our select line. Here's what the SQL looks like, then I'll explain:\n\nSELECT sitenum, (select max(b.inspdate) from inspect b where a.sitenum=b.sitenum) as lastinspect FROM inspect a GROUP BY sitenum\n\nThe select line within the parentheses will look for the largest (or in this case most recent) inspection date and the \"where\" clause specifies that it will compare only those records that have the same site number. If you were to leave that off, the query will run but it will give you the exact same date for all records (the most recent date that exists within the whole table). You need to GROUP BY in order for your results to yield just one record for each facility.\n\nIn this case I've named the field where my subquery results will show up as \"lastinspect,\" and I used aliases for each of the table names simply to have less typing. (\"a\" and \"b\" are the aliases). I'll use this type of naming in the next example as well.\n\nDiscipline\n\nThis one involves a couple steps. There are two tables, \"discipline\" and \"promotions.\" The promotions table lists the badge number, the rank and date of promotion; there might be multiple records for the same person if they were promoted more than once. The discipline table, which also might have multiple records, lists the badge number, the discipline action and date it occurred.\n\nThe first thing we need to do is to create a new table that lists start and end dates for each rank the person held. So for each badge number that shows up in the database, this query will look at the rank and list the promotion date as the \"start date\" and then look for the next rank (chronologically by date) and subtract one day to come up with the \"end date.\" Then it goes on to that second rank and does the same. In cases where there isn't another rank, it will leave the \"end date\" blank.\n\nHere's the SQL:\n\nSELECT a.badge, rank, a.promodate as start, (select min(b.promodate)-1 FROM promotions b where b.badge=a.badge and b.promodate>a.promodate) as end FROM promotions a\n\nCreate this into a new table. Here I'm going to call it \"dates.\" The next step then is to match this new \"dates\" table with the discipline table to put the correct rank with the date the officer was disciplined. This uses an inner join, which is easily accomplished in Access by drawing the line between the two tables in the grid view. Join it between the badge field on each table. Then when you open the SQL view, you can add in a few pieces of information to come up with the following:\n\nSELECT discipline.badge, start, end, rank, actdate, action FROM DISCIPLINE INNER JOIN dates ON DISCIPLINE.BADGE = dates.badge WHERE (actdate>start) and (actdate<end or end is null)\n\nIn addition to joining on the badge number, the WHERE criteria also specifies that we want to match up the discipline action date where it falls after the \"start\" date and before the \"end\" date in the dates table.\n\nThe resulting query yields the answer to our original question.\n\nOther uses\n\nAnother use for subqueries is to find any records from one table that do not show up in another table. Conversely you could also use it to find those records that match.\n\nFor example, if you slice records out of one table and create a new table. Then you want to go back and find those records from the original that did not end up in the new table. It works best if you have a unique ID that shows up in both tables. Here's the syntax:\n\nSELECT * FROM table1 WHERE uniqueid not in (SELECT uniqueid from newtable)\n\nTo find the records that match you would simply remove the \"not\" from the where clause.\n\nThere is more information about subqueries in the Microsoft Access help file and in the Microsoft Knowledge Base at http://support.microsoft.com/directory/. There you can also find details on how to run these queries in the grid view.\n\nMaryJo Sylwester can be reached by e-mail at maryjo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-mark-your-calendars","slug":"2000-november-mark-your-calendars","url":"https://dwillis.github.io/uplink/article/2000-november-mark-your-calendars.html","year":2000,"month":"November","monthNum":11,"headline":"Mark your calendars","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The article announces two upcoming journalism conferences: the National Computer-Assisted Reporting Conference in Philadelphia in October 2001 and the Investigative Reporters and Editors Conference in Chicago in June 2001, with details about hotel reservations and contact information.","keywords":["conference","journalism","registration"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"The National Computer-Assisted Reporting National Conference 2001 will be held Oct. 11-14, 2001 in Philadelphia, Penn., at Loews Philadelphia Hotel.\n\nFor hotel reservations call: (215) 627-1200 (reservation cut-off is Sept. 4, 2001). Rooms cost $155 single/double plus tax.\n\nThe Investigative Reporters and Editors National Conference 2001 will be held June 14-17, 2001 in Chicago, Ill., at the Hyatt Regency Chicago - the Riverwalk. For Hotel Reservations call 1(800)233-1234 or direct at (312)616-6864. Rooms cost $179 for single, $215 for double, $235 for triple, $260 for quadruple, plus tax. Reservations must be made by May 15, 2001. For more information about either of these conferences, contact IRE at (573) 882-2042. Stay tuned to the IRE web site, www.ire.org/training, for conference registration forms and other details about both of these conferences.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-november-bits-bytes-and-barks","slug":"2000-november-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2000-november-bits-bytes-and-barks.html","year":2000,"month":"November","monthNum":11,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The NICAR Database Library has expanded its collection with new datasets including Storm Events, National Practitioner Databank, DOT Truck Census and Accidents, and Airmen Directory, while also announcing upcoming training bootcamps for journalism educators and students.","keywords":["databases","journalism","training"],"topics":["Journalism & Newsroom","Data & Databases","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"The NICAR Database Library has expanded its collection, adding two new datasets and bringing back three that have been unavailable for a couple years.\n\nStorm Events: This is the official U.S. government database of storm events around the country, including tornadoes, hurricanes, tropical storms, droughts, snowstorms, flash floods, hail, wild/forest fires, fog and avalanches. Fields include: date and time; states and counties involved; latitude and longitude; property and crop damage; and injuries and fatalities.\n\nNational Practitioner Databank: This database contains information about doctors and other health care practitioners who have had medical malpractice suits filed or adverse action taken against them. Although names are not included, some news organizations have been able to use this with other public records to identity individual practitioners.\n\nDOT Truck Census and Accidents: These two databases, which have not been available through NICAR for the last two years, are a good starting point for looking at motor carriers and truck accidents on U.S. roadways. The accidents data includes details on accidents involving vehicles owned by motor carriers. The census data allows you to find more details about those carriers, including the type of cargo they transport.\n\nAirmen Directory: Because of a fight over privacy, this database has not been available from the FAA for a couple years. A law passed in April 2000 once again made the data public, but airmen had the opportunity to withhold their names. The data currently holds names for about 65 percent of the airmen - both pilots and non-pilots - who have active certificates. Nothing is available on those airmen who opted for privacy. More information, including sample slices, record layout and costs, are available at www.ire.org/dadalibrary/databases. To order, call (573) 884-7332.\n\nBootcamp for educators\n\nIRE and NICAR will hold a Bootcamp geared for journalism educators, Jan. 14-17, 2001, in Columbia, Mo. The four-day seminar will teach computer-assisted reporting techniques and how to teach them. Instructors include IRE and NICAR Executive Director Brant Houston and Jo Craven McGinty, who teaches CAR at the Missouri School of Journalism.\n\nMore information, including registration details, is available at www.ire.org/training/jan14.html\n\nIRE Student Conference\n\nSponsored by IRE and NICAR and hosted by the Missouri School of Journalism, this one-day conference for journalism students will be held Saturday, March 31 in Columbia, Mo.\n\nIt will feature panels on topics that include investigating politicians and local government, investigating nonprofits, using the FOIA and local sunshine laws and producing the story for TV. Hands-on sessions on Internet searching, spreadsheets and database managers will also be held.\n\nMore information is available at www.ire.org/training/2001studentconfistucontraining.htm","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-the-race-factor","slug":"2000-december-the-race-factor","url":"https://dwillis.github.io/uplink/article/2000-december-the-race-factor.html","year":2000,"month":"December","monthNum":12,"headline":"The race factor","kicker":"RACIAL PROFILING","author_name":"Dan Browning","author_title":"Reporter","affiliation":"Minneapolis Star-Tribune","summary":"The Minneapolis Star-Tribune investigates whether Minneapolis police use racial profiling. The report analyzes booking, jail, and sentencing data to determine if more black suspects are arrested than whites.","keywords":["racial profiling","police","Minneapolis"],"topics":["Crime & Justice","Investigative Methods","Demographics & Census"],"technologies":[],"full_text":"Minneapolis police say they don't use racial profiling to stop alleged lawbreakers. They rely on experience, aided by computer-driven analysis that points them to trouble spots. But their computer-assisted hunches lead them to more black suspects proportionately than whites. Are police using good judgment? Or are they trapping relatively more black criminals because they're sweeping up higher percentages of blacks than whites? Philip Meyer, a journalism professor at the University of North Carolina at Chapel Hill and one of the leading proponents in the use of quantitative analysis by journalists, suggested a kind of cost-benefit study. Assuming that the judicial system is fair, even if police rely heavily on race or ethnicity when deciding whom to arrest or ticket, and the proportion of minorities who are convicted of crimes is similar to the proportion of whites who are convicted, then the police strategy might be reasonable, Meyer said. Conversely, if blacks are arrested or ticketed at higher rates than whites for similar crimes, but are less likely than whites to be convicted, then police judgment is arguably unsound. Such a finding would support claims that racial prejudice was resulting in unfair, inefficient and costly enforcement practices, he said. The Star-Tribune tested this theory and on July 23 published a special report, 'Presumed Guilty Until Proved Innocent.' When studying racial profiling, or police bias, reporters ideally would study stops. But Minnesota lacks such data. Next might come searches, or tickets. But that data, too, was unavailable. That leaves bookings. Hennepin County is the one Minnesota jurisdiction using a computer system that allows easy tracking of an offender's case from arrest to adjudication. It was suitable for our analysis because nearly two-thirds of all blacks in Minnesota live in that county. (Almost half of all blacks in the state live in Minneapolis.) We took the county's jail data from 1994 through 1999 and linked it to sentencing data, then flagged each booking record that resulted in a sentence of some kind. The court data covered sentences only; it did not include diversion, probation before conviction or other types of less serious legal mechanisms used to resolve some cases without a conviction. We analyzed only those charges that were identified by the county as the 'main charge' when the suspects were booked; we ignored subordinate charges - including warrants which often get dropped in plea bargains or relate to other minor charges, especially traffic and parking citations. Although our analysis examined all races and Hispanics, we focused on blacks and whites because their numbers are larger, and because we believed the races were less likely to be misidentified. Our analysis examined the bookings in two groups: those originating in Minneapolis and those originating in all other communities within the county combined. This helped minimize the effects of chance or the effects of a particularly aggressive police officer in some of the smaller communities in the county. The results for Minneapolis and the balance of the county were similar. Whites were less likely than blacks to be sentenced for most felonies. But blacks were less likely to be sentenced for minor or petty charges than were whites. This pattern was true in nearly two-thirds of gross misdemeanors, and two-thirds to three-fourths of misdemeanors. Those crimes that typically require more police discretion were less likely to result in a sentence for blacks. Indeed, some crimes, such as lurking with intent to commit prostitution or to sell drugs, rarely resulted in sentences for anyone, black or white. But such charges are used far more often for blacks than for whites. Such disparities were more pronounced in Minneapolis than in the balance of the county. To calculate booking rates, we had to estimate the annual Minneapolis population by race and by age, since no official numbers exist. We used census data and annual population estimates for the county and the state. We derived the city's population figures by adjusting them to the proportions and trends in the census data and other official sources. The population figures are a rough estimate. But after comparing them to the estimates from an economic forecasting group, we concluded our figures were reasonable and conservative. Besides, the gap between black and white booking rates was so wide that even if our population estimates were slightly off the mark, it would not have appreciably changed the findings. After adjusting for population differences, blacks were arrested more often for most crime categories than were whites. And for crimes requiring police discretion, they were generally less likely than whites to be sentenced. While these results do not demonstrate 'racial profiling' by police, they do support claims that bias often results in unfair treatment for many blacks. Our analysis should not be misinterpreted as an assessment of the police motivation for the arrests, however. We could not say whether police stopped someone on suspicion of driving without a license because he was black, or arrested someone for lurking because he was belligerent or simply because he broke the law. But we could measure the outcome of the arrests. If sentences were a measure of the quality of the arrests, then police were less effective when arresting blacks on minor crimes. We computed one cost of that inefficiency by multiplying the number of arrests per crime category that failed to result in convictions by the average booking costs. The human cost - measured in terms of resentment, public humiliation and private suffering - is incalculable.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-december-nursing-mistakes","slug":"2000-december-nursing-mistakes","url":"https://dwillis.github.io/uplink/article/2000-december-nursing-mistakes.html","year":2000,"month":"December","monthNum":12,"headline":"Nursing mistakes","kicker":"HEALTH CARE","author_name":"Mike Berens","author_title":"Reporter","affiliation":"Chicago Tribune","summary":"A Chicago Tribune investigation used the FDA's MAUDE database combined with multiple public records sources to uncover how inadequately trained and overwhelmed nurses cause thousands of patient deaths and injuries annually as hospitals prioritize profits over safety.","keywords":["nursing","hospitals","databases"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["MAUDE","Access","Excel"],"full_text":"MAUDE was my first source. She led me to a largely hidden world of death and injury inside America's hospitals. She provided scant details. Sometimes she was simply wrong. But she was unerringly on target where it counted most. That information became the foundation of a Chicago Tribune series (Sept. 10-12) which detailed how overwhelmed and under-trained nurses kill and injure thousands of patients every year as hospitals sacrifice safety for an improved bottom line. MAUDE is available as your source, too. Officially, MAUDE is the Manufacturer and User Facility Device Experience database offered freely by the U.S. Food and Drug Administration (www.fda.gov/cdrh/databases.html). It is also available from NICAR. With more than one million records, the database tracks adverse events involving medical devices. With a twist in thinking, it can also be used to track cases involving human error. Amidst the dozens of fields are two 1,000-character text fields, which provide mini-narratives of each incident. In thousands of cases plucked out through Access queries nurses were believed responsible for fundamental errors. How many patients have been accidentally overdosed through the use of infusion pumps? How many patients were fatally trapped in their bed rails? How many patients connected to life-saving machinery died unnoticed in their hospital beds? MAUDE was just one of a dozen major sources for this project. Despite the narratives, MAUDE typically offers vague, inconclusive information. There are no patient or hospital identifiers. Fortunately, there are companion paper records that can yield more details and I pulled many thousands of reports - but MAUDE is more of a pointer than a destination. I exported Access query files to Excel and began building a custom database beginning with the sketchiest of clues gleaned from MAUDE. 'Blanks' were filled through the use of a dozen state and federal public record sources, from federal healthcare investigative reports to court records. My custom database became a giant jigsaw puzzle. MAUDE helped define a border, but other databases and paper records filled in the picture. I launched into this subject with little more than a realization that the nursing profession was a largely unexamined, essential component of the U.S. healthcare system. There were no signature lawsuits or events. The Tribune encourages reporters to scout out general themes. I initially looked at workplace safety issues surrounding the way medications are labeled, but I quickly settled into the challenge of quantifying nursing-related errors. Exploiting a back-door FOIA strategy, I obtained five years of nurse disciplinary records spanning every state and U.S. territory. The National Council of State Boards of Nursing (www.ncsbn.org) compiles this non-public database. However, paper copies of the database are regularly sent to state licensing agencies, which turns the privately compiled data into a public record. Illinois reluctantly turned over the massive files, which were scanned into Access. Violations receive numerical tags as well as descriptions, such as drug offense, medical error or felony conviction. After identifying key cases, I filed additional FOIAs for full investigative files from each state. These records can provide a startling picture into how patients are killed and injured. Records revealed that some states withhold information about death or injury from public files as part of a plea bargaining process with nurses who agree not to contest administrative sanctions. For an airtight picture of Illinois, I computerized 10 years worth of disciplinary records after spending two weeks combing through musty file cabinets located in the bowels of a state licensing agency. Many states offer limited records online through boards of nursing agencies (www.ncsbn.org/files/boards/boardswebsites.asp). MAUDE coupled with disciplinary records began to provide a clearer, provable picture of medical errors linked to nursing. But a Lexis-Nexis search of lawsuits provided more cases and filled more blanks. Another gold mine of solid information is found in federal investigative reports by the Health Care Financing Administration, part of the U.S. Department of Health and Human Services. HCFA provided computer searches from their extensive database. For example, I wanted lists of every hospital that had received a routine annual survey or had been the focus of a complaint investigation. In most states, HCFA investigations are conducted by state public health officials who contract with the federal government. Reports are kept by HCFA and the state. FOIA reports from the state; it's much quicker. Federal law designates these files as public records. Even if there is a state law banning disclosure of hospital records (as in Illinois), federal public record laws supersede state law. A handful of states post information on the Web. The best is Colorado's Department of Public Health and Environment (www.cdphe.state.co.us/hflhfd.asp), which provides searchable access and mini-narratives involving hospital investigations. Most hospitals maintain private databases. Some facilities turned over their data for inclusion in my custom database. For example, one hospital acknowledged dozens of medical errors monthly. The Tribune also created custom databases to track staffing levels and other criteria, such as how many nonprofit hospitals had been purchased by for-profit companies. The Volunteer Trustees (www.volunteertrustees.org/index.html), a national organization of not-for-profit hospital and health system governing boards, maintains a database of conversions. Likewise, dozens of healthcare organizations have accumulated databases of information. Many will share portions of their records, which can provide pointers to court cases or state and federal documents.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2000-december-check-sheets","slug":"2000-december-check-sheets","url":"https://dwillis.github.io/uplink/article/2000-december-check-sheets.html","year":2000,"month":"December","monthNum":12,"headline":"Check sheets","kicker":"RACIAL PROFILING","author_name":"Jim Adams","author_title":"Reporter","affiliation":"The Courier-Journal","summary":"A Louisville news organization investigated whether police were disproportionately stopping African-American drivers by matching police dispatch records with handwritten dispatcher logs containing driver demographic information. The journalists developed a novel data-matching methodology using unit numbers and timestamps to cross-reference traffic stop records with license checks that included racial identifiers.","keywords":["profiling","investigation","data"],"topics":["Investigative Methods","Crime & Justice","Demographics & Census"],"technologies":["SPSS","Excel","floppy disk"],"full_text":"When Louisville's mayor suggested last spring that some city police officers were stopping African-American drivers unfairly, he was giving voice to a suspicion - but he had no hard facts to support his belief.\n\nWe promptly went looking to see if there were any. And at first, we came up as empty as the mayor's assertion.\n\nWe were told, for example, that the Louisville Police Department does not enter any data on traffic citations into its computer system, instead sending all citations directly to the state courts system. The courts system does enter the data. But that turned out to be useless to us because of the impossibly complex search required to separate traffic offenses from other types of citations.\n\nOn the verge of abandoning the idea, we turned one last time to some helpful veterans inside the police department for guidance. Could we get a detailed listing of all traffic stops? Yes, we were told, every reported stop is stored in the department's computer-assisted dispatching (CAD) system.\n\nOn the verge of abandoning the idea, we turned one last time to some helpful veterans inside the police department for guidance.\n\nIs the driver identified in those records in any way? No, we were told. Is there any record anywhere identifying drivers stopped? Well, actually, there is one place, one officer said: The \"check sheets.\"\n\n\"Check sheets\" are handwritten logs kept by dispatchers. When an officer stops a car in Louisville and then radios information from the driver's license, a dispatcher writes that information down, including race, before checking to see if the license is valid and if there are any arrest warrants outstanding for the driver. The department retains those sheets for a year.\n\nMatching information\n\nThe Louisville police CAD reports and the \"check sheets,\" we learned, held two pieces of common information that would enable us to match a traffic stop with a license-and-warrant check: police car \"unit number\" and time of day.\n\nFor example, if the CAD report showed that unit 315 had a car stopped between 10 a.m. and 10:20 a.m. on a given date, and the \"check sheet\" for that date showed that unit 315 requested a license-and-warrant check on a white female at 10:07 a.m., we would have a match.\n\nBut even if we could develop this information, would it be meaningful to a community wondering about the attitudes within its police department? Experts in racial-profiling research we consulted thought it probably would be - not necessarily as proof that racial profiling does or does not exist, but at least as a worthwhile indicator.\n\nThis approach also would yield information about innocent drivers who were stopped but not cited, they said, making it in one way broader than studies drawn solely from records of arrests or citations.\n\nUsing the state's open-records act, we requested the CAD reports of all traffic stops and all \"check sheets\" for 30 separate days. We selected the 30 days randomly from a 12-month period using Statistical Package for the Social Sciences (SPSS).\n\nThe police department eventually handed us a six-inch-high stack of \"check sheets,\" and another six-inch-high stack of computer printouts of traffic stops covering all our requested days. The city also was able to give us the CAD information covering 13 of our 30 requested days on a floppy disk.\n\nUnfortunately, since then the police department has no longer decided that \"check sheets\" are public records.\n\nDoing the legwork\n\nThe job of cross-referencing these two information banks was fundamentally a manual task, as tedious as a Florida recount although it was considerably easier for the 13 days the city gave us on the floppy disk. Mark Schaver, The Courier-Journal's CAR director, transferred that information to an Excel spreadsheet, which enabled me to search the CAD records of traffic stops for individual patrol car unit numbers. That was far more efficient than my eyeball scan of paper for the other 17 days. The entire job took me, working alone, about three weeks. City officers made 6,211 traffic stops on those 30 days, and viewing the information in the most conservative way, we believed we had reliably identified the race of 1,647 of those drivers, about one-fourth. In the end, we found that 44 percent of those \"race-identified\" drivers stopped and checked for warrants were African American, in a city where an estimated 27 percent of the driving-age population is black. Experts say that justifies a conclusion that blacks in Louisville were stopped and checked for warrants at a rate twice that of whites. We also found that in the city's police district with the highest percentage of whites, 23 percent of the drivers stopped were African American four-and-a-half times the district's white population. Which raised another question: Regardless of the residential population, what if 23 percent of the district's actual drivers are African Americans?\n\nCounting traffic\n\nThe experts we consulted thought it was imperative that we confront that question by actually counting traffic in that district. Which raised yet another difficult question: Where, exactly, do we go to do that?\n\nMapInfo software showed us the answer at a glance. In our sample of 1,647 \"race-identified\" stops, 267 were within that district - and one-third of those were along one major surface street that bisects the district. Mirroring the district numbers overall, 22 percent of our \"race-identified\" stops along that street were of African Americans.\n\nWe dispatched four teams of reporters to count traffic along a four-mile stretch of that street over five days during the hours when most of the stops in our sample occurred.\n\nThe result: Of more than 3,800 drivers we observed, 7.6 percent were black - one-third the percentage of blacks stopped and checked for warrants in the district. This was important confirming information, the experts said. It was a real-world observation that tended to reinforce a major question raised by the overall numbers. Given the nuances inherent in racial-profiling research, these numbers as a whole proved nothing, but they were persuasive enough to raise a reasonable question, our experts said.\n\nWhen an editor and I arrived in the police chief's office to interview him, the chief had a guest waiting with him - an assistant professor in justice administration from the University of Louisville.\n\nThe Louisville Police Department, however, didn't have any interest in addressing that question. When an editor and I arrived in the police chief's office to interview him, the chief had a guest waiting with him - an assistant professor in justice administration from the University of Louisville.\n\nShe proceeded to explain why our research was invalid. Her primary criticism concerned our sample. Rather than drawing 30 days at random from a year, she said, we should have drawn randomly from all 75,000 traffic stops made during that year. Ideally, she argued, we would have drawn 10 percent of them about 7,500.\n\nOur experts disagreed. So did another expert we called at the suggestion of the Louisville professor herself.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-race-and-tickets","slug":"2000-december-race-and-tickets","url":"https://dwillis.github.io/uplink/article/2000-december-race-and-tickets.html","year":2000,"month":"December","monthNum":12,"headline":"Race and tickets","kicker":"RACIAL PROFILING","author_name":"Amber Arellano and Victoria Turk","author_title":"Staff writer and Database specialist","affiliation":"Detroit Free Press","summary":"The Detroit Free Press investigated racial profiling in suburban police departments by manually collecting and analyzing 3,000 traffic tickets from Harper Woods, Michigan, finding evidence of disparate ticketing patterns between African American and white drivers. The journalists overcame significant obstacles including restricted courthouse access and data organization challenges to prove statistically what residents had long complained about.","keywords":["profiling","tickets","analysis"],"topics":["Demographics & Census","Investigative Methods","Crime & Justice"],"technologies":["Excel"],"full_text":"African Americans have complained about racial profiling in Detroit's suburbs for generations. They say suburban police unfairly target them for tickets when they venture outside of predominantly black Detroit and into the metro area's mostly white suburbs.\n\nNo one had ever been able to prove at least statistically - that Detroit's suburban police racially profile, however. To do it, the Detroit Free Press had to show a pattern of disproportionate ticketing by race - and in the process, take on a recalcitrant judge and 3,000 traffic tickets, among other challenges.\n\nWe chose to look at Harper Woods, a suburb that borders Detroit's east side, because of a good tip we had received. Plus, unlike most Detroit suburbs, Harper Woods cops track race on their tickets, making it possible to analyze their racial ticketing pattern.\n\nThe first question was how to do it. Harper Woods court data isn't computerized, so we had to manually sort the tickets. With piles of complex information to analyze, we knew we needed to use a database.\n\nGetting the data\n\nWith our computer-assisted reporting expert Heather Newman's help, we set up Microsoft Excel spreadsheets with the categories we wanted to track: the driver's name and address, age, race, gender, location of ticket, infraction, time and date, car type and color, fine paid, and the officer's name. We decided 3,000 tickets was a good amount - about six months worth, a respectable sampling.\n\nBeyond race, we wanted to see what the trend was in terms of location. That is, whether drivers coming from Detroit, or driving closer to Detroit, were more likely to be ticketed.\n\nThen there was the question of access. By law, journalists can look at courthouse files without filing a FOIA. But we encountered numerous roadblocks.\n\nThey wouldn't allow us to photocopy the tickets to speed up the process, then one day the courthouse administrator kicked us out for talking to a court officer who happened to come in the room. The judge eventually conceded and allowed us back in, but with restrictions that included allowing only one person to look at files at the same time and requiring his workers to review our work. Later, he allowed us to bring in a portable copier. By then we were three weeks behind schedule.\n\nSetting up the database\n\nMeanwhile, Database Specialist Victoria Turk had been setting up the database queries based on a list of initial questions: the racial and gender breakdowns of people ticketed, the most common tickets for African Americans vs. whites, which intersections were most heavily ticketed, etc.\n\nWhen Turk finally received the raw data, she had about 80 percent of the database queries set up. The results of Turk's ongoing analysis helped dictate some of the Arellano's reporting.\n\nWe often had mix-ups in the data, such as accidental duplicates or the wrong spellings of police officers. We also had to be careful when it came to the racial category. Officers check a driver's race based on their presumption, not by asking the individual. So when an officer didn't know, or put \"O\" for \"Other\" (they told us that sometimes happens with drivers of Hispanic or Arabic names or appearance, for example) we made sure they were counted that way, so we could track people by race as unambiguously as possible.\n\nMost of the data inconsistencies resulted from the manual input process.\n\nFour different individuals keyed in the data, and although they were working from the same categories and code list for offenses, there were inconsistencies where abbreviations were used for city names, spellings of street names, and virtually every data input item entered from the ticket.\n\nThese weren't necessarily typos; the ticketing officers themselves misspelled information or abbreviated creatively, or simply had poor penmanship. In retrospect, some of the data inconsistencies could have been avoided if clear and explicit directions had been given to our data keypunchers at the start of the project.\n\nWe also had some problems reaching our total sample goal of 3,000 tickets. Either through the keypunch process, or in the process of being provided the paper tickets from the court, we somehow managed to get some duplicates in the data that needed to be eliminated. If a ticket was inadvertently entered twice, it would affect our analysis to double count it. Once we eliminated the duplicates, we ended up having to return to the court to pick up a few more to meet our sample quota. This could have been avoided by keying in a few extra tickets, in case some turned out to be unusable for one reason or another.\n\nLater, Turk saw that she could perform some calculations more easily in Excel. The calculated fields, as well as the original input data, were then exported to an Access database.\n\nNo one had ever been able to prove - at least statistically - that Detroit's suburban police racially profile, however. To do it, the Detroit Free Press had to show a pattern of disproportionate ticketing by race.\n\nThe queries performed tried to explore the role that race played in ticketing, if any. We looked at race differences for nearest intersection, offense, average dollar amount and total tickets.\n\nEventually, we looked at race differences by the ticketing officer. We looked at ticket dollar amounts by car model year to see if older cars were ticketed more because of defective equipment. We also took a look at the most heavily ticketed intersections, time of day, and model years to give our readers a feel for the way the Harper Woods police patrolled.\n\nTraffic study\n\nWe had one more hurdle: whether black drivers were being disproportionately ticketed. To find this, we had to figure out the number of black drivers on Harper Woods' roads and to compare it to the ticketing pattern.\n\nTo do that, we conducted a traffic study, similar to one done in New Jersey by investigators who looked into the state police's ticketing patterns. Arellano sat at Harper Woods' top ten most ticketed intersections and documented the race of almost 8,000 drivers. A profiling expert, a Temple University professor, advised us how to do it.\n\nIt took a few hours every day for about five days. If we couldn't tell what the driver's race was - because of a dirty windshield, or someone whose race wasn't obvious - we didn't count them.\n\nFinally, after three months of off-and-on work, we published the package June 1. The lead 1A story showed that Harper Woods Police are overwhelmingly and disproportionately ticketing African Americans.\n\nSuggestions\n\nIf you're thinking about a similar database investigation, here are our suggestions:\n\nPlan ahead with your editors and coworkers, especially when it comes to setting up the database, and give yourself extra time.\nIf you run into access problems, be diplomatic but firm. In this case, only the threat of a good fight would budge the court judge. And of course, know the law.\nDuring a difficult project where you and others are encountering many hurdles, encourage and reward people throughout the process. Good guacamole, brownies ever you make well - and small gifts are appreciated and deserved.\nFact check the database results as many times as necessary, and by different people.\nAvoid Harper Woods District Court.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-tipsheets","slug":"2000-december-tipsheets","url":"https://dwillis.github.io/uplink/article/2000-december-tipsheets.html","year":2000,"month":"December","monthNum":12,"headline":"Tipsheets","kicker":"RACIAL PROFILING","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The article provides a list of IRE Resource Center tipsheets offering guidance on building, maintaining, and managing newsroom intranets, including practical advice from industry experts and survey results.","keywords":["intranet","newsroom","tipsheets"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet"],"full_text":"For more on building and maintaining newsroom Intranets, check out these tipsheets, available from the IRE Resource Center, (573) 882-3364.\n\nTipsheet #895. NewsEngin's George Landau offers tips on getting started with an Intranet (1999, Boston).\n\nTipsheet #894. Offers tips on running an Intranet (1999, Boston).\n\nTipsheet #817. This includes useful information to include in a newsroom Intranet (1998, New Orleans).\n\nTipsheet #1328. Results of on an Internet survey on NICAR-L concerning newsroom Intranets (2000, Lexington).","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-data-and-privacy","slug":"2000-december-data-and-privacy","url":"https://dwillis.github.io/uplink/article/2000-december-data-and-privacy.html","year":2000,"month":"December","monthNum":12,"headline":"Data and privacy","kicker":"NATIONAL CAR CONFERENCE","author_name":"Paul Monies","author_title":"Associate Editor","affiliation":"IRE and NICAR","summary":"Media experts discuss the paradox of American privacy attitudes, where consumers claim to want privacy but willingly share personal data with corporations, while also debating the importance of keeping government records public versus protecting personal information.","keywords":["privacy","journalism","data"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"While privacy on the Internet has garnered a lot of attention these days, the whole issue of privacy needs to be explored more by the media, especially in light of some conflicting views by the public.\n\nSpecifically, polls suggest that while nine out of 10 Americans want more privacy, many people willingly surrender personal information to large corporations in return for a discount on groceries or for a few extra air miles.\n\n\"We talk out of both sides of our mouths when it comes to privacy,\" said Charles Davis of the University of Missouri's Freedom of Information Center. Davis shared his views at a panel on \"New Media Ethics and Privacy\" at the 2000 National Computer-Assisted Reporting Conference in Lexington, Ky., held in September. He was joined by Rebecca Daugherty, director of the Freedom of Information Service Center at the Reporters Committee for Freedom of the Press, Robert O'Harrow of The Washington Post and Dan Gillmor of the San Jose Mercury News.\n\nConsumers may gain extra goodies by sharing private information, but marketers gain much more comprehensive data on consumer behavior. At the same time, however, journalists have a tough time convincing the general public that personal information, laid out in government databases, should be open and accessible to all.\n\nDavis said journalists should remind their readers of the difference between data collected by corporations and data collected by government agencies. \"We have to tell the public the way we use data is different from how Procter & Gamble uses it,\" Davis said.\n\nHowever, O'Harrow admits that journalists haven't even laid enough groundwork to educate readers on why public records are important, let alone private ones. Davis said that critics want to close off information being posted by government agencies on Web sites because of privacy concerns.\n\n\"Unfortunately, the government is using scare tactics to close off public records,\" Gillmor said.\n\nIn one more example, a member of the Lexington audience told of the unintended morphing of privacy legislation in Colorado.\n\nThe 1994 Driver's Protection Act prohibits state DMVs from reselling certain driver's license data, but since local police departments in Colorado collect some of the same information, they conclude they are arms of the DMV.\n\n\"That's a great example of what's going wrong with privacy,\" Gillmor said.\n\nNoted Davis: \"Privacy is becoming an industry. It's not just a topic anymore. There's a whole cottage industry emerging to protect personal privacy.\"\n\nO'Harrow, who covers privacy issues for The Post, said that the fledgling privacy beat is much like the early days of civil rights reporting and environmental reporting.\n\nSome issues of concern on the privacy beat include when companies legitimately get public records and turn around to resell and append it with other information, and the lack of individual control over information once it is out there. For example, drug companies can buy prescription information from pharmacists. O'Harrow said that while most people think medical records are confidential, in truth they are among the least protected records in America. Specifically, HMOs, banks and security firms all have access to insurance records, which ultimately lead to medical records.\n\n\"When bad medical things are happening, public records are important,\" Daugherty said. \"The First Amendment should be a huge defense for that.\"\n\nBut Davis says that there are various uses for data, and it's not just journalists who can claim a public interest. Since researchers and historians all use public records and government data, Gillmor said he sees a problem with setting journalists apart from the general public. \"What are the definitions of a journalist in the Internet era?\" Gillmor asked. \"Serious problems arise when government defines what a journalist is.\"\n\nNoted O'Harrow: \"Privacy is a great story. These stories have a very real impact on very real people. There's room for explaining the issue in real ways.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-web-links","slug":"2000-december-web-links","url":"https://dwillis.github.io/uplink/article/2000-december-web-links.html","year":2000,"month":"December","monthNum":12,"headline":"Web links","kicker":"NATIONAL CAR CONFERENCE","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The Freedom of Information Center at the University of Missouri maintains a comprehensive library of over one million articles on government information access, while the Reporters Committee for Freedom of the Press provides resources for journalists seeking to understand their rights.","keywords":["freedom","information","journalism"],"topics":["Government & Politics"],"technologies":["World Wide Web"],"full_text":"The Freedom of Information Center at the University of Missouri is a reference and research library that has more than one million articles on access to governmental information. Its Web site is http://web.missouri.edu/~foiwwwl\n\nThe Reporters Committee for Freedom of the Press www.rcfp.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-tipsheets-2","slug":"2000-december-tipsheets-2","url":"https://dwillis.github.io/uplink/article/2000-december-tipsheets-2.html","year":2000,"month":"December","monthNum":12,"headline":"Tipsheets","kicker":"NATIONAL CAR CONFERENCE","author_name":"","author_title":"","affiliation":"IRE","summary":"The article provides a list of tipsheets from the IRE Resource Center offering guidance on building, maintaining, and running newsroom intranets, covering topics from initial setup to best practices and survey results.","keywords":["intranet","newsroom","tipsheets"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet"],"full_text":"For more on building and maintaining newsroom Intranets, check out these tipsheets, available from the IRE Resource Center, (573) 882-3364.\n\nTipsheet #895. NewsEngin's George Landau offers tips on getting started with an Intranet (1999, Boston).\n\nTipsheet #894. Offers tips on running an Intranet (1999, Boston).\n\nTipsheet #817. This includes useful information to include in a newsroom Intranet (1998, New Orleans).\n\nTipsheet #1328. Results of on an Internet survey on NICAR-L concerning newsroom Intranets (2000, Lexington).","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-equity","slug":"2000-december-equity","url":"https://dwillis.github.io/uplink/article/2000-december-equity.html","year":2000,"month":"December","monthNum":12,"headline":"Equity","kicker":"","author_name":"Mike Fish","author_title":"Reporter","affiliation":"Atlanta Journal-Constitution","summary":"The Atlanta Journal-Constitution conducted a comprehensive investigation into gender equity in Georgia high school athletics by analyzing education department civil rights cases, surveying coaches and administrators, and obtaining financial and participation data from the Georgia High School Association.","keywords":["athletics","investigation","gender"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"application forms to determine how the money was allocated by sport and gender.\n\nThe newspaper also obtained from the U.S. Department of Education an electronic copy of every civil rights case filed since 1985 involving an education institution. The information was split into two databases and coded. With some programming, the databases were merged, the codes were turned into English and the results were posted on ajc.com in a searchable form.\n\nSeparate mail surveys were sent to all (more than 3,500) varsity high school coaches in Georgia, to 323 principals, 180 superintendents and to the 39 members of the Georgia High School Association, asking about budgets, participation rates, their teaching and coaching backgrounds, athletic facilities and gender issues.\n\nNinety-three percent of superintendents, 92 percent of executive committee members, 67 percent of the principals and more than 26 percent of the coaches participated. For some of the surveying, the newspaper worked in cooperation with sports researchers at Georgia State University.\n\nThe newspaper also developed a survey form that was sent to each of the state high school and activities associations in the country, and it polled the 46 private schools in Georgia that are members of the Georgia High School Association.\n\nFinally, the newspaper obtained access to the Georgia High School Association's financial records for the past five years, and it obtained in electronic form the GHSA's 1998-1999 athletic participation survey for every sport at every high school, broken down by gender. The association initially resisted providing the data, but complied after its attorney concluded the GHSA would lose an open-records case if it went to court. The data was converted to a searchable form and included among the offerings on ajc.com.\n\nMike Fish can be reached by e-mail at mfish@ajc.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-social-science","slug":"2000-december-social-science","url":"https://dwillis.github.io/uplink/article/2000-december-social-science.html","year":2000,"month":"December","monthNum":12,"headline":"Social science","kicker":"","author_name":"Paul Monies","author_title":"","affiliation":"NICAR","summary":"Journalists discuss their investigation into rampage killings, detailing the challenges of defining terms, accessing incomplete news databases, and lack of social science research, while emphasizing the importance of combining database analysis with traditional reporting methods.","keywords":["shootings","databases","research"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Lotus 1-2-3","NewsEngin","Nexis","Dow Jones"],"full_text":"tallized after several mass shootings in 1999, including Columbine High School and the Atlanta day trader shootings.\n\n\"There was a notion in people's heads that there's a strong sense of concern about this 'new' violence,\" he said. \"We set out to explain it and it started as a regular, narrative newspaper story but we decided to go deeper.\"\n\nFessenden said the biggest problem the team faced was defining the terms used for researching rampage killings. The Times used its news research department as well as the AP's state wires to gather information.\n\n\"News databases are nowhere near as complete as you think,\" he said. \"There's lots of holes in coverage and big differences between Dow Jones and Lexis-Nexis.\"\n\nTo manage the huge amounts of information, the team turned to NewsEngin and Lotus databases for data entry and analysis.\n\nFessenden said the series faced hurdles because there was a lack of quality social science research on rampage killings to give context to their findings. The series used the FBI's Supplemental Homicide Reports for details on each killing.\n\nTo balance news databases such as Lexis-Nexis and Dow Jones, which have limited information prior to the mid-1980s, Meyer suggested getting back to people. \"Identify that 'old guy' in the newsroom and ask them what they remember,\" he said. \"Ask the librarians and compare it with what you have for the 1980s to give you some way to estimate the error rate.\" In response, Fessenden said he was limited by the data they had access to. \"If we had gone further, it would have been a geometric increase in work,\" he said. \"But we didn't base any data over time on our database-that was from the Supplemental Homicide Report.\" While both Reeves and Fessenden admitted the stories were starting points, Meyer congratulated them for the work and their ability to convince management that the projects were worth doing. \"This is social science because it tells you what they did and how they did it,\" Meyer said. Paul Monies can be reached by e-mail at paulm@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-tapes-of-panels-from-the-national-car-conference","slug":"2000-december-tapes-of-panels-from-the-national-car-conference","url":"https://dwillis.github.io/uplink/article/2000-december-tapes-of-panels-from-the-national-car-conference.html","year":2000,"month":"December","monthNum":12,"headline":"Tapes of panels from the National CAR Conference","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The National CAR Conference in Lexington offers recorded panel tapes available through Sound Images, covering topics such as journalism combined with social research methods, case studies on rampage killers and priests with AIDS, and new media issues including privacy and ethics in online data.","keywords":["journalism","conference","data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Tapes of panels from the National CAR Conference in Lexington, including those discussed in this issue, are available from Sound Images, (303) 649-1811.\n\nMore information, including tape numbers for each panel, is available at www.ire.org/trainingl audio.html\n\nWhen journalism meets social research methods and a look at two case studies: Rampage killer and priests with AIDS: Tape #CAR00-45.\n\nNew Media and CAR: Privacy, ethics and online data: Tape #CAR00-48","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-tread-carefully","slug":"2000-december-tread-carefully","url":"https://dwillis.github.io/uplink/article/2000-december-tread-carefully.html","year":2000,"month":"December","monthNum":12,"headline":"Tread carefully","kicker":"RACIAL PROFILING","author_name":"","author_title":"","affiliation":"","summary":"Journalists investigating racial discrimination in traffic stops must carefully account for multiple confounding variables including the racial demographics of highway users, differences in ticket-challenging behavior between groups, and socioeconomic factors that may influence citation patterns.","keywords":["profiling","data","demographics"],"topics":["Investigative Methods","Demographics & Census","Government & Politics"],"technologies":[],"full_text":"Tread carefully\n\nA question posted to NICAR-L in March asking for warnings about interpreting race and traffic stops prompted a flood of discussion. Answers to this question varied, but the general consensus was that analyzing traffic stop data for any indication of race discrimination is a thorny proposition. This is a condensed version of that discussion. To see the full thread, search the NICAR-L archives at www.ire.org/datalibrary/nicarl.htm\n\nTerminology\n\nDistinguish between traffic stops and tickets. Robert Bennicasa, of Gannett News Service, wrote: \"But I'd be wary of inferring that tickets were 'legitimate' or not, or constituted harassment, on the basis of conviction rates between two groups. The groups might tend to have different reasons for and rates of challenging tickets, with some factors being lack of time, lack of faith in the system, ability to pay the fine, ability to get time off work to go to court, etc. Also, one group might have a lower threshold for challenging tickets, pleading not guilty to 'legitimate' tickets more frequently, and the other group might only challenge those that are more defensible in court.\"\n\nDemographics\n\nDetermining the demographics of highway users is necessary in order to put the data into context. If there is a higher proportion of tickets given to minorities, that would not be unusual if the particular geographical area in question is predominately minority. Determining this, however, could be time consuming. The best suggestion offered was to collect the information yourself by sitting alongside a roadway for a few days. Steve Doig, of Arizona State University and formerly of The Miami Herald, did just that for a traffic ticket study of a one-main-street small town in South Florida. Another reporter and Doig spent a day counting heads of drivers at the main intersection. \"The biggest problem with racial profiling stories is data tends to be incomplete and unless you're willing to do the field collection that Steve Doig suggests, it's hard to prove the police's excuses are wrong,\" wrote Drew Sullivan, formerly of The Tennesseean. Sullivan wrote a racial profiling story in February 2000 (www.tennesean.com/sii/00/02/20/my profile 20.shtml).\n\nTo figure out who is on the road, Sullivan said you can't look at Census numbers because blacks tend to be younger and the number of black drivers is less than their population as a whole. If you look at drivers in a city or county, you're not getting the commuters or people driving on the Interstate. Then there's the problem of whether or not to include drivers with suspended/rev or cancelled licenses.\n\nMinor offenses\n\nBeware that socio-economics may play a role, i.e. tickets for minor offenses such as equipment problems, making it inaccurately appear race is a factor.\n\nFrank Kummer, of the Courier-Post in Cherry Hill, N.J., wrote: \"I would guess that if the city in question is mostly minority, the tickets would reflect that. So, a story saying more blacks are ticketed might be incorrect. Secondly, tickets for so-called small offenses can be misinterpreted as simple harassment. For example, you see a ticket issued to a black motorist for 'improper use of horn.' You might assume then that he was stopped simply for being black. You might be right. But, you also don't know if the cop actually 'downgraded' from a bigger offense to give a driver a break. This happens quite frequently. \"I think it's easy for us not to like cops and assume the worst. Indeed, that's easy to do so given our experiences. But I also believe you should approach it objectively.\"\n\nA few last bits of advice: Study which officers gave the tickets, and study the types of offenses. And one more from Drew Sullivan: \"Be careful how you use the term racial profiling as well. It has a specific meaning using race as part of a profile of a person likely to commit a certain type of crime. I'm guessing it came about when drug interdiction programs set up specific profiles of drug couriers. If race was part of that profile, it was racial profiling and the ACLU got mad. The point is the police are looking for a specific crime usually drugs. It's not the same as a cop giving a ticket to a black motorist because the motorist is black. That's more akin to racism than racial profiling.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-stories","slug":"2000-december-stories","url":"https://dwillis.github.io/uplink/article/2000-december-stories.html","year":2000,"month":"December","monthNum":12,"headline":"Stories","kicker":"RACIAL PROFILING","author_name":"","author_title":"","affiliation":"","summary":"The IRE Resource Center provides several investigative journalism stories examining racial profiling by police departments, including traffic stops in Seattle, profiling practices in New Jersey, and the phenomenon of 'driving while black.'","keywords":["profiling","police","race"],"topics":["Crime & Justice","Demographics & Census","Investigative Methods"],"technologies":[],"full_text":"For more on racial profiling, take a look at these stories available from the IRE Resource Center, (573) 882-3364.\n\nStory #16331. The Seattle Times takes a look at traffic citations and race in Seattle over a five year period.\n\nStory #16502. The Village Voice goes beyond the typical \"numbers\" story and profiles black and white police officers as well as victims of profiling.\n\nStory #16874. The New York Times reports that senior officers knew of racial profiling by police in New Jersey at least as far back as 1996.\n\nStory #16661. This Newark Star-Ledger story was one of the first on the widely reported problem of racial profiling in New Jersey.\n\nStory #15652. Emerge magazine looks at the \"Driving while black\" phenomenon.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-intranet-set-up","slug":"2000-december-intranet-set-up","url":"https://dwillis.github.io/uplink/article/2000-december-intranet-set-up.html","year":2000,"month":"December","monthNum":12,"headline":"Intranet set up","kicker":"TECH TIP","author_name":"Alan Cox","author_title":"Computer-assisted reporting specialist","affiliation":"WCCO-TV, Minneapolis","summary":"A journalist recounts the challenges of implementing computer-assisted reporting in a newsroom, from the chaos of managing shared databases on CDs to eventually building a successful intranet system using SQL Server and UniQuery software.","keywords":["database","newsroom","technology"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["FoxPro","CD-ROM","SQL Server","Access","NewsEngin UniQuery","World Wide Web"],"full_text":"I knew our computer-assisted reporting was getting out of control on \"The Day of Eleven Names.\"\n\nWe had what I suspect is the typical experience of newsrooms that stick with CAR for a few years. We collected a few gigabytes of data for investigative projects. The same data would be a blessing for daily news - if we could figure out how to access it within a day.\n\nWhen CD recorders became affordable, I began pressing the data onto discs. We bought a second copy of the database program FoxPro and put it on a communal computer at the assignment desk. But almost everyone was put off by FoxPro's complexity (or maybe it was the length of the instructions I wrote: \"Step 1: Use the mouse and click twice on the little picture of a So lookup requests kept coming to me.\n\nThe worst day\n\nI kept a count on what I remember as the worst day: in eleven separate requests, reporters brought me names to \"check on your computer.\" I had to spend 20 minutes on each name, switching CDs, watching the query progress bar slowly creep along, printing out the results and delivering them to the reporters (which I came to think of as \"putting the facts on a silver platter for the ingrates\"). Soon, we had to back off our efforts to spread the use of CAR.\n\nToday, technology has become cheap enough that the single-computer approach might work. Thirty-gigabyte hard drives are reasonably priced, eliminating the need to shuffle CDs. And a few months ago I saw a demonstration by the CAR specialist at one of our competitors. He created attractive search screens within FoxPro that are customized for major databases.\n\nBut I was a non-computer-assisted reporter long enough to know that you shouldn't expect reporters to abandon the telephone at their desks to look for something on a shared computer. And assignment people have enough to worry about without having to serve as librarians for the staff.\n\nEventually we experienced a wonderful coincidence: the station's budget for buying computers had a small surplus, and NICAR announced a seminar for creating newsroom Intranets. My news director gave me both. There were two challenges. One was getting the new server to operate on the station's computer network. For this I had generous help from our information technology manager. In fact, I'm not sure how generous it was. He used to be the engineer who ran our satellite truck, and after years of watching me do live shots, I don't think he was going to let me come near plugging a computer into our network. The other challenge was the industrial-strength database program, Microsoft SQL Server. It works like FoxPro or Access, but has a number of devilish differences in the details. And, at least in the version we bought three years ago, it generates the most indecipherable error messages I've ever seen.\n\nWhat made it work for me was the program that serves as an intermediary, NewsEngin's UniQuery. Often praised in these pages, UniQuery creates search forms for the databases loaded on the server. It installs easily. It creates forms on Web pages, and by now almost every reporter has experience with that interface.\n\nUsing UniQuery\n\nBest of all, UniQuery creates a unified search page so reporters can hit every database at once. We have 70 queries loaded on our Intranet now, and SQL Server is powerful enough to complete them in a half minute. There's a wonderful serendipity in finding extra details about a person in databases you'd never have time to check otherwise - a phone number on a hunting license or an employer in campaign contribution lists. Once UniQuery was up and running, I ventured into creating my own Web-based search pages for other data, using training aids created by Tom Torok, now of The New York Times. I created profiles of public schools drawing data from three different sources. I listed city data from FBI crime reports. Even simple Intranet features have their benefits. I got permission from the state to post the Minnesota criminal code (not that I fully accept why I need permission for that). It's copied straight from the state legislature's Web site. But the indexing built into the server's operating system is simpler to use and is up and running more often than the legislature's search engine. We've even branched out to other features used by the entire station, like a Web form to send messages to our alphanumeric pagers. The most widely used features within the station are a form to fill out expense reports on a spreadsheet and our overnight TV ratings.\n\nLessons from the Webmaster\n\nAll of which means I've turned into a Webmaster. That's taught me lessons that may apply as we all try to create versions of our work for the Web as well as for broadcast.\n\nFirst, not that a TV reporter should need to be reminded to be concise, but I thought the Intranet was supposed to be different the more details crammed in, the better. Instead, I've been surprised to see how slow users are to \"drill down,\" to click repeated links to get \"deeper\" into the data for more details. I've even watched people who seem reluctant to scroll down a single page to look for data. We're about to receive a new version of UniQuery that should simplify the look of the results.\n\nI've learned not to be cavalier about letting people figure things out for themselves. One day I watched an assignment person struggle on the Intranet to calculate how old a person was from the date of birth. I smugly shouted out an answer in an arrogant \"I got straight-As in arithmetic\" tone, and then had to go back in a minute when I realized I'd figured it wrong, too. I've now added an \"AgeToday\" column to the query results for databases that include birth dates. Let SQL Server worry about the math.\n\nAnd although I no longer have the need to tell people to click twice on the little fox, I have tried to make instructions clearer based on user feedback (often shouted at me across the newsroom when I've got my own deadline to worry about). I assembled a PowerPoint tutorial on the use of our Intranet that I've updated several times to reflect complaints and cautions. One is, \"These databases were collected from government agencies, and governments can make mistakes.\" That may seem too rudimentary even for Journalism 101, but I think it's a helpful reminder for people impressed by the presumed power of computerized data.\n\nBut maybe I contribute to that image of power. I keep a running tally of the size of the Intranet on its homepage, something like the McDonald's hamburger signs. Right now it reads \"Now featuring 41.9 million records.\" Eleven names? Bring 'em on.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-tipsheets-3","slug":"2000-december-tipsheets-3","url":"https://dwillis.github.io/uplink/article/2000-december-tipsheets-3.html","year":2000,"month":"December","monthNum":12,"headline":"Tipsheets","kicker":"TECH TIP","author_name":"","author_title":"","affiliation":"IRE","summary":"This resource lists IRE tipsheets providing guidance on building, maintaining, and utilizing newsroom intranets, with materials covering setup, management, content, and survey results from 1998-2000.","keywords":["intranet","newsroom","tipsheets"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet"],"full_text":"For more on building and maintaining newsroom Intranets, check out these tipsheets, available from the IRE Resource Center, (573) 882-3364.\n\nTipsheet #895. NewsEngin's George Landau offers tips on getting started with an Intranet (1999, Boston).\n\nTipsheet #894. Offers tips on running an Intranet (1999, Boston).\n\nTipsheet #817. This includes useful information to include in a newsroom Intranet (1998, New Orleans).\n\nTipsheet #1328. Results of on an Internet survey on NICAR-L concerning newsroom Intranets (2000, Lexington).","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-web-links-2","slug":"2000-december-web-links-2","url":"https://dwillis.github.io/uplink/article/2000-december-web-links-2.html","year":2000,"month":"December","monthNum":12,"headline":"Web links","kicker":"TECH TIP","author_name":"","author_title":"","affiliation":"","summary":"A curated list of resources and software platforms for intranet development, including NewsEngin, FileMaker Pro, SQL Server, and ColdFusion with their respective website URLs.","keywords":["intranet","software","resources"],"topics":["Technology"],"technologies":["FileMaker Pro","SQL Server","ColdFusion"],"full_text":"For more on Intranet software, check out the following Web sites:\n\nNewsEngin\nwww.newsengin.com. Under \"Free Tools,\" you'll find a link to Tom Torok's tools and handouts from training seminars.\n\nFileMaker Pro\nwww.filemaker.com\n\nSQL Server\nwww.microsoft.com/sql\n\nColdFusion\nwww.coldfusion.com/products/coldfusion","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2000-december-bits-bytes-and-barks","slug":"2000-december-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2000-december-bits-bytes-and-barks.html","year":2000,"month":"December","monthNum":12,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article discusses upcoming Census 2000 data release plans, provides technical solutions for importing fixed-width text into Access, and introduces the NICAR Database Library's federal campaign finance data subscription service.","keywords":["census","database","data"],"topics":["Data & Databases","Journalism & Newsroom","Technology"],"technologies":["FTP","CD-ROM","Excel","Access","Word","WordPad","FoxPro"],"full_text":"Bits, Bytes and Barks\n\nCensus Data\n\nThe big question these days is, \"How will I get the Census 2000 immediately after it's released?\" The first data, state-by-state redistricting data, will come out sometime in March. The Census Bureau plans to make it available in several ways, including on the Web, via FTP and CD-ROM. IRE and NICAR and the Associated Press are both studying ways to get the information to journalists the same day it is released. Details have not been worked out yet. However, NICAR's plan will include producing a standard set of extracted data, offering the most immediately useful summary levels (like state, county, place and tract). These would be extracted into Excel files for each state and put on an FTP server accessible to subscribers.\n\nStay tuned to Uplink and the NICAR-L and IRE-L listservs for more details about the NICAR Census plan. More information about Census 2000 for journalists is available at Steve Doig's Web site, http://cronkite.pp.asu.edu/census.\n\nAccess Tech Tip\n\nHere's a question posted to the NICAR-L list recently with a common, but annoying, importing problem: \"I'm trying to import fixed-width text into Access, using the import wizard, but before the process begins, an error message appears telling me that the first row on my records is too large. The record length is about 320 characters. The text I'm bringing in includes about 300,000 records. Where have I missed the boat?\"\n\nThe problem was that the file either didn't have end-of-record characters, or had ones that Access doesn't recognize. As a result, Access was thinking the entire 300,000-plus-record file was one big record and it choked.\n\nThere are several ways to fix this problem:\n1) If the file isn't too large, you can open it in Word or Wordpad and save it as MS-DOS text.\n2) If you have FoxPro in your office, another option is to open it in FoxPro's text editor and saving it as a new file. FoxPro will automatically add the end-of-record characters.\n3) There are many text editing programs available, including demo and shareware versions. These will allow you to add in the necessary characters. Two possibilities are EditPad, www.jgsoft.com, or UltraEdit32, www.ultraedit.com.\n\nCampaign Finance Data\n\nThe NICAR Database Library offers a monthly subscription to federal campaign finance data from the Federal Elections Commission. Each month, you would receive four tables of data, including individual contributions, committee contributions and other transactions, a listing of candidates and a listing of committees.\n\nThe subscription costs $100 for news organizations with circulation under 50,000 or 50-200 market; $200 for circulation 50,000 to 100,000 or 25-50 market; and $225 for circulation over 100,000 or top 25 market.\n\nMore information about this data is available at www.ire.org/datalibrary/databases or to place an order call NICAR at (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-measuring-foster-care","slug":"2001-januaryfebruary-measuring-foster-care","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-measuring-foster-care.html","year":2001,"month":"January/February","monthNum":1,"headline":"Measuring foster care","kicker":"CHILD WELFARE","author_name":"Jason Method","author_title":"","affiliation":"Asbury Park Press","summary":"Despite a significant reform effort by New Jersey's child protection agency, children are spending more time in foster care. The article explores the state's lack of progress against its reform goals.","keywords":["foster care","reform","child welfare"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["SPSS"],"full_text":"Despite New Jersey's child protection agency's two-year, $128 million reform effort, children who are wards of the state are spending more time in foster care, group homes and institutions than just a few years ago. That was the key finding when the Asbury Park Press decided to find out how much, if any, progress the state had made in its reform goals. The child protection agency is veiled under confidentiality laws, and New Jersey has one of the weakest public access laws in the nation. DYFS officials turned down requests for statistical reports. Fortunately, a federal database helped pierce through this confidentiality. The Adoption and Foster Care Analysis and Reporting System (AFCARS) tracks 600,000 children under state foster care. This database includes detailed information on each child such as birth date, sex, race, disabilities, entry and exit dates from care, reasons for removal, and more. Despite confidentiality, AFCARS data provides essential insights, such as the number of placements and removals from homes. Unfortunately, the data has its limitations like missing details on some critical aspects. However, as AFCARS data from 1999 and 2000 was obtained, it demonstrated that New Jersey's system wasn't significantly reformed yet, and black children seemingly suffer worse care. This evaluation in SPSS led an impactful report showing stagnancy in reform efforts.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-januaryfebruary-race-problem-solutions","slug":"2001-januaryfebruary-race-problem-solutions","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-race-problem-solutions.html","year":2001,"month":"January/February","monthNum":1,"headline":"Race problem solutions","kicker":"CENSUS 2000","author_name":"Steve Doig","author_title":"","affiliation":"Arizona State University","summary":"The article discusses the complexities in comparing racial data from the 1990 and 2000 Census due to changes allowing multiracial categories, and solutions to overcome these issues.","keywords":["Census 2000","racial data","multiracial"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"A year ago, I discussed looming issues with comparing racial and Hispanic origin data between the 1990 and 2000 Census. No solutions were initially offered, but now that the problem is here, solutions must be found. The 2000 Census allowed respondents to choose multiple racial categories, ballooning the categories from 10 in 1990 to 126 in 2000. This presents issues in making valid comparisons for questions related to racial fairness, such as education and health access. Solutions vary from ignoring multi-racial categories where they are small, to creating a unified 'bridge' method for analysis, which distributes multiracial individuals proportionally across traditional racial categories. Alternatively, combining them through sophisticated dataset manipulation could provide a refined comparison to the simpler 1990 categories. While more solutions like definitive bridge categories from Census Bureau data might be years away, these suggested methodologies can offer interim plans for handling these disparities.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-januaryfebruary-car-newsletters","slug":"2001-januaryfebruary-car-newsletters","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-car-newsletters.html","year":2001,"month":"January/February","monthNum":1,"headline":"CAR newsletters","kicker":"","author_name":"Richard Dalton","author_title":"","affiliation":"Newsday","summary":"This article provides practical tips for creating an in-house computer-assisted reporting (CAR) newsletter, including featuring This-Is-How-I-Did-It articles, Q&A sections, software tips, and highlighting CAR resources to help journalists learn reporting techniques.","keywords":["newsletter","reporting","software"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Excel","Access","HTML"],"full_text":"So you do computer-assisted reporting at your news organization and you've decided that with all the spare time on your hands, you might as well start an in-house newsletter to spread the CAR gospel to the rest of the newsroom? Great. You've joined a handful of other newsrooms. Even with a limited amount of time, you can produce a CAR newsletter that is both practical and inspirational. Here are some tips on creating CAR newsletters. CAR enthusiasts sent in some tips, while I culled others from reading newsletters.\n\nFor some examples of in-house newsletters, including Car Talk, Byte Me and Mouse Droppings, check out www.kricar.com.\n\nThis-Is-How-I-Did-It articles\n\nThe best way to provide both practical advice and inspiration is the TIHIDI article the This-Is-How-I-Did-It piece. Uplink is chock full of TIHIDI articles, and for good reason. Such pieces show how you can replicate the story in your area and help you brainstorm. And the articles offer the Wow factor: \"Wow, it's amazing what you can do with computer-assisted reporting.\"\n\nEven with a limited amount of time, you can produce a CAR newsletter that is both practical and inspirational.\n\nEven a small in-house newsletter should contain at least one TIHIDI article. And the best part is that you can delegate this task: Ask other staff members to write articles describing how they wrote a CAR story. If you're part of a chain, ask other reporters in your chain to contribute.\n\nThe piece should briefly describe the CAR story, indicate what software was used, what databases were analyzed and how they were obtained. A box accompanying the story should list some of the key fields in the database. If CAR reporting is quite active in your newsroom, run a roundup of CAR stories.\n\nCAR Q&A\n\nAnswer questions that reporters submit to the newsletter. Or if you help out another reporter in person, turn the help session into a Q&A for the newsletter.\n\nSoftware Tips\n\nMicrosoft Excel and Access are the most commonly used software tools in CAR. Focus on those products initially, starting with simple tips: subtotals, importing data from HTML documents, etc.\n\nDon't let 'em lose it\n\nYou know how easy it is to lose something amidst the mounds of papers on a reporter's desk. So, Jennifer LaFleur, of the St. Louis Post-Dispatch recommends printing the newsletter on obnoxiously bright paper, so it won't get lost in the shuffle. In her previous job in San Jose, she gave out binders for reporters to store tip sheets and copies of the newsletter.\n\nPoint out CAR resources\n\nHighlight useful Web sites for finding anything from phone numbers to government data. What databases does your news organization have? Focus on a few, listing possible story ideas. List CAR resources reporters can turn to, including possibly yourself. Does your news organization offer training in Microsoft Excel or Access? List the schedule of classes. Or hold some classes yourself. Also point out books or videos on CAR software. Do you have tech-support people who can answer questions about software? If so, point reporters in their direction (and save yourself some time.)\n\nHighlight Math Tips\n\nInclude tips on calculating percentage change, per-capita rates inflation adjustments, etc.\n\nHow often to publish\n\nJanet Roberts of the St. Paul Pioneer Press went from writing all of the articles to relying almost entirely on contributions from reporters, tech-support staff librarians. Even so, she publishes just once a quarter because she \"found it impossible to find the time to produce it more often than that,\" indicating just how time-consuming a newsletter can be.\n\nSee how long it takes to produce your first newsletter to gauge how often you'd like to issue your newsletter.\n\nLet's face it. Writing about Excel can be boring. But if your newsletter is fun to read, more people will get through it. Even fun graphics you can pull from clip art liven up the page.\n\nOnline versions\n\nProvide a paper edition and online edition, if possible. Remember to include links for data and Web sites related to the story.\n\nRichard Dalton can be reached by e-mail at rdalton@newsday.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-historical-data-sources","slug":"2001-januaryfebruary-historical-data-sources","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-historical-data-sources.html","year":2001,"month":"January/February","monthNum":1,"headline":"Historical data sources","kicker":"CENSUS 2000","author_name":"Neill A. Borowski","author_title":"Database Editor","affiliation":"The Philadelphia Inquirer","summary":"A journalist explains how to build historical census data archives for local newspapers, providing practical tips for accessing demographic information from past decades through libraries, online databases, and the Census Bureau.","keywords":["census","archives","data"],"topics":["Data & Databases","Demographics & Census","Journalism & Newsroom"],"technologies":["ArcView","CD-ROM","9-track tape"],"full_text":"Historical data sources\n\nBy Neill A. Borowski\nThe Philadelphia Inquirer\n\nMy squirrel-like habits of burying historical census data like acorns to be uncovered in the future began in the days of paper.\n\nAnd my prized \"database\" is a booklet called Recent Historical Trends, published by the Philadelphia City Planning Commission. How \"recent\" are the trends? The report was published in 1966 and long ago fell apart (it's held together with a binder clip). But that report is packed with detail on population and race in parts of the city going back to 1930.\n\nShould you be as obsessed? If you're so inclined, your newspaper could benefit greatly from historical local census data online - available well in advance of the flood of 2000 data.\n\nElectronic data archives should be easier to build than paper ones. But they aren't. I discovered this recently when I began to ask around about the availability of 1970 and 1980 census data in computer form. A typical answer from one agency in Pennsylvania that hands out such data was \"We don't have anything that old.\"\n\nIf you build a historical census archive for your area, you may have the only such archive around. You will be building something useful to reporters and editors that will be around long after you're gone.\n\nHowever, you may find yourself behind closed doors - as I have been - reconciling an ArcView census tract map on your computer screen with an old paper map from 1950. Yet, imagine how powerful a graphic we'll have to accompany a story on population change in a given Philadelphia neighborhood: 50 years of change, from 1950-2000.\n\nYou may not want to get so nerdy that you include every census tract from 1960 in your archive. But you may want to have the populations of minor civil divisions (townships, cities, boroughs) or counties going back 50 or more years.\n\nA few ideas to get started in your quest to pin the wonk-meter's needle:\n\nVisit your local library. Tucked away on shelves you'll often find dusty printed volumes of census results going back decades. Photocopy the pages and have the data input to build a database.\n\nStop by the \"United States Historical Census Data Browser\" at http://fisher.lib.virginia.edu/census/. You can pull county-level data going back to the first decennial census in 1790.\n\nPurchase the 1980 and 1990 data in bulk from the Census Bureau. Go to www.census.gov/mp/www/rom/ to check out pricing and availability. For example, the 1980 STF-3A file for the nation costs $1,500. However, the cost plunges to $150 if you're after only one state (the CDs contain several states each). The 1990 STF-3A file for the nation costs $4,340. Some single states fit on one CD at $100 each. Larger states, such as Pennsylvania, fill several CDs. Pennsylvania, for example, is a three CD set costing $225. Texas is a four-CD set costing $300. The PL 94-171 file for 1990 costs $550 for the nation (10 CDs) or $100 for a single CD.\n\n\"Your newspaper could benefit greatly from historical local census data online - available well in advance of the flood of 2000 data.\"\n\nBuy the 1970 block-level data from the National Archives at www.nara.gov/nara/electronic/cen1970.html. The price ranges from $90 for a state to $1,315 for the nation.\n\nPurchase 1940 through 1970 tract-level data from the National Archives at www.nara.gov/nara/electronic/bogue.html. This dataset, called the \"Bogue File,\" was created by keypunching paper data as part of research at the University of Chicago. Not all areas of the country are represented, although the largest cities are in the data. Each year is $80 on nine-track tape.\n\nBuy the CensusCD from GeoLytics Inc. (www.geolytics.com/). National data for a full year is available on one CD. This private firm's slogan is \"Making Demographics Easy to Use!\" and using the CensusCDs is far easier than rummaging through the Census Bureau's data disks. However, queries using the CensusCD set were designed to go through the front end.\n\nYou can extract bulk data, but it can be cumbersome. If you can't afford mapping programs, the CensusCD+Maps product does a solid job of making thematic maps (the maps can be exported to ArcView Shape and MapInfo MIF files).\n\nCosts: CensusCD+Maps (to block group geography), $500; CensusCD Blocks, $1,000 for nation; $500 for a single state; CensusCD 1980, $800 each; $400 for a single state. The company also has announced a CensusCD1970, although the introduction date and pricing weren't available at press time.\n\nOther useful sites for census data include Center for International Earth Science Network (CIESIN) at www.ciesin.org/ and Office of Social and Economic Data Analysis at the University of Missouri (OSEDA) at www.oseda.missouri.edu/. Also check the site for your state data center (www.census.gov/sdc/www/) or state, regional and local planning agencies.\n\nNeill Borowski can be reached by e-mail at inborowski@phillynews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-felons-and-nurses","slug":"2001-januaryfebruary-felons-and-nurses","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-felons-and-nurses.html","year":2001,"month":"January/February","monthNum":1,"headline":"Felons and nurses","kicker":"FIRST VENTURE","author_name":"Randy Travis","author_title":"","affiliation":"WAGA-TV, Atlanta","summary":"A Georgia news station used computer-assisted reporting to investigate certified nurse aides with criminal records working in nursing homes, discovering that state law allows hiring convicted felons despite background check requirements. The investigation required matching databases from the Department of Human Services and Department of Corrections, cleaning messy data, and eliminating false matches to identify 5,595 CNAs with criminal histories.","keywords":["investigation","nursing","database"],"topics":["Crime & Justice","Investigative Methods","Health & Environment"],"technologies":["Access","Nexis"],"full_text":"We started with separate open records requests to the state Department of Human Resources (DHR) and the Department of Corrections (DOC). DHR maintains a database of all certified nurse aides, both past and current. DOC keeps track of all criminal convictions, both felony and misdemeanor. We got the complete DOC database without any trouble, and no charge. Our first roadblock came in getting what we needed from DHR.\n\nWhat intrigued us here in Georgia is a unique twist in our law. The state certifies nursing home workers, called certified nurse aides (CNA). Then, Georgia nursing homes must do a criminal background check before they are hired. But the law doesn't preclude the nursing home administrator from hiring a convicted felon. With staffing shortages so critical, we suspected CNAs with criminal records were still being hired. Turns out, we were right.\n\nWe've worked long-term investigations here before, but nothing compares to my station's first venture into computer-assisted reporting. As it turned out, the story was well worth the wait. Even if it took nearly a year.\n\nOur investigation began with an issue some reporters in other parts of the country have already tackled: people with criminal records working in nursing homes.\n\nWe felt Social Security numbers provided the most accurate way to match up names between the two databases. No problem, DOC told us. They gave us everything. But DHR balked at the idea of providing us what they considered private information. Our state's open records law does not directly address this issue. It took four months of phone calls to the attorney general's office and DHR before we finally convinced them the information was subject to the open records law. The disk cost us only $10. But by then, our plans for a May sweeps series were gone.\n\nDirty data\n\nIt didn't take long, though, for us to realize that the hard work hadn't even begun. The data from both sources were dirty. In one table, the Social Security numbers were separated by dashes. In the other, the numbers weren't separated at all. First and last names were in separate fields in one but combined in the same field in the other. That's why we turned to NICAR. They were able to clean up the data and do the match between databases. Of the 73,410 nursing aides, NICAR found 5,595 had a criminal record. But there was more work to do. The CNA database included people who hadn't been certified in years. We didn't want them included in this investigation. Secondly, some of the CNAs committed their first crime after they were certified. Can't blame a nursing home for that. Yet they still showed up as a match. So, we began eliminating matches. Using Microsoft Access, we first kicked out all CNAs whose certification expired before 2000. That gave us a list of current criminal CNAs. Then we took out all CNAs who were convicted after their certification date. That eliminated all those CNAs who had clean records when they were hired, but committed a crime later on. We wound up with 219 felons and 2,942 probationers currently allowed to work in Georgia nursing homes. Now it was time to find them.\n\nWe felt Social Security numbers provided the most accurate way to match up names between the two databases. No problem, DOC told us.\n\nGeorgia does not keep track of where certified nurse aides actually work. So it was impossible for us to say how many of the criminal caregivers actually had access to the elderly. The DHR database did include the CNA's address at the time of certification. So, we started there. Thanks to the Web site healthgrades.com, we used those addresses to get a list of every nursing home nearby. Then we started calling asking to speak to someone to verify employment. When we got a hit, we confirmed it by Social Security number. We focused on the small communities that didn't have as many nursing home choices. We found more than a dozen criminal caregivers that way.\n\nCrimes at nursing homes\n\nThe other part of our investigation looked at whether criminal caregivers are actually more likely to commit crimes once they get inside the nursing home. The state Department of Human Resources also maintains an abuse registry. When a CNA harms a resident, they wind up on the registry. We merged the registry with the Department of Corrections table and found 82 criminals. Many of them committed the same crime inside the nursing home that they had on their record when they were hired: burglary, assault, theft, etc.\n\nIt took us even longer to track down those victims or their families. By law, they're not listed in the abuse registry. That meant contacting local police departments and asking for incident reports of those crimes. But often, the nursing home didn't bother to call the police. Months later, we managed to find family members to react to our investigation. They had no idea the nurse aide who had harmed their elderly relative was a criminal before he was ever hired.\n\nOur series finally aired November 8 and 9, 2000, nearly a year after we made our first request for the data. The investigation is expected to prompt a change in the law when the state general assembly meets in January. Unfortunately, in an ironic twist, state lawmakers may also vote to make Social Security numbers exempt from the open records law. If you're planning a similar investigation, fight for that key data. Without it, our investigation would certainly have taken even longer.\n\nRandy Travis can be reached by e-mail at rtravis@fox5atlanta.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-enterprise-joins","slug":"2001-januaryfebruary-enterprise-joins","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-enterprise-joins.html","year":2001,"month":"January/February","monthNum":1,"headline":"Enterprise joins","kicker":"TECH TIP","author_name":"Ron Nixon","author_title":"NICAR instructor","affiliation":"NICAR","summary":"Enterprise joins allow journalists to match databases from different sources to uncover investigative stories, but reporters must use careful methodology to avoid false matches, including standardizing data and verifying results through additional reporting.","keywords":["databases","verification","matching"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["FoxPro","Access","Paradox"],"full_text":"Enterprise joins - matching databases that weren't designed to be matched - is one of the most powerful features of relational database managers like FoxPro, Access and Paradox. Using enterprise joins, reporters across the country have successfully matched databases of school bus drives against people with driving convictions, found dead people voting in elections, felons with hunting license, and deadbeat parents in state government.\n\nWhen seeking databases, reporters should always think about ways they could join databases from different agencies or across different fields of study to get stories. Unfortunately, databases rarely have identifying codes like Social Security numbers or some form of employee id. In cases where you don't have this type of information you have to come up with proxies. Some identifiers could be birth dates or address like the person's street number, city or zip code.\n\nAlthough enterprise joins can produce great stories they can be fraught with danger. A wrong match can give you incorrect information. You could, for example, get a match for a deadbeat parent working at the agency that collects delinquent childcare payments. But since you only matched on the person's name, the join could be wrong.\n\nAlways remember that a name is never a unique identifier. How may Ron Nixon's are there or John Smith's? There are probably thousands. Another thing to remember is that the names have to match in both. Ron Nixon is not the same as Ronnie N. Nixon to the database, even though they might be the same people. You will have to standardize the names to get the matches you want.\n\nFinally, always verify, verify, and verify that the name you have is really the person you think it is. The join is just the start. Once you have that the real reporting begins. This not only ensures that you get the story right, it also protects you and your paper from a potential lawsuit.\n\nRon Nixon can be reached by e-mail at ron@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-stories","slug":"2001-januaryfebruary-stories","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-stories.html","year":2001,"month":"January/February","monthNum":1,"headline":"Stories","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"An AP investigation using computer-assisted reporting reveals that despite the American Indian gambling industry's explosive growth from $100 million to $8.26 billion between 1988-1998, poverty and unemployment rates in tribal communities remained largely unchanged. A Rhode Island Journal-Bulletin investigation into state lottery corruption prompted multiple investigations and the dismissal of the lottery director.","keywords":["gambling","investigation","corruption"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":[],"full_text":"For more on gambling, call the IRE Resource Center, (573) 882-3364.\n\nStory # 17010. In this article, the Associated Press uses computer-assisted reporting to find \"the American Indian gambling industry has boomed, with annual revenues increasing from $100 million in 1988 to $8.26 billion in 1998. Poverty and unemployment rates changed little during that period.\"\n\nStory # 13853. The Journal-Bulletin (Providence, R.I.) investigation into the state lottery leads to the dismissal of director of the lottery, investigations by the attorney general, state auditor and state Ethics Commission and the cancellation of a contract that would have put the state gambling agency in the television bingo business.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-valuable-accident-data","slug":"2001-januaryfebruary-valuable-accident-data","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-valuable-accident-data.html","year":2001,"month":"January/February","monthNum":1,"headline":"Valuable accident data","kicker":"TRANSPORTATION","author_name":"Hurst Laviana","author_title":"","affiliation":"The Wichita Eagle","summary":"A Kansas newspaper obtained a traffic accident database revealing that some drivers had numerous accidents and convictions despite lax enforcement of license suspension laws, prompting investigation into why dangerous drivers remained on the road.","keywords":["accidents","drivers","database"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["CD-ROM","Access","Excel"],"full_text":"Wayne Louden has been involved in 37 traffic accidents since Jan. 1, 1990, making him the most accident-prone driver in Kansas. Thomas Spicer has been involved in 27 wrecks, 21 of which resulted in inattentive driving tickets.\n\nLouden and Spicer were key players in a series of stories about accident-prone drivers in Kansas. The series was based on a Kansas Department of Transportation 1990-1999 database that covered 703,000 traffic accidents involving 1.1 million drivers.\n\nAcquiring the database was a struggle, but it has proved to be one of the most valuable data sets in our newsroom. It has yielded stories about the growing number of car-deer accidents in the state, the reluctance of drivers in the nation's \"beltless belt\" to use seatbelts and the fact that accidents involving police chases doubled in Kansas during the 1990s. The data also allows us to quickly generate lists of accidents that have occurred at any intersection in the state.\n\nAcquiring the database was a struggle, but it has proved to be one of the most valuable data sets in our newsroom.\n\nBut the biggest reader reaction came from the story about accident-prone drivers. Many were surprised to learn they were sharing the roads with 20 Kansas drivers who had 10 or more accidents in the 1990s. And some of those drivers had dozens of traffic convictions.\n\nHow did they keep driving?\n\nIt turned out that the Kansas Division of Motor Vehicles hasn't been enforcing a law that allows it to suspend the license of any driver convicted of three moving violations in a 12-month period. Instead, the DMV only takes action after the fifth conviction.\n\nIt also turned out that Spicer's inattentive driving tickets - as well as 70,000 other inattentive driving citations issued each year by Wichita police - don't even count against a driver's record. Inattentive driving is a ticketable offense in many Kansas cities, but state law does not prohibit it. So the state doesn't count it.\n\nDoing the stories\n\nThe seeds for the story were planted more than a year ago when Crime & Safety team leader Jim Lewers asked KDOT for five years of accident data. KDOT mailed him copies of the 1990 through 1996 annual traffic accident reports.\n\nLewers tried again, this time asking for a comprehensive electronic database with detailed information about each accident. KDOT's response: for $51,710 we could have the minus the drivers' names and accident locations. KDOT, which wanted 5 cents a line for just more than 1 million lines of data, said giving us the names and accident locations could result in the information being used against it in court.\n\nKDOT held its ground until February, when a judge ruled that the agency was improperly withholding railroad crossing accident data from another Kansas newspaper, the Garden City Telegram. KDOT eventually gave us the accident data - names and all - for $87.11.\n\nCrunching numbers\n\nThe file arrived in the newsroom on a CD-ROM in the form of a 550-megabyte Microsoft Access database, which included separate tables for such variables as drivers, accidents and vehicles. My three-gigabyte hard drive was no match for that much data, but we did turn around some quick stories using pieces of the data.\n\nOn March 22, the day the Kansas House of Representatives approved a bill designed to cut car-deer accidents by increasing the number of out-of-state hunting permits, we sliced the deer accidents from the accidents table and loaded them into an Excel spreadsheet.\n\nExcel's Chart Wizard produced an interesting graphic that showed a steady rise in the number of car-deer accidents. It also revealed a sizable blip every November, when the males were out seeking mates, and a smaller blip every May, when females were out looking for places to give birth.\n\nOur lead: \"Cars, trucks and buses crashed into more than 10,000 deer in 1999, a record for the state.\"\n\nOnce a bigger hard drive was installed in my computer, we set out to find the accident-prone drivers. We needed the extra room in order to link the tables so we could track such variables as injuries, fatalities and accident causes.\n\nA single \"group-by\" query on the driver's license field generated a list of hundreds of Kansans who had been involved in three or more accidents during the decade. The process was complicated by the fact that Kansas changed its driver's license numbering system in the mid-1990s, so most drivers had two driver's license numbers.\n\nOther \"group-by\" queries on the firstname, lastname and DOB fields turned up another batch of candidates. We ended up working with a list of 1,960 drivers who had five or more accidents in the 1990s. Their 11,165 accidents left 50 people dead (including 14 of the accident-prone drivers) and 5,171 injured.\n\nWe eventually decided to focus on the six drivers who had 12 or more serious accidents during the decade.\n\nDoing the legwork\n\nThe next step was to ask KDOT for hard copies of the accident reports for the six drivers. We also asked local law enforcement agencies for copies of reports for non-injury accidents involving less than $500 in damage - accidents that aren't reported to the state. The written reports, which often had narratives and diagrams, included more detailed information than the KDOT database. Based on this information, reporters interviewed drivers.\n\nThe series ran in late September, and a week later we put the list of drivers with five or more accidents on our Web site with a disclaimer noting that the listed drivers were not necessarily at fault in any of the accidents.\n\nThe only negative reaction from readers came on the following Wednesday, when a woman who works at City Hall called to say our on-line information was wrong - that she hadn't been involved in six accidents. When given the dates and locations she retreated, then insisted that none of the accidents was her fault.\n\nHurst Laviana can be reached by e-mail at hlaviana@wichitaeagle.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-the-math-on-gambling","slug":"2001-januaryfebruary-the-math-on-gambling","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-the-math-on-gambling.html","year":2001,"month":"January/February","monthNum":1,"headline":"The math on gambling","kicker":"GAMBLING PART I","author_name":"Barnet D. Wolf","author_title":"","affiliation":"The Columbus Dispatch","summary":"The Columbus Dispatch investigated Ohio's gambling industry by analyzing lottery sales data mapped by ZIP code and estimating out-of-state casino gambling through multiple data sources, revealing that border regions experienced steeper revenue declines than central Ohio.","keywords":["gambling","lottery","data"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Access","ArcView"],"full_text":"Sometimes the germ of a project begins with something very common.\n\n\"Ohio Hedges Its Bets,\" a three-day series on the gambling industry published Oct. 27-29 in The Columbus Dispatch really started when a Business News copy editor began pooling money from staff members to drive two hours to Indiana and buy Powerball tickets.\n\nOne writer pointed out that ever since the multistate lottery games began in states adjacent to Ohio, the staff had rarely pooled its money for the Ohio Lottery.\n\nWhat was happening in Columbus, we thought, probably was happening all across the state, not only with the lottery but also with other types of gambling.\n\nJust how big is the legal gambling industry in Ohio and the nation, we wondered, and what impact have casinos, multistate lotteries and other forms of betting in neighboring states had on Ohio?\n\nSetting out to find data on the subject, we found that some statistics would be easy to uncover, because they were collected by the states. A large amount of data also was compiled by the casino industry.\n\nState lottery\n\nFor instance, some computer-assisted research was used to plot the impact of new border-state gambling options to the decline in Ohio Lottery (www.ohiolottery.com) ticket sales.\n\nBy using the state's own lottery sales numbers for several years, and then breaking them into ZIP codes, the figures could be plotted on a map.\n\nThe Ohio Lottery Commission provided an electronic database of sales for the past several years. That database was pretty thorough, with a breakdown of sales by each vendor (gas station, convenience store, etc.). Each vendor listing had a street address as well as a city and ZIP code for the business where tickets are sold.\n\nThat made it fairly easy to analyze with Microsoft Access. We basically grouped and totaled the sales figures by ZIP code, then moved that into our ArcView mapping software. Once we had a list of sales by ZIP code, it wasn't tough to map that across Ohio. The results were compelling: They showed that the sales decline was much more severe along the state's borders than in the middle of Ohio.\n\nThe state also keeps statistics for bingo games and horse racing, two other large gambling sectors.\n\nOut-of-state casinos\n\nBut other conclusions were much more difficult to determine, particularly trying to find out how much money Ohioans gambled at out-of-state casinos.\n\nThe result was a time-consuming exercise that required searching for financial and marketing data from public and private local, state and national sources, and then applying old-fashioned math to the numbers.\n\nThe findings could not be precise because of the limitations of the data available, but the approach produced a reliable estimate.\n\nFortunately, The Dispatch had the assistance of the Buckeye State Poll, conducted by Ohio State University in conjunction with the newspaper, to help serve as a secondary source to back up some of the numbers.\n\nThe questions posed by the poll, conducted in April, looked at the gambling opinions and habits of Ohioans and where - if anywhere - they gambled.\n\nGambling revenues\n\nStates maintain decent statistics when it comes to revenues from gambling after all, tax money is derived from it - so that's always a good place to start. Most states also provide the most basic of their lottery data online.\n\nIn a state like Indiana, which has riverboat gambling, a gaming commission maintains statistics about slot machines and gaming tables. West Virginia's devices - video slots only - are overseen by the Lottery Commission (www.state.wv.us/lottery).\n\nSometimes the casino information is easily accessible. For instance, Mississippi's gaming commission (www.msgaming.com) has data on its Web site that gives the percentage of casino business from residents of other states.\n\nFor the most part, however, states maintain data only on the amount of money wagered and lost by individual casinos. Once you obtain those results, it takes a little legwork to get numbers for a specific state. Indiana has riverboat casinos, and its gaming commission (www.state.in.us/gaming/) reports monthly on the amounts of money gambled at and won by each casino for slot machines and table games. The commission's annual report - also online - includes the total numbers for the previous year at each casino. For all intents and purposes, the amount won by the casino is the amount that is lost by its customers.\n\nThe legwork comes in when you need to find out how much of that money came from Ohioans. According to several statisticians and business experts, the easiest method for determining those amounts is to get market-share data.\n\nMost of the border-state casinos were willing to divulge the percentage of their customers who visit from Ohio. When the market share was not provided by a casino, it was determined from other sources, ranging from estimates by state officials to a study that one university conducted by counting license plates in casino parking lots.\n\nLas Vegas\n\nCalculating the Ohio gambling numbers from Las Vegas was a bit more complex.\n\nThe Las Vegas Convention and Visitors Bureau (www.lasvegas24hours.com) is a good source of information on the amount of money won by casinos in the city, as well as airplane traffic and visitor profiles. According to the bureau, 87 percent of people who visit Las Vegas gamble. Those who gamble spend an average of $559 per visit. So, how many of those people are from Ohio? As with other airports, McCarran International Airport (www.mccarran.com) maintains statistics regarding the true origin and destination sites for commercial flight passengers traveling to and from Las Vegas. Using these numbers, it's possible to get a total number of Ohioans coming to Las Vegas via commercial flights (since the number will be for arrivals and departures, it must be divided by two to get only arrivals).\n\nBy taking the Ohio number against all commercial passengers, it's possible to get a percentage number of Ohio passengers that then can be applied to the charter flight statistics, in which origin destinations are not calculated.\n\nAdd up all those passengers and then multiply it by .87 (remember, 87 percent of all visitors gamble). Then multiply that number by $559 (the average lost per visit) and the total is the amount that Ohioans lost gambling.\n\nSince the amount of money lost is about 8 percent of all money bet at the casinos, the total that Ohioans bet can be ascertained by multiplying the amount lost by a multiple in the neighborhood of 12.5 percent.\n\nThis calculated number of Ohioans gambling in Las Vegas meshed with the numbers determined in the Buckeye State Poll. Thus the amount of people gambling at casinos in other locations can be determined by using the poll.\n\nAdding it all up, The Dispatch was able to estimate conservatively that Ohioans bet $11.7 billion and lost $1.8 billion on legal gambling in 1999. And a large chunk of that all that money gambled outside the state - resulted in tax revenues for states other than Ohio.\n\nOur approach in tackling the complex topic of gambling can be applied to other subjects in which precise data does not exist: With a little creative thinking, reasonable estimates based on a solid foundation can be made.\n\nBarnet D. Wolf can be reached by e-mail at bwolf@dispatch.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-stories-2","slug":"2001-januaryfebruary-stories-2","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-stories-2.html","year":2001,"month":"January/February","monthNum":1,"headline":"Stories","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article examines the growth of the American Indian gambling industry and investigates state lottery misconduct, revealing that despite gambling revenues increasing dramatically, poverty and unemployment rates remained stagnant.","keywords":["gambling","investigation","industry"],"topics":["Investigative Methods","Government & Politics","Demographics & Census"],"technologies":[],"full_text":"For more on gambling, call the IRE Resource Center, (573) 882-3364.\n\nStory # 17010. In this article, the Associated Press uses computer-assisted reporting to find \"the American Indian gambling industry has boomed, with annual revenues increasing from $100 million in 1988 to $8.26 billion in 1998. Poverty and unemployment rates changed little during that period.\"\n\nStory # 13853. The Journal-Bulletin (Providence, R.I.) investigation into the state lottery leads to the dismissal of director of the lottery, investigations by the attorney general, state auditor and state Ethics Commission and the cancellation of a contract that would have put the state gambling agency in the television bingo business.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-tribes-and-casinos","slug":"2001-januaryfebruary-tribes-and-casinos","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-tribes-and-casinos.html","year":2001,"month":"January/February","monthNum":1,"headline":"Tribes and casinos","kicker":"GAMBLING PART II","author_name":"David Pace","author_title":"","affiliation":"The Associated Press","summary":"The Associated Press conducted a comprehensive study comparing quality of life metrics on Indian reservations with and without casinos during the 1990s, finding that welfare participation grew less in gaming communities despite historically persistent unemployment and poverty problems.","keywords":["gambling","reservations","welfare"],"topics":["Data & Databases","Government & Politics","Demographics & Census"],"technologies":["county business pattern records"],"full_text":"A steady drumbeat of news stories throughout the 1990s documented the rapid growth of casino gambling on American Indian reservations. But lingering questions remained.\n\nHow had the billions of dollars in gambling revenues changed the lives of Indians on reservations, historically among the nation's poorest areas? Was gambling solving the unemployment, poverty and welfare problems that have plagued reservations? The Associated Press set out last spring to answer those questions by gathering data on a broad range of quality of life variables during the 1990s and then using it to compare reservations that have casinos with those that don't.\n\nThe immediate problem in designing such a research project was the scarcity of data collected at the reservation level. The 1990 Census provides a detailed snapshot of reservation life at the beginning of the casino boom, from education levels to poverty rates to housing conditions.\n\nBut comparative information from the 2000 Census won't be available for several more years, and the Census Bureau collects no reservation-level data in the years between the decennial census. We found only two other government sources of reservation-level data: The biennial workforce report compiled by the Bureau of Indian Affairs from unemployment data provided by tribes, and the Agriculture Department's Food Distribution Program on Indian Reservation.\n\nImprovising\n\nWe used the BIA workforce reports for 1991 and 1997, and the Agriculture Department's FDPIR data for 1990 and 1998 as two comparative variables. We also obtained from the Department of Health and Human Services a breakdown by race of welfare caseloads in each state. That enabled us to compare American Indian welfare participation during the 1990s in states that permit casino gambling with those that don't.\n\nFinally, we asked the Census Bureau's geography division to create a database of all the counties that are part of Indian reservations. That enabled us to use the bureau's 1989 and 1995 county income and poverty estimates, and its 1990 and 1997 county business pattern records, for comparative analysis.\n\nThe next major hurdle was dividing tribes into gaming and non-gaming categories. To operate a Las Vegas-style casino, the National Indian Gaming Regulatory Act requires that a tribe first negotiate a compact with the state. From the National Indian Gaming Association, we obtained a current list of Indian gaming operations and used it to divide tribes into gaming and non-gaming categories.\n\nWe divided them based on the legal definition, designating as gaming tribes those that operate Class III casinos under compacts with states. Tribes with small-stakes bingo games were combined, with those that have no gambling operations, into the non-gaming category.\n\nThe analysis found that welfare participation on Indian reservations with casinos grew far less during the 1990s than on other reservations. But the historically high unemployment and poverty levels on reservations changed very little during the 1990s, despite the influx of gambling money.\n\nThe analysis found that welfare participation on Indian reservations with casinos grew far less during the 1990s than on other reservations.\n\nBetween 1991 and 1997, when the U.S. unemployment rate dropped from 6.9 percent to 4.9 percent, the unemployment rate for 146 tribes with casinos declined from 55.9 percent to 52.2 percent, and it actually increased slightly among the 55 tribes with casinos that opened before 1992. Among the 144 tribes without casinos, the unemployment rate increased from 43.6 percent in 1991 to 48.3 percent six years later. Similarly, the average poverty rate in counties where there are gaming tribes declined only gaming tribes, the poverty rate increased slightly, from 18.2 percent to 18.4 percent. In the U.S. as a whole, the poverty rate increased from 12.8 percent to 13.8 percent during that period.\n\nTo understand why gambling revenues weren't having more of an impact, we tried to determine where the money was going. We filed a Freedom of Information Act request with the National Indian Gaming Commission - the agency with regulatory oversight of the Indian gambling industry - asking for a revenue breakdown of the casinos.\n\nThe commission refused to release financial data on individual casinos, saying it was proprietary information. But it did classify the casinos into six different revenue categories, from those making more than $100 million a year to those making less than $3 million a year, and provided a cumulative revenue total for each category.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-maps-to-go","slug":"2001-januaryfebruary-maps-to-go","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-maps-to-go.html","year":2001,"month":"January/February","monthNum":1,"headline":"Maps to go","kicker":"MAPPING","author_name":"John Maines","author_title":"","affiliation":"South Florida Sun-Sentinel","summary":"A journalist describes building an inexpensive GIS mapping system for under $300 using free public domain data and off-the-shelf software, which was successfully deployed to analyze the 2000 Florida presidential election and voter precinct data.","keywords":["mapping","elections","software"],"topics":["Technology","Data & Databases","Journalism & Newsroom"],"technologies":["ArcView","MapInfo","Maptitude","GIS","PC","Internet","Intranet","World Wide Web"],"full_text":"There are two truths about buying good computerized mapping software:\n\n1. Single-user versions are expensive.\n2. Internet or Intranet versions are very expensive.\n\nThree years ago, I decided to see if there was a way around the high cost of commercial mapping programs. The idea seemed to make sense: Build an inexpensive, simple mapping system and pack it with census stats and anything else we could get our hands on, especially free public domain data from federal, state and local government agencies. We've done it. So can you. For less than $300, it's possible to build a fairly powerful program that can be installed on a PC, a laptop, or launched on the Internet or company Intranet. At the South Florida Sun-Sentinel we have test versions running now. We should have the bugs in time for the March release of Census 2000 data on race and Hispanic origin. To do this, we combined our own software know-how with some off-the-shelf mapping software designed for people who want to build their own Geographic Information Systems (GIS) software. We created a program that can be tailored any way we want it and launched on the Internet or re-deployed as a stand-alone application for installation a PC or laptop.\n\nThe acid test\n\nThe 2000 presidential election provided a great acid test.\n\nOne of the map prototypes was a precinct map of South Florida, which displays a color-coded map and spreadsheet-like data grids. When a user clicks on a precinct, the map pulls up demographics and voter registration data for election precincts in the Sun-Sentinel's circulation area: Broward, Miami-Dade and Palm Beach counties.\n\nThe precinct borders were drawn from files that we downloaded - for free - from a Web page run by a state research agency.\n\nSo it was pure luck that, when the election went to hell, we had a fully-functional computerized precinct map armed and ready. We changed the map constantly in the days following the election, each time reloading an updated version into our company Intranet, Sunspot, for use by reporters and editors.\n\nThe first maps we created were simple showing who won each precinct. Red for George W. Bush, blue for Al Gore. When reports came in that voters in Palm Beach County had accidentally voted for Pat Buchanan instead of Al Gore because they misunderstood the \"butterfly ballot,\" we mapped out where Buchanan got the most votes. When we found out that most of the disqualified votes were in black precincts, we mapped that out as well, adding new map layers on top of the old. Our art department polished up our graphics for use in the newspaper. Len De Groot, a senior editorial artist at the Sun-Sentinel, made a terrific map that showed undervote rates in all 1,754 election precincts in the three South Florida Counties. It ran the entire length of a newspaper page, and covered about one-third the width. (See for yourself at www.electricalfire.com/sprecincts.htm).\n\nPros and Cons\n\nI'm not saying that newsrooms should toss the programs that they've already bought and build a Frankenstein. Arcview, MapInfo, Maptitude or some other powerful program will still be the best way to analyze Census 2000. But a simple custom-made system provides a terrific secondary mapping system for the newsroom, because it does something that the big bruisers don't do very well: Create dynamic maps on the Intranet that can be used by even the most non-technical people.\n\nConsider two popular brands of high-quality Geographic Information System (GIS) software. Arcview 3.2a ($1,195) and MapInfo Professional ($1,495) are built for use by one person per copy of software. That person builds the maps and distributes them to others. Both companies have viewers that can be used on any PC and are available for free. But the viewers can't do much more than zoom in or out of a map and identify what's on it.\n\nThe viewers also require some effort by the end user. They need to read the instructions. That's minimal, but still enough to send many people running. A good general-use mapping program needs to be \"Web page simple,\" allowing the user to learn by pushing buttons and generally messing around.\n\nInternet mapping software can be incredibly expensive. One of the best deals on Internet-based mapping software is $2,995 for Caliper's Maptitude for the Web license, MapInfo's MapX product line starts at about $5,000 and ESRI's ArcIMS comes in at $7,500. Yikes.\n\nThose costs are probably far beyond what most newspapers are willing to spend. So very low cost was one of five requirements in developing our software. Here are the other four:\n\nIt had to be extremely easy to use. The maps would come with no little or no instructions, allowing users to wing it.\nIt had to read many kinds of commercial software, such as Arcview shapefiles or MapInfo's MIF files. It also had to be able to read data from programs used in most newsrooms, such as an Excel spreadsheet or an Access database.\nIt had to be available in a desktop version, or on the Internet.\nIt had to be of \"rapid-deployment\" nature, producing maps for deadline stories.\n\nCreating a hybrid\n\nEarly on, it became clear that the system would be some sort of hybrid - parts of the software would be designed by us, and part would be purchased off-the-shelf. I thought it would be best for me to build the control panel that surrounded the map; the components that would allow reporters to push buttons and insert all the parameters for data analysis.\n\nBut I really didn't want to build the part of the program that would actually draw the map's lines, points and polygons. For that I needed some sort of \"engine.\"\n\nSeveral companies make such software. There's ESRI's MapObjects, MapInfo's MapX. But these cost thousands of dollars to buy and maintain and are really built for someone who builds high-quality maps for a living. Newsrooms don't need that degree of complexity. One product that looked very good - at first - was ESRI's MapObjects LT, a trimmed-down version of MapObjects. Three years ago, the software only cost $295 and produced some great-looking maps. But it lacked one very important feature. There was no easy way to put the names on roads in your map. It was a critical flaw in an otherwise good, inexpensive product. (The newest version of MapObjects LT has nice labeling capabilities. But it now costs $1,000).\n\nAfter abandoning MapObjects LT, we tried about a half-dozen inexpensive GIS programs found on the Web. They all bombed for one reason or another. Some were too slow. Others couldn't handle geocoding. Some were just plain junk. Finally, in a late-night search of the Internet last February, I discovered a great little program called SylvanMaps.\n\nA solution at last\n\nSylvanMaps is sold by Sylvan Accent (www.sylvanmaps.com), a small company based in Taos, New Mexico. If you call Sylvan, there's a pretty good chance that the guy answering the telephone is Roger Bedell, the company president. The software is easy to use and comes with clearly-written user manuals. There are three versions. The first format, SylvanMap: Net, is a $249 design-time control for Active Server Pages (ASP), the popular Web page program. Users build the map in Microsoft Frontpage or Visual Interdev and launch the finished product on the Internet. For another $496 you can buy the second version, which has more features than the first.\n\nThe third version, SylvanMaps OCX, creates maps in Visual Basic, Microsoft Access, C++ or Delphi, and you can install the program on the Internet or build a desktop program. Version three is the most expensive, $795. But that price allows you to build unlimited applications, which can be resold or distributed to co-workers and sister companies without fooling around with annoying royalties or redistribution fees. All three versions are easy to use.\n\nA beginner can put together a simple map, and work upward from there. The better your programming skills, the more sophisticated the maps you'll make. A very nice feature is that an entire map - including the roads, census tracts, and other features - is built from data stored in any standard relational database, including Microsoft Access, SQL Server, or Oracle. There's no \"hidden\" proprietary software.\n\nSylvanMaps can also read or translate Arcview shapefiles, MapInfo MIF files, or the Census Bureau's TIGER line files. If translated, the map data can easily be installed in a database on your company's web server.\n\nSuch versatility is great, because it means you can use maps already created by other people - the Census Bureau, the state department of transportation, your local planning department, or whatever. The street maps we use at the Sun-Sentinel are Broward County's 911 maps, used for dispatching fire trucks and ambulances to emergencies. The maps are constantly updated, and much more up-to-date than anything that you can buy. And they're free. Mapping software is changing rapidly. Chances are that everything on the market now will be considered junk in a few years. We're going to continue to refine SylvanMaps for now, but at the same time testing other software that comes along as we search for the best product around.\n\nMeantime, the data that goes into the maps keeps getting better and cheaper. For $20, we recently bought a CD-Rom from the University of Florida with all kinds of interesting public domain maps on it. These include hiking trails, bike routes, the paths of Florida hurricanes over the last 75 years and the locations of dozens of shipwrecks in the Atlantic Ocean and the Gulf of Mexico.\n\nWe plan to put those on our Intranet. Shipwrecks aren't as important as elections or Census 2000. But they're fun. And hey, that's Florida.\n\nJohn Maines can be reached by e-mail at jmaines@sun-sentinel.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-parking-tickets","slug":"2001-januaryfebruary-parking-tickets","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-parking-tickets.html","year":2001,"month":"January/February","monthNum":1,"headline":"Parking tickets","kicker":"FIRST VENTURE","author_name":"Jason Callicoat","author_title":"","affiliation":"South Bend Tribune","summary":"A journalist describes using data journalism techniques learned at NICAR Boot Camp to investigate whether bad weather correlates with increased parking tickets by combining weather data from the National Weather Service with parking ticket records from a city's Ordinance Violations Department.","keywords":["data","weather","tickets"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Access","dBase","CD-ROM"],"full_text":"One of the first stories I wanted to do after returning from a NICAR Boot Camp last spring involved linking a database of local parking tickets with a database of weather and climate information. I wanted to see whether there was a correlation between bad-weather days and the number of parking tickets handed out.\n\nThis was a story idea that was mentioned as an example during one of the boot camp presentations.\n\nI decided to look at data from Jan. 1, 1998, to as close to the present as I could get.\n\nWeather data\n\nThe weather database was fairly easy to get. The National Weather Service already had a table of each month's weather data saved as a text file. They maintained it this way so they could print it each month in a weather newsletter.\n\nThe weather service had about 18 months' worth of weather data saved on a disk, which they let me borrow - for free - if I promised to mail it back to them.\n\nThere was some prohibition on them giving away data (they were supposed to charge for it), but the person I was dealing with was very accommodating. I couldn't copy the disk, or have the disk, but I could \"borrow\" it if I mailed him back the original.\n\nFor the months before the weather service began storing the data electronically, he gave me the newsletters in which they published the final data. Again, I couldn't have copies of these, but I could take them all back to the office with me if I mailed him back the originals.\n\nI copied the text files for each month that they had saved on disk, transferring them into an Access database. Then I typed in about 12 months' worth of daily data from the weather service's paper newsletters. I only had about 10 fields, so it didn't take that long.\n\nParking tickets\n\nThe parking tickets database was a little harder to get, but I used some of the tricks I learned at boot camp.\n\nThe clerks in the city's Ordinance Violations Department said their software was 15 years old and not compatible with Microsoft.\n\nThey offered to make me a printout of all the parking tickets from the past two and a half years (roughly 50,000 of them), but it would take several days of print time for their dot-matrix printer to finish with that job.\n\nI asked who was in charge of backing up their data every night, and they gave me the name of the information technology director.\n\nI got in to see him fairly easily, and he looked through the backup files and found one for parking tickets. He had no problem with burning me a CD of the file for free. It was stored in DBase format, which is easily converted into Access.\n\nBut when I got it converted and reorganized the data, I found it was a couple years old and didn't have the tickets that matched up with my weather data. When I went back that afternoon, the IT director found the right backup file and burned another CD. It had about three years' worth of tickets, up through the tickets that were given the previous day.\n\nOn to analysis\n\nI linked the tables by date and found that the 10 percent of the days with the most tickets were generally nice-weather days, and the 10 percent of the days with the fewest tickets were generally bad-weather days. The story ended up taking about six or seven weeks to complete, although I was pulled away from it several times to do daily stories. The CAR portion of it took about two-thirds of that time. The parking tickets database also proved useful, beyond the original story, because it listed license-plate numbers and registered owners of all the cars (tens of thousands of them) that had gotten a parking ticket in South Bend in the past several years. I ended up doing a few searches for another reporter who had a group of license plates he wanted to trace. Also, while working with the parking tickets database, I found quite a few government cars listed. I ran a query to pull them all out, and I found that almost none of the tickets on these government cars had been paid. The cars belonged to a group of armed forces recruiters who had offices down the street from our building.\n\nTogether, they owed more than $4,000 in tickets and late fees. I did some checking, and it turned out that the city had issued a legal opinion that was preventing Ordinance Violations from collecting on these tickets.\n\nThe weather service had about 18 months' worth of weather data saved on a disk, which they let me borrow - for free - if I promised to mail it back to them.\n\nThe opinion said that armed forces recruiters could not be taken to small claims court for unpaid parking tickets the way anyone else would be. That's because it would involve one branch of the government suing a larger branch of the government, which is prohibited by the Supremacy Clause of the U.S. Constitution.\n\nEventually, these recruiters had figured out nothing ever happened if they failed to pay their parking tickets. (They preferred to park on the street outside their offices, even though the government rented parking spaces for them in a garage right across the street. But the spaces were on the 6th floor, and the recruiters didn't like driving all the way up there.)\n\nAfter a couple years of unsuccessful attempts, the city stopped trying to collect the money for the recruiters' tickets.\n\nI wrote a story about it, and the recruiters were all very contrite when I talked to them. The Marines sent a colonel up from Indianapolis to pay $1,000 worth of their tickets a day or so after I interviewed the recruiters. Some of the other recruiters paid smaller amounts of their tickets, as well.\n\nTips for others\n\nI'd say the parking tickets/weather story is a good one to do as a \"first venture.\" The National Weather Service was not at all possessive with its data, and the people I dealt with there actually seemed pleased that someone was taking interest in their work. They went out of their way to make their data available.\n\nIn South Bend, the records of our parking tickets are maintained by the Ordinance Violations Department. It's not a city department we report on very regularly, which seemed to work to my advantage.\n\nThey hadn't been burned and didn't seem suspicious at all, just a little curious. They didn't react badly when I gave somewhat vague answers to their questions of what I wanted the data for. The only hurdles with this department were technical: working around their 15-year-old software and identifying the correct back-up file.\n\nAfter returning from NICAR's Boot Camp, I found these stories were a good way to get my feet wet with CAR. I'd recommend similar stories to anyone as a \"first venture.\"\n\nJason Callicoat can be reached by e-mail at jcallicoat@sbtinfo.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-census-listserv","slug":"2001-januaryfebruary-census-listserv","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-census-listserv.html","year":2001,"month":"January/February","monthNum":1,"headline":"Census listserv","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Instructions for subscribing to Census-L, a listserv focused on census topics and mapping discussions.","keywords":["Census","listserv","subscription"],"topics":["Demographics & Census"],"technologies":["Internet"],"full_text":"The Census-L listserv is a discussion list about census topics, and occasionally mapping. To subscribe, send an e-mail to: majordomo@nicar.org\n\nIn the body of the message type: subscribe census-I youremailaddress\n\nYou will receive an e-mail confirming your subscription, and start receiving e-mails shortly thereafter.\n\nMore information is available at www.ire.org/membership/subscribel census-l.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-restaurant-inspections","slug":"2001-januaryfebruary-restaurant-inspections","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-restaurant-inspections.html","year":2001,"month":"January/February","monthNum":1,"headline":"Restaurant inspections","kicker":"PUBLIC HEALTH","author_name":"Rick Linsk","author_title":"Investigative team member","affiliation":"St. Paul Pioneer Press","summary":"A St. Paul Pioneer Press investigation examined restaurant inspection records across the Twin Cities to determine whether establishments were following health codes and whether consumers had access to safety information, facing significant delays from government agencies reluctant to release data.","keywords":["restaurants","inspections","investigation"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":[],"full_text":"Restaurant inspections\n\nBy Rick Linsk\nSt. Paul Pioneer Press\n\nIt took a strong stomach to successfully pull off our \"Restaurant Reservations\" series (Oct. 23-27, 2000) - not to mention oodles of computer-assisted reporting resources and considerable tenacity.\n\nFirst, a disclosure: We didn't invent this wheel. Restaurant inspections are a longtime staple of investigative reporting. Television stations in the Twin Cities and other cities, sometimes aided by hidden cameras, have reported filthy or unsafe practices. Newspapers in Toronto, San Jose, Charlotte and St. Louis, among others, have also examined restaurants. Our series built on the techniques of our predecessors, and in some ways went even farther.\n\nWe expected to be finished by the end of 1999. But slow-moving government bureaucracies, dirty data, and other factors would confound our timetable.\n\nThe editors and reporters central to the project - Projects Editor Jeff Kummer, Food Editor Kathie Jenkins, restaurant beat reporter Gita Sitaramiah and myself, a member of the investigative team - began in mid-summer 1999 with several fundamental questions. We wondered whether area restaurants were generally following or failing the rules; whether the government's system of overseeing restaurants was working; and whether the average consumer had any way to distinguish between the safe and unsafe, the clean and unclean.\n\nWe expected to be finished by the end of 1999. But slow-moving government bureaucracies, dirty data, and other factors would confound our timetable.\n\nGetting the records\n\nFrom the outset, we faced an obstacle: Fragmentation of oversight. Across the nation, restaurants are inspected by a patchwork of state, county and local government health agencies. In Minnesota, ultimate responsibility rests with the state Department of Health, but the department delegates its powers to dozens of counties and cities. In the end, we studied restaurant inspections by MDH and five local agencies in the Twin Cities that were willing and able to give us computerized records.\n\nSome government agencies resisted our request for inspection data and only relented after official requests, meetings, and follow-up phone calls and e-mails. Even then, some took weeks or months to provide the records.\n\nThe most notorious agency was the state Department of Health, which oversees food establishments in Dakota County and most of outstate Minnesota. MDH failed to provide complete inspection data for six months, and only after the Pioneer Press noticed that a computer file initially provided by the department was incomplete. The department also took two months to provide computer records about foodborne outbreaks, and only after \"sanitizing\" the records of information that could identify victims. (More on that below.)\n\nIn one county, officials gave us the computer data from inspectors' portable computers, but said their contract with a software vendor prevented them from providing any background on how to read the data.\n\nAnalyzing the data\n\nAs the old saying goes, be careful what you ask for, because you just might get it. That's how we felt after the various agencies' data arrived. The reason: Regulatory agencies use computers to help keep track of inspection schedules and compliance, not sift trends from the data as we wanted to do. The six databases had several different formats. Some were missing records. Others were littered with duplicate records. One county had only entered about half of all restaurant visits into the computer system - the \"bad\" inspections, leaving the better visits documented only in hundreds of paper files.\n\nSt. Paul's data came from an ancient computer system (finally replaced in the middle of our project) with some confusing twists - among them, taking violations by restaurants long since out of business and attributing them to the building's current occupant. St. Paul also was unable to give us a list of restaurants issued tickets, something other agencies were able to do.\n\nMost of our tasks had to be multiplied by six - for instance, six sets of queries for the worst and best restaurants; six sets of phone calls to officials to request paper records, confirm our findings, ask about \"good guys,\" and other needs; six different sets of preparation materials for interviews with officials, and so forth.\n\nOne of the dilemmas we wrestled with was how to analyze the inspection scores. There's a strong argument that scores can be misleading. A restaurant can get a mediocre score, say in the low 80s, for a bunch of minor violations that involve \"floor, walls and ceilings\" but not food safety. On the other hand, a restaurant can get a score of 90 while committing one or two really gross violations, like improper food temperatures and cross-contaminating cooked food with bacteria from raw food - things that can make people sick.\n\nSo from early on, to avoid this controversy, we never contemplated actually publishing the averages, or scores from a given year, etc. Instead, we made plans to count up and rank restaurants based on how many \"critical\" violations they received.\n\nIn total, we found Twin Cities restaurants had violated food-safety rules more than 130,000 times from 1995 to mid-1999, including more than 20,000 violations that government agencies classify as \"critical\" because they can make diners sick. We also obtained a state database of foodborne illness outbreaks, many of which had never come to light before. As for enforcement, we pieced together agencies' records - sloppy in most cases - and found penalties against violators were rare, even when they were proven to have made people sick.\n\nTips for others\n\nAdvice for this or other health and safety related stories: Carefully document, for instance in a spreadsheet, all your requests for information - when, to whom, how officials responded, and where things stand. It gets harder to keep track as the weeks and months go by, and multiple requests pile up.\n\nTo locate victims, consider such public records sources as lawsuits, foodborne illness complaints, death certificates, hospital discharge records, even letters to the editor.\n\nThink early, and if possible decide early, how to document and illustrate your findings regarding specific restaurants. We began by looking at bests and worsts, but had concerns about whether the data would be accurate, timely and complete enough. Late in the game, we shifted gears and wound up publishing inspection results for the most popular restaurants (as measured by the Zagat Survey).\n\nRick Linsk can be reached by e-mail at rlinsk@pioneerpress.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-januaryfebruary-bits-bytes-and-barks","slug":"2001-januaryfebruary-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2001-januaryfebruary-bits-bytes-and-barks.html","year":2001,"month":"January/February","monthNum":1,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR Database Library announces updated datasets on hazardous materials incidents, AIDS cases, and NEA grants, while highlighting recent computer-assisted reporting investigations by news organizations on gun trafficking, insurance fraud, and educational disparities.","keywords":["databases","investigation","journalism"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"Fresh data\n\nThe NICAR Database Library has recently updated the U.S. DOT Hazardous Materials Incidents, the NEA Grants and the AIDS Public Information databases.\n\nThe Hazardous Materials data contains the incident reports of unintentional releases of hazardous materials for all modes of transportation (air, highway, railway, water) from 1971 through May 2000. Cost is $50, $100, or $125 depending on size of your news organization.\n\nThe AIDS Public Information Data Set contains information from Acquired Immunodeficiency Syndrome cases reported to state and local health departments. The most recent data available from the Centers for Disease Control is for 1998. Cost is $25, $50 or $75.\n\nThe National Endowment for the Arts grants database contains all organizations or individuals who've received money from the NEA since the program began in fiscal year 1987. The most recent data available is for fiscal year 1999. Cost is $25, $50 or $75.\n\nMore information about these datasets, including sample data and record layouts, is available at www.ire.org/datalibrary/datasets. To order the data, call the NICAR Database Library at (573) 884-7711.\n\nRecent CAR stories:\n\n\"Gunrunner's Paradise,\" Dayton Daily News, Dec. 10-13, 0 uwww.activedayton.com/partners/ddn/local/projects/guns/1210gunmain2.html\nThe paper's analysis found Ohio ranks behind only Florida, Georgia, Virginia and Texas for guns used in crimes out of state. The data, obtained from the Bureau of Alcohol, Tobacco and Firearms, show that in 1997 and 1998 more than 1,000 guns used in crimes nationwide came from Ohio.\n\n\"Crashing for Cash,\" Sun-Sentinel, Dec. 17-20, 2000 www.sun-sentinel.com/crashingforcashl\nTo study the dimensions of auto insurance fraud, the Sun-Sentinel analyzed thousands of state Department of Motor Vehicles records and other government and insurance industry claims data.\nThe Sun-Sentinel built a database that captured the driving and accident histories of hundreds of people identified in arrest reports or other investigative files.\n\n\"Destined to Fail,\" The Beacon Journal, Dec. 10-13, 2000\nwww.obio.com/specials/buchtel_seriesl\nA six-month Akron Beacon Journal examination revealed factors that have nothing to do with a child's family income or race help explain why poor minority children in Akron perform behind nearly every other group of students.\n\nGlobal conference\n\nIRE and NICAR are among the sponsors of the first global journalism conference on investigative reporting and computer-assisted reporting, April 26-29, 2001 in Copenhagen, Denmark.\n\nThe 4-day conference will focus on great stories, the latest techniques and the effect of media convergence on investigative journalism. More information is available at www.ire.org/training/denmark/2000conf","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-car-in-the-aftermath","slug":"2001-marchapril-car-in-the-aftermath","url":"https://dwillis.github.io/uplink/article/2001-marchapril-car-in-the-aftermath.html","year":2001,"month":"March/April","monthNum":3,"headline":"CAR in the aftermath","kicker":"ELECTIONS","author_name":"Geoff Dougherty","author_title":"Reporter","affiliation":"The Miami Herald","summary":"Miami Herald journalists conducted a four-month investigation analyzing over 50 databases to examine the legitimacy of Florida's 2000 presidential election, which hinged on just 500 votes, investigating voter eligibility and ballot counting procedures.","keywords":["election","investigation","Florida"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":[],"full_text":"On the morning of Nov. 8, I sat in The Miami Herald's cafeteria with three colleagues. Bleary eyed from watching Florida's presidential election returns roll in until 3 a.m., we talked over the results. Five hundred votes, in a state known for ballot fraud, would determine the presidency.\n\nIt seemed clear: We needed to take a comprehensive look at the Florida election. Who had voted? Did they do it legally? How had their ballots been counted (or discarded)? Were there signs of organized fraud?\n\nAnswering those questions would take us thousands of hours over the next four months. Over that time, we had cranked up a data collection and analysis effort of unprecedented scope.\n\nDatabases galore\n\nWe've gathered and analyzed more than 50 databases, including:\n\n• Information from the Florida Secretary of State's office on each registered voter in Florida, and whether they cast a ballot on Nov. 7.\n\n• Precinct-level election result files for all of the more than 5,000 precincts in Florida, including information on the number of undervotes and overvotes. These came in both electronic and paper format (that was then hand-entered) from county election supervisors.\n\n• Databases from the court and police agencies in Dade, Broward and Palm Beach counties, identifying potential voters who were logged into jail as illegal immigrants, people who had vouched for their non-citizen status on jury selection forms and people declared incompetent to vote by the courts.\n\n• A database from the Florida Department of Corrections identifying millions of people who had served state prison time after being convicted of felonies.\n\n• A database of all convicted felons who had been granted clemency and had their right to vote restored by the Florida Office of Executive Clemency.\n\n• A database from the Florida Department of Health of each person who had died in the past year in Miami-Dade County, along with databases from the Florida Secretary of State identifying voters who were targeted for removal from the rolls. They were being re-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-everday-methods","slug":"2001-marchapril-everday-methods","url":"https://dwillis.github.io/uplink/article/2001-marchapril-everday-methods.html","year":2001,"month":"March/April","monthNum":3,"headline":"Everday methods","kicker":"FIRST VENTURE","author_name":"Joe Ellis","author_title":"","affiliation":"KMOL-TV, San Antonio","summary":"A journalist reflects on attending a computer-assisted reporting bootcamp that challenged their mental capacity, and describes how CAR techniques have since become an integral part of their daily reporting practice.","keywords":["bootcamp","reporting","records"],"topics":["Investigative Methods"],"technologies":[],"full_text":"I thought I'd be eager to get back to my computer immediately after bootcamp, but that was before bootcamp. I remember my brain feeling like it had run a week's worth of PT. And it had...following the cadence of Sgt. Tom McGinty and company for six days, 10 hours a day. It showed me just how out of shape my mind really was.\n\nI took what I felt was a well-deserved weekend away from my thoughts of work, news and computer-assisted reporting. But it wasn't long before I was finding ways to use my newfound skill.\n\nSince then, CAR has become an everyday method for me.\n\nWhen I'm looking for information, among the first things I ask the \"Public Interference Official\" is \"how are these records kept?\" \"In what form are they","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-uplink-improvements","slug":"2001-marchapril-uplink-improvements","url":"https://dwillis.github.io/uplink/article/2001-marchapril-uplink-improvements.html","year":2001,"month":"March/April","monthNum":3,"headline":"Uplink improvements","kicker":"ANNOUNCEMENT","author_name":"MaryJo Sylvester","author_title":"Managing Editor","affiliation":"NICAR","summary":"Uplink journalism newsletter is transitioning to a bi-monthly publication schedule with expanded content, themed issues focusing on topics like education and transportation, and enhanced resources including databases, sources, and story ideas for computer-assisted reporting. The redesigned format will include regular features, an online searchable archive, and email news flashes to keep journalists informed about CAR developments.","keywords":["newsletter","journalism","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["World Wide Web","Internet"],"full_text":"For more than a decade, Uplink has brought you inspiring tales, technical how-to's, story ideas, tips for finding information and more. The newsletter has chronicled the evolving realm of computer-assisted reporting and offered support and guidance to hundreds of journalists worldwide.\n\nNow we'd like to do more.\n\nStarting with this issue, we will publish Uplink on a bi-monthly schedule (every other month). In the next issue, you'll see even more content, with at least 28 pages and supplementary materials on the Web.\n\nEach issue will have a central theme – i.e. education in one issue, transportation in another – giving journalists not only the inspiring tales, but also the means to pull them off in their own community. There will be sidebars listing databases and where to find them; Web sites and phone numbers to sources; and listings of other stories on the same or similar topics.\n\nIn addition to the theme stories, each issue will include regular features such as First Ventures, Mapping how-to's and Tech Tips.\n\nBenefits to the bi-monthly schedule are to provide greater time and space to develop each theme and to be able to coordinate more with The IRE Journal.\n\nOur plan includes design improvements and to make available to subscribers an online, searchable archive of past articles. Subscribers will also receive news flashes via e-mail on developments in the world of data and hot new CAR stories available online.\n\nMaryJo Sylwester can be reached by e-mail at maryjo@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-census-maps-and-more","slug":"2001-marchapril-census-maps-and-more","url":"https://dwillis.github.io/uplink/article/2001-marchapril-census-maps-and-more.html","year":2001,"month":"March/April","monthNum":3,"headline":"Census, maps and more","kicker":"TRAINING","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering numerous training opportunities in 2000, including a spring mapping bootcamp focused on Census data, statistics seminars, and computer-assisted reporting training at conferences in the United States and Europe.","keywords":["training","mapping","census"],"topics":["Journalism & Newsroom","Technology","Data & Databases"],"technologies":["ArcView"],"full_text":"Census, maps and more\n\nIRE and NICAR has a calendar full of opportunities this year, including a spring mapping bootcamp and CAR training sessions in Denmark and London.\n\nWe're also planning to schedule several more Census workshops in various locations of the country. Stay tuned for dates and places.\n\nThe mapping bootcamp, recently added to the schedule, will have a special emphasis on mapping Census data. It will be held May 4-6, in Columbia, Mo., at the Missouri School of Journalism.\n\nSessions by Andy Lehren of Dateline NBC and Jennifer LaFleur of The St. Louis Post-Dispatch will include thematic mapping, overlaying, spatial analysis and address geocoding. Participants are asked to have a basic understanding of database managers.\n\nMore information, including registration forms, is available at www.ire.org/training/may4map.html.\n\nAlso in May, IRE and NICAR will sponsor the Advanced Bootcamp on Statistics at the University of North Carolina-Chapel. See page twenty-two for more about this seminar.\n\nIn June, the IRE National Conference in Chicago will include a full-day of panels dedicated to computer-assisted reporting topics. The conference runs June 14-17. More information is available at www.ire.org/chicago/index.html.\n\nAnd make sure to mark your calendars for the NICAR National Conference in Philadelphia, Oct. 11-14.\n\nFor European journalists, IRE and NICAR will offer CAR training at two conferences.\n\nIn April, IRE and other European journalism organizations will hold the Global Journalism Conference in Copenhagen. The conference, April 26-29, will include hands-on CAR training and panels on the latest reporting issues. More information is available at www.ire.org/training/denmark/2000conf/\n\nIRE and NICAR will also offer CAR training at NetMedia, a journalism conference July 4-6 in London.\n\nFinally, the usual lineup of NICAR Bootcamps are on schedule in Columbia, Mo. Seats are still available for the May, July and August sessions. More information is available at www.ire.org/training/bootcamps.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-floridas-dead-heat","slug":"2001-marchapril-floridas-dead-heat","url":"https://dwillis.github.io/uplink/article/2001-marchapril-floridas-dead-heat.html","year":2001,"month":"March/April","monthNum":3,"headline":"Florida's dead heat","kicker":"","author_name":"Stephen K. Doig","author_title":"Professor","affiliation":"Arizona State University","summary":"An in-depth investigation into the Florida presidential election, focusing on voter fraud and undetermined ballots.","keywords":["Florida","election","fraud"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Excel"],"full_text":"I left The Miami Herald for Arizona State University more than four years ago. But in November, for the first time, I wished I were back working in The Herald newsroom. The reason was the dead-heat Florida presidential election. This political nightmare was a journalist's dream, and I knew that computer-assisted reporting would play a big role in the coverage. As the media frenzy began to build, I sent a 'gee, if there's anything I can do to help, just yell' e-mail to Herald city editor (and IRE chairman) Judy Miller. My eagerness to have a piece of this story must have been evident. A few days later, Judy called. We talked about different angles The Herald troops were pursuing, including an analysis underway by CAR specialist Geoff Dougherty indicating that hundreds of felons had voted in violation of state law. But in light of the evident confusion in so many Florida polling places, the burning problem at the moment was 'discerning the mind of the voter' - particularly the clouded minds of the 185,000 or so Floridians who had gone to the polls but had failed for whatever reason to cast a valid ballot. I did have an idea, albeit unformed, about how to attack the question. I had already been intrigued by how a smart statistician at Carnegie Mellon had demonstrated convincingly how vanishingly small was the probability that so many voters in Palm Beach County, home of the infamous 'butterfly' ballot, would deliberately vote for Pat Buchanan. We should gather precinct-level returns to see what they tell us. I didn't have a very clear idea of what I might do with such data, but it somehow seemed useful at the time. It never occurred to me how staggeringly difficult it might be for The Herald to canvass all 67 counties, which were using all manner of voting methods and at least one of which was still using paper ballots. However, one thing Herald editors excel at is getting bright young reporters to work really hard when yet another 'Story of the Century' breaks in South Florida. And so it happened with this one. Two reporters, Anabelle de Gale and Lila Aruza, had worked long hours on the phones wheedling elections departments across the state, pulling together what I so casually asked for. Sometimes they got it as a nice, neat e-mail attachment. More often, it came as a fax from technologically backward - or deliberately obstructive - officials that had to be tediously retyped into a spreadsheet. Making a table As they completed each county, Anabelle and Lila e-mailed me the spreadsheets and I stitched those into one master Excel table that would cover the state's 5,885 precincts. For each precinct, the table showed the total vote for Bush, Gore, the combined other presidential candidates, and the total number of people who had cast ballots. By the middle of the week after Thanksgiving, the pressure had shifted to me and they wanted a story for that weekend. And I didn't have anything to say. My favorite [reaction] came from the Bush spokesman who dismissed it all as 'voodoo statistics'. My initial plan had been to merge the results table with voter demographics from other sources, in the hope that we could make some definitive statements about who had been most affected by the ballot problems. But the demographic merge turned out to have huge problems caused by the varying ways each county named its precincts. It was a story that would be done later by Geoff, but the necessary cleanup put it out of reach for that weekend. For a while, all I could envision was some sort of squishy look at such trivialities as the number of uncounted votes in each precinct. Eureka! Finally, inspiration struck. Years of working with both elections and census data taught me that people behave pretty much like their neighbors do. A precinct, almost by definition, is a neighborhood. So it dawned on me to take the voting behavior of those who had successfully voted in each precinct, and just apply it proportionally to the uncounted votes. Thus, if 60 percent of the vote in a given precinct went to Bush, then I would give Bush 60 percent of the uncounted votes, and so on. At that point, the analysis became a straightforward spreadsheet exercise, simple enough that I'm going to have my students duplicate it this semester. I did the formulas, copied them down the columns, hit the recalculation button, and saw the lead: Gore would have won by about 24,000 votes, instead of losing by about 500, if all those uncounted ballots had been cast correctly. Reporter Curtis Morgan began bouncing our results off different sources for reaction. My favorite came from the Bush spokesman who dismissed it all as 'voodoo statistics'; I think I'm gonna have a T-shirt made with that slogan. Although we were still refining the findings, Curtis managed to craft a nicely readable account with these initial interviews. He also came up with the clever analogy of this analysis to a public opinion survey. Normally, a small sample is used to gauge the opinion of a large group. In this case, I was using the demonstrated behavior of the large group to project the behavior of a small subset. Some critics retorted that a perfect election is impossible, and that therefore my analysis was meaningless. Others argued strenuously about the so-called 'protest vote' of those who go to the polls but deliberately don't vote in a race because they don't like any candidate.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-marchapril-four-day-success","slug":"2001-marchapril-four-day-success","url":"https://dwillis.github.io/uplink/article/2001-marchapril-four-day-success.html","year":2001,"month":"March/April","monthNum":3,"headline":"Four-day success","kicker":"ELECTIONS","author_name":"Nancy Cook Lauer","author_title":"Bureau Chief","affiliation":"Tallahassee Democrat","summary":"A small-circulation newspaper in Tallahassee used basic spreadsheet analysis to uncover a correlation between county income levels and uncounted votes during Florida's 2000 election dispute, completing the investigation in just four days despite the reporter's other duties.","keywords":["analysis","election","data"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel","PDF"],"full_text":"Here's proof that you don't have to be a big newspaper with a lot of expensive database software and unlimited staff time to come up with a compelling story. From inspiration to publication, this CAR project took just four days - during which the reporter also churned out front-page dailies from Ground Zero of the 2000 post-election imbroglio.\n\nAs a newly promoted bureau chief at a small (just over 50,000 daily circulation) Knight Ridder newspaper in Tallahassee, Fla., I was up to my eyeballs in Florida's hanging chads and other controversies when a call came out of the blue. A civic activist I had known at a previous job five years ago and 400 miles away had driven to the state capital to get some information from the state Division of Elections. She saw my byline and called to tell me she'd picked up an interesting report.\n\nBecause the information wasn't available on the state Division of Elections' Web site, I had a copy of the report faxed over. It was a compilation of the presidential votes cast and counted in each of the state's 67 counties.\n\nLike the rest of the world, we'd been hearing that voters in counties with antiquated punch-card voting systems had fewer of their votes counted, compared to those using optical scanners. We'd also been hearing complaints from minority groups that poorer counties had the fewest votes counted because they had the bulk of the old equipment.\n\nI used Microsoft Excel 97 (for PCs) and compiled a spreadsheet listing the state's 67 counties in one column and the percent of votes not counted in another. Generally, I prefer using raw numbers, not computed ones, in spreadsheets, but in this case, the percentages were all I needed and I was in a hurry to make a Sunday story out of information I'd just discovered on Thursday.\n\nBecause I was in a hurry and on the cheap, I went to the University of Florida's Bureau of Economic and Business Research site (http://www.bebr.ufl.edu/) and printed out a free report on population and income. Since it was a PDF, I had to key the per-capita income data back in instead of just downloading it.\n\nI had to do the same with the information about which counties used what voter system. That's available on the Division of Election's Web site (http://election.dos.state.fl.us/votemeth/cvs.shtml), but not in any downloadable form.\n\nOnce the data were input and double-checked, there were several ways I could have approached the analysis. I could have done more intensive number-crunching on it such as Chi-square, but in the time frame allotted, I thought just ranking the counties by income and slicing them in half - high income vs low income - would offer a new perspective and serve the purposes of the article.\n\nThe resulting story \"Incomes, discarded votes may be linked\" (http://web.tallahasseedemocrat.com/content/tallahassee/2000/12/03/local/1203.loc.votes.htm) showed that voters in Florida's poorer counties were more than twice as likely as those in more affluent ones to have their votes for president disregarded. That held true regardless of whether they voted on optically scanned ballots or punch card ones.\n\nThe story inspired debate about the get-out-the-vote campaigns that had been launched by the National Association for the Advancement of Colored People, unions and Democrats, all of whom tend to draw lower-income voters. Advocates used the article to talk about the need for voter education and the equitability of having the state subsidize new voting machines. Even Gov. Jeb Bush found the article of use and had a copy faxed to him.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-election-machine","slug":"2001-marchapril-election-machine","url":"https://dwillis.github.io/uplink/article/2001-marchapril-election-machine.html","year":2001,"month":"March/April","monthNum":3,"headline":"Election machine","kicker":"ELECTIONS","author_name":"Neil Reisner","author_title":"","affiliation":"Miami Daily Business Review","summary":"This article provides a step-by-step guide for election reporters on how to use spreadsheets, databases, and mapping software to analyze election results by precinct, revealing voting patterns across different demographic groups and communities.","keywords":["election","precinct","spreadsheet"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel","Access","ArcView"],"full_text":"Now that Chad-Fest 2000, otherwise known as the presidential election is over – I suppose I could call it a Chad Row, but awful puns like that are just so tacky – it might be a good time to come back to earth and think about the kinds of elections we normally cover.\n\nSmall stuff: town council, school board, county commissioner, maybe governor.\n\nAnd lots of computer savvy reporters figured out a long time ago that a spreadsheet is an election reporter's best friend, ideal for tabulating votes, percentages and winners and able to produce neat little graphs lickety-split.\n\nBut as soon as results are in, the questions start coming:\n\nWhat happened? How'd she win? or How'd the Black community/Hispanic neighborhoods/the rich/the poor/home owners/condo residents/gays/swing precincts vote?\n\nUsually that means making a quick checklist of key precincts, scanning the spreadsheet and tweaking out whatever patterns may be visible. It's not easy and there's a better way. A map of the election will often let the patterns jump out at you and your colleagues whose eyes may glaze over when they look at a table of numbers. It's not hard, but to make it happen requires some front-end work before the election. I call it the \"Election Machine.\"\n\nYou'll need:\n\n• Excel, Access and ArcView or some other spreadsheet, database manager and mapping program.\n\n• Precinct-by-precinct voter registration data, in the spreadsheet.\n\n• Tables of key voting precincts, in the database manager.\n\n• A map layer showing election precincts.\n\n• Election results in the spreadsheet.\n\nHere's a step-by-step guide to how it all fits together using Excel, Access and ArcView. Looking at real examples will make it a little easier to follow; if you want to try it for yourself, download a PowerPoint demonstration and sample data in Excel, Access and ArcView from the NICAR Web site: www.nicar.org/electmach/index.html.\n\nStep 1: Voter Registration\n\nBuild a one-page workbook showing the number of Democrats, Republicans, etc. in each precinct or election district, as well as the number of blacks, whites, Hispanics and any other ethnic or demographic information you can obtain. The best source for this usually is the county supervisor of elections, who should keep this information on each precinct; other sources might be local political consultants or university political scientists or demographers. Figure the percentages of each political or demographic group: Democrat, Republican, black, white, etc.\n\nStep 2: Key Precincts\n\nCreate a database and build a table identifying key precincts, bellwether precincts that are predominantly Democratic, Republican, Black, white and the like.\n\nStep 3: Election Results\n\nBuild a spreadsheet workbook with one page for overall returns, precinct-by-precinct and a separate page for each election that you want to analyze. Why? If you're lucky, you can get results in some electronic format, through e-mail, a download or on a floppy disk.\n\nAsk the election folks for a dummy file before – the same layout you'll see on Election Day, but without numbers – and build a dummy page mimicking that layout. Then it becomes a (reasonably) simple matter to paste the results onto the page. Try this in advance to make sure it works. Worst-comes-to-worst, build the page in advance and type results into it.\n\nIf you're dealing with multiple elections – several municipalities or commission seats, etc. – copy results for each election in which you're interested to separate pages, one page per election. Paste the results from the results page as a live link by clicking on \"Edit,\" \"Paste Special,\" \"Paste Link.\" When you do it this way, the results of each individual election will flow automatically from the results page to the election pages.\n\nBuild formulas, calculating what you want to know about the election: the turnout, the winner, etc.\n\nHow Come...\n\n• ...it's important to keep registration numbers in a separate workbook? Registration data will change election-to-election and you'll sometimes have to re-calculate percentages. Keeping it separate lets you build a registration template, updating it for each election as the numbers change\n\n• ...I should keep overall election results on one page and each election on others? Because it lets you paste/type all the results on one page instead of having to do it on multiple pages. Because it lets you type all results on one page.\n\n• ...I should keep key precincts in a database? Because key precincts are not likely to change and you can use the same base data for each election.\n\nStep 4: Name Spreadsheet Election Pages\n\nOn each individual election's page, select the range containing the results, including headings. Give it a name – City Council, for example – by clicking \"Insert, Name,\" \"Define\" and \"Add.\" Do the same for the voter registration page. You'll see that the name of the range will appear in a small window left of Excel's formula bar.\n\nOpen Access and click \"File,\" Get External Data,\" and \"Link Tables.\" Select the election results workbook. When the Wizard opens, click on \"Show Named Ranges.\" You'll see that the ranges you just named in Excel show up as items you can link to Access. Follow the wizard's instructions and link each Excel range to Access as a table.\n\nDo the same in the registration workbook: Name the range containing the demographic percentages in which you're interested and link it to Access as a table.\n\nStep 5: Build and Run Queries\n\nBuild queries to look at winners, losers, ethnic votes or anything else of interest. Link to swing precincts winner, Black vote or anything else of interest. Link precinct-by-precinct registration numbers to precinct-by-precinct results table or swing precinct table to determine voting patterns. You can build the queries in advance, linking to your dummy data. That makes things faster on deadline.\n\nStep 6: Export Queries\n\nExport the results of each relevant query in dBase IV (*.dbf) format by clicking on \"File,\" and \"Save As/Export.\" Choose \"To an External File or Database\" and follow the wizard. This will let you import the results into ArcView.\n\nStep 7: Build Maps\n\nOpen ArcView, where you've already built a precinct map. You may be able to get a precinct map from your local election supervisor, city or county planning department, the state legislative staff responsible for redistricting, political consultant or political scientist may have one.\n\nJoin the *.dbf tables you exported from Access to the map.\n\nThe example here joins a \"Winner\" table to the precinct map. It shows that Rubble didn't get beyond his political base, while Flintstone took his base in Central Bedrock and more.\n\nSumming up\n\nUsing a spreadsheet, database manager and mapping software you can quickly create a picture of what happened in an election. Just gather the basic data that you'll need in advance and build spreadsheets, tables and queries.\n\nBuild dummy spreadsheets for the election results you won't get until Election Day. When results","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-car-in-election-2000","slug":"2001-marchapril-car-in-election-2000","url":"https://dwillis.github.io/uplink/article/2001-marchapril-car-in-election-2000.html","year":2001,"month":"March/April","monthNum":3,"headline":"CAR in Election 2000","kicker":"ELECTIONS","author_name":"Ron Nixon","author_title":"","affiliation":"IRE and NICAR","summary":"Investigative reporter Thomas Hargrove discovered that voting irregularities and uncounted votes were not unique to Florida's 2000 election, but rather a widespread systemic problem across the United States. Computer-assisted reporting by major news organizations revealed that millions of votes went uncounted in previous elections, with faulty registration lists and inadequate resources in county clerk offices disproportionately affecting minority and low-income communities.","keywords":["voting","election","investigation"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["SPSS"],"full_text":"As the 2000 Presidential election dragged on due to lawsuits filed over accusations of missed and uncounted votes, Thomas Hargrove, a reporter for Scripps Howard, wondered if the 2000 Florida fiasco was an isolated problem.\n\nTo test his theory, Hargrove went to Election Data Services, a Washington group that keeps election results, for data from the 1996 election.\n\nPulling the data into SPSS, a statistical software package, Hargrove ran queries to see if there were indeed undercounts in the contest between Bob Dole and Bill Clinton.\n\nThe results were startling.\n\n\"Our investigation showed that in the Dole-Clinton race, millions of votes were not counted,\" he said. \"In one West Virginia County, the county clerk had just found 1,000 votes that Bob Dole should have gotten.\n\n\"It wasn't enough to change the election because Clinton had such a lead over Dole,\" Hargrove said.\n\nThe News Service also found other problems: official registration lists in the United States have become so faulty that scores of counties claim to have more voters than actual adult population.\n\nSloppy bookkeeping led 190 counties and the state of Maine to appear to be more than 100 percent registered in the 1996 presidential election, according to the Scripps Howard News Service study of election records in 3,189 counties and voting districts.\n\n\"Our studies did show that there were numerous problems in voting across the country and not just Florida,\" Hargrove said.\n\nComputer-assisted reporting played an important role in coverage of the 2000 presidential election.\n\nHundreds of newspapers and other media across the country found problems in the election system that went beyond Florida, which had gathered most of the attention in the presidential race. Among the papers and news organizations were The New York Times, The Wall Street Journal, The Miami Herald, The Palm Beach Post, The Chicago Tribune, and The Tallahassee Democrat, among others.\n\nKey findings\n\n• The long-held belief that every vote counts is a myth. Media organizations found that in each election, thousands of votes are thrown out or not counted.\n\n• The 3,067 individual counties in the United States have few national standards for determining how votes get counted. The job of counting the 100 million or more votes often falls on poorly trained and understaffed county clerk offices with few resources. A Wall Street Journal investigation found that county governments typically spend about 3 percent of their budgets on the clerk offices.\n\n• Voting failures often fall on those who can least afford it: Minority and low-income communities. Investigations by The New York Times and several papers in Florida show that voting systems in these communities tend to be older and less reliable than those in high-income communities.\n\n• Registration lists were often flawed. News organizations found that many people were turned away from polls because their names weren't on the list, and a Miami Herald investigation found that hundreds of felons voted in the presidential election.\n\n• In Florida, poor ballot designs may have given thousands of votes to the wrong candidate. Another Miami Herald investigation found that a third party candidate's gain in votes in a heavy Democratic county was a statistical anomaly.\n\nFaulty machines\n\nMost of the problems with under-votes were attributed to faulty voting machines, an investigation by a media organization found.\n\nOne of the first newspapers to look at the voting machine issue was The New York Times. The Times and the Scripps Howard post-election analysis will be available soon at the IRE Resource Center.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-tipsheets-for-local-elections","slug":"2001-marchapril-tipsheets-for-local-elections","url":"https://dwillis.github.io/uplink/article/2001-marchapril-tipsheets-for-local-elections.html","year":2001,"month":"March/April","monthNum":3,"headline":"Tipsheets for local elections","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A collection of tipsheets providing guidance on using computer-assisted reporting (CAR) and databases to investigate local elections, including lobbyist influence, official salaries, and campaign contributions.","keywords":["elections","reporting","investigations"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Tipsheets for local elections:\n\nTipsheet # 923. Francis Gilpin of the Tampa Tribune examined how CAR can help discover the power of local lobbyists in elections. (1999, Boston).\n\nTipsheet # 924. David Washburn offered tips on local reporting including past stories on commissioner salaries and elections. (1999, Boston).\n\nTipsheet # 478. This tipsheet includes a list of election story ideas as well as tips on setting up a contributions database.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-other-local-election-stories","slug":"2001-marchapril-other-local-election-stories","url":"https://dwillis.github.io/uplink/article/2001-marchapril-other-local-election-stories.html","year":2001,"month":"March/April","monthNum":3,"headline":"Other local election stories","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This item highlights three investigative journalism stories about local election issues including political fundraising, absentee ballot fraud, and forged petition signatures from newspapers in Buffalo, Indiana, and New York.","keywords":["elections","fraud","fundraising"],"topics":["Government & Politics"],"technologies":[],"full_text":"Other local election stories\n\nRose Ciotta of the Buffalo News (Buffalo, N.Y.) examined political fundraising for a 1995 county executive race. Story # 12358.\n\nIn 1994, the Times (Munster, Ind.) investigated absentee ballot vote fraud in the spring primary election. Story # 10944.\n\n\"GOP-petition signatures questioned,\" by Greg Moran of the Post-Star (Glens Falls, N.Y.), Aug. 1987, found forged signatures on the petitions to get local GOP candidates on the ballot. Story # 5119.\n\nCopies of these stories can be ordered from the IRE Resource Center by calling (573) 882-3364. Search the story database at www.ire.org/resourcecenter.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-the-tallahassee-democrat-stories-online","slug":"2001-marchapril-the-tallahassee-democrat-stories-online","url":"https://dwillis.github.io/uplink/article/2001-marchapril-the-tallahassee-democrat-stories-online.html","year":2001,"month":"March/April","monthNum":3,"headline":"The Tallahassee Democrat stories online","kicker":"","author_name":"","author_title":"","affiliation":"The Tallahassee Democrat","summary":"The Tallahassee Democrat published investigative pieces examining potential links between voter income levels and discarded ballots, as well as analyzing legal funding sources in the 2000 presidential election dispute. These stories are available online and can be ordered through the IRE Resource Center.","keywords":["election","investigation","funding"],"topics":["Investigative Methods","Government & Politics"],"technologies":["Internet","World Wide Web"],"full_text":"The Tallahassee Democrat's \"Incomes, discarded votes may be linked\" is available online at http://web.tallahasseedemocrat.com/content/tallahassee/2000/12/03/local/1203.loc.votes.htm\n\n\"Bush wins battle for legal funds\" is also available online at http://web.tallahasseedemocrat.com/content/tallahassee/2000/12/18/local/1218.loc.contribs.htm\n\nThese stories will soon be available from the IRE Resource Center. Search the story database at www.ire.org/resourcecenter for order numbers. Call (573) 882-3364 to order these and other investigative stories.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-marchapril-the-new-york-times-and-scripps-howard-post-election-analysis","slug":"2001-marchapril-the-new-york-times-and-scripps-howard-post-election-analysis","url":"https://dwillis.github.io/uplink/article/2001-marchapril-the-new-york-times-and-scripps-howard-post-election-analysis.html","year":2001,"month":"March/April","monthNum":3,"headline":"The New York Times and Scripps Howard post-election analysis","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The New York Times and Scripps Howard post-election analysis will be available at the IRE Resource Center, which can be contacted at (573) 882-3364.","keywords":["election","analysis","resources"],"topics":["Investigative Methods","Government & Politics"],"technologies":[],"full_text":"The New York Times and the Scripps Howard post-election analysis discussed in Ron Nixon's story on page eight will be available soon at the IRE Resource Center. The contact number to the Resource Center is (573) 882-3364","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-firestone-and-explorers","slug":"2001-mayjune-firestone-and-explorers","url":"https://dwillis.github.io/uplink/article/2001-mayjune-firestone-and-explorers.html","year":2001,"month":"May/June","monthNum":5,"headline":"Firestone and Explorers","kicker":"TRANSPORTATION","author_name":"Josh Barbanel","author_title":"Reporter","affiliation":"The New York Times","summary":"Investigates issues with Firestone tires on Ford Explorers, identifying a surge in tire-related fatal accidents and highlighting potential oversight by government and manufacturers.","keywords":["Firestone","Explorer","accidents"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["spreadsheet"],"full_text":"It was late in the day. We were up to our ears in other projects when a business editor interrupted with one of those vexing requests. It was a nuisance - hardly worth the attention of one, not to mention two, highly significant reporters in the new Investigative Cluster of The New York Times. \"Can you help us open up this spreadsheet,\" the editor asked. Sigh. The spreadsheet turned out to be a messy table with the first release of government complaints about deaths attributed to Firestone tires on Ford Explorers. We clicked on the table icon in the email, counted the cases and were done. Back to our very important work. But the next day, as my colleague Ford Fessenden likes to say, serendipity happened. We were discussing what we would have said, if the editors had seen us as something more than keyboard jockeys - and actually asked for help in thinking about and dealing with the rapidly developing story of the recall of more than 14 million Firestone ATX and Wilderness tires. Well, we both had worked with government accident data files. We knew that Ford Motor Company was using warranty claims data to monitor the problem (and make its case that the problem was all the fault of Bridgestone/Firestone). We knew the federal government was relying on complaints called in by survivors, lawyers and the auto companies. If these complaints involved real people killed in real fatal accidents, shouldn't we be able to track at least some of the same cases in the rich federal database of fatal accidents? The editors never actually assigned us to look into the Ford Explorers. But we poked around, and they were fascinated when we told them we were able to clearly trace a significant surge in tire-related fatal accidents in Ford Explorers, not found in other sport utility vehicles. This was remarkable (some may say debatable) because reports of tire-related deaths were rare events. At the time, the federal government had complaints of 88 deaths in tire-related accidents in the Ford Explorer, including some going back many years.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-mayjune-census-2000-heavy-lifting-ahead","slug":"2001-mayjune-census-2000-heavy-lifting-ahead","url":"https://dwillis.github.io/uplink/article/2001-mayjune-census-2000-heavy-lifting-ahead.html","year":2001,"month":"May/June","monthNum":5,"headline":"CENSUS 2000 Heavy lifting ahead","kicker":"CENSUS 2000","author_name":"Richard O'Reilly","author_title":"","affiliation":"Los Angeles Times","summary":"The Census 2000 data release presents increasing complexity for journalists using computer-assisted reporting, with upcoming Summary File 1 containing vastly more variables and detailed information than the initial PL94-171 data.","keywords":["Census","data","journalism"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"The first wave of data from Census 2000 has washed ashore, providing some with an exhilarating ride surfing the leading edge of computer-assisted reporting It may have been more terrifying than thrilling for others, however.\n\nWhichever situation you identify with, upcoming data releases will be much, much more challenging.\n\nThe data dictionary for PL94-171's four tables of race and race and Latino/Hispanic ethnicity was 10 pages long. For Summary File 1 coming this summer, the preliminary description runs 110 pages, and the final detailed listing will be many times that length.\n\nPL94-171 had a total of 288 population count variables, plus the 80 geography variables.\n\nSummary File 1 will have the same 80 geography variables plus 8,132 population and housing count variables. That includes 3,078 variables down to the block level plus another","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-fatal-highway","slug":"2001-mayjune-fatal-highway","url":"https://dwillis.github.io/uplink/article/2001-mayjune-fatal-highway.html","year":2001,"month":"May/June","monthNum":5,"headline":"Fatal highway","kicker":"TRANSPORTATION","author_name":"Tony Manolatos","author_title":"","affiliation":"FLORIDA TODAY","summary":"A Florida newspaper's investigation into deadly crashes on Interstate 95 revealed that fatalities had doubled over five years, prompting state and local officials to take action after the publication of a comprehensive 10-page report analyzing nearly 30,000 crashes.","keywords":["safety","investigation","highways"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":[],"full_text":"Fatal-crash stories along Interstate 95 were almost becoming routine. During a six-month period last year, 12 people were killed in 11 separate accidents on Brevard County's 73-mile stretch of Interstate 95. One New York family, who lost their parents in a March crash, wrote FLORIDA TODAY and pleaded with reporters and editors to take the coverage beyond spot-news stories. They wanted to know the reasons why their parents and many others were dying on I-95. They wanted the newspaper to illustrate the human toll. We, too, saw the potential of the story. But first we had to determine just how dangerous the interstate, one of two main arteries in Florida, really is. A team of reporters and editors began culling and analyzing state data from nearly 30,000 crashes over five years along Florida's 382 miles of I-95. In an initial 10-page report on Dec. 3, 2000, we delivered the news that the number of people killed on the Florida highway had doubled over five years.\n\nGathering data for I-95 required working with four state transportation districts and compiling data from each district into one database.\n\nThe data were startling enough, but the process also was revealing and newsworthy. We learned that money was the biggest culprit impeding enhancements to the road. But we also learned that while state transportation officials collect crash data, an aging computer system does not allow those officials to thoroughly analyze the information to identify the causes of crashes.\n\nIn the 10-page report (called \"I-95, More Dangerous Than Ever\") and the dozens of stories, editorials, reader letters and a public forum that followed, enough urgency was raised to compel state and local officials to take action. It was impossible to ignore the findings of the report, including:\n\nIn 1996, there was a sudden and dramatic increase in the number of fatal crashes on I-95. That was the year the speed limit was raised to 70 mph from 65 mph.\n\nThe death rate per I-95 mile driven in Brevard County, home to FLORIDA TODAY, is more than twice the national average for interstate highways.\n\nAlmost a third of the fatal crashes in Brevard County occur when vehicles veer out of control across grass medians and into oncoming traffic. Those \"crossovers\" lead to more multiple-death crashes than any other type of accident. They could be prevented by erecting median barriers in high-accident locations - a fairly simple, low-cost solution.\n\nComparisons\n\nOne thing the project did not do was compare I-95 to another road. We did gather fatality rates for U.S. interstates and Florida interstates and we compared those to I-95. But we did not, for example, look specifically at Interstate 75 in Florida.\n\nWe chose to compare I-95 to itself because it is Florida's eastern gateway and the most heavily traveled road in the state. However, taking the time to compare it to I-75, Florida's other major highway, would have added depth and perspective to our report.\n\nGathering data for I-95 required working with four state transportation districts and compiling data from each district into one database. Because each district is independent of the other, no one at the state level can effortlessly answer basic questions - such as the number of people killed in any given year - about I-95, I-75 or any other road that runs through multiple transportation districts.\n\nState transportation officials said a new central computer system, expected to be in place by July, would give them a better and quicker understanding of state-maintained roads.\n\nWhile the National Highway Traffic Safety Administration tracks all fatal wrecks in its Fatal Accident Reporting System database, the agency essentially relies on police reports for the information. Those reports often list multiple names of one road and NHTSA's database does not take that into account. In other words, you would have to know every possible name that law enforcement might use for a road in order to properly crunch the numbers.\n\nAnalysis\n\nAnthony DeBarros, a database editor for USA TODAY, cleaned up the data, compiled it into one database and assisted with the number crunching. We used Microsoft Access. The database was 13 megabytes. We also used Microsoft Excel for some import work. The data was supplied to us at no cost.\n\nState transportation officials said a new central computer system, expected to be in place by July, would give them a better and quicker understanding of state-maintained roads.\n\n\"The data was in fixed-width text files, but we discovered that one of the state zones was missing a couple of years of data, so we had to get that data and add it to our database\" DeBarros said.\n\n\"In terms of cleaning, all the data was presented in code, and I spent considerable hours going through and converting those codes into meaningful text that the reporter could understand. Also, we had to figure out how to piece together (separate data from four districts) so we could get a picture from end to end.\"\n\nThe CAR work took about two months to complete as a side job. The project took about four months to complete.\n\nDeBarros mentioned that while we did compile traffic data for I-95, using it to determine fatality rates, reporters who do similar projects should keep in mind that traffic volume must be figured into the equation before a stretch of road can be labeled the \"deadliest.\"\n\nReaction\n\nReaction to the coverage has been swift and substantive. Here's what has resulted so far:\n\nBrevard County legislators pledged that I-95 safety would be their top priority during the 2001 legislative session. They backed that pledge by setting a meeting in January with the state secretary of transportation to push for median barriers along I-95 in Brevard County.\nThe transportation secretary hired a traffic engineering company to analyze Brevard County's 73-mile stretch of I-95. The company is expected to examine 10 years of crash records.\nThe Metropolitan Planning Organization, the chief transportation planning agency in Brevard, said it would use our report to build a public awareness campaign to improve safety on Brevard's stretch of I-95.\nIn South Florida, Broward County officials initiated a public awareness campaign to alert drivers to the dangers of I-95 in that county. Transportation officials in Broward are continuing to study I-95 crash records in an area the newspaper identified as the stretch with the most fatal wrecks and the most traffic.\nAnd state highway patrol officials are using the newspaper's finding to lobby vigorously for more high-visibility enforcement along the entire interstate, including more marked and unmarked cars, increased DUI enforcement and aircraft speed-enforcement details.\n\nTony Manolatos can be reached by e-mail at tmanolat@flatoday.net\n\nTHE FLORIDA TODAY story, \"I-95: More Dangerous Than Ever,\" is available from the IRE Resource Center, story 17007. To order, call (573) 882-3364.\n\nData: Accident reports from four Florida transportation districts\nSoftware: Microsoft Access and Excel","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-truck-safety","slug":"2001-mayjune-truck-safety","url":"https://dwillis.github.io/uplink/article/2001-mayjune-truck-safety.html","year":2001,"month":"May/June","monthNum":5,"headline":"Truck safety","kicker":"TRANSPORTATION","author_name":"Andy Lehren","author_title":"Reporter","affiliation":"Dateline NBC","summary":"An investigation into trucking safety reveals that many trucks avoid inspections by using back roads, and analysis of multiple federal and state databases shows that the official one-in-five out-of-service rate significantly underestimates actual safety problems in the industry.","keywords":["trucking","inspections","safety"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"During a lull at a truck inspection station in rural Florida, I took a drive a few miles up the road. It wouldn't take much guesswork to figure out what I saw. Trucks. Lots of trucks. Some pulled over the side of the road. Others getting off at exits that had little discernible lure for commercial traffic.\n\nThey were doing whatever they could to avoid inspections. They were driving down tiny main streets, past scattered homes. They trundled by schools and farms. You could hear them speaking on CB radio about the best way to navigate the back roads so they could pop back on the interstate, past the inspectors. Some would play possum by pulling into a parking lot, waiting for the inspection station to close.\n\nWelcome to one of the things you won't see when you look at truck inspection data. When trucks are inspected, one of five is pulled out of service. A lot of the time it's for brakes that won't work right when there's an emergency; stuff that leads to accidents. When you see trucks pulling off the road to avoid inspections, you know there are a lot of inspections that never happen. That one-out-of-five figure is only a baseline.\n\nDateline NBC obtained more than a dozen databases to examine the problems with trucking safety. These included federal and state databases on accidents, inspections, truck carrier profiles, safety ratings and enforcement actions. The sources included the U.S. Office of Motor Carriers (now called the Federal Motor Carrier Safety Administration), the National Transportation Safety Board, the U.S. Bureau of Transportation Statistics, and transportation and public safety agencies in several states, including Florida, Missouri, Texas and California.\n\nThe reason we undertook such a thorough approach was to overcome some of the incompleteness and unreliability that happen if you relied solely on one of these databases. The result allowed the news magazine to more fully document the problems in trucking safety, in a story produced by Lynn Keller with correspondent Steve Daniels. It documented how many trucking companies systematically ignored safety rules and put bad trucks on the road, and how regulators fall short monitoring them.\n\nStudies and reports\n\nA starting point for our work was collecting various reports and studies that had been done on trucking safety, to establish sound methodologies. It also helped us gauge how well different states handle truck safety. California, for instance, is much better than Arkansas.\n\nThe simplest analysis was tracking which companies had unusually high out of service rates when their trucks were inspected. This is where having state data becomes important. Federal truck inspection data is only required to include interstate carriers. One in ten trucks on the road are intrastate truckers. They aren't licensed to go into other states. So their inspection records are not included in federal databases even if they are part of a subsidiary that's an interstate carrier. These trucks tend to face less scrutiny. It's also one of the things you won't see when you check a company's history off the U.S. Department of Transportation's web site (www.safersys.org).\n\nSome states offer far more details in their inspection and accident data. Those additional columns of data may tell you more about the driver or more easily identify a particular truck. The federal Fatality Analysis Reporting System is also good for double-checking truck accident data.\n\nSafety scores\n\nSeveral of these databases were obtained under the Freedom of Information Act and state open records laws. The most significant victory was a FOIA request that Dateline NBC won, on appeal, with the U.S. Office of Motor Carriers. We were the first news organization to obtain a recently developed database known as SafeStat, short for the Motor Carrier Safety Status Measurement System. It's worth getting, and should be easy to obtain for any news organization seeking it these days.\n\nSafeStat includes numerical grades on the safety of trucking companies. The scores are on a sliding scale, and allow you to compare different companies with scores for four areas: driver, vehicle, accident and safety management.\n\nDateline NBC used the FOIA to also obtain the safety classifications kept by federal regulators in a separate, older system, as part of the federal motor carrier census. These are three letter grades - satisfactory, conditional and unsatisfactory. Just like the statistics books tell you, there's a lot of subtlety that's lost when data is categorical. Out of service rates, accident rates and SafeStat scores give you a much better sense of how much safer one trucking company is versus another. That sense of proportion is lost when information is telescoped into three grades.\n\nBut the older three-grade system is still worth a look. The grades are one more way to cross-check the different rates. It also gives you a sense of a company's history. Very few are unsatisfactory. But experts say that in this somewhat lenient system, keep your eye on trends. If a company is unable to maintain a satisfactory grade consistently over the years - say, it goes from unsatisfactory to satisfactory to conditional and then again conditional - it's worth checking out.\n\nOne way we used SPSS was to run a linear regression comparing the size of truck companies versus their serious accident rate, then comparing companies against like competitors. Reporters should look for outliers - which companies have higher than expected accident rates. (There is much in the literature on using numbers of trucks versus miles traveled, generally a less reliable number. Reporters should be aware of these factors, run the data both ways, and compare results).\n\nMapping\n\nWe used mapping software several ways. Perhaps the most interesting was plotting accident and inspection data to figure out which inspection stations we wanted to visit, to see how regulators handled their workload and dealt with companies that had a pattern of problems. That meant we were there when a military munitions truck was pulled out of service for not being safe to drive.\n\nMapping also helped us analyze whether a trucking company was inspected more in certain states, and whether those states had lax or stringent inspections. One large trucking carrier, for instance, scored well in its home state, and usually scored much worse elsewhere in the country.\n\nHaving all this data allowed us to challenge trucking industry claims about safety. We knew which companies' paper records for on-site inspections we wanted to obtain. Enforcement data helped underscore the contrast between the problems on the roadways and the declining actions taken by regulators.\n\nThe data also helped us make sure we had a strong reason for highlighting the different companies and accidents that were profiled in the story.\n\nAndy Lehren can be reached by e-mail at alehren@nbc.com\n\nThe Dateline NBC story, \"Truck Stop,\" is available on videocassette, story #16152.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-bus-driver","slug":"2001-mayjune-bus-driver","url":"https://dwillis.github.io/uplink/article/2001-mayjune-bus-driver.html","year":2001,"month":"May/June","monthNum":5,"headline":"Bus driver","kicker":"FIRST VENTURE","author_name":"Dan Feldstein","author_title":"Projects Desk Reporter","affiliation":"Houston Chronicle","summary":"A journalist used database analysis to investigate overtime practices at Houston's Metropolitan Transit Authority, discovering drivers were working excessive hours and earning over $100,000 annually, raising questions about the legality and safety of the practice.","keywords":["overtime","database","safety"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["Excel","Access","SQL"],"full_text":"William \"Bones\" Pierson is a legend among bus drivers at Houston's Metropolitan Transit Authority.\n\nHe routinely drives 19-hour days and seven-day weeks. He didn't take a day off between Aug. 21 and Nov. 4.\n\nHe earned $109,000 last year, more than tripling his base salary through overtime. Dozens of other drivers were close behind.\n\nI hadn't known about Bones or the others until the computer tipped me off. Fresh out of NICAR's January boot camp, I wanted something simple and comfortable for my first venture.\n\nI ranked them in descending order and found my first real problem: The top guy drove 26 hours in one day. Good trick.\n\nI covered transportation before I went to the Chronicle's projects desk, so I picked on Metro - the Metropolitan Transit Authority. I'd heard that some of its drivers were racking up big bucks driving overtime.\n\nI had four questions: How much were they earning? How much were they driving? Was this legal? Was this safe?\n\nI requested the record layout of any database that contained time records, pay records or accident records. Neatly, they had all three. Each was separate, but each had a field for driver ID and a field for date. I requested them all for the calendar year 2000, plus a week on either end.\n\nThey came in Excel worksheets and I carefully put them into Access. The pay was trickier than I had expected. There were nearly a million records for only about 1,500 drivers because each type of pay was listed in a different record. On any one day, a driver could have one record for regular pay, one for overtime pay, one for a monthly bonus, etc. Some simple SQL gave me a list of the top earners.\n\nList of adjustments\n\nThe time records were next. They came by driver, by day. These were \"platform minutes,\" the actual time the driver was on the street, behind the wheel. I ranked them in descending order and found my first real problem: The top guy drove 26 hours in one day. Good trick.\n\nOh yeah, said the Metro computer guy. You need the \"list of adjustments.\" Every bus route has a set time. But if you get stuck in traffic, time can be tacked on. And if you get ill and leave early, time can be deducted. Plus other errors are corrected.\n\nIt turned out there were roughly 50,000 adjustments for about 350,000 records, which is when I learned about \"outer joins.\" Using driver ID and date, I tacked the adjustments onto my time table in a new field. Then I summed that field with the original time field in a query to create a new master table.\n\nIt turns out the top shift was 21 hours behind the wheel, not 26 - so much more reasonable.\n\nChecking the law\n\nWas all this driving legal? The feds allow interstate commercial drivers no more than 10 hours behind the wheel each day, or 60 hours in seven days. Texas allows intrastate drivers 12 hours a day or 70 in seven days. But both governments exempt government drivers.\n\nStill, 16 percent of all shifts driven by Metro last year would have violated the federal safety standard, and 5 percent would have violated the state standard.\n\nFor the single-day standard, it was easy to calculate. How many records were over 10 hours, over 12 hours?\n\nThe seven-day rolling total was wicked, however. It didn't exist, and I would have to create it. On every given day, how many hours had the driver been driving in the previous seven?\n\nHelp from NICAR-L\n\nI turned to the NICAR listserv. Three people offered suggestions. Tim Henderson of The Miami Herald offered this:\n\nSELECT a.driver_id, .work_date, sum(b.minutes) as rollingminutes FROM Platform as a, Platform as b WHERE ((([b].[driver_ID])=[a].[driver_ID]) and ((b.work_date) between ([a].[work_date]-7) and ([a].[work_date]-1))) GROUP BY a.driver_id, a.work_date\n\nIt worked! I also tweaked it slightly to get the last six days plus the current day. Finally came the question of whether the excessive driving was more dangerous than regular shifts. Experts said yes, but was there statistical evidence? Actually, it didn't show up in my crude analysis. I joined the time and accident tables, using driver ID and date. I found that, generally, the percentage of accidents attributed to drivers who had exceeded drive-time standards was the same as the percentage of shifts that were driven in excess of those standards.\n\nJumping into a project immediately after boot camp really solidified SQL in my mind. It would have escaped otherwise.\n\nAnd frankly, it took me as long to refine that statement into a single sentence as it did to make the calculations.\n\nLessons learned\n\nThis story could be replicated in a lot of cities. Transit systems have a tough time making new hires because of drug tests and a good economy. They are as happy with overtime as the drivers are.\n\nI learned several lessons in this project. While the data looked nice and neat, there were certainly some individual entries that had errors. Some of the accidents simply didn't join with the time table, for instance. The buses weren't being driven by phantom drivers - people had entered wrong dates, wrong driver IDs, etc., into the database.\n\nI kept my analysis broad because of that. And when I generated a list of the top 10 longest shifts that I was going to print, I gave it to Metro to check. Sure enough, they found a couple that were in error - a guy got sick, someone else hopped on his bus and their times were merged, etc. But most were correct.\n\nSQL solidified\n\nOther important things: Jumping into a project immediately after boot camp really solidified SQL in my mind. It would have escaped otherwise.\n\nAlso, what little I know about databases puts me one step ahead of most flaks. I made my best progress by suggesting a conference call among the flak, Metro's computer guy and me. It helped enormously for the flak to hear the computer guy say, \"Oh that's easy.\" It took two or three weeks to get the data, free of charge.\n\n\"Bones\" owns a nightclub and a Lexus that he doesn't get to drive much. Sadly, his wife told him to choose between marriage and driving so much overtime.\n\nThe new bachelor still has his pedal to the metal.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-water-roads-and-more","slug":"2001-mayjune-water-roads-and-more","url":"https://dwillis.github.io/uplink/article/2001-mayjune-water-roads-and-more.html","year":2001,"month":"May/June","monthNum":5,"headline":"Water, roads and more","kicker":"TRANSPORTATION","author_name":"Devon Hammes","author_title":"","affiliation":"IRE and NICAR","summary":"A Fort Worth Star-Telegram reporter used transportation datasets to identify the most dangerous driving routes in Texas by normalizing accident data against traffic volume, demonstrating how data journalism can uncover meaningful safety insights.","keywords":["transportation","data","accidents"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["FoxPro","Excel"],"full_text":"If you are seeking out a topic for an in-depth investigative piece, or if you just need that nugget of information to follow up on the most recent plane crash, transportation data has you covered.\n\nStory ideas abound in data such as hazardous materials, boating accidents, truck accidents or any of numerous aviation datasets.\n\nReaders of the Fort Worth Star-Telegram learned where some of the most dangerous routes in Texas were thanks to one reporter's use of data.\n\nJeff Claassen used Microsoft FoxPro and Excel to examine the dangers of driving on Texas highways and interstates. The paper specifically wanted to find where drivers in north Texas and statewide were most likely to encounter a bad accident.\n\nWith help from transportation experts, Claassen devised a method to use average daily traffic figures to calculate a rate of serious wrecks per volume of traffic.\n\nOne thing to keep in mind when working on any story that relies on statistics is that it is necessary to normalize the data. To normalize his data, Claassen took into consideration the number of vehicles travelling on a specific roadway.\n\n\"Other newspapers had analyzed Texas accident data, but focused on the total number of fatalities or fatal wrecks for each segment of a freeway or highway, called a 'control section,'\" Claassen said. \"That seemed incomplete because the fact that one roadway had more fatalities or fatal wrecks than another could just be the result of having more traffic. Also, the control sections varied in length from as short as a mile to more than 30 miles, making them difficult to compare to each other.\"\n\nWith help from transportation experts, Claasen devised a method to use average daily traffic figures to calculate a rate of serious wrecks per volume of traffic. He also broadened the story by including serious injury accidents.\n\nThe paper used accident report data from the Department of Public Safety, which consisted of about 1.9 million records from 1993 to October 1999 (the most recent available). The Fort Worth Star-Telegram also obtained the Road Inventory file from the Texas Department of Transportation, which provided average daily traffic figures for the roadway \"control sections.\"\n\nBut this is where some problems began. \"Our first roadblock was incomplete data. The DPS data notes the location of some accidents using 'control sections,' and others using city accident-location codes. That omitted important places like Houston and San Antonio from the mix, so we requested more data to fill in the holes,\" he said.\n\nClaassen then focused the investigation by analyzing wrecks from January 1997 to October 1997 to give the study more accuracy. Transportation analysts told him that time frame was an accurate look at the average number of accidents. State officials warned Claassen that the distance of control sections sometimes change. So, with help from the transportation institute, he was able to develop a method to calculate the rate of serious injury and fatal wrecks.\n\nCable cars\n\nEven if a story doesn't start from the data, transportation stories can be found anywhere in the city. For example, Dan Noyes of KGO-TV in San Francisco took an in-depth look at cable car safety. The story found that the cable cars are not as safe as many people might think.\n\n\"I decided to look into the cable car story after seeing several accidents over the months,\" Noyes said. \"I wondered how this piece of San Francisco history compared to other forms of transportation.\"\n\nUsing accident reports from the California Highway Patrol, Noyes was able to put together a spreadsheet in Excel. With that spreadsheet, he analyzed how many accidents occur per million passenger miles traveled for the different modes of transportation -- car, light rail, bus and cable car. The biggest problem KGO-TV had was getting authorities to realize that his station had a right to the information.\n\nBoating accidents\n\nKarl Ebert of The Northwestern in Oshkosh, Wis., used the federal Recreational Boating Accidents Database to look at the number of boating accidents on Wisconsin waters. He found several instances of accidents that were attributed to alcohol and the boater's inexperience in maneuvering the vessel.\n\n\"People up here love three things: booze, boats and speed. Each summer we report stories about boat accidents, most of them brought on by bad behavior of one sort or another, and decided that we needed to look into the issue to see just how big a problem there actually is out on the water,\" he said.\n\nThe data did help give the story more accuracy, but Ebert ran into a few problems along the way.\n\n\"The biggest problems were the inability and in some cases unwillingness of county officials and local wardens to provide information about tickets that had been written on the water. Because of time and publication scheduling constraints we decided to go with more anecdotal information. That was frustrating, but I don't think it hurt the main thrust of the story,\" Ebert said.\n\nThere were also instances in which he found some problems in the data itself. There was a change in the data's format from several years ago, and because of that, not all of the fields matched up. He also found that a field indicating alcohol involvement was unreliable in historical data because it hadn't been consistently reported.\n\nDevon Hammes can be reached by e-mail at devon@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-data-galore","slug":"2001-mayjune-data-galore","url":"https://dwillis.github.io/uplink/article/2001-mayjune-data-galore.html","year":2001,"month":"May/June","monthNum":5,"headline":"Data galore","kicker":"TRANSPORTATION","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR offers transportation datasets including FAA and NTSB aviation records that journalists can use to develop stories, with pricing tiers based on news organization size.","keywords":["aviation","datasets","journalism"],"topics":["Public Safety","Data & Databases","Technology"],"technologies":["World Wide Web"],"full_text":"NICAR offers numerous helpful transportation datasets to generate story ideas, add depth and authority to daily stories, and to develop enterprise stories.\n\nPrices are based on the size of your news organization. Journalism professors and students fall into the smallest category.\n\nMore information about these datasets, including sample slices and record layouts, is available at www.ire.org/datalibrary/datasets\n\nAir\n\nFAA Enforcements: Actions against airlines, pilots, mechanics and their designees, including all types of planes. Data are available from 1962 through August 2000 for $25, $50 or $75, depending on circulation or market size of your news organization.\n\nFAA Service Difficulty Reports: Contains information on mechanical difficulties collected by the FAA for the purpose of tracking repair problems with commercial, private and military aircraft and aircraft components. Reporting is voluntary, so the database is considered incomplete. Data is available from 1974 through December 2000 for $65, $130 or $155.\n\nFAA Accidents and Incidents: Contains information mainly about U.S. flights where there was an accident or an incident, such as crashes, collisions, deaths, injuries, major mechanical problems or costly damage. Data is available from 1973 through about April 2000. Cost for the most recent data, 1990 through 2000 is $40, $80 or $105.\n\nFAA Aircraft Registry: Contains a listing of all aircraft and aircraft owners registered in the United States. This dataset also includes tables on registered aircraft dealers and individuals or companies that reserved the N-number for their plane. The registration table in this dataset can be used to identify the registered owner of an aircraft by searching on the registration number (commonly called the N-number). NICAR updates its data every three months. The most current version, February 2001, is available for a one-time purchase for $25, $50 or $70 or an annual subscription, which includes four updates, for $50, $100 or $125.\n\nFAA Airmen Directory: A listing of pilots and other airmen, including the type of certificate(s) they hold and their ratings. As a result of a new law that allows airmen to have their information withheld from the public, this listing is incomplete. This data, current as of October 2000, includes about 65 percent of the total. NICAR updates its data every six months. The most current, November 2000, is available for $30, $60 or $80.\n\nNASA Air Safety Reporting System: Anonymous reports of airplane safety submitted by pilots, flight attendants, air traffic controllers and passengers. Anyone is eligible to file an ASRA report, and pilots appear to be the most frequent filers. Reports might include month, year and day of the week, but it will not include the exact date. Data is available from 1988 through 1999 for $65, $130 or $155.\n\nNTSB Investigations of Aircraft Accidents: Contains information on civil aviation accidents and incidents, including pilot, crew and airplane descriptions and environment conditions within the U.S. its territories and in international waters. Data is available from 1984 through October 2000 for $75, $150 and $175.\n\nRoads\n\nDOT Fatality Analysis Reporting System: A good tool to analyze the number of fatal accidents on a specific roadway, in your state or in the nation. FARS is a nationwide database of fatal road-vehicle accidents, truck and trailer accidents are included. The data files cover 1975, when the DOT first began tracking fatal accidents, through 1999. It is available from NICAR for the entire U.S., most recent year at $40, $80 or $105. A file containing 1994-1999 data merged together by NICAR is available for $45, $90 or $115. Earlier years are also available separately.\n\nNHTSA Vehicle Recalls and Complaints: A database of vehicle complaints, recalls, service bulletins and inspections. This data is available for the entire U.S., all years and costs for $40, $80 or $105.\n\nDOT Truck Accidents: A database of accidents on U.S. roads involving a commercial vehicle weighing more than 10,000 pounds, including semi-tractor trailers, buses and hazardous material carriers. This data is available for the entire U.S., all years or by each state, all years and costs $90, $180 or $205 for the entire U.S. or $30, $60 or $85 for one state.\n\nTruck Census: This U.S. Department of Transportation database contains records on each company that has commercial interstate vehicles weighing more than 10,000 pounds. The data is available for the entire U.S. for $95, $190 or $215 or for one state for $30, $60 or $85.\n\nHazardous Materials Incident Report Subsystem: The database contains the incident reports of unintentional releases of hazardous materials for all modes of transportation (air, highway, railway, water). Among other things, the data allows reporters to determine which hazardous material carriers are having the most accidents and in which states. Chemicals most often released and exact accident details are also recorded. The report number column has unique case numbers, making it possible to join different databases for more in-depth investigations. NICAR has this data from 1971 through May 2000. This data is available for the entire U.S., all years, for $50, $100 or $125.\n\nWaterways\n\nBoating Accidents: The database contains information on recreational boating accidents in the United States. The data is available for the entire U.S., all years for $25, $50 or $75.\n\nBoat Registration: The database contains information on registered recreational and commercial boats.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-outside-the-gates","slug":"2001-mayjune-outside-the-gates","url":"https://dwillis.github.io/uplink/article/2001-mayjune-outside-the-gates.html","year":2001,"month":"May/June","monthNum":5,"headline":"Outside the gates","kicker":"FIRST VENTURE","author_name":"Brian D. Crecente","author_title":"Reporter","affiliation":"The Palm Beach Post","summary":"The Palm Beach Post's investigation found that the gated city of Atlantis issued traffic tickets almost exclusively to non-residents (99.3 percent) after taking control of three boundary roads, with fine collections jumping 1,067 percent in five years. The story required extensive data collection and database analysis from court records and police files to uncover the discriminatory ticketing pattern.","keywords":["tickets","investigation","database"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Access","Excel"],"full_text":"A year of investigation, database entry and query building boiled down to just two numbers: 1,067 and 99.3. But those numbers made the story. The Palm Beach Post found that in 1997 and 1998, the tiny gated city of Atlantis quietly took control of three major roads framing its boundaries and began churning out tickets to nearly everyone but its own residents. In one year's time, 99.3 percent of all tickets written by city police went to out-of-town drivers. In the past five years, Atlantis fine collections jumped from about $5,400 to more than $63,000 - a 1,067 percent increase.\n\nDespite Florida's wonderful sunshine laws, getting to those numbers wasn't as easy as it could have been. The story began as we followed a tip from a disgruntled ex-Atlantis cop who alleged tickets were being issued on racial lines. We later found that not to be the case.\n\nMy colleague, Matt Mossman, made dozens of requests for traffic ticket data from the city, but officials said tickets weren't tracked and that there weren't any electronic records. Finally we went to the source, the Atlantis police chief, and requested copies of all traffic citations and warnings issued by the department from July 1998 to June 1999. The result of the request came in four large boxes, 1,506 tickets sorted by date and several hundred warnings.\n\nConstruction\n\nDatabase editor Christine Stapleton constructed a database in Access which included fields for everything from the officer's badge number to the specifics of the citation and when and where it was issued.\n\nMossman spent the next few months entering the data and then as his data entry drew to a close, left the paper.\n\nI was left with the enviable task of sorting through a well-organized database, filled with the sort of numbers investigative reporters dream about. There were about 2,000 records between Access and Excel tables.\n\nI knew we were on to something, but I had no idea how dramatic the numbers were going to be until I ran my first query.\n\nDigging in\n\nAfter a little fiddling around, I typed a query that would show how many of the tickets issued were to people whose driver's license showed them living anywhere but Atlantis.\n\nOf the 1,506 tickets analyzed, only 11 went to residents. The other 99.3 percent went to non-residents.\n\nWe had our story. The database also showed that the city's police seemed to spend a majority of their time outside the walls of their city, with all but 60 of the tickets written being for traffic stops on the three main roads framing the city's walls.\n\nAtlantis residents were also more likely to receive warnings than tickets when pulled over. About 78 percent of the Atlantis residents who where pulled over received warnings, while only 16 percent of the out-of-towners pulled over received warnings.\n\nIt was important not to stop with just the data we collected on speeding tickets. I also needed to understand how the city was able to annex the three roads that framed the city's walls.\n\nAfter nearly a month of talking to city, county and state officials, I discovered that the county and the state either didn't officially allow the annexations or didn't fully understand what the annexations meant. According to city records, Atlantis didn't have to do anything on the three newly-annexed streets, but write tickets. The county commissioner, after hearing this, quickly launched an investigation into the annexation and the city's policy of writing tickets outside of their city but not working crashes on those same streets.\n\nCrashes\n\nNext I went to the Palm Beach County traffic division to collect data on the number of crashes that occurred on the streets annexed by Atlantis.\n\nAgain, while I found the officials all too willing to comply with the sunshine law, the data was not in an electronic form. I found myself thumbing through oversized notebooks copying the data into my notepad, which I later transferred to Excel.\n\nThe spreadsheet showed that while Atlantis police spent a lot of time patrolling the streets for speeders, they hardly ever worked crashes that occurred there. Of the 733 crashes that occurred on those three roadways from 1996 to March 2000, Atlantis police worked three, but none of the five fatal ones.\n\nIn an interview, the police chief didn't admit to the over-enforcement, but simply said his residents didn't speed. He added that the department's written policy stated officers are supposed to spend only 30 minutes per eight-hour shift parked outside the city walls. He also said that the reason for the city's enforcement was because they wanted to reduce crashes and that cars passing his city were not only speeding they were going well over the limit. After getting back to the office, I ran a few quick queries to check out the chief's answers.\n\nChief's answers\n\nI found that despite the increased traffic enforcement in the area, crashes along the three roads went up about 15 percent while the amount of traffic in that area went up by only 9 percent.\n\nI also found at least four officers who broke the 30-minute rule, showing up in the database for writing tickets over as much as a five-hour period.\n\nI set up another interview with the chief. When I told him what I found he said that his officers had gotten so over-zealous in their pursuit of traffic tickets he had to issue a moratorium on writing tickets.\n\nHe also said that inside the city's limits police follow a policy that allows officers to ticket someone going only 5 to 9 mph over the limit, if they have received a prior warning.\n\nLuckily, we had created a separate Access database of all of the warnings issued over the year, so when I got back to the office I ran a query for all tickets issued inside the city to people traveling less then 10 mph over the limit and then ran that list against our database of warnings.\n\nI found 14 occasions where officers ticketed drivers who were traveling 9 mph over the speed limit but had no prior warning.\n\nAgain I met with the chief. He said that 11 of the 14 tickets were written by two officers who are no longer with the department and that the problems were anomalies.\n\nThe end result of the data entry, queries, spreadsheets, and back-and-forth interviews was a two-story package.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-tread-dangers","slug":"2001-mayjune-tread-dangers","url":"https://dwillis.github.io/uplink/article/2001-mayjune-tread-dangers.html","year":2001,"month":"May/June","monthNum":5,"headline":"Tread dangers","kicker":"TRANSPORTATION","author_name":"David Raziq, Anna Werner, and Chris Henao","author_title":"","affiliation":"KHOU-TV, Houston","summary":"Journalists investigate a pattern of fatal accidents involving Firestone Radial ATX tire tread separations on Ford Explorers, compiling court records and accident reports after discovering government databases and manufacturer data were unavailable or proprietary.","keywords":["tires","accidents","investigation"],"topics":["Investigative Methods","Public Safety","Business & Economy"],"technologies":["World Wide Web"],"full_text":"The story began with a simple conversation: a lawyer told us of a fatal accident involving something called \"tread separation.\" In this case, he claimed a woman was driving her Ford Explorer down a Texas highway when suddenly one of the car's tires, a Firestone Radial ATX, simply lost its tread. At that point, it was just a tip, but we decided to look into it further. Additional contacts with attorneys and others revealed more cases involving the same tire and car combination: Firestone ATX tires and Ford Explorers.\n\nObviously, our curiosity was piqued: How big was the problem? We found that some traditional resources didn't provide the answer. For example, the government's Fatality Analysis Reporting System database did not list tire brands or makes. And the tire manufacturers, who collected tire return and failure claim data considered that information to be \"proprietary.\" (Although such information was often turned over to plaintiffs' attorneys during lawsuits, it might be protected by court order.)\n\nPaper trail\n\nSo we gathered our own data by using, among other documents, court records and accident reports. We also interviewed a number of experts who were in a hands-on position to know what the industry's data on these tires might be like.\n\nWe knew the limits of our approach and our data wouldn't allow for any kind of statistical analysis, but we decided that that was not our goal in any event. In the first place, any real \"answers\" about the allegations would only be obtained through an objective and scientific analysis of both a full body of the industry's data along with a mechanical investigation of both the tire types and perhaps even the SUV model.\n\nTherefore, the point of our story was that there clearly seemed to be enough allegations to warrant an in-depth investigative broadcast asking whether or not there was a problem and to bring it to the attention of Firestone, Ford, the appropriate government officials, and the public itself. If that happened, then perhaps it would lead to an open and scientific look at the issue.\n\nWe went to work and soon sat down to discuss the cases we had accumulated - at the time, more than two dozen deaths that victims' families claimed allegedly resulted from tread separations on Firestone ATX tires mounted on Ford Explorers.\n\nA spokesperson for The National Highway Traffic Safety Administration told us agency officials were unaware of any pattern of accidents involving the tire and SUV, despite the dozens of complaints from Ford Explorer owners we had found on the agency's own Web site. (Although many of those complaints did not identify a specific model or even make of tire, by this time we knew that the Radial ATX had been original equipment on thousands of Explorers and many of the complaints appeared to be about tread separations.)\n\nOur findings were further supported in the first story, Feb. 7, 2000, by a former employee of Firestone's North Carolina tire plant, who told us about what he felt were quality control problems at the plant.\n\nResults\n\nThree nights later, as we were about to air a follow-up report, the president of our corporation received a multi-page letter from Firestone disputing our story. The letter called our report \"misleading.\"\n\nThe letter also called our story unbalanced and stated \"the broadcast may be grounds for finding of actual malice.\" We included the company's response in our February 10 story, reporting their most damaging allegations in the interests of fairness. We also posted the full letter on our Web site for viewers to read.\n\nHowever, we continued to investigate, eventually learning that NHTSA was doing its own internal investigation of complaints reported by our viewers to the agency. By May, the agency announced it was beginning a defect investigation into the Firestone ATX, ATX2, and Wilderness AT tires.\n\nAs we monitored the investigation, there was another break in the story. On July 30, a source called to let us know that since late May, Ford had been recalling ATX and Wilderness tires from Ford Explorers in Venezuela because of accidents similar to ones in the U.S. We even found out that Ford was planning to include similar recalls in Ecuador and Columbia. Within days, the national media jumped on board and coverage of the issue became widespread. Finally, just weeks later Firestone made the announcement it would voluntarily recall more than six million ATX, ATX II, and Wilderness AT tires.\n\nObviously the story didn't stop there. We also broke the story that Venezuela was considering criminal charges against both Ford and Firestone. And, of course, events here eventually led to congressional hearings, and the recent passage of a new law that among other things sets criminal penalties for manufacturers who don't address defects and forces them to tell the federal government when they recall products in other countries.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-ire-awards-car-winners","slug":"2001-mayjune-ire-awards-car-winners","url":"https://dwillis.github.io/uplink/article/2001-mayjune-ire-awards-car-winners.html","year":2001,"month":"May/June","monthNum":5,"headline":"IRE AWARDS CAR winners","kicker":"IRE AWARDS","author_name":"","author_title":"","affiliation":"IRE","summary":"Several IRE Award winners used computer-assisted reporting techniques including databases, mapping software, and data analysis to investigate stories ranging from insurance company practices to legislative conflicts of interest across multiple news organizations.","keywords":["databases","investigations","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["ArcView"],"full_text":"Several winners and finalists in this year's IRE Awards used computer-assisted reporting as a key part of their investigations. Two of those stories that won certificates, \"The Body Brokers\" by The Orange County Register, and \"Treading on Danger?\" by KHOU-TV in Houston, are featured in this issue [see pages twelve and eighteen].\n\nOther certificate winners: MEDIUM NEWSPAPERS: \"Detroit Fire Department: Out of service,\" The Detroit News, by Melvin Claxton and Charles Hurt. The reporters documented irresponsibility and incompetence in the Detroit Fire Department, outlining myriad flaws - malfunctioning fire trucks, broken hydrants, closed stations - that contributed to deaths. They built databases off paper records that included dispatch tickets, run books, maintenance sheets, out-of-service reports and daily attendance records from each firehouse. IRE Resource Center story #16983.\n\nNETWORK TELEVISION: \"The Paper Chase,\" Dateline NBC, by John Larson, Lynne Dale, Allan Maraynes, Neal Shapiro, Andy Lehren and Mable Chan. Dateline's investigation documented how insurance companies used people with no medical training to write up reports that routinely denied patient claims after accidents. Data work included creating databases of internal company documents and medical reports, obtained from confidential sources. They used ArcView to analyze maps of State Farm district and regional offices. A free-text base was used to look for patterns in paper documents. IRE Story #17394.\n\nONLINE: \"Our Private Legislatures - Public Service, Personal Gain,\" Center for Public Integrity, by Diane Renzulli, Meleah Rush, John Dunbar, Alex Knott, Robert Moore and Ken Vogel. The Center for Public Integrity investigated conflicts of interest involving legislatures in all 50 states. It found more than 20 percent of state legislators sat on committees charged with regulating the legislators' professional or business interests. They built a database from financial disclosure statements for more than 7,000 lawmakers, and another database of employers of state-registered lobbyists. IRE #17325.\n\nSTUDENT: \"Cycle of Influence: How Campaign Contributions, Lobbyist Spending and Personal Financial Interests Affect the WV Legislature,\" Charleston Gazette/University of Missouri, by Scott M. Finn. The report showed through many different windows how personal and business conflicts affect legislation in West Virginia. For the data work, he used databases of personal financial interests of state legislators, West Virginia lobbyist spending, state lobbyists and their employers, and campaign contributions to state legislators. IRE story #17208.\n\nThe full list of winners and videostreamed excerpts of television category winners are available at www.ire.org/history/pr/2000winners.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-more-resources","slug":"2001-mayjune-more-resources","url":"https://dwillis.github.io/uplink/article/2001-mayjune-more-resources.html","year":2001,"month":"May/June","monthNum":5,"headline":"More Resources","kicker":"TRANSPORTATION","author_name":"Dawn Fallik","author_title":"Reporter","affiliation":"St. Louis Post-Dispatch","summary":"A tipsheet compiled by a St. Louis Post-Dispatch journalist providing useful transportation research websites and resources for journalists covering vehicle crashes, accidents, and trucking industry data.","keywords":["transportation","data","resources"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["World Wide Web","Internet"],"full_text":"Dawn Fallik of the St. Louis Post-Dispatch compiled a tipsheet of useful transportation Web sites for the 2000 Lexington CAR Conference. Below are some of the links she included. For a complete version of this tipsheet, call the IRE Resource Center, (573) 882-3364 and ask for tipsheet#1290. Or, download it from www.ire.org/reourrecenterl.\n\nGeneral Crash/Explosion/Accident data\n\nNational Transportation Data Archive (NTDA). www.bts.gov/ntdal\nUniversity of Michigan Transportation Research Institute: The institute has a lot of projects, data and research. wrwww.research.umich.edulresearchl proposals/proposal_deul UM_Resources/UMTRL.html\nVolpe Center: Part of the U.S. Department of Transportation's Research and Special Programs Administration. This Web site includes links to many government transportation Web sites. www.volpe.dot.gov/resrefllinksdir.html\n\nTrucking\n\nMotor Vehicle Carrier Association. Has statistics for commercial trucking. www.safersys.org\nBureau of Transportation Statistics: Provides North American merchandise trade data by commodity type and surface mode of transportation with geographic detail for the U.S. www.bts.gou/transborderl\nTanker advisory center (oil spills): 212-628-7686","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-crash-scams","slug":"2001-mayjune-crash-scams","url":"https://dwillis.github.io/uplink/article/2001-mayjune-crash-scams.html","year":2001,"month":"May/June","monthNum":5,"headline":"Crash scams","kicker":"AUTO ACCIDENTS","author_name":"Fred Schulte and Jenni Bergal","author_title":"Investigative Editor and Senior Writer","affiliation":"South Florida Sun-Sentinel","summary":"A seven-month investigation into Florida's auto insurance fraud revealed over 4,000 residents involved in at least 1,400 suspicious accidents during the 1990s, exposing a network of accident staggers, corrupt medical professionals, and lawyers exploiting high billing limits to cheat insurers and consumers.","keywords":["fraud","insurance","accidents"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Access"],"full_text":"Just about everybody in the auto insurance business in Florida has a story about fraud, from rings that stage fake accidents to crash-care clinics that bill huge sums for needless treatments.\n\nUnfortunately, most insurance companies are wary of talking to the media. They don't want to discuss fraud schemes for fear of drawing a slander suit from those involved, or out of concern that doing so would reveal how vulnerable they are to cheaters. But a number of public records exist to help reporters get an inside look at this multibillion dollar criminal enterprise that investigators say operates in some form in nearly every state.\n\nOur seven-month investigation identified more than 4,000 Florida residents who played roles in at least 1,400 suspicious auto accidents statewide during the 1990s.\n\nIn \"Crashing For Cash,\" a four-part series published in December, we exposed an underworld of traffic accident stagers, corrupt chiropractors and pain clinic owners, ambulance chasing lawyers, patient solicitors and medical testing brokers who band together to cheat insurance companies and consumers.\n\nUnder Florida law, a doctor or clinic can bill up to $10,000 for every crash patient treated. Lawyers can tap insurers for thousands of dollars more in personal injury settlements. This fast money drives the industry. Our seven-month investigation identified more than 4,000 Florida residents who played roles in at least 1,400 suspicious auto accidents statewide during the 1990s.\n\nHundreds of them were chiropractors or lay owners of crash clinics who had been reported to state fraud investigators for padding bills or running needless medical tests on uninjured accident victims.\n\nOthers belonged to crash rings, whose members often were related by blood, ethnicity or neighborhood ties. We entered each name we encountered into one of seven Microsoft Access databases that allowed us to keep a running total of crash-for-cash participants.\n\nData sources\n\nHere are some of the primary sources we used:\n\nState Department of Insurance files. To our surprise, the Florida Division of Insurance Fraud allowed us to visit five of their offices around the state, where we inspected more than 500 closed cases, dating to 1995. Most had been closed without an arrest.\n\nPatterns quickly emerged.\n\nDozens of chiropractors and clinics allegedly were paying \"runners\" to recruit patients, sometimes offering them $300 to $600 apiece, for instance.\n\nIn other cases, solicitors obtained the names of people in minor accidents by purchasing crash reports in bulk at local police stations and calling them at home with promises that they could make thousands of dollars by claiming injuries in a lawsuit.\n\nIn addition, some people had obtained drivers licenses under several aliases. We decided to check their driving backgrounds.\n\nDepartment of Highway Safety & Motor Vehicles. We accessed driving histories and accident records for people investigated for insurance fraud, accident staging or possessing a phony drivers license, the criminal charge that often gets filed against accident stagers.\n\nWe entered this information into a database, which included the name of the driver and passengers in the accident, the speed at impact, damage amount and age of the other driver.\n\nThe data showed us that the same drivers were involved in questionable crashes over and over. Some were members of the same family. Some targeted elderly drivers as victims. One man with six aliases was in three dozen crashes. Eight people living in Palm Beach County wracked up 139 crashes. One family-based ring showed up in 68 suspicious crashes.\n\nWe used standard industry criteria for determining whether a crash was suspicious. For instance, often the accident was little more than a fender bender and the estimated damage to the vehicles was under $500. Yet the accident stager and his passengers would immediately go to a chiropractor's office and begin a series of expensive treatments.\n\nMany other crashes in our sample occurred in parking lots at about 5 miles per hour.\n\nCivil court files. Increasingly, auto insurers are refusing to pay bills they believe are inflated or bogus and are pressing their cases in court. Records of these civil lawsuits, especially depositions of doctors and clinic owners, are invaluable, both for locating patients and establishing questionable billing practices and medical testing abuses.\n\nOther court cases can be helpful as well. Insurers in some states, for instance, have filed class action suits against chiropractors and clinics, claiming they have deliberately inflated costs or billed for services that weren't rendered. Some of the biggest cases have been filed in Chicago and New Jersey.\n\nDisciplinary and criminal case files. We obtained a list of all Florida Board of Chiropractic Medicine disciplinary cases in the last five years and requested hard copies of cases that involved insurance fraud or patient solicitation, including penalties imposed.\n\nMore work\n\nWe combined those named in disciplinary cases with other chiropractors, doctors and clinics whose names surfaced in state insurance fraud complaints, arrest records or civil court cases for suspected accident fraud.\n\nThis list grew to more than 60 medical clinics in Miami-Dade County alone and more than 400 chiropractors, many of them reported to the state repeatedly for accident fraud.\n\nWe also examined the applications of chiropractors with criminal histories and found that the board had granted licenses to dozens of chiropractors from other states with criminal convictions including insurance fraud and cocaine trafficking.\n\nChecking the criminal histories of accident clinic operators, we also found a number of owners with felony convictions, including ex-drug dealers who had migrated to the crash treatment racket.\n\nIn reaction to the series, Florida lawmakers are expected to take up a number of legislative fixes, including one that would regulate auto accident clinics and limit their ownership to medical professionals.\n\nOther proposals include increasing criminal penalties for insurance fraud and accident staging and cracking down on runners who exploit crash victims on behalf of doctors and lawyers.\n\nJudging from the early reaction among state officials, we're glad we told this story.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-body-parts-industry","slug":"2001-mayjune-body-parts-industry","url":"https://dwillis.github.io/uplink/article/2001-mayjune-body-parts-industry.html","year":2001,"month":"May/June","monthNum":5,"headline":"Body parts industry","kicker":"USING THE WEB","author_name":"Ronald Campbell","author_title":"","affiliation":"The Orange County Register","summary":"Reporters investigated the $500 million body parts industry, uncovering unethical practices between nonprofit tissue banks and for-profit processors through business records analysis. By examining SEC filings, corporate contracts, and nonprofit financial forms, they revealed that tissue banks sometimes harvested body parts without consent and made misleading safety claims.","keywords":["investigation","nonprofit","tissue"],"topics":["Business & Economy","Investigative Methods","Health & Environment"],"technologies":["Excel","EDGAR","Guidestar"],"full_text":"When a team of reporters began work in October 1999 on the story that became \"The Body Brokers,\" we started with a vague idea: People were buying and selling body parts donated at death. CAR played an important role in uncovering the dimensions and practices of the $500 million body parts industry.\n\nAmong the findings of the five-part series: Nonprofit tissue banks work hand-in-glove with for-profit processors, often through exclusive contracts; the pressure to get more bodies is so strong that tissue bank employees and researchers occasionally take body parts without asking; and although the Food and Drug Administration and the industry claim a near-perfect safety record for tissue products, they aren't looking very hard.\n\nAlthough we explored many issues in our story - legal, political, even moral and religious issues - The Body Brokers was at heart a business story. Like any business, tissue banks and processors had to find raw material, win financial backing, develop technology, deal with pesky bureaucrats and market their products. Those universal business needs created paper and online trails.\n\nNonprofits\n\nWe were dealing with the intersection of nonprofit groups and for-profit, often public corporations. As a result, we often could get information from corporate documents that was unavailable from the nonprofits, and vice-versa. For example, we learned early in our investigation that the American Red Cross was among the nation's largest tissue suppliers. We knew it had a relationship with Osteotech, a New Jersey-based public company. The Red Cross wouldn't give us a copy of its contract with Osteotech. But Osteotech had included the contract in a filing with the Securities and Exchange Commission. It took just a few minutes to find the contract on EDGAR, the SEC's irreplaceable online database of corporate documents (www.sec.gov).\n\n(A hint for those seeking obscure corporate documents: Read the footnotes at the end of a company's 10-K or 10-Q. These will list not only documents attached to the current filing but citations to documents that were filed years ago, including the name and date of the relevant filing. A recent Osteotech filing pointed us to a 2- or 3-year-old filing that contained the American Red Cross contract.)\n\nWhen we began our research, Guidestar (www.guidestar.org) - a wonderful source for reporters on the nonprofit beat - had just begun posting Form 990's for hundreds of thousands of nonprofits. We had to get our 990s the old fashioned way, through a massive request to the Internal Revenue Service.\n\nCAR played an important role in uncovering the dimensions and practices of the $500 million body parts industry.\n\nOnce we got sufficient 990s and 10-K's, we assembled them into a series of spreadsheets. Although this information was a year to 18 months out of date, it gave us an excellent idea of the industries' finances and of the nonprofit/for-profit relationships that made everything work. This put us in a strong position when we interviewed industry executives; we knew too much to be given the standard industry spin on the \"gift of life.\"\n\nFederal regulation\n\nThe Food and Drug Administration was an indifferent regulator of the industry. But its Web site (www.fda.gov) still provided many valuable clues. I strongly recommend the agency's weekly online bulletin, which lists every FDA-sanctioned recall. Its online list of warning letters also is valuable. After discovering that a simple search on \"human tissue\" or \"bone\" missed many relevant recalls, I settled down to spend a day and a half reading every FDA bulletin from the beginning of 1994 (shortly after FDA began regulating tissue banks) through the end of 1999. This provided the starting point for our story on FDA's half-hearted enforcement.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-medical-conflicts","slug":"2001-mayjune-medical-conflicts","url":"https://dwillis.github.io/uplink/article/2001-mayjune-medical-conflicts.html","year":2001,"month":"May/June","monthNum":5,"headline":"Medical conflicts","kicker":"HEALTH CARE","author_name":"David Heath","author_title":"Reporter","affiliation":"Seattle Times","summary":"Journalists used computer programming and automated data searches through SEC records, patent databases, and other public sources to uncover financial conflicts of interest among researchers at the Fred Hutchinson Cancer Research Center when traditional public records requests were denied.","keywords":["conflicts","databases","research"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Visual Basic","EDGAR","Internet"],"full_text":"Once officials at the Fred Hutchinson Cancer Research Center figured out that we were working on a story that might not be flattering, they became quite secretive.\n\nWe wanted to know more about financial conflicts of interest in experimental trials, yet they flatly refused to give us any information about the financial holdings of top researchers or even the center itself. The fact that some of these doctors had financial entanglements with the treatments they were researching - or that patients died from these treatments - didn't sway them at all.\n\nThe state open records act was useless because the Hutch is a private institution, even though nearly 70 percent of its funding comes from federal grants. And an intriguing amendment to the federal Freedom of Information Act - one intended to open records on projects funded with government grants - was rendered worthless as well when research lobbyists prevailed in getting new regulations that severely undercut the law.\n\nWe seemed to be stuck.\n\nUnfortunately, there are no off-the-shelf databases on physicians' financial holdings. We knew that there was a lot of information to be gleaned from Securities and Exchange Commission records, patent records and the like, but we faced the nightmare scenario of reading through thousands - perhaps hundreds of thousands - of pages looking for names.\n\nThen it hit us. With a little bit of computer programming, reading through those documents could be automated.\n\nIt wouldn't be as easy as joining the sex-offender database to the day-care worker database. But with the wealth of data easily accessible on the Internet and the sophisticated tools we have available today, it was clearly doable.\n\nThe results appear as part of our five-part series, \"Uninformed Consent: What patients at the Hutch weren't told about the experiments in which they died.\" The series, written by Duff Wilson and myself and edited by David Boardman, documents two failed clinical trials in which doctors had financial interests and in which patients died from risky treatments without being properly informed of the risks.\n\nTo get information about financial conflicts, first we needed researchers' names. Then we would search for those names in a slew of financial documents from the SEC, an insider-trading database and the U.S. patents full-text database. We'd finally add our results to the few financial disclosure forms that we were able to obtain.\n\nHow we did it:\n\nFirst we headed to the library to gather up every biennial scientific report the Hutch has ever published. These included the names of their entire faculty, including post-doctoral fellows. We sent photocopies to a vendor to be keyed into a database. That was the easy part.\n\nNext we tackled SEC documents. Those are free through the Edgar database at www.sec.gov. But we had two problems: How do we download all documents for companies involved in the health-care industry? And how do we search those documents for names?\n\nThe answer to the download problem was to write a program in Visual Basic. If I ever doubted the value of learning a bit of programming to do CAR work, this experience resolved those doubts. VB includes functions that allow you to go out onto the Web and grab files. Other languages do too.\n\nIt took me two solid days to write the program in one of those work binges where you forget if it's almost dawn. But the payoff was running the thing and actually seeing it work. I annoyed my wife and plenty of friends by showing off the finished product.\n\nThe list of all health-care companies came from Edgar's own documentation pages. Using my program, I grabbed every annual report and proxy statement for all years and put them into one massive text file.\n\nThe next problem was what to do with this massive file. I could have written another Visual Basic program to search for each researcher's name in the file, but that would have been deadly slow. What I really needed was a text-indexing tool to make the search speedy. My first thought was to use Lotus Notes but I was lucky to have a tool called Texis, a document retrieval database that we use at the Seattle Times for our own electronic clips library.\n\nI won't bore you with the details here. I'll just say that Texis is extremely powerful and fast. Best of all it's a relational SQL database that can do joins. So once I got the text file loaded, parsed by company name, document type, year and the full text of each document, I could run a fairly simple join with the researchers' names. The query would search each document's full text looking for name matches, like an automated clip search. We searched for 1,800 names.\n\nI had a big assist on this from research editor Tom Boyer, who developed our electronic library. For more information on Texis, go to www.thunderstone.com.\n\nThe result of the query was a new file with a large chunk of text on either side of a name match. I loaded that dataset into SQL Server and then started the manual process of verifying each match. With a few exceptions, this was easier than it sounds because typically the Hutch would be mentioned somewhere in the text.\n\nPatents\n\nThe next challenge was to get all patents by these researchers. The data are available for free at www.uspto.gov/patft/index.html Again, a modification of the Visual Basic program allowed me to grab off the Web all patents in which Hutch researchers were listed as inventors. I limited this search to inventors from the state of Washington, which helped to produce more clean matches.\n\nAgain, I had to weed out false hits manually. If the person invented a new type of typewriter ribbon or garden tool, I automatically eliminated that patent. However, a patent for radiation delivery to lymphoid and marrow tissues was a good candidate for a true hit. Finding true matches wasn't terribly difficult.\n\nWe searched an insider-trading database with the help of a company called First Call, www.firstcall.com One last piece came from investigative reporter Deborah Nelson, who unearthed these stories but left for The Washington Post before she could complete the project. She had requested all financial disclosure forms for medical faculty at the University of Washington who also had positions at the Hutch. That information - useful but incomplete - was then keyed into a database. All of these data were loaded into SQL Server. I developed an interface in Access to pull all the data we had on each researcher together on one page. We created an 'A' list of the researchers that we wanted to focus on. As we learned of other financial connections, we were able to key these in manually through the Access interface.\n\nA quick note: to add researchers to the 'A' list later, I used an incredible tool in SQL Server called triggers. FoxPro has the same capability. The trigger would automatically launch a stored procedure to create a unique ID for any name we added to the 'A' list and then automatically search all datasets and add that unique ID to other tables to allow for joins. In short, you type in a name and suddenly all the data about that person magically appears.\n\nTriggers may be routine for database developers, but I'd never heard of them being used by CAR folks. I was awestruck by them and encourage you to investigate.\n\nMedline\n\nIn the course of our reporting, we also downloaded abstracts of medical journal articles written by Hutch researchers in the past decade. These data are available through a National Institutes of Health site called Medline, www.ncbi.nlm.nih.gov/pubmed/\n\nDownloading them involved a total of about 18,000 searches. My original program ran as fast as possible, and I hadn't really thought about the demand that this would put on NIH servers until the annoyed database administrator blocked our access. I called to let him know what we were doing and he gave us access again providing we spaced our searches five seconds apart and ran the program after 9 p.m.\n\nWe pulled full copies of articles for researchers with financial interests the old-fashioned way, photocopying them at a medical library. This effort produced a mere paragraph in our series about how researchers don't disclose their interests in medical journals, even when the articles are about their patents.\n\nWe also made heavy use of our death certificate database in tracking down patients. Other data included all grants from the NIH and all clinical trials ever advertised by the Hutch on the National Cancer Institutes' Web site.\n\nAfter doing the CAR work, I assumed a more traditional role of a shoe-leather reporter on the series. This project is a good example of the seamless blending of CAR work with investigative reporting. Even though the series was aided by the most sophisticated data work I've ever done, the focus was not on data but on the compelling stories of the patients, their families, a whistleblower, a financier, a Nobel Prize winner and the Hutch's doctors.\n\nDavid Heath can be reached by e-mail at dheath@seattletimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-preparation-pays-off","slug":"2001-mayjune-preparation-pays-off","url":"https://dwillis.github.io/uplink/article/2001-mayjune-preparation-pays-off.html","year":2001,"month":"May/June","monthNum":5,"headline":"Preparation pays off","kicker":"CENSUS 2000","author_name":"Robert Gebeloff","author_title":"Database Editor","affiliation":"The Star-Ledger","summary":"A journalist describes how extensive preparation, including data processing systems and pre-planned story angles, enabled his newsroom to execute comprehensive 2000 Census coverage efficiently on deadline.","keywords":["preparation","Census","deadline"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["ArcView","TIGER","Access","SQL Server","ASP"],"full_text":"Preparation pays off\n\nBy Robert Gebeloff\nThe Star-Ledger\n\nLet's be frank - the 2000 Census is a four-alarm headache.\n\nThere's a mountain of data to climb, and at the summit sits some of the most nondescript field names in the history of data processing.\n\nQuick - how many P0010005s did you have in your state? And what did they call P001005 in 1990 and 1980? Then there's the matter of mapping. Learning how to use the canned maps that come with ArcView is no easy feat. But learning how to convert Tiger maps and tie them to the Census data - that's an entirely new skill.\n\nAnd then there's the none-too-small matter of journalism - You've conquered the data, now what's the story? Can you transform this mountain of information into news, news that would qualify as left(news,3), rather than a rehash of trends you already reported to death?\n\nBe prepared\n\nAll of this requires preparation - the Census is just too big to do well on deadline. At The Star-Ledger, we took all the standard measures. We sent people to IRE Census seminars, we acquired and processed historical data, we did battle with the Tiger Maps, and we devised and revised our list of stories, graphics and maps throughout the weeks leading up to C-Day. But we also made a major investment in what I'll call logistical preparation. We cover the state and compete in eight local markets as well. On Census release day, we would need to get data that was cleaned and analyzed to our bureaus in Washington and Trenton, not to mention Woodbridge, West Caldwell, and Morristown - on deadline. What we came up with was a somewhat Rube Goldberg-esque system of Access tables, SQL Server Views and intranet Active Server Pages. The idea was that the data would arrive on Census day, a button would be pushed, and poof the 2000 numbers would flow into place, fuel for thousands of preprogrammed calculations.\n\nSmooth sailing\n\nI expected, of course, that the system would cough and wheeze a few times along the way, but to my amazement, it worked without a hitch, and enabled us to execute our coverage plan pretty much to the letter. The New Jersey files arrived on my hard drive at about 3:15 p.m. on a Thursday afternoon. By the time the last edition was hitting the presses, we had managed to get seven stories in the paper, illustrated by four photos, five maps, and 15 charts. The stuff was flowing through production so fast that in one early edition, a map got printed twice, but otherwise, the paper came out looking like it was the result of a month-long project. And then we were well positioned to come back Sunday with more.\n\nAll of this requires preparation - the Census is just too big to do well on deadline.\n\nI mention quantity, but having prepared was a matter of quality too. For example, we got the 1990 PL 94-171 data and went to the voting precinct level to generate demographics for the current state legislative districts. Then we coded each district based on the make-up of the three-member delegation. This enabled us to do precision reporting on political shifts.\n\nWhile the competition noted generally that population growth appeared to be greater in Republican-controlled districts, we reported precisely that 422,000 of the state's 684,000 population increase occurred in districts with all-Republican representation.\n\nShowing growth\n\nSame idea with sprawl. It was obvious by looking at county numbers that the growth in New Jersey was occurring in places that were mostly rural and suburban. But why approximate? We got a list of Urban/Rural/Suburban town classifications as labeled by the state planning department. We simply tied that list to the Census data and voila we were able to say precisely that the population growth... \n\nCensus Lessons:\n\nWe'd like to hear from those of you who had interesting experiences in working with the PL94 Census Data that was released in March.\n\nAre there lessons you learned that would be beneficial to others?\n\nIf this was your first experience with Census data, was there something you wished you had known earlier?\n\nDid your news organization come up with a unique story based on the data?\n\nWe'd like you to share your experiences in an upcoming issue of Uplink, preferably by writing a piece yourself. We could also arrange for a NICAR staff member to interview you for a story.\n\nThe stories can be anything from a few paragraphs to a few pages - whatever it takes to share what you learned.\n\nPlease contact us as soon as possible if you're interested.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-census-lessons","slug":"2001-mayjune-census-lessons","url":"https://dwillis.github.io/uplink/article/2001-mayjune-census-lessons.html","year":2001,"month":"May/June","monthNum":5,"headline":"Census Lessons","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR invites journalists to share their experiences and lessons learned from working with PL94 Census Data released in March, either through written submissions or staff interviews for an upcoming Uplink issue.","keywords":["Census","journalism","data"],"topics":["Demographics & Census"],"technologies":[],"full_text":"Census Lessons:\n\nWe'd like to hear from those of you who had interesting experiences in working with the PL94 Census Data that was released in March.\n\nAre there lessons you learned that would be beneficial to others?\n\nIf this was your first experience with Census data, was there something you wished you had known earlier?\n\nDid your news organization come up with a unique story based on the data?\n\nWe'd like you to share your experiences in an upcoming issue of Uplink, preferably by writing a piece yourself. We could also arrange for a NICAR staff member to interview you for a story.\n\nThe stories can be anything from a few paragraphs to a few pages - whatever it takes to share what you learned.\n\nPlease contact us as soon as possible if you're interested. Contact Mike Sherry at mikes@nicar.org or (573) 884-7332.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-census-stories-online","slug":"2001-mayjune-census-stories-online","url":"https://dwillis.github.io/uplink/article/2001-mayjune-census-stories-online.html","year":2001,"month":"May/June","monthNum":5,"headline":"CENSUS STORIES ONLINE","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A collection of online census coverage from major regional newspapers across the United States, providing local reporting and analysis on census data and its impact on communities.","keywords":["census","newspapers","coverage"],"topics":["Demographics & Census"],"technologies":["Internet","World Wide Web"],"full_text":"CENSUS STORIES ONLINE\n\nThe Los Angeles Times: www.latimes.com/news/nation/reports/census/\n\nDaily Camera (Boulder, CO) www.thedailycamera.com/census\n\nTribune-Review (Western Pennsylvania) www.triblive.com/census/\n\nNewsday (Long Island, NY) www.newsday.com/census\n\nJournal News (Northern New York City suburbs) www.thejournalnews.com/census\n\nThe Post and Courier (Charleston, SC) www.charleston.net/census2000/\n\nThe Roanoke Times (Virginia) www.roanoke.com/roatimes/census","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-lots-of-kids","slug":"2001-mayjune-lots-of-kids","url":"https://dwillis.github.io/uplink/article/2001-mayjune-lots-of-kids.html","year":2001,"month":"May/June","monthNum":5,"headline":"Lots of kids","kicker":"CENSUS-L","author_name":"","author_title":"","affiliation":"","summary":"Journalists discuss using PL94 census data to identify neighborhoods with unusually high concentrations of children, with tips on verifying findings and where to expect such demographics.","keywords":["census","children","neighborhoods"],"topics":["Demographics & Census","Investigative Methods","Data & Databases"],"technologies":["Excel"],"full_text":"Lots of kids\n\nThe following is a recent discussion on CENSUS-L concerning a sliver of the PL94 data that could produce an interesting story:\n\nI'm just starting to tackle a back-burner story from the PL94 data: Neighborhoods with lots of kids. I found one census tract in my area that had more kids than adults. Then I ran the numbers for the entire state, and found that this is the only census tract in all of Virginia with more kids than adults. My question: Has anyone else run across tracts in their area that have more kids than adults? That would be tracts with more than 50% people age 17 and under. I'm guessing this would be found in inner-city, public housing neighborhoods, which is where my special tract was.\n\nA word of caution: make sure there aren't group homes in that tract. You might want to contact social service agencies (state and local) to find out for sure.\n\nI'm working on a version of the where's the kids story, and two trends have come up. First, we're seeing significant flight away from our growing cities. Second, the kid concentrations are turning up in the same block groups where the Mormon temples or Indian reservation towns are.\n\nThe SF (short form) data that starts coming out this summer will be released to news media on an embargo basis (about 3-5 days).\n\nStarting in early May, the Census Bureau planned to start releasing summary information from this short form data. These will be in one-page Excel spreadsheets or PDF images. Samples are available through Steve Doig's Reporting Census 2000 Web site, http://cronkite.pp.asu.edu/censusi (check under \"What's New)","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-mayjune-bits-bytes-and-barks","slug":"2001-mayjune-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2001-mayjune-bits-bytes-and-barks.html","year":2001,"month":"May/June","monthNum":5,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR has updated several journalism databases including FBI crime reports, bridge inspection data, and FAA accident records, with pricing tiered by organization size. Resources from an IRE reporting conference for student journalists are also now available online, covering topics like property records, FOIA laws, and business reporting.","keywords":["databases","reporting","journalism"],"topics":["Data & Databases","Journalism & Newsroom","Public Safety"],"technologies":["CD-ROM"],"full_text":"Bits, Bytes and Barks\n\nData updates at NICAR\n\nThe following datasets have recently been updated at the NICAR Database Library: The 1999 FBI Uniform Crime Reports: Includes summary information from law enforcement agencies throughout the country on index crimes. There are six tables to the database, including one on homicides and another on arsons. Cost for the entire U.S., all databases: $160, $320, $345 depending on size of news organization. State slices are available for: $55, $110, $135. You can also purchase just one of the six databases. More information is available at www.nicar.org/data/fbil\n\nThe 2000 National Bridge Inventory: Includes the most recent inspection for all bridges in the United States. Bridges include those for highways, railroads, pedestrian-bicycle, overpasses, and those designated as historically significant. The data has been used by many news organizations to document problems with bridges in local areas. It is also useful to have available in case of a breaking news story involving a bridge. Cost for entire U.S.: $40, $80, $105. A single state is: $15, $30, $55. More information is available at www.nicar.org/data/bridges/\n\nFAA Accidents and Incidents: Includes all accidents and incidents in the United States involving all types of aircraft. The most recent incidents included in the data are from early 2001, however not all incidents from 2000 were entered into the database by the FAA by the time NICAR received the data. The most current data, 1990 through 2000, costs: $40, $80, $105. NICAR also has data going back to 1973 that is available separately. More information is available at www.nicar.org/datalfaaailindex.htm\n\nReporting Conference Tipsheets\n\nTipsheets from the IRE Reporting Conference, held March 31 in Columbia, Mo., are available online at www.ire.org/training/2001studentconfltipsheets.html The one-day conference featured a variety of topics geared toward students and young journalists. Panels included:\n\nFinding out about licensed professionals\nUsing property records\nUsing FOIA and local sunshine laws\nCovering cops and courts\nProducing the story for television\n\nAlso available, is a transcript of the keynote speech by Diana B. Henriques of The New York Times. Her speech discussed, \"What every reporter should know about business.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-dispelling-a-myth","slug":"2001-julyaugust-dispelling-a-myth","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-dispelling-a-myth.html","year":2001,"month":"July/August","monthNum":7,"headline":"Dispelling a myth","kicker":"TEST SCORES","author_name":"Betsy Hammond","author_title":"","affiliation":"The Oregonian","summary":"The article discusses how Oregon's suburban schools are not as exceptional as often claimed. Using analysis that accounts for family backgrounds, the research identifies schools that underperform despite privileged demographics.","keywords":["Oregon","schools","test scores"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["SPSS"],"full_text":"Oregon education officials have emphatically touted our state's public schools as among the very best in the world, and parents in most Portland suburbs say they moved to their particular part of our metro area for the great schools.\n\nBut our newspaper was able to use computer-assisted reporting, supplemented with a lot of in-the-field reporting, to establish that Oregon's suburban schools are in fact overwhelmingly mediocre - and to name which schools have done the least to help students reach their academic potential. Our series - \"Why Not the Best?\" (http://www.oregonlive.com/special/oregonian/education) drew hundreds of calls and e-mails from outraged readers.\n\nThe Oregonian used SPSS to run regression analyses on student test scores, using family background statistics to \"level the playing field\" among schools and see how each local school performs, given the students who come through their doors.\n\nMany newspapers issue \"report cards\" on schools in their communities. These typically pull together existing data, including test scores, and organize in reader-friendly fashion. The problem is that, inevitably, schools with more educated and affluent parents get the top grades; schools with uneducated and poor parents the worst.\n\nA few newspapers go beyond that by conducting regression analyses. Those efforts all are grounded in the same reality: A school's average test score tells you more about how wealthy and well-educated the parents are than how effective the school is, so it's essential to systematically discount the effects of family background to measure school performance.\n\nThe Oregonian added some new twists to its particular analysis, so that we can stake a real claim not to have let our suburban schools off the hook. We focused not on finding schools in impoverished neighborhoods that do well, a story that local educators were eager for us to tell. Instead, we took a hard look at schools in the most privileged neighborhoods and told readers about the ones that were coasting or failing. And we anchored our findings to national data, from the SAT and the National Assessment of Educational Progress, to provide a data-grounded reality check for parents who were hearing a relentless drumbeat of boosterism about Oregon schools being \"world-class.\"\n\nSPSS is easy to use, and we relied on NICAR's Advanced Bootcamp on Statistics to get the refresher we needed in how to use it right.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-a-new-wrinkle","slug":"2001-julyaugust-a-new-wrinkle","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-a-new-wrinkle.html","year":2001,"month":"July/August","monthNum":7,"headline":"A new wrinkle","kicker":"CRIMINAL TEACHERS","author_name":"Dianna Hunt","author_title":"","affiliation":"Fort Worth Star-Telegram","summary":"A Fort Worth high school football coach resigned after sexual assault allegations in a deal involving a $100,000 payout, prompting investigation into Texas's teacher licensing system which was found to have a massive backlog allowing credentialed teachers with serious criminal histories to remain teaching for years.","keywords":["teachers","credentials","investigation"],"topics":["Education","Investigative Methods"],"technologies":[],"full_text":"It started with a high-profile football coach and allegations of sexual assault.\n\nThe story made headlines for weeks in the Fort Worth area, as a local school district investigated the allegations against a popular high school coach.\n\nThe coach eventually resigned from the district after officials agreed to pay him $100,000 and drop the investigation. He was never charged with assaulting a female coach, but was charged a few months later with harassing another co-worker and her boyfriend.\n\nIt wasn't until Star-Telegram education reporter Mart Frazier called the state teacher licensing board, however, that the bigger story began to emerge.\n\nThe agency, the State Board for Educator Certification in Austin, Texas, acknowledged it was investigating the coach's teaching credentials. But beleaguered investigators also acknowledged that their workload was so high they had a backlog of cases that could take years to resolve.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-law-creates-shortage","slug":"2001-julyaugust-law-creates-shortage","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-law-creates-shortage.html","year":2001,"month":"July/August","monthNum":7,"headline":"Law creates shortage","kicker":"TEACHERS","author_name":"Russell Clemings","author_title":"","affiliation":"Fresno Bee","summary":"California's class-size reduction law five years ago created over 25,000 new teaching positions, but this rapid expansion has led to a persistent teacher shortage that disproportionately affects low-income and rural schools with underqualified educators.","keywords":["teachers","shortage","schools"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["FoxPro","9-track tape"],"full_text":"With tax revenues rising and a state law requiring a fixed percentage to be spent on education, the California Legislature five years ago reduced class sizes in three primary grades from an average of more than 30 to a maximum of 20.\n\nAlmost overnight, California's elementary schools created more than 25,000 new classrooms and hired a like number of new teachers. The result was a statewide teacher shortage that persists to this day and which has burdened low-income and rural schools with large numbers of underqualified teachers.\n\nIn December, the Fresno Bee analyzed five years of teacher census data. We determined that, for the poorest one-quarter of California schools, the teacher shortage - as measured by the percentage of teachers working without a full credential, the basic teacher license - has grown worse each year since class-size reduction began.\n\nThe PAIF data is derived from an annual teacher census. Each record includes details about a single teacher, such as education level, credential status, experience and demographic background.\n\nThroughout the state, schools that have trouble competing in a tight teacher market are relying on ever-larger numbers of teachers on temporary permits, internships and waivers - teachers who, in a normal market, would never be hired.\n\nThe data came from the California Department of Education's Professional Assignment Information Form and CalWORKS Children/Meal Programs databases.\n\nThe PAIF data is derived from an annual teacher census. Each record includes details about a single teacher, such as education level, credential status, experience and demographic background. Data on teacher assignments is also included. The CalWORKS/Meals database contains demographic information on each school's students, including the percentages on welfare or receiving free or reduced-price lunches.\n\nPersonal identifiers are either not collected or purged from the public data releases, so it's not possible, for example, to track an individual teacher's movements. But the data does provide rich detail on populations of teachers and students at the individual school and (for teachers) grade level.\n\nMany states collect similar data, and the National Center for Education Statistics (http://nces.ed.gov) has nationwide datasets and surveys covering some of the same subject matter.\n\nMuch of the California data was available online at the department's Web site, www.cde.ca.gov/demographics/files/paif.htm, for the most recent PAIF data and www.cde.ca.gov/demographics/files/afdc.htm for the welfare/meals data. The California Department of Education provided the two earliest years of PAIF data at no charge on nine-track tape.\n\nCollating and analyzing the data was not easy. Midway through the five-year period, the structure of the PAIF data changed from a flat file with room for multiple assignments for each teacher to a relational database with a flat file of teacher records linked to a table with one or more assignment records for each teacher.\n\nAdditional reference tables were needed to translate assignment codes into English terms such as \"self-contained first-grade classroom.\" And in determining which assignment codes to use and how to define credentialed and uncredentialed teachers, studies from the state Class Size Reduction Consortium (www.classize.org/), the Public Policy Institute of California (www.ppic.org/), and the Center for the Future of Teaching and Learning (www.cftl.org/) were invaluable.\n\nIt took a 290-line FoxPro script and a week of work to reduce the disparate databases, which totaled more than 400 megabytes, to a manageable flat file of 6,295 records, one for each California school that had at least one teacher in a self-contained elementary classroom.\n\nEach record in that table contained five years of data, and for each year there were five data fields: (a) Percentage of students on welfare, (b) Percentage of students on free/reduced price lunches, (c) Total teachers, (d) Fully credentialed teachers, (e) Less-than-fully credentialed teachers, and (f) Teachers for whom no credentialing information was available.\n\nWe published tables with the local data, and our Web designers created a searchable interface for the entire state table at www.fresnobee.com/man/projects/teachers.\n\nStatewide, a simple line graph told the story. For the wealthiest one-quarter of California schools, the proportion of uncredentialed teachers rose from zero in 1995-96 to 5 percent in 1999-2000. In the poorest schools, the proportion rose from 3 percent to 23 percent over the same five-year span.\n\nReaction\n\n\"No one intended for the poorest schools to have the least-qualified teachers, but that seems to have been the effect\" of class-size reduction, one researcher said.\n\nClass-size reductions have been an undeniable boon for most California schools. Test scores are rising, pupils are getting more individual attention, and parents are pleased. But for those schools that can't compete in the tight teacher market, the data shows a decidedly bleaker situation.\n\nThe Fresno Bee story on the teacher shortage is available from the IRE Resource Center (Story 17883). To order, call (573) 882-3364.\n\nRussell Clemings can be reached by e-mail at Clemings@cris.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-teacher-student-exodus","slug":"2001-julyaugust-teacher-student-exodus","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-teacher-student-exodus.html","year":2001,"month":"July/August","monthNum":7,"headline":"Teacher, student exodus","kicker":"SCHOOLS","author_name":"David Knox","author_title":"","affiliation":"Akron Beacon Journal","summary":"A reporter at the Akron Beacon Journal used computer-assisted reporting and data analysis to investigate why one school cluster dramatically underperformed others, challenging assumptions that poverty alone explained the gap and focusing instead on teacher transfer policies within school administrators' control.","keywords":["education","data","schools"],"topics":["Education","Investigative Methods","Demographics & Census"],"technologies":["Excel"],"full_text":"Computer-assisted reporting is no different than other journalism in an important way: Often the best stories start with a reporter's gut instinct. When Reginald Fields took over the education beat at the Akron Beacon Journal in June 1998, it didn't take him long to learn that the city's public schools weren't equal in quality. Among the district's eight clusters - each composed of a high school and its feeder middle and elementary schools - Buchtel High School was considered by parents, teachers and students far and away the worst.\n\nThe evidence\n\nEvidence backing that poor reputation was easy to find. During the final grading period of the 1999-2000 school year, 36 percent of the students at Buchtel earned a D letter grade - or worse.\n\nComparisons to other schools were equally dismal. Scanning a sorted listing of Ohio's proficiency test scores found almost all the Buchtel cluster schools near the bottom.\n\nBut grades and test scores didn't explain why one group of schools performed so poorly.\n\nFields soon came to question the conventional answer - that Buchtel's students were too economically disadvantaged (read, too black) - to do better.\n\nTrue, Buchtel schools were by far the most segregated in the city - 97 percent black compared to a district average of 47 percent. But the neighborhoods where they were located were much more racially diverse. Furthermore, visits to the Buchtel schools took Fields down streets lined with many tidy, attractive homes near well-kept parks - hardly a portrait of urban blight.\n\nFields also heard a steady drumbeat of complaints from parents and students about the low experience of the teachers at the Buchtel schools. He wondered whether the school district's liberal policy toward teacher transfers - by contract, teachers can ask to move after a single year of service - might contribute toward Buchtel's poor academic performance.\n\n\"Destined to Fail,\" a six-month project sparked by Fields' skepticism, provided a challenging opportunity to use multiple analytical tools and strategies to go behind the issue of poverty and search for other factors under the control of school administrators that can affect educational quality.\n\nThe Ohio Department of Education publishes reams of readily available, easily imported data - almost all useless for the Buchtel project.\n\nAs always, the first task was getting the data.\n\nThe Ohio Department of Education publishes reams of readily available, easily imported data - almost all useless for the Buchtel project because none of it was at the cluster level. With the exception of proficiency tests information, which included the number of students taking and passing each test in each school, all the other \"building-level\" data came in the form of percentages and averages. Those numbers couldn't be brought up to the cluster level because their denominators - the number of students and teachers in each school - varied widely.\n\nWhat was needed was student- and teacher-level data that could be added together, and then calculated for each cluster.\n\nThat information was available from the Akron Public Schools - but mainly as paper records.\n\nCalculating poverty\n\nFor the issue on student poverty, that wasn't a problem. All that was needed to calculate the percent of low-income students was the enrollment and number of students eligible for free or reduced lunch for each of Akron's 48 elementary and middle schools. (High schools were left out because of the tendency of older teen-agers not to apply for free and reduced-cost lunches out of embarrassment. Two other schools were excluded because they are magnet schools that draw from all parts of the city.)\n\nMeasuring teacher inexperience took a little more work. The numbers of years of service for each of the 2,023 teachers in the schools were input into a Microsoft Excel spreadsheet and a percentage of those with less than six years calculated for each building and cluster.\n\nThe results showed that nearly half the 285 teachers in the Buchtel cluster's eight schools had less than six years experience. The cluster average was 9.7 years of experience, compared to the district average of about 14.4 years.\n\nSPSS, a statistical analysis program, was used to gauge the possible impact of teacher inexperience on student performance. A multiple regression analysis found that low teacher experience and poverty together could explain as much as 54 percent of the difference in the proficiency test results among Akron's elementary and middle schools. The probability of the relationship being a coincidence is less than 1 in 1,000. A limitation of regression analysis is that, while it shows correlations, it doesn't prove cause-and-effect - a point stressed by several researchers at two local universities who were asked to review the data. Teacher inexperience may contribute to low test scores, but a lack of experienced teachers also could reflect an exodus of veterans who choose to transfer to better performing schools.\n\nThat scenario would be more likely if poverty is considered an invariable factor - beyond the influence of school policy. That's because a single-variable regression analysis showed poverty alone could account for 48.7 percent of the difference in the test scores, compared to 43.1 percent for teacher inexperience considered as the sole independent variable.\n\nIn other words, the conventional wisdom still works: poverty could be the dominant factor in explaining low test scores.\n\nBut a key finding of the Buchtel data analysis was that the school district's \"open enrollment\" policy, which encourages students to leave their neighborhood schools in search of a better education, might be altering the family-income demographics of the schools. Such student flight might contribute to poor test scores if the students leaving a cluster are more likely to do better on the exams than the classmates they leave behind. To investigate that possibility, another methodology - geocoding - was used to roughly gauge the performance potential of students leaving and staying in each school cluster by determining the household income of their neighborhoods. The Beacon Journal obtained from the school district the addresses of more than 25,000 students - about 84 percent of all students enrolled in Akron's public schools. In accordance with Ohio law, the only addresses omitted were those of students whose parents signed forms requesting the information be withheld.\n\nMicrosoft Word's global search and replace functions were used to massage the text files into delimited columns and rows for import into a database program, Microsoft Access 97.\n\nInitially, school officials said the data, which included which school each student attended and grade level, was available only on paper because the district couldn't afford the cost of programming. Officials relented after the paper pointed out that redirecting the ASCII text output of the report program to a floppy disk entailed no additional cost.\n\nMicrosoft Word's global search and replace functions were used to massage the text files into delimited columns and rows for import into a database program, Microsoft Access 97.\n\nThe addresses were then geocoded into census block groups that included 1999 estimates of median household income, as provided by Claritas, a national marketing research service used by the Beacon Journal's advertising department.\n\nThe goal was to determine where the students would go to school in the absence of the open enrollment program by matching their addresses to the district's 317-page directory of school attendance zones.\n\nThe job was made manageable by sorting the database on street number and name - allowing a dozen or so records to be updated with their assigned schools in a single query.\n\nThe completed database had all the information needed to determine the impact of the district's open enrollment program: what school each student was assigned and the school actually attended.\n\nThe analysis found that 20 percent of the children in the Buchtel district were going to schools in other clusters, compared to a district average of 13 percent. At the high school level, one of every three Buchtel students had left the cluster.\n\nComparing the average neighborhood incomes of students staying and leaving provided evidence that the student flight might contribute to keeping test scores down. Students fleeing the Buchtel cluster were found to have neighborhood incomes more than $3,000 higher than those who remained. Only one other cluster - Central-Hower, the next poorest performing cluster - showed a net loss of income - about $1,800. And what was the reaction of the school officials to the series? They didn't challenge the findings of the statistical analysis. That changed soon after \"Destined to Fail\" was published, December 10-13. Within a week, the school board president called for a panel of parents, school administrators and community leaders to address issues raised in the series.\n\nWithin a month, the school administration presented the board with over a dozen recommendations to better the Buchtel cluster, including several proposals for upgrading the quality of teaching in the schools.\n\nDavid Knox can be reached by e-mail at Dknox@thebeaconjournal.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-education-tipsheets","slug":"2001-julyaugust-education-tipsheets","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-education-tipsheets.html","year":2001,"month":"July/August","monthNum":7,"headline":"Education tipsheets","kicker":"","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"The IRE Resource Center offers education reporting tipsheets covering investigative angles on education data and teacher quality, including resources and best practices for education beat reporters.","keywords":["education","reporting","investigation"],"topics":["Education","Investigative Methods"],"technologies":[],"full_text":"Education tipsheets\n\nThe following education tipsheets are available from the IRE Resource Center, (573) 882-3364:\n\nTipsheet #957 outlines new angles for reporting on the same old education data. Also included are tips for putting together your own local education investigation.\n\nThe Herald-Leader (Lexington, Ky.) ran a four-part series that examined teacher quality by evaluating Kentucky's teaching programs and certification process. Tipsheet #892 contains the resources used in the series as well as a list of do's and don't's for tackling the education beat.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-separate-and-equal","slug":"2001-julyaugust-separate-and-equal","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-separate-and-equal.html","year":2001,"month":"July/August","monthNum":7,"headline":"Separate and equal?","kicker":"SCHOOLS","author_name":"Doug Haddix and Bill Bush","author_title":"reporters","affiliation":"The Columbus Dispatch","summary":"A Columbus newspaper uncovered racial and economic inequities in the school district by obtaining electronic records that revealed unequal opportunities across elementary schools, requiring persistent negotiation and discovery of Ohio law to access usable data with proper identifiers.","keywords":["segregation","investigation","records"],"topics":["Investigative Methods","Data & Databases","Education"],"technologies":["mainframe","CD-ROM","Access"],"full_text":"Like many government offices, the Columbus school district maintains its electronic records in a mainframe computer system that makes it nearly impossible for outsiders to analyze.\n\nInformation is kept on separate tables that can be linked only by a confidential field such as student identification number or teacher Social Security number.\n\nThe moment we saw the record layouts for the district's master student file and master employee file, we saw red flags. Those databases, we realized, would play a pivotal role in the newspaper's investigation into whether the return to racially segregated schools in Columbus had produced unequal opportunities for poor and minority students.\n\nGlaring inequities\n\nThe four-day \"Dividing Lines\" series, published in June 2000, uncovered glaring inequities among the district's 88 elementary schools. Those findings came despite a school board policy approved four years earlier, promising that the end of cross-town busing would not produce separate and unequal schools.\n\nSeven weeks after our request for the district's master files, we finally obtained the electronic data in a usable format but only after finding a smoking gun in Ohio law. At four separate meetings with the school district's data crunchers, we repeated our request that programmers randomly generate a unique identifier for each of the 65,000 students and 11,000 employees. We assured district officials that we had no interest in identifying particular students or obtaining Social Security numbers of any teachers, principals or administrators. We simply wanted to be able to link the district's 19 separate data tables.\n\nWithout a link, for instance, we had a table that included teacher salaries but not their names, which were kept in a separate file.\n\nSix weeks after our initial data request, the district provided its master student and employee tables on a CD-ROM, complete with a nifty jewel case cover. The only problem: no unique identifiers. The tables were useless.\n\nWe scheduled a fifth meeting with top district officials to press our case. The day before the meeting, we discovered through a source in the Ohio Department of Education that all public school districts are required by law to generate a unique student identifier when they submit data to the state. Armed with that ammunition, we met with the district's top brass. Immediately, they agreed to provide the data with a randomly generated identifier created just for The Dispatch - the missing link we needed. Without that discovery, we might have been forced into taking legal action. Ohio's public record law sets no firm deadline for completing requests; it merely requires public bodies to make records available in a \"reasonable\" period. It took the district another week, however, to turn over data that it had maintained all along.\n\nSeven weeks after our request for the district's master files, we finally obtained the electronic data in a usable format - but only after finding a smoking gun in Ohio law.\n\nOnce the district's usable master student and employee tables arrived on a new CD-ROM, we did a bit of tedious work to move the 57 MB of data into a Microsoft Access database. The 19 tables had an unfamiliar \".JUN\" extension, so we saved them as flat text files, then moved them into Access.\n\nWe used the advanced import function to save the import specifications. That saved a lot of time because we had requested master files for the last year of cross-town busing (1995-96) and the most current data for the return to neighborhood schools (1998-99). It took about 20 minutes to type in the import specs for each table for 1995-96, so saving them slashed the import time for the 1998-99 tables. In addition, the saved specs will make it faster to import data for subsequent years, provided that the data fields don't change. Because the Columbus district moves slowly on requests for information, we filed simultaneous data requests with the Ohio Department of Education, which maintains information on the more than 3,600 public schools across the state data.\n\nThe judge who originally ordered desegregation of Columbus schools in the 1970s termed the new zones \"curious.\"\n\nWhile not as extensive as the Columbus district's master files, the state data gave the project team helpful information on enrollment, per-pupil spending, proficiency test scores, median teacher salaries and other topics that we could query by individual schools and by district.\n\nUnlike the Columbus district, the Education Department responded with amazing speed, sometimes e-mailing Microsoft Excel spreadsheet files to us within hours of a request. The school district maintained it didn't have the staffing to provide information that quickly.\n\nTo get the fullest picture possible of changes in Columbus elementary schools since the end of busing for integration, we obtained from the district and analyzed several other types of data:\n\nStudent school assignment records.\n\nWe received one of those \"you can't get there from here\" responses from the district when we first asked for records showing Elementary Attendance Zones, which are geographic boundaries that determine each student's \"home school.\"\n\nFirst, the district provided a huge paper file that included an elementary-zone breakdown of the number of students living in each assignment area, their race, their gender and the school they attend. In Columbus, students can attend their home school or try to transfer to another school through a lottery.\n\nWith 65,000 students to track, the paper files were useless for analysis because of their size. In a bizarre twist, the school district official who created the paper report maintained that he did not keep it electronically; rather, he insisted, he printed the report and then deleted the electronic version from his PC. (Don't ask.)\n\nAfter some wrangling, we were able to obtain the underlying records to recreate a version of this report ourselves using a Microsoft Access database. We obtained an electronic database in which each record represented a student; each record contained demographic information, as well as that student's \"home school\" and the school the student actually attended.\n\nBecause we had data for the last year of cross-town busing and for the most recent year of neighborhood schools, we were able to analyze changes in the assignment boundaries for elementary schools. Those attendance zones, it turned out, were being grouped together in many cases along racial lines.\n\nThe judge who originally ordered desegregation of Columbus schools in the 1970s termed the new zones \"curious.\" One school assignment plan formed a sideways L, picking up predominantly white neighborhoods while avoiding minority neighborhoods. Another linked white neighborhoods that were separated by a railroad switchyard and a freeway, again avoiding adjacent black neighborhoods.\n\nUniform school accounting system database.\n\nThis huge district database tracked dollars by various expenditure codes by building. That allowed us to analyze how much money was actually flowing to schools with different racial and economic compositions, and how those funds were being spent. Among other uses, we were able to gauge a school's ability to raise money through parent-teacher groups.\n\nFree and reduced-price lunch program participation by school. For whatever reason, neither the state nor the district kept this information in computer form. The district's food services office maintained paper records, which we entered into a Microsoft Excel spreadsheet. We linked that table using the school building identification number to other data tables so that we could explore the relationship between student poverty and items such as test scores, per-pupil spending and teacher experience.\n\nIn addition to basic spreadsheet and database analysis, the newspaper used GIS mapping software to look for geographic patterns involving the 88 elementary schools. Seeing the data on a map helped us spot several important changes, most notably the concentration of experienced teachers in the inner-city and near-downtown schools. Mapping also vividly showed how dramatically many of the schools changed in racial composition of the student body.\n\nThe reporting explored the key data findings:\nSchools with high concentrations of poor students generally have the least-experienced teachers.\nElementary school assignment boundaries in some neighborhoods closely match the lines once deemed unconstitutional by a federal judge.\nAlthough schools with high numbers of poor students receive hundreds of thousands of federal dollars, overall spending by school bears little relation to the poor student population.\nSchools with high concentrations of poor students score far below other schools on the statewide fourth-grade proficiency test.\nElementary schools with low numbers of poor students qualify for additional resources because their students fare better on the proficiency test.\n\nThe work on \"Dividing Lines\" continues to benefit The Dispatch because we all learned about data sources and techniques that we never knew existed.\n\nDoug Haddix can be reached by e-mail at dhaddix@dispatch.com. Bill Bush can be reached by e-mail at bbush@dispatch.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-non-traditional-licenses","slug":"2001-julyaugust-non-traditional-licenses","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-non-traditional-licenses.html","year":2001,"month":"July/August","monthNum":7,"headline":"Non-traditional licenses","kicker":"EDUCATION","author_name":"Alice Warchol and Philip Walzer","author_title":"","affiliation":"The Virginian-Pilot","summary":"Virginia schools are increasingly hiring teachers with alternative licenses who lack traditional qualifications like passing the PRAXIS I exam, prompting concern from officials and educators about whether these temporarily licensed teachers should be in classrooms. Journalists investigated the scope and placement of alternatively licensed teachers across school districts to determine if they were assigned randomly or concentrated in lower-performing, poorer schools.","keywords":["licenses","teachers","schools"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["Excel","Internet","World Wide Web"],"full_text":"Non-traditional licenses\n\nBy Alice Warchol and Philip Walzer\nThe Virginian-Pilot\n\nIn the scramble to hire teachers, school divisions in Virginia are now employing educators who do not have traditional teaching licenses.\n\nSome of them have yet to take or pass the PRAXIS I - a national teaching exam. Others have not completed college-level coursework required by teacher-preparation programs. But they are allowed to teach for three years with varying types of alternative licenses.\n\nWhile administrators defend their hiring practices, a slew of elected officials and teaching professionals say these temporarily licensed teachers should not be in the classroom.\n\nWhile administrators defend their hiring practices, a slew of elected officials and teaching professionals say these temporarily licensed teachers should not be in the classroom.\n\nInvestigating dispute\n\nThat was one of the disputes we set out to answer in our story. We also wanted to know how many teachers being hired had some type of alternative license. Where did they work? Were they assigned randomly, as administrators claimed, or were they mostly working in poor schools or schools with low test scores?\n\nTo begin, we started with the Virginia Department of Education. We learned that there are two types of alternative licenses: provisional and conditional. We also found out that half of the new licenses issued last year were alternative ones. (We wanted to gauge if this was a growing trend, but the state just started tracking this information.)\n\nOur next step was to ask each of the five local school systems that we cover how many provisional and conditional teachers they employ, and more importantly, where they worked. We used Virginia's Freedom of Information Act to request the numbers. We encountered little resistance. We were not charged.\n\nOrganizing data\n\nTo organize our data, we used Excel and created one worksheet for each city. Then, we downloaded from the Department of Education's Web site school enrollment numbers and free- and reduced-price lunch data a national indicator of poverty. We were lucky because the state posts its enrollment and lunch program information in Excel.\n\nFor every school in each city, we had columns for enrollment, percent of children eligible for free- and reduced-price lunch and number of alternatively licensed teachers. Then we added another column for the test score information. That came from our colleague Matthew Bowers who had created an Excel spreadsheet when the state's test results were announced last year. Each school in each city was assigned a number between 1 and 4. The number 1 meant the school was fully accredited by the state; a 4 indicated the lowest rating: accredited with warning.\n\nTo determine whether one school had more alternatively licensed teachers than another, we inserted a column that gave us the rate of teachers per 1,000 students for each school. (We divided each school's number of alternatively-licensed teachers by its student enrollment and then multiplied that result by 1,000.) At this point, we sorted each worksheet by the rate per 1,000. That showed us the top 10 schools in each city with the largest number of alternative teachers taking into account their student enrollment.\n\nTop 10 list\n\nAll of the hiring folks we interviewed said they placed alternatively licensed teachers wherever needed. True, we found poor and wealthier schools among our list of top 10. But when we looked at how well those schools did on the state's exams, our analysis showed that most of the schools did not meet Virginia's standards.\n\nWhen we published our story, we arranged to post part of our database online. Parents could log on to our Web site and search by city for their child's school. The online database shows enrollment, number of alternatively licensed teachers and the rate per school. We e-mailed the file to our online news coordinator, Kerry Sipe, a couple of days in advance to publication. He arranged to have it posted on the Web site.\n\nOur advice to fellow reporters is to repeatedly proof your work. Have someone else look over your computations.\n\nFor the print version of our story, we ran a large graphic listing the same info for the top 10 schools.\n\nNumbers give structure\n\nThe numbers gave us a structure for the story. But most of our reporting focused on why teachers weren't qualified for a full license and whether that was hurting kids. We found alternatively licensed teachers ready to defend themselves. We showed what it's like to be in their classroom. We also talked to education professionals who worry about the quality of education someone can bring to the classroom when they haven't student-taught or can't pass the PRAXIS I - a basic-skills test.\n\nOur last step was to think like parents. Would a school tell them if their child's teacher had a temporary license? The conflicting yes and no answers we received from school divisions only made our story better.\n\nAlthough the project was a lot of work and took several weeks to complete - we did it in between covering our beats - it was fairly simple as far as CAR endeavors can go. Our advice to fellow reporters is to repeatedly proof your work. Have someone else look over your computations. And when you're sending Freedom of Information Act requests to several school systems, it's imperative to ask for the same information.\n\nAlice Warchol can be reached by e-mail at awarchol@pilotonline.com. Philip Walzer can be reached by e-mail at walzer@pilotonline.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-funding-schools","slug":"2001-julyaugust-funding-schools","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-funding-schools.html","year":2001,"month":"July/August","monthNum":7,"headline":"Funding schools","kicker":"SPENDING","author_name":"Holly Hacker","author_title":"CAR specialist","affiliation":"St. Louis Post-Dispatch","summary":"St. Louis Post-Dispatch education reporters analyzed school funding disparities in Missouri and Illinois by adjusting for geographic costs and special student needs, using data analysis techniques to determine if districts spent equitably per student.","keywords":["funding","analysis","education"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["Excel","Access","SPSS"],"full_text":"In 1993, the courts threw out Missouri's school funding system. The judge in the case declared that because of huge gaps in spending, public schools ranged from \"golden\" to \"god-awful.\" So the state created a new funding formula that was supposed to be more equitable. Still, over the years, many educators and lawmakers complained that disparities still existed. The education reporters at the St. Louis Post-Dispatch decided to tackle the school funding issue last fall. We knew it was going to come up for debate in the next legislative session. Meanwhile, across the Mississippi River, Illinois lawmakers also grappled with the issue. So we wanted to do two separate analyses, one for each state.\n\nHere was the perfect opportunity for me to test my skills as the education team's CAR specialist. I had used Microsoft Excel and Access extensively and recently learned SPSS. Still, I would suffer plenty of headaches and emit several primal screams before our three-part series ran, after about three months of work.\n\nWe wanted to know if school districts spent about the same amount on each child. Sure, one could just get official spending figures for each district and compare. But those numbers don't tell the whole story. Just as it costs more to live in Chicago than in Peoria, some districts have to pay more for teachers and resources based on their location. Plus, children with disabilities or other special needs cost more to educate. So, we wanted to compare districts by taking those differences into account.\n\nI called various school finance experts for help and struck gold with the National Center for Education Statistics. A researcher in the NCES school finance division pointed me to a bunch of studies and sources. I learned about something called the geographic cost of education index, which lets you adjust for cost differences by school district. The index is based mostly upon salaries for teachers with comparable experience, plus other costs.\n\nThrough NCES I also found a way to adjust for differences in student costs. Generally, children with disabilities cost 2.3 times as much to educate, while poor children and children with limited English cost about 1.3 times as much. So if a district has 1,000 students and 200 of them are in special education, it's the equivalent of educating 1,260 \"regular\" students. That is, 800 + (200x2.3) = 1,260.\n\nGetting the data\n\nTo do this project, I pulled data from several sources.\n\nI downloaded the cost-of-education index from the NCES Web site.\n\nThe Illinois education department e-mailed me spreadsheets with finance information, student enrollment, and the number of special education, limited-English and poor students in each district. I defined \"poor\" students as those who get free or low-cost meals through the federal school lunch program.\n\nLuckily, officials at the Missouri and Illinois education departments not only provided data willingly, but also helped me make sense of the numbers.\n\nMissouri's education department e-mailed me a monster (16.8 megabytes) text file with detailed financial information. For reasons discussed later, I needed Missouri school district spending broken down in ultra-specific categories.\n\nI pulled student information for Missouri districts from an Access database I had already created.\n\nIt seems that on every CAR project, I inevitably run into more problems than I ever expected. This one proved no different. For starters, we had to determine how to count students. Illinois gives districts money based on student attendance. Missouri uses \"eligible pupils,\" which is like attendance, but with some differences. We picked student enrollment as our measure, because even if kids don't come to school every day, the district still has to pay for their teachers, textbooks, utilities, etc. Plus, poor districts tend to have lower attendance, thereby inflating the amount spent per pupil. We also had to define expenditures. We used operating expenditures, which includes costs to run the district, but not capital costs. To calculate Missouri district spending based on enrollment instead of \"eligible pupils\" (love that education jargon), we had to add some spending categories and subtract others. That's why we needed detailed spending files. I spent two weeks just importing and cleaning those files. Luckily, officials at the Missouri and Illinois education departments not only provided data willingly, but also helped me make sense of the numbers and financial terms.\n\nNumber Crunching\n\nI imported the various spreadsheets and tables into Microsoft Access, then joined them so I had a single table for each state. From there, I divided each district's operating expenditures by its cost-of-education index to get adjusted expenditures. Next, I adjusted student enrollment using the weights for special education, limited-English and poor students. Then I divided the adjusted expenditures by adjusted enrollment to get adjusted spending per pupil.\n\nActual expenditures = Adjusted expenditures Cost-of-education index\n\nRegular students + 2.3*special ed students) + (1.3* students) + (1.3*limited-English students) = Adjusted enrollment\n\nAdjusted expenditures = Adjusted spending per pupil Adjusted enrollment\n\nSo you might have two districts that spend $5,000 per student in actual dollars. But factor in location and student need, and one district could have more \"buying power\" than the other - say, $5,200 per student vs. $4,600 per student.\n\nNow that we had adjusted spending for every district, I turned to SPSS for further analysis. I calculated the minimum, maximum and average spending per student for each state, in both actual and adjusted dollars, weighting for district size. One school finance expert suggested we also look at the coefficient of variation, which is the standard deviation divided by the average spending per pupil. This is the range that about two-thirds of students fall within.\n\nIn both Missouri and Illinois, the gap between the highest- and lowest-spending districts shrank using adjusted dollars, but not entirely. This table summarizes the results for Missouri:\n\nSpending per pupil for Missouri school districts, 1998-99\n\nActual dollars / Adjusted dollars\nMinimum: $3,500 / $3,250\nMaximum: $10,500 / $9,500\nAverage: $5,400 / $4,400\nCoeff. of variation: 20% / 15%\n\nIn actual dollars, about two-thirds of Missouri students are in districts that spend within 20 percent of the average - that is, from $4,320 to $6,480 per child. In adjusted dollars, those students are within 15 percent of the average - from $3,740 to $5,060.\n\nAnswers\n\nOk, so we know the gaps still exist. Why? For answers, I ran some regressions in SPSS of school spending against district poverty rates, tax rates, property tax bases, and other variables.\n\nWe found that local tax rates explained about a third of the variation in Missouri district spending. The property tax base explained about a third of the variation in Illinois. In our story, we explained that Missouri rewards communities willing to tax themselves, while in Illinois, what a district spends depends more upon the property wealth of the community.\n\nWe also found that in Missouri, poor children weren't necessarily trapped in the lowest-spending districts. But in Illinois, the lowest-spending districts had more poor students, even after adjusting for student need. Small districts in both states tended to spend more because transportation, administration and other costs eat more of the budget.\n\nI also tried to see if differences in spending had any effect on test scores, but I didn't find a strong relationship. Perhaps the relationship does exist, but one would need to break down spending into more specific categories - say, dollars spent on classroom instruction. Plus, because test scores can vary from school to school, next time I might try analyzing on the school level, not the district level.\n\nAnother issue we ran into was how to report our findings. Because I defined pupils differently, the actual spending did not match district figures. Would readers get the distinction? In the end, we decided to group districts as high, medium and low spenders based on actual spending, then on adjusted spending.\n\nAdvice\n\nHere's some advice if you attempt a similar project:\n\nFollow what I call the Rule of 3.7 - Estimate the time you think the project will take, then multiply by 3.7. Or more.\n\nBe shameless in consulting with experts. Get their advice on how to design your analysis. What you're doing has been done before, so you can learn about pitfalls and caveats up front.\n\nGet those experts to review your findings.\n\nWork with graphics editors early and often. This is complicated stuff, and you should present your findings in a way that makes sense to readers but doesn't overwhelm them. CAR directors probably know this already, but as a reporter doing CAR, I made this crucial discovery too late.\n\nOnce you've analyzed the numbers, get into the classroom. So one district spends $5,000 per child and another spends $10,000. How does that play out in the classroom? Two education reporters spent several days in schools getting vivid examples to illustrate the spending gaps.\n\nHolly Hacker can be reached by e-mail at hhacker@post-dispatch.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-checking-the-grades","slug":"2001-julyaugust-checking-the-grades","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-checking-the-grades.html","year":2001,"month":"July/August","monthNum":7,"headline":"Checking the grades","kicker":"SCHOOLS","author_name":"Jeff Thomas","author_title":"Reporter","affiliation":"The Gazette","summary":"A Colorado reporter obtained the state's school grading formula and used Excel and statistical software to calculate what grades schools would receive under a proposed law, revealing results the governor's office had kept secret during legislative debate.","keywords":["formula","grading","transparency"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["Excel","SPSS"],"full_text":"It took some computer power to pull this one off, but the crucial ingredient was a reporter refusing to let her sources control the story. Colorado, like other states, has placed academic standards into statute. And like many states, Colorado uses a test to determine if schools are meeting those standards. Last year, lawmakers wanted to go a step further and translate the sometimes-murky test results into terms that millions of taxpayers statewide could rapidly understand. They agreed to issue each school an annual report card, complete with letter grades, A through F, based primarily on test scores. Colorado drew up a formula that, for each school, blended the scores of the separate reading, writing, math and other tests taken by kids in the various grades. The result was a single number for the school that, in turn, would be translated into a letter grade. Inside the statehouse committee rooms, debate revolved around the very concept of handing out As or Fs to schools. Outside the Capitol, the question was more visceral: What grade would my school get?\n\nInside the statehouse committee rooms, debate revolved around the very concept of handing out\n\nThe governor's office wasn't saying. It had drawn up the formula and had tried it out on some test results, but wasn't sharing the results. Here was a proposed law whose precise effects could be determined, yet those effects were closely guarded, even as the legislation moved toward a vote.\n\nOne of The Gazette's Denver bureau reporters, Michele Ames (now with the Denver Rocky Mountain News), took matters into her own hands. She obtained a copy of the formula and reviewed it with the authors. Test scores were readily available from the state education department. She cracked open Excel, and began pouring test results into spreadsheets.\n\nI'll spare you the specifics of the formula, but here were the basic steps:\n\n1. Assemble test scores for each school in the state. Also for each school, assemble the number of pupils enrolled and the number taking each test. The raw data were available in flat files and were assembled in Excel.\n\n2. The formula required adjusting each test score using the percentage of pupils taking the test. This also was done in Excel.\n\n3. At this point, the formula wandered into statistics. Michele brought her disk to The Gazette newsroom in Colorado Springs, where we fed the numbers to SPSS, which is statistical software. The basic idea was to determine the z-score of each test score in each academic subject, in each grade level, at each school. A z-score tells you whether a test score falls within the average amount of variance around the mean test score, or if it falls somewhere above or below that average amount of variance.\n\n4. Having produced a series of z-scores for each school, one for each of the tests, the formula blended the z-scores in such a way that produced a single numeric value for each school. These values themselves were compared to produce yet another set of z-scores, one for each school. This was done in SPSS.\n\n5. Under the proposed legislation, schools with the top 8 percent of the z-scores would receive A's. The bottom 2 percent would receive F's. B's, C's and D's would be distributed to the 90 percent of schools in between. We moved the final z-scores from SPSS back to Excel. We created a new column, into which we deposited a nested \"IF\" statement that read the z-score and issued the proper letter grade for each school.\n\nLocal schools\n\nFor our own story, we extracted the schools and grades in the school districts within our readership area, and published those in the paper. Meanwhile, we put the entire list of schools statewide on our Web site and asked the Associated Press bureau in Denver to send an alert to Colorado members, directing them to our Web site to find schools in their area. Newspapers across the state published portions of the list, along with localized stories of their own making. We also posted a nerd box that other papers were welcome to use. A sidebar to Michele's story noted that 75 percent of the As would go to schools in just five affluent school districts. Most of the Fs, meanwhile, would go to urban schools in low-income parts of Denver. Judging by the solid week of phone calls and e-mails that rained down upon Michele, and by the legislators who brought her story to the floor during debate, this could be called impact journalism. Computers made it possible, but a reporter's instincts made it happen.\n\nJeff Thomas can be reached by e-mail at jeff@gazette.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-legislator-gifts","slug":"2001-julyaugust-legislator-gifts","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-legislator-gifts.html","year":2001,"month":"July/August","monthNum":7,"headline":"Legislator gifts","kicker":"STUDENTS","author_name":"Robb Crocker","author_title":"Legislative reporting student","affiliation":"Virginia Commonwealth University","summary":"Virginia Commonwealth University journalism students analyzed state legislator gift disclosures and found lawmakers received approximately $118,000 in gifts from special interests, creating a searchable database and writing stories that raised ethical questions about potential conflicts of interest.","keywords":["gifts","legislators","database"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["Excel"],"full_text":"Legislative reporting students at Virginia Commonwealth University took a close look this spring at the gifts state legislators receive. We found that Virginia's lawmakers got about $118,000 in gifts last year from special interests.\n\nAdd in the gifts to the governor, lieutenant governor and attorney general, and the haul totaled almost $185,000.\n\nThe 13 students in the class, myself included, culled the information from the conflict-of-interest forms state officials filed in January. We entered the information into a spreadsheet so we could analyze it and identify the biggest givers and recipients.\n\nOnline searchable database\n\nWe wrote more than a dozen stories and put them online - along with a searchable database detailing the gifts reported by each legislator. We highlighted the biggest gifts including trips to hunt caribou in the Arctic Circle and quail in Georgia. And we raised ethical questions: Should lawmakers be accepting such freebies from businesses, interest groups and lobbyists seeking to influence the legislative process?\n\nThe project drew an enthusiastic response. AP picked it up, and we received several complimentary e-mails - especially from citizens thanking us for putting the records online.\n\nFocus on General Assembly\n\nWe initially focused on the 140 members of the General Assembly. The project involved the following steps:\n\nWe obtained the forms, called Statements of Economic Interests, from the clerks of the House and Senate.\n\nEach student was given the conflict-of-interest forms for 10 legislators. (Jeff South, who teaches the course, also took a batch of 10 forms.)\n\nWe entered the information for each legislator into Excel. For every gift, we recorded the description, the value, the giver's name and city and the recipient.\n\nUsing cut-and-paste, we combined the data from each student into one large spreadsheet.\n\nWe cleaned the data a bit - especially to standardize the names of givers. (The state's major utility company, for example, had been entered several ways: as Virginia Power, VA Power, Dominion Power, Dominion Resources and other variations.)\n\nWe analyzed the data, using Excel's PivotTable Report feature to subtotal it by givers and by recipients.\n\nWe highlighted the biggest gifts - including trips to hunt caribou in the Arctic Circle and quail in Georgia.\n\nThe class held a brainstorming session to generate story ideas. Some students decided to profile the most generous gift-givers - like Philip Morris, which gave legislators nearly $26,000 in trips, meals and other gifts. Other students chose to profile the top recipients. One student offered to write about quirky gifts, like WWF tickets; another, about legislators who refuse all gifts. One student decided to investigate the gifts received by Gov. Jim Gilmore, Lt. Gov. John Hager (who presides over the Senate) and Attorney General Mark Earley. As we did with the legislative gifts, the class combined forces to input the gifts records for Virginia's top state officials.\n\nFor our individual stories, we each did additional reporting, interviewing officials and tracking down more records. Then we started writing.\n\nThe stories were later published at http:// During the legislative session, we used the site to report on the daily happenings at the General Assembly.\n\nLearning spreadsheets\n\nAs a participant in the project, I learned quite a few skills that I plan to utilize in the future. First, I learned how to compile data into a spreadsheet - and why that's important: All told, we needed to tally up about 1,000 gifts, and you can't do that in your head. Excel helped us \"interview\" the data so we could discover things that weren't obvious, like the average haul in gifts for male legislators ($912) versus female legislators ($441).\n\nIn addition, I got my first exposure to working as part of a large group on a big project. That takes collaboration and teamwork. If any member of the group screwed up - made errors inputting data, for instance - it would undermine all our work. So reliability and attention to detail were critical.\n\nAs a participant in the project, I learned quite a few skills that I plan to utilize in the future.\n\nI was honestly surprised at what our class found, and at the public's response. I think people are interested in gifts to politicians for several reasons:\n\nMany gifts have a cachet that campaign donations don't. A lobbyist may get more bang for the buck with a $100 steak dinner than with a $100, or even $1,000, check to a campaign committee. In Virginia, where $5,000 campaign contributions are pretty common, an interest group could spend less money but make a bigger impression by sending a key legislator on a vacation.\n\nGifts often guarantee face time with lawmakers - giving lobbyists an opportunity to state their case for or against legislation. We found that legislators received $26,000 in meals and $24,000 in trips, with lobbyists usually serving as hosts.\n\nConstituents may view certain gifts more suspiciously than campaign contributions. People might sympathize with politicians' need for money to run for office. But they might think it's grubby for Virginia lawmakers to accept free Washington Redskins tickets, golf equipment and eyeglasses - things ordinary folks must buy.\n\nThe rules governing gifts may be looser than the rules on campaign donations. In Virginia, lawmakers cannot accept campaign contributions during the legislative session - but they can accept gifts any time. (Virginia has no limits on the amount or size of gifts or campaign donations legislators can receive. Steve Calos, executive director of Common Cause of Virginia, describes the state as the \"Cayman Islands\" of campaign finance.)\n\nCampaign donations often are online (in Virginia, thanks to the Virginia Public Access Project www.vpap.org). Not so for gifts; those records had been available only on paper in Richmond. Our project made the gifts data available to the public for the first time over the Internet.\n\nComparing regions\n\nMy story compared the regions of Virginia and the amount of money legislators received. To do this, I split the state into five regions: northern, eastern, central, southwest and western. Next, I determined which legislators go where, using Excel to tally my results.\n\nI found that legislators from Eastern Virginia - including Norfolk and Virginia Beach - received the most overall gift money and had the highest average per legislator. They were followed by the legislators from central and Northern Virginia.\n\nThat was surprising because densely populated Northern Virginia has the most lawmakers - and a reputation for being highly courted by lobbyists.\n\nFrom the results of my research, I also created a couple of bar graphs showing legislative gifts by region.\n\nExperts' opinions\n\nNext, I contacted Stephen Medvic, an assistant professor of political science at Old Dominion University in Norfolk, and Nelson Wikstrom, a political science professor at VCU. I told both experts about my findings and asked their opinions on why Eastern Virginia legislators received the most gifts.\n\nCalos, the government watchdog, told me that political donors use gifts and campaign contributions to gain access to politicians. \"Money does buy access,\" he said. \"Access is the gateway to influence.\"\n\n(Jeff South, an associate professor in the VCU School of Mass Communications, contributed to this article.)\n\nRobb Crocker can be reached by e-mail at cobb9crock@msn.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-two-useful-datasets","slug":"2001-julyaugust-two-useful-datasets","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-two-useful-datasets.html","year":2001,"month":"July/August","monthNum":7,"headline":"Two useful datasets","kicker":"FIRES","author_name":"Raquel Dixon","author_title":"","affiliation":"IRE and NICAR","summary":"The U.S. Forest Service maintains databases on forest fires and arsons that journalists can use to investigate fire incidents, their causes, and impacts on communities. Two main databases—NIFMID for wildland fires and an arson database—provide detailed information about fire occurrence, weather conditions, and fire characteristics, though both have limitations in coverage and timeliness.","keywords":["fires","database","investigation"],"topics":["Data & Databases","Investigative Methods","Public Safety"],"technologies":["9-track tape","CD-ROM"],"full_text":"Each year, the U.S. Forest Service reports about 14,000 forest fires. Perhaps more startling, half-a-million arsons occur annually as well. These fires damage property, cost firefighters time and taxpayers money, and can sometimes kill.\n\nThe government is trying to understand how, why and when these fires occur. So they keep these incidents as records in databases. You can use the databases to find out if your city's firefighters are up to speed, if your state has to pay year after year to restore damage caused by forest fires or even who is to blame for the fire.\n\nThe National Interagency Fire Management Integrated Database (NIFMID) stores data on wildland fire occurrence and fire weather observations.\n\nThere are two databases that can of these questions and lead to stories that reveal the facts about your city and its fire protection.\n\nForest fires\n\nFirst, the forest-fire data. It's important to understand that only fires that occur within national forests are included.\n\nThe National Interagency Fire Management Integrated Database (NIFMID) stores data on wildland fire occurrence and fire weather observations. Weather data is collected from federal, state, and local fire-management agencies from throughout the United States. Fire-occurrence data includes Forest Service fire reports from 1970 to the present.\n\nWith the weather data, observations are received hourly for approximately 800 stations. Another 800 or so stations observe weather once per day. The fire-occurrence data is updated periodically throughout the year. \"Officially\" the fire data is loaded \"within 10-days of being declared out,\" but more often it is entered after the local fire season. So the agency tries to get all the data for the calendar year entered by the end of February, so it can run year-end reports.\n\nAll fires have an entry in the fire-occurrences table. Other tables offer more information about the events. However, not every fire in the occurrences table will be included in the report tables because filling out this information is optional. Most fires do have entries in the fire-resources-used, fire-events, and fire-causes tables. Other tables handle information generally only collected on large fires or for special purposes.\n\nThese tables join by the fire_id, which is unique to each fire. Some interesting information contained in fields is: land ownership, acres burned, agency responsible for reporting fire and working the fire, the cause of the fire, the intensity of the fire and the vegetation that it killed. All this information comes from fields that must be filled out.\n\nArson data\n\nThere is some incredibly detailed data found within three different tables, but there are some holes. Only 14,000 departments, in 42 states, out of 30,000 report. Also, the data is current only through 1998.\n\nThe data helps answer questions like: How many fires occurred on Fridays, Saturdays, etc.? How many fires occurred each hour of the day or month of the year? What was the average response time? How much did response time vary by station location? What was the average amount of time spent at the fire scene and how much did that time vary depending on the type of fire?\n\nFor more information, go to wwww.usfa.fema.gov/pdfdah.pdf\n\nObtaining the information may be costly. The Federal Fire Prevention and Control Act of 1974 (P.L. 93-498) authorizes the National Fire Data Center in the United States Fire Administration (USFA) to gather and analyze information on the magnitude of the nation's fire problem, as well as its detailed characteristics and trends. The system that gathers the data is called the National Fire Incident Reporting System (NFIRS).\n\nThe NFIRS Master, Incident and Equipment/Casualty tables can be ordered from the National Technical Information Service on magnetic tape (9-track, EBCDIC character set) and CD-ROM.\n\nFile descriptions and prices are: Master File, which includes fire department identification, report of submitted incidents, incident, mobile property, hazardous materials incident, equipment, civilian casualty, and fire service casualty record types. Cost: $454. Incident File, which includes fire department identification, submitted incidents and basic incident records. Cost: $454. Equipment/Casualty File, which includes fire department identification, mobile property, equipment, civilian casualty, and fire service casualty records. Cost: $305.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-teaching-car","slug":"2001-julyaugust-teaching-car","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-teaching-car.html","year":2001,"month":"July/August","monthNum":7,"headline":"Teaching CAR","kicker":"CLASSROOM","author_name":"Jeff South","author_title":"Associate Professor","affiliation":"Virginia Commonwealth University","summary":"The article describes 'stealth CAR' (computer-assisted reporting), a practical approach to teaching data journalism skills by integrating them into existing courses rather than offering standalone classes, using real-world examples and simple tools to help students understand data analysis as a fundamental reporting skill.","keywords":["CAR","journalism","data"],"topics":["Journalism & Newsroom","Investigative Methods","Education"],"technologies":["Excel"],"full_text":"At a computer-assisted reporting conference several years ago, Stephen Miller, a technology expert at The New York Times, said he took an under-the-radar approach to CAR training in his newsroom. Talking to reporters, he found out what tasks they were doing by hand - like calculating how a government budget had changed from one year to the next. Then Miller let slip that it would be easier and faster to do such computations with a spreadsheet, and would the reporter like to know how?\n\nMiller called this approach \"stealth CAR,\" and I have found it's a good model for classrooms as well as newsrooms.\n\nAt VCU\n\nVirginia Commonwealth University doesn't offer stand-alone CAR courses. Instead, faculty members integrate CAR into existing newswriting and reporting courses. That approach has certain advantages: It signals to students that CAR should be part of everyday journalism - not an esoteric specialty. Here are some tips on doing \"stealth CAR\" in J-schools:\n\nBegin with paper. Before crunching census data with Excel, for example, give students a sheet of paper with a few columns of population numbers, and have them compute the percentage change by hand or with a calculator. This will help students understand and visualize what they want the computer to do.\n\nKeep it simple. Don't start with an outer join involving three tables and a nested query. Start with sorting data in a spreadsheet and doing simple calculations. It may be better to teach a kludge in a familiar program than to teach an elegant solution with more complicated software.\n\nKeep it real. Use real data - crime, traffic deaths and other stats - available online from government agencies. Taking advantage of student-power, obtain paper records (like health inspection reports for restaurants around campus), and have your class input them into a computer.\n\nRelate CAR to other reporting skills. Describe CAR as another way of interviewing - but interviewing data instead of people. Your goal is to find bottom-line trends and then examples that illustrate the trends.\n\nPreview, present, review, practice. Tell students what you are going to teach them; tell them what they are learning as you teach them; then tell them what you just taught them. Have students practice a particular skill, like the AutoSum button and other bottom-line functions, on a variety of datasets.\n\nEmphasize people, not numbers. Reporters are in the business of telling stories, and data alone do not make a story. Discuss how to humanize the trends you discover with CAR. Then have students shut down their computers and go interview people.\n\nJeff South can be reached by e-mail at jcsouth@vcu.edu","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-uplink-story-ideas","slug":"2001-julyaugust-uplink-story-ideas","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-uplink-story-ideas.html","year":2001,"month":"July/August","monthNum":7,"headline":"Uplink Story Ideas","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The journalism newsletter invites readers to submit story ideas, technical insights, and beat suggestions related to computer-assisted reporting (CAR), with contact information provided for the associate editors.","keywords":["journalism","reporting","data"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Have you or one of your colleagues recently published a story using CAR that has not been done before or involved particularly difficult data work?\n\nDo you know of a technical problem (or its solution) that others may like to hear about?\n\nOr is there some issue or beat that we haven't covered?\n\nIf you have a story idea, we'd like to hear from you. Please contact associate editors Amy Sherrill and Mike Sherry. Amy may be reached at amys@ire.org and Mike can be reached at mikes@nicar.org. Both can be reached by phone at (573) 884-7711","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-money-and-politics","slug":"2001-julyaugust-money-and-politics","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-money-and-politics.html","year":2001,"month":"July/August","monthNum":7,"headline":"Money and politics","kicker":"STUDENTS","author_name":"Scott Finn","author_title":"","affiliation":"Charleston Gazette","summary":"A journalist investigates how special interest money influences West Virginia's Legislature by analyzing campaign contributions, lobbyist spending, and conflicts of interest to show voters why lawmakers often ignore public opinion on key issues like gambling and tobacco taxes.","keywords":["lobbying","legislature","influence"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["dBase","Excel","Access","Paradox"],"full_text":"I have a friend whose first newspaper job was covering West Virginia's Legislature for a conservative, family-owned newspaper. He sat down with the editor at the beginning of the session to talk about topics he'd like to cover, such as Medicaid funding and health care reform.\n\nThe editor soon cut him off, and said to go to the legislators with one question: \"Why aren't you doing the will of the people?\"\n\nAt first, I thought this was a ridiculous question, but the more I dug into the inner workings of our Legislature, the less paranoid this question seemed. Two examples:\n\nEven though a majority of West Virginians opposes any expansion of gambling, a bill legalizing coin-drop slot machines passed the Legislature.\n\nEven though a majority of state citizens supported a smokeless tobacco tax, it died in a Senate committee year after year.\n\nMoney influence\n\nOne obvious reason the Legislature didn't respond to what voters wanted was the influence of special interest money over the legislative process. But that's a vague answer. How can voters picture something like \"influence\" in their minds? I wanted to show how the money worked, and tell it to readers in a way that wouldn't make them sleepy.\n\nI spent a lot of time stumbling around before I finally figured out how to tell the story of how money warps the legislative process. I completed the series \"Cycle of Influence\" for the Charleston Gazette as my master's degree project from the University of Missouri. I was new to reporting, and what I knew about the Legislature came from what I read in the newspaper and heard from friends.\n\nI was fresh out of two Missouri classes investigative reporting and a computer-assisted reporting \"boot camp.\" I wanted to incorporate what I had learned into the series.\n\nI read through the IRE archives and found a lot of stories about money's influence in state legislatures. The stories tended to fall into three categories: campaign contributions, lobbyist spending, and personal conflicts of interest.\n\nIn West Virginia, different groups already had computer databases that tracked all three of these types of money in our Legislature:\n\nCampaign Contributions: A non-profit group called the People's Election Reform Coalition (PERC) had campaign contributions to state legislators going back a decade. PERC includes the state chapter of Common Cause, the West Virginia Citizens' Action Group, and other reform-minded groups.\n\nLobbyist spending: Two databases existed at the state Ethics Commission - one of lobbyists and their registration, the other of lobbyist spending on meals, gifts, and campaign contributions.\n\nConflicts of Interest: Fortunately, the 50 States Project of the Center for Public Integrity had just completed its nationwide searchable database of every personal disclosure form of every state legislator in America (www.50statesonline.org).\n\nThe campaign contributions database was in Dbase and Microsoft Excel, and both translated easily into Microsoft Access, which I used to analyze the data. The lobbyist databases were in Paradox, but also were translated into Access.\n\nI had no trouble getting access to any data. Lucy Suchy at the state Ethics Commission even gave me free disks. She said no one had ever asked for her databases before.\n\nOnce I began to analyze these databases, I saw some disturbing trends. For example, gambling and tobacco industry donations were increasing - more than 200 percent for gambling and 800 percent for tobacco in just two years.\n\nI struggled with how to tell these stories so readers would understand and not get bogged down in numbers. The best stories in the IRE archives tied one of these money issues into a \"real\" issue people cared about - like a bill about filling in wetlands, or locating a toxic dump near a low-income community.\n\nGambling and tobacco\n\nI focused on two gambling and tobacco bills because of the disconnect between public opinion and what the Legislature actually did.\n\nThe Gazette's writing coach, Kate Long, helped me think of ways to show readers how things worked instead of telling them. I picked people on the losing side of the gambling and tobacco bills and reconstructed their experiences in fighting against the bills.\n\nOne anti-gambling activist, a minister, provided some great examples of how lobbyist money buys influence. A gambling lobbyist hangs his coat in a legislator's office and walks in without invitation, while the minister has to wait in the hall. Teams of lobbyists with cell phones tag-team different hearings and meetings with legislators, while the minister wears lace-up shoes instead of loafers so he literally can run from the House to Senate chambers during certain critical hearings.\n\nThe computer-assisted analysis gave flesh to the anecdotes. Pro-gambling lobbyists outspent anti-spending lobbyists by a ratio of 66-1. Legislators who voted to legalize slot machines in the state on average received more than four times as much in campaign contributions from the gambling industry as those who voted against the bill.\n\nIn a story about the failure of a smokeless tobacco tax, an anti-smoking activist told the story of her father's cancer death from chewing tobacco.\n\nShe then talked about her sense of betrayal as a legislator who knew both her and her father voted to keep the smokeless tobacco tax bottled up in the Senate Finance Committee.\n\nThe CAR analysis showed this committee had six of the top ten recipients of tobacco industry contributions in 1998. The committee chairman has a personal financial interest in several convenience stores that sell tobacco products.\n\nImpact\n\nSince the series ran, the Legislature passed a smokeless tobacco tax, after more than a decade of defeat. I would like to think the series helped put pressure on legislators to pass the bill.\n\nThe gambling industry suffered its first major defeat in a decade last fall when voters in Greenbrier County defeated a measure that would have allowed casino gambling at The Greenbrier, a four-star resort.\n\nThe results of \"Cycle of Influence\" and follow-up stories were part of the debate over gambling's increasing influence in the state.\n\nI have continued to follow the gambling industry and its influence on politics in West Virginia. In October, I published a story showing that contrary to popular belief, the Republican governor who publicly opposed gambling quietly was raking in tens of thousands from the gambling industry in campaign contributions. He lost in a close election.\n\nBut gambling forces did help pass a bill to increase the minimum slot machine bet from $2 to $5 in this legislative session, showing their increasing clout among legislators, if not voters. Another bill passed that legalized, taxed and limited the number of video poker machines. During that debate on video poker, an anti-gambling legislator was almost censured by the House of Delegates for what he said during a debate: that there is a correlation between gambling contributions and how legislators voted.\n\nAs long as the Legislature ignores the will of the people, the Gazette will continue to show how money warps the political process.\n\nScott Finn can be reached by e-mail at sfinn@wvgazette.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-information-manager","slug":"2001-julyaugust-information-manager","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-information-manager.html","year":2001,"month":"July/August","monthNum":7,"headline":"Information manager","kicker":"PROGRAMS","author_name":"Mark Schaver","author_title":"federal courts reporter","affiliation":"The Courier-Journal","summary":"Mark Schaver reviews Zoot, a powerful shareware information management tool that helps journalists organize and retrieve interviews, documents, web pages and other research materials efficiently.","keywords":["software","organization","journalism"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Zoot","askSam","Info Select","Word","Microsoft Outlook","Access","PACER","Internet"],"full_text":"Information manager\n\nBy Mark Schaver\nThe Courier-Journal\n\nA frequent question on journalism discussion groups is: What software will organize the flood of interviews, phone numbers, e-mail, documents, Web pages and other information I'm inundated with every day? Some use the free-form database askSam. Others like Info Select. Still others just dump everything into Word files.\n\nTool of choice\n\nFor years, my tool of choice has been an obscure but powerful shareware program called Zoot. Zoot's lone programmer, Tom Davis, whose nickname is \"The Admiral,\" calls it an \"information processor.\" Zoot makes it easy to vacuum up large quantities of information from a variety of sources, organize it automatically, and then find it fast days, weeks or years later. Davis, who taught himself to write code, said he felt compelled to develop Zoot because the first computer program he used to gather information, Microsoft's cardfile, was so inept, and there was nothing else that served his needs. \"Information management is the only reason I use a computer,\" Davis said in an e-mail interview.\n\nZoot useful everywhere\n\nEvery day of my working life is spent in Zoot. I use Zoot to keep my task list and my phone contacts. It's closely integrated with Microsoft Outlook, and information kept in one program can be synchronized with the other. You can also synchronize with Outlook's calendar, notes, journal and e-mail functions, although I personally don't use Zoot for those.\n\nZoot places a small Z - called the Zooter - on the Window's title bar. The Zooter gives you access to Zoot from any open application on your desktop. You can then rapidly grab text from other programs - either by selecting text or grabbing a whole page - and dump it in Zoot.\n\nYou can choose to either automatically add what you gather to a single folder within Zoot as you cut and paste while Web surfing, or you can open a window that allows you to assign your information to multiple folders, categorize it by subject or type notes. Folders can be further divided into separate \"databases,\" which can then be grouped together as \"projects.\"\n\nYou can also use Zoot in reverse, taking text already stored in Zoot, and inserting it into a document while using another program. This is great for adding boilerplate text to Freedom of Information Act requests and the like.\n\nZoot keeps track of every Internet site you visit. To return to a site takes one click on an Internet address in Zoot, which will open up your browser and return you to that site. Double click on an e-mail address, and it will open up your e-mail program.\n\nI use Zoot to gather Web sites useful to reporters, quickly typing in descriptions with the Zooter. Later I import them into an Access database, which can be searched over our newsroom Intranet.\n\nBut Zoot's inner beauty is that you can customize it in limitless ways. You can use what Zoot calls \"rules\" and \"actions\" to organize text once it's in Zoot.\n\nZoot makes it easy to vacuum up large quantities of information from a variety of sources, organize it automatically, and then find it fast days, weeks or years later.\n\nIf you are working on a long project, you can set up Zoot to automatically sort any notes, articles or other information you gather using what Zoot calls \"smart folders.\" For example, you can create a folder for every major figure in an investigation, and any information you gathered would instantly be placed in the appropriate person's folder.\n\nTracking court cases\n\nWhen I was a federal courts reporter, I used our court's Web version of PACER and Zoot to grab the court dockets for cases I was following. To check on whether there was any new action on a case, I would just click on the item in Zoot and it would return me to the case docket, where I could see if there was any new activity.\n\nAnd I configured Zoot to automatically sort cases by prosecutor and judge, so I could see at a glance who was handling what.\n\nEach nugget of information in Zoot is called an item. Keyboard shortcuts make it easy to quickly read through them. You can also create links between items, so clicking on a button will take you to other, related items. It's also easy to create new items, combine them, or drag text from one item to create another. You can also configure Zoot to automatically change an item's color based on text contained within, such as turning all items with the word \"urgent\" in them red.\n\nSearching tools\n\nZoot has multiple, fast search tools. A \"quick query\" box lets you type keywords and instantly see any items containing them. You can filter or search items by numerous criteria, such as the date it was created, the last date it was modified or its subject. A more advanced search will show you the first few lines of each match. There are also a number of buttons and shortcuts that make it easy to navigate through the program.\n\nZoot can be obtained at http://www.zootsoftware.com and costs $99.\n\nThat buys you the right to install it on both your home and work PCs, which is good because another Zoot feature is the ability to automatically synchronize information between your home and work PCs by e-mail or \"sneaker net\" - nerd slang for floppy disk.\n\nNot always perfect\n\nZoot is not perfect. For the uninitiated, Zoot can be difficult to learn, and documentation for the latest version, 4.0, a radical redesign from previous versions, is incomplete at this writing, although Davis says he is working on it and will be finished soon.\n\nItem sizes are limited, so large documents are split into multiple items that are given sequential numbers. You can print Zoot items to a printer or a file, or create a simple Web page out of them, but formatting options are few. Recently Davis added the option to print through Internet Explorer or Outlook.\n\nZoot also makes a poor substitute for a word processor because there is no text formatting. Zoot also doesn't handle graphics although you can configure it to use an external file viewer. You can also drag an Adobe Acrobat, PaperPort or similar file into Zoot, which will automatically create a link to them on your hard drive. You can then type notes and click on the file link, which will call up the associated program.\n\nZoot has been remarkably stable on my Windows 2000 and Windows 98 machines. But as you would expect from any program this complicated developed by one man, there are small bugs.\n\nYou can also configure Zoot to automatically change an item's color based on text contained within, such as turning all items with the word \"urgent\" in them red.\n\nNevertheless, Davis is exceptionally responsive and fixes them quickly. He is constantly tweaking the program, and often makes changes requested by users. He also maintains a Zooters mailing list to get feedback and post information about Zoot.\n\nCult-like following\n\nDavis doesn't advertise Zoot, but he has developed a cult-like following. One of his biggest boosters has been the writer James Fallows, who has touted Zoot in The Atlantic Monthly and the Industry Standard.\n\nFallows was one of many participants in a forum on CompuServe for the now-defunct program Agenda, another information manager whose loyalists turned to Zoot after it was abandoned by Lotus in the early 1990s. On that forum Davis picked up his nickname as The Admiral, but he can't explain why, except that it has something to do with his favorite book, A Confederacy of Dunces.\n\nZoot is named after the 1976 Frank Zappa album, Zoot Allures, the meaning of which remains equally obscure. Some fans say Zappa's album paid homage to the Zoot suit. Others say it is derivative of the French expression, Zut Alors, which translates as a mild exclamation like \"Dammit\" - which just happens to be the typical expletive of reporters overwhelmed with information on deadline.\n\nMark Schaver can be reached by e-mail at inschaver@courier-journal.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-internet-health-data","slug":"2001-julyaugust-internet-health-data","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-internet-health-data.html","year":2001,"month":"July/August","monthNum":7,"headline":"Internet health data","kicker":"THE WEB","author_name":"Devon Hammes","author_title":"","affiliation":"IRE","summary":"The Health Care Financing Administration provides publicly accessible Medicare data through its website that journalists and the public can use to research healthcare providers, costs, and facility performance. These datasets, including information on renal providers, nursing home comparisons, and home healthcare agencies, serve as valuable starting points for reporting stories and preparing interviews.","keywords":["Medicare","healthcare","data"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"The Internet is host to many different options when looking for information quickly, and one of those options is health data.\n\nThe Health Care Financing Administration (HCFA) has several public-use files that can be helpful for looking into Medicare or useful supplements to other health care articles.\n\nThere is a catalogue of all public data Medicare provides available for downloading at www.hcfa.gov. The catalogue includes information on the different types of data available, pricing and how the data is available.\n\nSeveral of the datasets would be a great tool for the general public and reporters alike to compare service, statistics and overall operations of Medicare. Of course, you can't write a story on data alone, but some Web sites can be a great starting point and can be used as tipsheets to prepare for interviews.\n\nSome Web sites can be a great starting point and can be used as tipsheets to prepare for interviews.\n\nSome of the HCFA data includes information about providers, cost limits and payment rates. The data can be accessed at www.hcfa.gov/stats/pufiles.htm. Some of the provider data includes the End Stage Renal Disease (ESRD) Renal Provider File, which contains information about Medicare-approved providers who supply kidney dialysis and/or kidney transplant services. The data includes the location of the providers and the range of renal services available at those providers.\n\nHCFA also offers the (ESRD) Renal Facility Survey. This data is collected annually by HCFA from all facilities certified to provide Medicare-covered renal dialysis and transplantation. The survey includes the entire U.S. and lists the facilities by zip code. Each record contains information about the facility and the number of patients that have been treated, the number of dialysis treatments provided and the number kidney transplants performed. The data includes services to both Medicare and non-Medicare patients.\n\nThe Medicare home page at www.medicare.gov provides a lookup function that will compare nursing homes. It is possible to search for a specific Medicare- and Medicaid-certified nursing home in the U.S. You can compare nursing homes in an area or find information on a specific one.\n\nThe database contains information like the percent of residents with behavioral symptoms and other information about the specific nursing home. There is also an option of comparing it to the average of residents with behavioral symptoms in the U.S. and in the state where the nursing home is located.\n\nThis look-up page also lets you view recent inspection results, with a column that measures the level of harm the deficiency could have caused. This table breaks it down into categories such as: mistreatment deficiencies, quality care deficiencies and resident assessment deficiencies.\n\nThe future\n\nHCFA has implemented new regulations for all home healthcare agencies. The agencies will have to obtain information from all adult, non-maternity patients and submit the data to HCFA. Home-health agencies use a standard core assessment dataset, the Outcome and Assessment Information Set (OASIS). OASIS contains reports on all home-health agency patients, and HCFA uses it to monitor home healthcare quality and to determine payments for individual patients.\n\nHCFA is still debating whether to release the data to the public. It would offer valuable information on how home-health agencies measure up. This data would be a great tool to compare service and statistics. The agencies, however, are also required to ensure the privacy of many pieces of information kept within this data, so many of the fields will be redacted to ensure the anonymity of the patients.\n\nDevon Hammes can be reached by e-mail at devon@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-dealing-with-race","slug":"2001-julyaugust-dealing-with-race","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-dealing-with-race.html","year":2001,"month":"July/August","monthNum":7,"headline":"Dealing with race","kicker":"CENSUS","author_name":"Erin McCormick","author_title":"","affiliation":"San Francisco Chronicle","summary":"Journalists covering census data between 1990 and 2000 must navigate new multiracial categories and measurement challenges by identifying clear trends rather than claiming scientific precision, while being careful about census tract boundary changes that can distort findings.","keywords":["census","demographics","methodology"],"topics":["Demographics & Census","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"There is no way to measure exactly how much racial groups changed between 1990 and 2000 because of new categories in the census. The fact that respondents can choose more than one race creates a whole new playing field. The best hope is to identify clear trends, then convey them in a way that doesn't mislead readers into thinking that it's an exact science.\n\nHere are a few simple techniques for looking at change:\n\nCompare the percentage of 'minorities.' Look at how many people list themselves as something other than non-Hispanic white to get a sense of whether the community has gotten more or less diverse. Let readers compare the apples and oranges for themselves.\n\nLook at how many people list themselves as something other than non-Hispanic white to get a sense of whether the community has gotten more or less diverse.\n\nPut multiracial respondents in their own category. Then, run pie charts with the new 2000 categories and the old 1990 categories side by side. Readers will see that things have changed, but they will also get a sense of the general trends.\n\nWhen studying a single race, look at the range of potential change. Say you want to look for changes in the American Indian population alone. Look at both the number of people who described themselves as being American Indian \"alone\" and the larger group of those who describe themselves as being partly or wholly of American Indian heritage. Compare that to the 1990 population. If both comparisons show an increase or a decrease, you have a clear trend. If not, it's impossible to conclude what's really happening.\n\nLook at Hispanics as if they were a race. Build 1990 and 2000 pie charts showing Hispanics displayed as a race, alongside non-Hispanics in each race (non-Hispanic whites, non-Hispanic blacks, non-Hispanic Asians, etc.). This works well in areas like the West where most Hispanics are white. If there are many Hispanic blacks, it will result in an undercount of blacks.\n\nPitfalls to avoid:\n\nSometimes it will look like the population of a race has decreased when you compare the number of people who describe themselves as being of that race alone to 1990 figures. But, the population will appear to have increased when you compare those who chose that race alone or in combination with another race. This is a statistical tie. Because of the sometimes huge margin of error brought in by the new mixed-race categories, it's impossible to tell what's really going on.\n\nCensus tract boundary changes\n\nWhen comparing 1990 census tracts to 2000, make sure the Census Bureau hasn't quietly rejiggered the tract boundaries. Usually, when the Census Bureau splits or combines tracts, it changes the tract number, so it's easy to tell there has been a change. But, we found many cases where the bureau shaved off a tiny sliver of a tract with no announcement. It often seemed to be a sliver that had a lot of the population in it, making it easy to draw some pretty wild conclusions about unexplained population declines.\n\nErin McCormick can be reached by e-mail at emccormick@sfchronicle.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-julyaugust-bits-bytes-and-barks","slug":"2001-julyaugust-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2001-julyaugust-bits-bytes-and-barks.html","year":2001,"month":"July/August","monthNum":7,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR has updated its U.S. Coast Guard boating accident and boat registration databases, which journalists can use to analyze local boating safety data, with pricing based on circulation size. The newsletter also highlights a new IRE Beat Book on using math and statistics in journalism and provides access to tipsheets from the recent IRE National Conference.","keywords":["boating","databases","journalists"],"topics":["Data & Databases","Journalism & Newsroom","Public Safety"],"technologies":[],"full_text":"Bits, Bytes and Barks\n\nNICAR Data Updates\n\nThe summer boating season has started and that means it's time for accidents. You can get a better picture of accidents, injuries and the boats in your area by analyzing the U.S. Coast Guard's boating accident and boat registration databases. The NICAR Database Library recently updated both datasets.\n\nThe boat registration data is current as of December 2000. The boat accidents data runs from 1969 through 1999, the most current information available from the Coast Guard.\n\nThe accidents database includes all recreational boat accidents involving: death, injury requiring more than first aid, vessel damage greater than $500, or the disappearance of a person from the vessel under circumstances that indicate death or injury.\n\nThe boat registration database has owner information and other details about currently registered commercial vessels and large recreational vessels (26 net tons or greater). A second table in the database includes all vessels, expired registrations and smaller recreational vessels.\n\nMore information about both datasets, including sample tables and record layouts, is available at www.ire.org/datalibrary/databases\n\nPrices for the databases are:\n\nBoat accidents. Entire U.S., all years:\n\nCirculation below 50,000 or market 50-200 (university freelance): $25\nCirculation 50,000 to 100,000 or market 25-50: $50\nCirculation above 100,000 or top 25 market: $75\n\nBoat registration. Entire U.S.:\n\nCirculation below 50,000 or market 50-200: $40\nCirculation 50,000 to 100,000 or market 25-50: $80\nCirculation above 100,000 or top 25 market: $105\n\nTo order, call (573) 884-7332. Or you can download an order form from www.ire.org/datalibrary/orderform/orderform.pdf\n\nMath for journalists\n\nThe fourth volume in the IRE Beat Book Series is titled \"Numbers in the Newsroom: Using Math and Statistics in News.\" The guide, written by Washington Post database editor and former IRE training director Sarah Cohen, focuses on putting numbers into perspective for stories. It's also a great deadline guide.\n\nOther books in the series: \"Understanding Crime Statistics,\" \"Covering Aviation Safety,\" and \"Home Mortgage Lending: Detecting Disparities.\"\n\nAdditional information and ordering details can be found at: www.ire.org/store/books/math.html\n\nTipsheets from IRE National Conference in Chicago\n\nTipsheets from panels at the June IRE conference are available at www.ire.org/resourcecenterlinitial-search-tipsheets.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-septemberoctober-naming-debtors","slug":"2001-septemberoctober-naming-debtors","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-naming-debtors.html","year":2001,"month":"September/October","monthNum":9,"headline":"Naming debtors","kicker":"TAXES","author_name":"Howard Fielding","author_title":"Publisher/Editor","affiliation":"Waterbury Republican American","summary":"An investigation into Waterbury's fiscal crisis reveals over $55 million in unpaid taxes. A published list highlights delinquent real estate taxpayers, questioning if the city's financial woes could be solved by collecting these debts.","keywords":["taxes","debt","Waterbury"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":[],"full_text":"In the Tax Collector's Office in Water- bury, Conn., there's a box about the size of a file cabinet drawer. It weighs roughly 50 pounds and is filled with accordion-folded printouts of delinquent real estate, auto and personal property tax bills totaling almost $55 million. That's roughly a million dollars per pound of good stories in this city, where a fiscal crisis forced a state takeover of the budget process last year. The Republican-American has covered news of the city's budgetary woes as it developed: the liens that were sold to a company that itself went bankrupt; the contract and pension disputes; the state's oversight board. The city, which is facing a staggering two-year deficit of about $75 million, received so much state aid this legislative session that one lawmaker jokingly referred to the collection of provisions as \"Waterbury welfare.\" Unanswered questions Several questions kept nagging readers - and the publisher and editors - of this family-owned paper: How much did the delinquent taxes contribute to the money pit, and would the city be solvent if it could collect all the outstanding debt? We decided to obtain a list of the delinquent taxpayers, analyze it, and publish it in a format that people could understand. Better yet, who are the deadbeats? Inquiring minds want to know. We decided to obtain a list of the delinquent taxpayers, analyze it, and publish it in a format that people could understand. The result was a June 17 cover story by investigative reporter Darlene McCormick and a six-page special section that ran in the city and suburban editions on June 17. Among her findings: The top 10 real estate debtors combined owed more than $6.4 million but the city is unlikely to collect any of that soon because of court actions, environmental issues, and the sale of liens. The property that showed the biggest outstanding debt - $1.3 million in bills going back to 1985 - was sold that year to the state, and meanwhile all the bills were going to the wrong place. A large part of the problem was poor record-keeping by the city. The special section listed some of the delinquent taxpayers and how much they owe, based on records the Tax Collector's Office at City Hall gave to the newspaper in March. Naming names We published the names of 2,159 of the more than 9,000 delinquent payers of real estate taxes only. Delinquencies on personal property, such as business equipment and fixtures, or on motor vehicles were not included because real estate makes up more than half the outstanding balance and is much easier to track. Months of research went into formatting the records we received from the city. The list covered a 15-year period from 1985 to 1999 and reflected all payments made through March 22, 2001. The newspaper requested and received names and amounts of every delinquent, even if they owed a nickel. Because of space limitations, only delinquents who owed $1,000 or more on real estate were listed in the paper. Also, most delinquents owing less than $1,000 owed only for the 1999 supplemental assessment or owed taxes on properties of very little value. The project started last summer, when the newspaper first requested a list of delinquent taxpayers for analysis. The data the city provided then went back only a few years. It was not enough to publish a full and accurate list, but was enough to help uncover some of the problem spots in the system. The newspaper made its second request in February, this time intending to get a full list for analysis and possible publication. The Tax Collector's Office was willing to cooperate, but asked if we could wait until the end of March because it was swamped with a wave of payments from January bills. We agreed. We asked for all data available on delinquent tax bills, going as far back as possible. Computerized records of tax bills are available back to 1982. On March 23, the tax collector's office exported this data, reflecting all payments made through March 22, into four Excel spreadsheets - one for real estate property, one for business fixtures subject to personal property tax, one for motor vehicles through 1998, and one for motor vehicle bills delinquent from the 1999 regular and supplemental levies. We decided to concentrate on the real estate list. Using Excel, we sorted the spreadsheet data by map, block and lot number and ran subtotals for each of the 9,354 delinquent properties. First, we had to clean the data because the information we wanted to publish - the owner's name - was spread out over two fields in the spreadsheet. This had to be searched and checked manually to ensure consistency, and this step was done over the course of a week. The real estate property tax list consisted of more than 30,000 bills going back as far as 1982, listing taxpayers alphabetically. At first this seemed to be exactly what was needed; we could search for a name, group all the outstanding bills and subtotal them. We discovered it wasn't that simple. When we checked on the name of one businessman on the oversight board, for example, we found a large outstanding balance. When we double-checked with the taxpayer and the city tax records, however, we found that he had sold the property listed in his name and had paid the remainder of his taxes. This made it clear we had to sort first by some unique key for the property-we chose map, block and lot number and find a sub-total for each property. Then we'd needed to identify the last known owner listed for that property. Using Excel, we sorted the spreadsheet data by map, block and lot number and ran subtotals for each of the 9,354 delinquent properties. Then we copied the owner's name from the last 1999 listing for each of the properties. This step, plus editing the names again for consistency and typographical errors, took more than a week. The backbone We then opened the Excel 2000 spreadsheet, with records for each bill and subtotals for each property, into a Word 2000 table. This allowed us to run a macro that edited out the individual bill records, leaving only the subtotals for each property. We saved this document as a text file and opened it in Access 2000 to give us a working database of the outstanding debt for each property. This database created the Top 25 properties list and formed the backbone for the delinquent properties database that was published on the Web site. That got us only halfway to the list in the special section, which provided combined totals for each taxpayer (because many own more than one property). Shifting software To get that result, we took the properties list, sorted it alphabetically by owner in Access, and saved the result as a text file. We opened that table in Excel, subtotaled it by owner, and saved the result as a text file. That table included individual property balances and the subtotals by owner. We then imported that text file into Word 6.0 for publication. We used Word to change the names from all-capital letters to Title Case, and to clean up known errors. Some long names may have been truncated or shortened in moving the tables from Excel into Access; otherwise, spellings and listings are in the format originally provided by the tax collector. Work turning the spreadsheet into the list took about three weeks. Only after it was done did we discover that under state law, bills that have been outstanding more than 15 years are not recoverable. We went back to the original spreadsheet to identify the 54 properties that were delinquent that far back, subtotaled the outstanding debts from 1982-84 for each, and manually subtracted those sums from the final list. In the week afterward, hundreds of complaints filtered into the city's tax office and the newspaper, but city officials said few who owed taxes paid their bills. Most complaints came from those who said they didn't owe because their property had been sold, but they were still listed as the owner. Others said they paid their bills after March 22, the cutoff date for publication. Then there were those who admitted owing back taxes, but complained that the newspaper had no business printing their names. Howard Fielding can be reached by e-mail at hfielding@rep-am.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-mapping-molesters","slug":"2001-septemberoctober-mapping-molesters","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-mapping-molesters.html","year":2001,"month":"September/October","monthNum":9,"headline":"Mapping molesters","kicker":"SEX OFFENDERS","author_name":"Mark Douglas and Rocky Glisson","author_title":"Database Manager","affiliation":"WFLA-TV","summary":"A CAR investigation exposes how more than 100 sex offenders in Florida live too close to schools and day care centers, violating the 1000-foot rule intended to protect children.","keywords":["sex offenders","Florida","mapping"],"topics":["Crime & Justice","Investigative Methods","Technology"],"technologies":["GIS","MapInfo","Access","ASCII"],"full_text":"Six years ago, Florida lawmakers passed legislation designed to protect kids from child molesters. We used CAR techniques to show that law was a farce. Our WFLA-TV investigation uncovered more than a hundred molesters on probation living close to schools and day care centers in violation of Florida's \"1000-foot rule,\" a measure that's supposed to help separate predators from their prey. Using Geographic Information Systems (GIS) software we mapped one county's population of child molesters to look at their proximity to schools and child-care centers and determined there was a pattern of violations. We then targeted a few of the worst violators for narrative storytelling. The project began with a phone call last August to the Florida Department of Law Enforcement (FDLE) in Tallahassee, which maintains a database of sex offenders online. The FDLE Web site enables anyone to search for sex offenders by name, address, etc. For our purpose, we needed more information. We asked for the FDLE's raw database so we could correlate offender addresses with schools and day care centers using our GIS software. FDLE's data managers balked at our initial request so we followed up with a letter. Apparently, we were the first journalists to ask for the underlying data (always an encouraging sign). Within two days, FDLE's legal counsel agreed to release it under Florida's Public Records Law. We received data for thousands of registered sex offenders via e-mail in ASCII delimited text (which we easily converted to a Microsoft Access table). We also asked for and received a key to explain some of the more confusing fields critical to our project (such as dates of arrest versus dates of offense, codes that indicated victims under 18, and so forth). Step by step, we refined the data using Access software to meet our exact needs: 1) We selected all of the sex offenders living in Pinellas County. 2) We deleted everyone except offenders who had committed crimes against kids. 3) We ordered the data by date of offense and deleted child molesters who had committed offenses before October 1995, when the \"1000-foot rule\" took effect. 4) We selected only the offenders currently on probation. These four steps gave us an offender database of about 300 individuals precisely matching our purpose. By this time we had already acquired the other two databases needed for our project: The School Board supplied us with the names and street address of every public school in Pinellas County. The Juvenile Welfare Board sent us the location and operator of every licensed day care center or family-care home. All three databases arrived by e-mail in the form of text tables that were easily imported to Microsoft Access. WFLA-TV Database Manager Rocky Glisson then plotted the most updated address on each of the sexual offenders we'd filtered from the FDLE text file. He also plotted the addresses of each licensed childcare center, and all public schools in Pinellas County. All of the plotting was done using the MapInfo program. With this software, we could plot the addresses from Microsoft Access directly into MapInfo. Sex offenders in the \"red zone\" Next, using. MapInfo's radius tool, Glisson determined how many of the sexual offenders were within the 1000 foot \"red zone\" of any of the day-cares and schools. He found plenty. Out of approximately 300 molesters, Glisson plotted over 100 in apparent violation of Florida law. Because of the number of violators, the limitations of MapInfo, and Glisson's attention to detail, mapping these offenders became the most tedious and time-con- suming part of the project. Glisson verified by hand any offender that lived near a day-care or school. He repeated the process at least four times to validate his results. Results surprised reporters Those results surpassed our expectations, and established the underlying CAR basis for Mark Douglas's narrative story. Douglas began putting faces on the data by using the courthouse computer system to locate individual cases. He then manually researched the hard copy case files, with the help of News Videographer Eric Hulsizer and Special Projects Producer Marie Hickman. To simplify the process for the file research team, Douglas created a hard copy form that uniformly noted basic information about each offender for later comparison. Old fashioned footwork In Florida, court clerks \"sanitize\" criminal files to remove identification of sex crime victims, especially if they are children. To work around this, we studied the witness lists for related names that often are a tip-off to child-parent relationships, as an indirect way of determining the victims' identity (none of which we revealed in the stories). In the end, we focused more on the criminals, the probation officers who supervised them, and the judges and prosecutors who were supposed to ensure the 1000-foot rule was invoked and enforced. We also talked to parents, neighbors, educators and day care operators who expressed concern about the system failures our CAR project had uncovered. Measuring the distance. We headed into the neighborhoods located by our MapInfo program, and manually verified the proximity between kids and molesters with a handheld measuring wheel borrowed from our maintenance department. The measuring wheel, which we rolled from point to point, turned out to be a good storytelling device and made our lawyers happy. Our reporting eventually took us to the Pinellas Sheriff's Office where deputies were testing a new computerized tracking program of their own. We asked them to plug-in the names of some of the sex offenders we'd targeted as a third way to verify our results. Our lawyers loved that, too. Multimedia Our multimedia reports took the form of TV stories, several articles in the Tampa Tribune, and an online map featured on our web site TBO.com. (http:// ports.tbo.com/reports/nov2000loffendersl) That online graphic enabled viewers to open smaller maps detailing the location of offenders in their neighborhoods relative to schools and day care centers. All three versions of our \"convergence\" reports on child molesters unfolded five days before Florida's presidential election. Despite the obvious political distractions, the investigation became a Tampa Tribune front-page story, lead our TV newscasts, and hit the AP wire several times. The public response was pronounced and immediate. Florida Governor Jeb Bush fired off a letter to the chief justice of the Florida Supreme Court asking for reforms. The chief circuit judge in Pinellas County took action to fix the loopholes and mistakes that had allowed hundreds of molesters to avoid the 1,000-foot restriction in Pinellas County. Probation officials began searching our online database to correct their own mistakes. Undeniable proof The use of CAR techniques on this topic delivered undeniable proof of a systematic problem in Florida's criminal justice system. Three months after that first phone call to FDLE, our CAR project coupled with plenty of old-fashioned courthouse research and field reporting generated some compelling stories that helped reform how Florida's justice system manages child molesters. Some tips to follow: Plan well ahead to allow for slow response from the data keepers. Talk directly to the data managers and ask for information in a form that's easily importable to Access, and whichever GIS program you're using. Make sure you get an accurate data key to eliminate confusion over dates and codes. Consider adding more databases such as school bus stops, parks and pri- vate schools. Make sure you have the most current offender data. Verify with other sources like the Department of Corrections, as this population tends to be transient. Look for a GIS program that will automatically plot designated radii. As always with a CAR project, be suspicious if your findings look too good to be true. Find other ways to verify the results. Remember, the CAR data may give you supporting data, but let people tell the story. Mark Douglas can be reached by e-mail at mdouglas@wfla.com and Rocky Glisson can be reached at rglisson@wfla.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-tax-breaks","slug":"2001-septemberoctober-tax-breaks","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-tax-breaks.html","year":2001,"month":"September/October","monthNum":9,"headline":"Tax breaks","kicker":"CORRUPTION","author_name":"David Herzog","author_title":"investigative reporter","affiliation":"The Providence Journal","summary":"An analysis of Palm Beach County's affirmative action program reveals significant increases in contracts awarded to minority and women-owned businesses over a decade, despite political pressure to terminate it.","keywords":["affirmative action","contracts","minorities"],"topics":["Investigative Methods","Business & Economy","Government & Politics"],"technologies":["Access","GIS","spreadsheet"],"full_text":"More than two years ago, FBI agents raided Providence City Hall and lugged box loads of documents out of the tax assessor's office as part of a probe of corrupt city officials. It was a reporter's nightmare: the documents that we would need to check into the crooked tax deals that we were starting to hear about were in the firm grip of the feds. The agents also took school department lease records. Using databases, GIS and spreadsheets we were able to bridge gaps in the paper trail and come up with some great stories. When the Providence Journal's four-member investigative team was assigned to stay on top of the federal probe while developing our own stories, we knew that computerized records would play a key part in our reporting. Using databases, GIS and spreadsheets we were able to bridge gaps in the paper trail and come up with some great stories. Among other things, we used CAR to report stories about a mob associate and his wife getting improper tax breaks and a career criminal getting millions from City Hall in lease and property sale payments. Learning the ropes Early in our reporting, we knew that the FBI had a keen interest in investigating whether city tax board members had squeezed bribes from property owners for property-assessment reductions. For property owners, any decrease in the assessed value meant a lower tax bill. The question was: Who got the improper tax breaks? We also know that before jumping into any stories we had to learn how the city's tax system is supposed to work. Like in many other jurisdictions, we found that the city assessor, or its contractor, sets the value of every property in the city based upon the features of the property (size, number of bathrooms, improvements on the property). If a property owner believes the assessor made an error in the assessment (say, by miscalculating the square footage of the lot) the property owner may ask the assessor's office for a corrected assessment. In other cases, where the property owner and assessor's office disagree about the value of the property, based on fair market value judgements, the property owner can appeal to the city tax board. Property owners make their cases before the board at public hearings, and then the board decides whether to grant a reduction. First step Our first mission was to get hold of the Providence tax board's database of assessment hearings. The only problem was, it didn't exist. The board's secretary kept handwritten hearing dockets that were barely legible, sometimes cryptic and often incomplete. We had to find another way to zero in on the reductions, and decided that database matching would work, at least in a crude way. We already had an extract of the assessor's database from 1997 in Microsoft Access format that we had obtained at no cost from the city Department of Planning and Development. All we needed was a more recent copy of the database, so we could see which properties had declined in assessed value. The 10-mb database contained the key information about all 47,000 properties in Providence, from the newest condos to sprawling industrial complexes vacated years ago. And it was very clean, no doubt because the city used to it generate tax bills. By looking at the table, we could see the street address of the property, current value (for the land and improvements), name of the owner, the tax-bill address, parcel type and, in many cases, the most recent sale date and price. Also, the database contained fields for the plat, lot and unit of the property. We got an updated copy of the database at no cost from the city Planning and Development office after sending a public-records request. We wanted to join the two tables in Access and construct a query that returned just those properties with a decrease in value. First, we needed to create a unique identifier for each property so Access could match the properties one-to-one. To do this, we created a new field in the tables called \"PID\" for property ID. Then, we created update queries that filled the PID fields with a unique value concatenated with the plat and unit numbers and hyphens. One name jumped out: Paul Calenda, a mob associate who spent time in prison for possessing an Uzi machine gun. With a unique field, we were ready to roll. We constructed a query that joined the two tables on the PID fields and, using the expression builder in Access, selected only the records of properties whose current assessment was lower than the earlier assessment. That left us with more than 1,300 properties: a lot, to be sure, but not an unmanageable number. We built a report in Access that listed each reduction on one line. Which reductions were improper? The data couldn't tell us. So we scanned the names on the list, looking for patterns or the politically connected. One name jumped out: Paul Calenda, a mob associate who spent time in prison for possessing an Uzi machine gun. All told, we found three properties owned by Calenda or his wife that had gotten assessment reductions. Seemed like a good candidate for a story. Building property profiles Normally, we could have gotten detailed information about the properties and assessment reductions from the property file folders (arranged by plat, lot and unit) in the assessor's office. But when we asked for the folder, clerks told us that the folders were gone - the FBI had taken them. Now we knew we were onto something. We created Excel spreadsheets to create profiles of each property, using information we collected from the annual tax rolls and tax bill printouts from the city collector's office. The tax rolls told us the land and improvements value of the properties over the years, and the tax bill printouts showed us the history of payments and credits resulting from reductions granted by the tax board. We discovered that all three of the Calenda properties had credits from the tax board applied on the same day. There's nothing fancy about this, but it's just a simple way to track information about one property in one place, so we all could learn a lot at a glance. We also used these sheets to calculate the one-year tax savings the Calendas got from the reduction, nearly $17,000. With a solid foundation built with our CAR work, we were able to weave in all the great information we were getting from interviews and documents: how the mayor had invited Calenda onto his boat in Newport for a brief tour and that the tax board chairman had wanted the reductions to go through - Calenda was a friend of his. Calenda's lawyer ducked our interview requests. And, it turns out, he had good reason. Five months later, he admitted in court, that he had delivered a $5,000 bribe from Calenda's wife to the tax board chairman. Mapping the way to leases Next, we decided to dig into the dealings between Eddie Voccola, a 71-year-old career criminal (notorious for an auto-body insurance scam) and City Hall. We knew that Voccola managed to get at least $1 million from the city, for renting his former garage near a strip club to the school department for a student registration center. How much was Voccola getting from the city in how many deals? We couldn't tell because Voccola had placed his interests behind so many corporate entities. So we used ArcView GIS mapping software and the assessment data to find his property holdings, The beauty of ArcView and other GIS programs is you can select the data you want and display the results on a map. We already had the assessment data, but needed an ArcView shapefile of all the city's parcels. We requested that from the city Department of Planning and Development, which gave it to us at no cost. In ArcView, we created a unique property ID number for the parcels layer, so we could join it to the assessments data with the same field. Once that was done, we queried the table for every property that had a bill going to the mailing address we had for one of Voccola's companies. ArcView showed us, on the parcel map, the location of Voccola's properties. Then, by clicking on the parcels highlighted by the map, we were able to look at data from the table, including the name of the company owning the property. Once we had that list in hand, we got paper copies of payment records from the city treasurer and entered those into an Excel spreadsheet, finding leases that we hadn't even known about, including some that hadn't been approved by the city contracts board. When we summed it all up, Eddie Voccola's grand total hit more than $2.2 million during the 1990s. After our story ran in the summer of 2000, the city stopped leasing property from Voccola. He's now facing racketeering charges, along with the mayor, for allegedly paying bribes to obtain city leases. David Herzog can be reached by e-mail at dherzog@projo.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-program-study","slug":"2001-septemberoctober-program-study","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-program-study.html","year":2001,"month":"September/October","monthNum":9,"headline":"Program study","kicker":"COUNTY CONTRACTS","author_name":"Trebor Banstetter","author_title":"Reporter","affiliation":"Palm Beach Post","summary":"Former Camden Mayor Milton Milan is sentenced for corruption. Persistent coverage by the Courier-Post, using CAR techniques, highlights misuse of campaign funds and connections with the mob.","keywords":["corruption","mayor","Camden"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"Affirmative action was under fire in Palm Beach County. Sentiment was growing among several county commissioners to junk the south Florida county's 10-year-old Minority and Women Business Enterprise program, which attempted to steer a percentage of county contracts to firms owned by women, blacks and Hispanics. Data from the U.S. Census Bureau added depth, by detailing the growth in minority and women- owned businesses and population in the county. While the contracting industry, which wielded considerable political clout, largely supported getting rid of the county's MWBE program, minority residents - represented by a single African-American commissioner - were pushing to keep the initiative in place. Across the state, affirmative action was also facing a possible electoral challenge by activist Ward Connerly, who had chosen Florida as the next battleground for his state-by-state attack on government and university affirmative action programs. And Gov. Jeb Bush enacted a plan that eliminated racial quotas at state universities and ended the program for state contracts in departments that answered directly to the governor. Answering the question Lots of passionate, sometimes angry rhetoric flowed on both sides of the issue. But no one seemed to be able to answer a basic question: Had the county program shown benefits over the past decade that made it worth saving? The county's only minority commissioner proposed having staffers conduct a study of the program's effectiveness. That idea was rejected by the white commissioners. At the time, I was a reporter covering county government at the Palm Beach Post. My editors and I decided that if the county wouldn't do an analysis of its own program, the newspaper would. Readers would then have some hard information on what the program had really accomplished and could make their judgment about whether it was worth preserving. Taking advantage of Florida's broad public information laws, I requested access to all records pertaining to the program. The response was surprising. Although commissioners refused to study it, the county had collected reams of data on the program over the years. And they weren't opposed to us examining it. Lots o' files County staffers brought me to a large room filled with filing cabinets, and let me loose for several days to rummage through 10 years worth of contracts, program applications, reports and correspondence. Not only did the county have information on the MWBE program, but it also had files dating back to nearly 10 years prior to the program's start. That gave us a good basis for comparison. Unfortunately, all of this wonderful information was on paper. So we set up a laptop in the county office and began entering contract data by hand into a large Excel spreadsheets. From the paper reports, we focused on two key items - contracts awarded to women- and minority-owned businesses and contracts awarded to white-owned firms, both before and after the affirmative action program was put in place. The data entry took some time. But it ultimately ended up as a multiple-worksheet spreadsheet that totaled the number of contracts awarded per group per year and compared racial and gender groups. Data from the U.S. Census Bureau added depth, by detailing the growth in minority and women-owned businesses and population in the county. The Census information also helped us counter the idea that the county awarded few contracts to minorities prior to the program because there were very few minority businesses. In fact, there were plenty - they just weren't being hired by Palm Beach County. The analysis was bolstered by traditional shoe-leather reporting, including interviewing business owners, elected officials, legal experts and activists on both sides of the issue. This type of story is by no means unique to Palm Beach County, or even Florida. Many counties and cities across the nation adopted such programs in their contracting departments during the 1970s and 1980s, but they don't always study them to see how businesses actually fared. CAR helps reporter That's where a reporter with some CAR skills can step in and analyze a program local government officials might not. If you're lucky, the county's contracting department produces an annual report that summarizes contracts and contractors by total amount, race and gender. Be prepared for lots of additional work if you find you have to tabulate contracts by hand. In either case, the spreadsheet is probably your most valuable tool for tabulating the data. Basic Excel skills taught in newsrooms or at NICAR's boot camps are enough to calculate the data over time and see very quickly and clearly whether an affirmative action program has really improved the picture for minority businesses, or whether it is just lip service used when campaigning in black and Hispanic neighborhoods. Findings Our findings in Palm Beach County were described in the resulting story: From 1980 to 1989, black, Hispanic and women-owned businesses received a yearly average of just 4.6 percent of all county contracts. That's about $4.4 million, compared to the average $96 million. In 1991, when the program was launched, the share of contracts going to women and minorities leaped to 15 percent, or about $25 million a year. By 1998, the latest year for which statistics are available, women and minorities earned 16 percent, or $34 million worth, of the county's business. But while the share going to women and minorities has increased, the vast majority of public dollars still flows to firms owned by white men. The program, the story concluded, had a concrete result that had boosted public dollars to women and minority businesses in an environment that heavily favored white-owned companies. That gave some substance to our coverage of the issue, and gave our readers more to contemplate than just the hot air being blown on both sides of the debate. The staffers who ran the county's affirmative action program lauded the Posts's story as a clear, understandable presentation of a complex issue. Ultimately, the piece was cited by many in the county's minority community who sought to defend the affirmative action program. And the county's sole black commissioner referred to our findings on several OC-casions in lieu of an official county report on the program. While Bush enacted his statewide plan despite strong opposition from the minority community, affirmative action remains in place today in Palm Beach County. Trebor Banstetter, who previously worked at the Palm Beach Post, is a reporter at the FortWorth Star-Telegram. He can be reached by email at tbanstetter@star-telegram.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-investigating-a-mayor","slug":"2001-septemberoctober-investigating-a-mayor","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-investigating-a-mayor.html","year":2001,"month":"September/October","monthNum":9,"headline":"Investigating a mayor","kicker":"CORRUPTION","author_name":"Frank Kummer","author_title":"Reporter","affiliation":"Courier-Post","summary":"A newspaper's computer-assisted reporting efforts, including analysis of property databases and public records, helped expose corruption in Camden's government, contributing to the conviction and sentencing of former Mayor Milton Milan for mob payoffs and fraud.","keywords":["corruption","database","investigation"],"topics":["Investigative Methods","Crime & Justice","Government & Politics"],"technologies":["AutoTrack","Ultra Edit","Access","9-track tape","CD-ROM"],"full_text":"Milton Milan, former mayor of Camden, was sentenced in June to more than seven years in prison for taking mob payoffs, stealing money from a campaign fund and even swindling an intern. I don't claim that my reporting - and that of three other reporters at our paper - was responsible for bringing the once-arrogant Milan to justice. But the sentence was vindication for the 150 or so stories our newspaper wrote over almost two years about corruption in his administration and others connected to him. The mayor once issued a public statement comparing our reporting to \"Nancy Drew.\" A database of the city's 32,000 properties proved to be a favorite resource - and, best of all, it was free. Our stories from August 1999 to his sentencing this summer were the result of knocking on doors, reading court records, scouring records and attending hearings. But computer-assisted reporting played a role in many as we combed through any relevant databases we could get our hands on. If not, we crafted our own, spending hours plugging in numbers. Online databases such as those provided by Auto TrakXP and the New Jersey Department of Treasury, both free, were invaluable in tracking down individuals and shell companies, such as corrupt cops and construction firms linked to the mob. A few examples I'll address just three instances of how we used CAR. A database of the city's 32,000 properties proved to be a favorite resource - and, best of all, it was free. I culled these data from the entire property database of all 21 counties in New Jersey, which we keep on hand and update yearly. This used to be a difficult database to handle because of its size and the nine-track tapes it used to be stored on. Now, it's shipped on CD-ROM. But it's still difficult to manage the 1.2-gigabyte database. We cleaned the data using Ultra Edit's search and replace function to eliminate odd characters that kept choking Access. Once we wrote the import specs, we were able to import any county's property records into Access on demand. By importing only those counties that we need at a given time, we wouldn't exceed Access 1 gig limit. We are working to put the data on our intranet. Regardless, we were ready to ready to sort, filter and group the properties block-by-block or owner-by-owner when stories began to break. We searched for any property with the name Milan, knowing he was once a housing contractor. But we found only one, his primary residence. We knew there had to be others. Using that database and others, we did various searches and traveled to city hall. Fellow reporter Clint Riley (now at The Virginian-Pilot in Norfolk,Va and I were familiar with Camden County's system of computerized property records. But we taught ourselves to become experts. Learning the way property records, mortgage and deed books are referenced in your coverage area is one of the best ways to boost the sophistication of your reporting. It worked. During a search of those records, Clint stumbled on what would turn out to be a crumbling, two-story row home in the city still owned by Santiago & Milan Construction Co., a defunct business. The property was in arrears. A trip to city hall showed $28,000 in back taxes were owed on it. We also did some checking on Milan's former partner in the venture, Frankie Santiago. It turned out that Santiago was a four-time felon when he became vice president of Santiago & Milan. His convictions ranged from receiving stolen goods to a drug offense. Milan was president of the company. In response to the article, the mayor denied having bilked the city, saying he had no responsibility for the properties, or the $28,000 in unpaid taxes. It was ironic. One of the mayor's key initiatives was to raze abandoned properties. Now that we had a second name, Santiago, we could search for that too. We discovered, in our database and at city hall, that the duo owned up to six properties at varying times, including one that was considered one of the most dangerous properties in the city because of its near state of collapse. Some action Three weeks after the city placed the property on its foreclosure list because Santiago & Milan failed to pay taxes on it, the company turned around and deeded it over to an individual for $1. Santiago was the individual. We also compiled campaign documents from his 1997 mayoral campaign. This took some time but was relatively easy as far as data work goes. We requested all campaign reports involving Milan. We checked to see what his treasurer said the campaign spent and took in. Then, we built an Excel spreadsheet with two columns, one for expenses and one for revenue, to see if the numbers added up. They didn't come close. Subtracting the two [columns] showed that the zero balance the campaign claimed on its [election commission] report was actually a $2,440 deficit. To do this, we entered all expenses the campaign said it had written for food, \"street money,\" entertainment and other related expenses. Street money is a payment to an individual during an election to help \"get out the vote.\" The expenses came from detailed line items filed with the state Election Law Enforcement Commission (ELEC) the campaign manager had to fill out. We then entered all revenue from other PACs, transfers and donations. All this took days and had to be triple checked. But by the end, we had recreated the books from his campaign and came up with a balance sheet. Simply summing the two columns showed what the campaign spent and took in. Subtracting the two showed that the zero balance the campaign claimed on its ELEC report was actually a $2,440 deficit. The mayor's campaign manager told ELEC he paid out $51,287 in expenses in the last few weeks of the campaign. Actually, checks that were written for payments totaled $58,755. Other problems The campaign also failed to list names and numbers for many of those receiving street money, including those who lived at a homeless shelter that accounted for some 200 checks. In fact, more than 200 other checks were never recorded at all. Overall, we found at least $10,000 in unaccounted funds and several violations of state election laws. The former campaign manager never gave a clear explanation of the discrep~ ancies, other than sloppy bookkeeping. Milan was never charged with crimes related to those discrepancies, but jurors did find him guilty of instructing the treasurer to take $7,500 from a fund and use it to pay for the mayor's personal vacation to Puerto Rico with friends and family. Again, we felt vindicated. We knew something wasn't right with the mayor's campaign books and were able to prove it. We also created a database of possible shell companies that mobsters or their associates were using, or suspected of using, to get city contracts. We got our information from sources then checked the companies with the state Treasury Department's online subscription site of Corpo- rate and Business Information Reporting. By downloading the standard information, we were able to assemble a standard form that included each company's name, it's identification number for tracking purposes, principals involved, agents, addresses, phone numbers, dates of incorporation, annual report filings and any pseudonyms the companies used. We used the information to spot patterns, see family connections, and track down officers, whose names were unknown to the public. We had solid hits with several companies, including Transaero Inc., an electrical contracting company. Transaero had connections with a mob associate, Daniel Daidone, who, it turned, out, opened a branch office in the mayor's former campaign headquarters. Staff writer Mike Franolich uncovered a trove of documents linking the mayor to Transaero. A source confirmed for me that Transaero was a mob front and that the goal was to install a minority as the company's head. Then, Transaero would be able to get city contracts through its friend, the mayor. One of the charges a jury convicted the mayor on was taking cash bribes from the Philadephia/South Jersey mob boss, Ralph Natale. More than a year after we wrote our Transaero story, Natale, a confessed killer, admitted on the witness stand that it was his key company and Daidone was his intermediary. They never did land contracts, but Milan was finally exposed. As of July, Milan's attorney was working to appeal the conviction. Frank Kummer can be reached by e-mail at fkummer@courierpostonline.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-calculating-bills","slug":"2001-septemberoctober-calculating-bills","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-calculating-bills.html","year":2001,"month":"September/October","monthNum":9,"headline":"Calculating bills","kicker":"TAXES","author_name":"Garry Lenton","author_title":"Reporter","affiliation":"Patriot-News","summary":"A Pennsylvania newspaper obtained early data on a county property reassessment and used Excel, mapping software, and publicly available millage rates to calculate how individual tax bills would change before the reassessment officially took effect in 2002.","keywords":["reassessment","data","taxes"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Excel","ArcView","DBF"],"full_text":"Early last year, the Board of Commissioners in Pennsylvania's Dauphin County voted to conduct the first real estate reassessment since 1973. It was a long-overdue move, designed to bring property tax values back to reality. From the beginning, our city desk figured this for a package story, with lots of graphics telling homeowners what to expect, how the reassessment would work, and how to appeal the results. A decision was made early to obtain the reassessment data when the work was completed. But that wasn't going to be for many months. And, since the county hired a private consultant to do the work, we were told that the newspaper would not get access to the new property values until the county took possession of the data. But the county's chief tax assessor agreed to provide us with the total real estate values for each of the county's more than 40 municipalities. By comparing those totals with the property values the county was currently using, we were able to get an advance look at where real estate values were changing the most. We wanted to tell our readers how their individual tax bills would change once the reassessment went into effect in 2002. The numbers were provided in a text file. It included the name of the municipality, the updated real estate value, and the old real estate value. It took about five minutes to dump the data into an Excel spreadsheet and calculate which municipality's property values increased most. Surprisingly, the biggest increases were in communities where population growth was the smallest and not in the sprawl zones around suburban Harrisburg as expected. The tax assessor wasn't surprised, however, and explained that the sparsely populated farming regions had a higher number of old properties whose values would have the farthest to go to catch up with 21st Century prices. Once the calculations were completed, we added the FIPS codes for each municipality, saved the spreadsheet as a .dbf file and loaded it into ArcView and mapped the results. The story, with various breakouts and the map filled a full page. But while telling readers where values increased the most was OK, there was more we could do with those numbers. We wanted to tell our readers how their individual tax bills would change once the reassessment went into effect in 2002. The only way to do that was to calculate new real estate tax rates, essentially beating local government bodies at their own game by months. Here's how. Step 1 We imported the real estate millage for each municipality and school district in the county, as well as the county tax rate. We got the data from the Pennsylvania Department of Community and Economic Development's Web site: (www.dced.state.pa.gov). DCED maintains an enormous amount of local government data on its site, mostly all of its transferable just by copying and pasting. Step 2 Use the millage rates to calculate the real estate tax revenue each government body is collecting. If you are new to this, here's how it works: One mill of tax = .001 of a dollar, or $1 of revenue for every $1,000 of property value. In other words, a $100,000 home taxed at 5 mills will net $500 in taxes (100,000 X 5). The formula in Excel would look like this if the property value was in cell A2 and the millage rate expressed in decimal points was in B2: =(A2/1000)*B2 The formula would be written in cell C2. Now you know approximately how much revenue each local government body was earning from real estate taxes. What you must figure out now is how the millage rate must change to generate the same amount of money when it is multiplied by the new real estate values. State law forbids local governments from gaining more than 10 percent revenues through reassessment. Step 3 Add a fifth column next to RE_millage and call it RE millnew. Copy and paste the contents of RE_millage into the new column, it's needed for the function Excel uses to do the math. Add another column next to RE_revOLD and call it RE_revNEW. In this column, calculate a new revenue total using the millage rates in column RE_millnew and the dollar amounts in a column called assessval_new. This will give you the amount of money each local government would get if it applied current millage rates to the new assessed values. The result is a number that is twice or more what they are getting now. Step 4 The final step uses Excel's \"goal seek\" function to calculate the a new millage rate that will keep revenues at current levels. Here's how. Click on the first value under the assessval_new column.. Go to the tools pull-down menu and select \"goal seek.\" A box will pop up. In the window that says \"set value\" type in the value generated by the old tax. In the window for \"by changing cell,\" Then select the cell under RE_millnew. (This cell should have a old millage rate as a value, but Excel will change the number when the operation is complete.) Hit \"enter.\" The value in the cell under RE_millnew will change, and the value under RE_revNEW should now equal the value in RE_revOLD. Unfortunately, goal seek doesn't work with the autofill function like formulas do. Each value has to be done separately. We calculated the rate for all of the affected government bodies and totaled the millage. We ran a chart showing the new taxes along with instructions on how readers could calculate their tax bill. Before we ran the story, I called the chief tax assessor and outlined this process. Better safe than sorry. \"That's exactly how we're going to do it,\" he said. It was just going to be another two months until they could get around to it. Garry Lenton can be reached by e-mail at glenton@patriot-news.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-adding-context-meaning","slug":"2001-septemberoctober-adding-context-meaning","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-adding-context-meaning.html","year":2001,"month":"September/October","monthNum":9,"headline":"Adding context, meaning","kicker":"LOCAL REPORTING","author_name":"Rob Gebeloff","author_title":"Reporter","affiliation":"Star-Ledger","summary":"The Star-Ledger developed a Standardized Town Reporting program to provide contextual, data-driven local government coverage across New Jersey's fragmented municipal system, using computer-assisted reporting to connect individual town stories to broader county and state trends.","keywords":["journalism","data","government"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["spreadsheet"],"full_text":"To an outsider, New Jersey seems like an incomprehensible jumble of pavement covered by too many buildings, too many cars and too many people. Which is pretty much how it seems to people who live here too. And when it comes to government, New Jersey, true to form, is ruled by a mind-boggling array of elected bodies - boards, councils, commissions - along with a maze of public authorities that wield power over a wide range of issues, from commuter trans- portation to real estate development in our great swamp, The Meadowlands Covering this smorgasbord of public bodies poses a special challenge. If you try to blanket every city hall, county agency and school board, your news organization will likely go bankrupt, either from the staffing costs, or from all the canceled subscriptions from readers who grow tired of the zoning board minutae you present each day. But providing superficial coverage doesn't work either. Readers want a watchdog - insightful and provocative. In this piece, I'll share with you two of the projects we published to meet that de- mand, then discuss the steps we've taken to build a comprehensive government data intranet to allow reporters to conduct their own simple data analysis, without having to learn spreadsheets and database software. Standardized Town Reporting The Star-Ledger last year launched a pro- gram called Standardized Town Reporting (STR). In short, the goal is to bring context and meaning to our local coverage, both on a daily basis and through special projects. Computer-assisted reporting is a major com- ponent in each area. Too often, local coverage fails to connect the dots. Reporters might cover their towns like a glove, but won't necessarily view their community within the context of the sur- rounding county and state. That flaw gets magnified in a place like New Jersey, where municipalia is an endless string of indepen- dent communities with no natural bound- aries, and where problems that crop up in one community are quite often an issue in 100 others. With STR, we're trying to institutional- ize this big-picture thinking. We want re- porters who come back from a meeting where citizens complain about property taxes in Piscataway to report where Piscataway ranks among its peers in average residential property tax bills. We want stories about the crime rate in Morristown to include data for the last 10 years, not just the last 2 years. We also want to break news by looking in depth at government institutions, but in new ways. Courting Cash The first STR project examined the mu- nicipal court system, an institution that most people encounter far more than city hall but that generally merits only a fraction of the coverage. In New Jersey, there are more than 500 of these local courts, which process all of the parking tickets, moving violations and bar-fight disorderly conduct cases that crop up when you cram 8.4 million people into 7,400 square miles. Most of our readers were probably aware of the municipal court system, and some probably had figured that these courts were partly about justice and partly about raising funds for the town treasury. We turned these vague notions into concrete. We obtained a database that summarized case load and revenues by courthouse, then crossed the figures with a database of town budgets and popula- tion figures. We put together a team of reporters from throughout Ledger-land and told them to ex- amine the numbers for their own communi- ties and then get out to the courts and observe. We then built a package, full of color and anecdote, around these findings: More than $300 million flows through the municipal court system each year, with municipalities keeping nearly three- fifths of that money. While court revenues generally represent a small fraction of a municipality's cash flow - 2.4 percent on average - some towns are downright dependent on court collections. At least 10 New Jersey towns use court money to fund 10 percent of their budgets, with one town using 17 percent. Justice often takes a back seat to conve- nience in municipal cases. In some cases, drivers looking to avoid points and insur- ance charges can plead guilty to municipal offenses that will leave their records clean on a state charge while increasing the municipality's share of the fine. Most of our readers were probably aware of the municipal court system, and some probably had figured that these courts were partly about justice and partly about raising funds for the town treasury. Numerous charts, including a town-by- towner that was locally zoned for eight dif- ferent editions of the paper, accompanied the stories. The Extracurricular Divide For our next STR project, we picked a topic closely aligned with a major ongoing news story. New Jersey courts had ordered a remedy for the wide gap in spending between wealthy and poor school districts. While the state had introduced a special funding program for the poorest of the poor districts, we wanted to take a broader look and compare districts in the bottom quarter of the economic strata with districts in the top quarter. We also wanted to throw extracurricular activities into the mix. Many educators con- sider extracurricular activities to be an essen- tial part of overall development, and after-school programs also are an important com- ponent of college resumes. So the question was natural - are kids reared by poor school districts being put at yet another disadvantage? Our plan of attack was similar to STRI. We assembled a team of reporters from throughout our coverage area, crunched school finance numbers, crossed them with poverty data, had team members analyze the local figures. Then we sent them out to report. Again, we presented a package of stories and graphics zoned for each of our editions Our conclusion: New Jersey has moved rapidly in recent years to close the gap in education spending between rich and poor school districts. But every afternoon, when the school bell rings and the instructional day ends, all the long-standing inequities in public education re-emerge. Even as New Jersey reduced the overall spending gap between rich and poor districts during the past decade - whittling the dis- parity from 18 percent to 7 percent - the gap in spending on extracurricular activities remained a staggering 58 percent. In the state's 100 wealthiest districts, more than $22 of every $1,000 spent goes toward extracurricular activities, we found. In the 100 poorest districts, it's $13. Getting the Data Out I've mentioned in each of these projects how we had team members examine num- bers for their own turf, and I've also men- tioned how we want beat reporters to regu- larly incorporate statistical analysis into their daily reporting. As anybody who's ever tried to get the newsroom hooked on Excel knows, this is a tall order. But we've been quite successful going another route- data intensive, value-added intranet. The key phrase here - and I hate using words like this but I think it applies - is value-added. Since arriving at the Ledger last year, I've devoted a lot of time to develop- ing a library of local government databases, cleaning them so they can be linked together, and putting them up on the newsroom intranet with a newbie-friendly interface. If a reporter wants to rank New Jersey towns by average property tax bill, or query Census data to find the county with the larg- est increase in the Asian-Indian population, they don't have to open a spreadsheet. They can do all that and more from their Web browser. It truly helps get reporters using numbers, and using them in a meaningful, guided way. In addition to a Census query interface (which includes 2000, 1990 and 1980 data), our reporters can browse and rank Claritas in- come data, crime stats, building permits, prop- erty tax data, and all kinds of school data. I make sure each dataset identifies towns by the five-digit Census ID, so linking towns across databases is a snap. In fact, I even built a town background page. A reporter picks a town, and gets back eight pages of data. To make it even more reporter-friendly, I put a link on the page that runs the town name through Mapquest, just in case the reporter needs driving directions. (It's New Jersey, don't laugh). For each of the STR projects, I built a tool specifically for the job. So when we were look- ing at school finance, the team members could call up a page that had all the relevant data for every district in their county. Or, they could rank districts in any single category, filtering for district size and poverty ranking. While it takes a bit of work to program all of this (I used VBScript and active server pages), the benefits are many. For one, it truly helps get reporters using numbers, and using them in a meaningful, guided way. It's like truly being the database editor - wading through the mounds of data and pulling out the key figures that are needed to compare and contrast. And in the long run, the work I put in up front saves me time later. I don't have to run Access queries for 20 reporters every day. In- stead, they can do a lot on their own through their Web browser. The bottom line with STR is that we have 566 towns, including 200 in our direct circu- lation area. By looking at the big picture, we're starting to put this mess of home rule into some perspective - exactly what readers want. Rob Gebeloff can be reached by e-mail at rgebeloff@starledger.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-uplink-story-ideas","slug":"2001-septemberoctober-uplink-story-ideas","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-uplink-story-ideas.html","year":2001,"month":"September/October","monthNum":9,"headline":"Uplink story ideas","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE newsletter invites journalists to submit stories involving computer-assisted reporting (CAR), technical solutions, or uncovered beats for potential publication.","keywords":["journalism","data","reporting"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Have you or one of your colleagues recently published a story using CAR that has not been done before or involved particularly difficult data work?\n\nDo you know of a technical problem (or its solution) that others may like to hear about?\n\nOr is there some issue or beat that we haven't covered?\n\nIf you have a story idea, we'd like to hear from you. Please contact associate editors Amy Sherrill or Mike Sherry either by e-mail at amys@ire.org or mikes@ire.org or by phone at (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-septemberoctober-ratio-study","slug":"2001-septemberoctober-ratio-study","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-ratio-study.html","year":2001,"month":"September/October","monthNum":9,"headline":"Ratio study","kicker":"PROPERTY ASSESSMENTS","author_name":"Mark Houser","author_title":"Reporter","affiliation":"Pittsburgh Tribune-Review","summary":"The Pittsburgh Tribune-Review used computer-assisted ratio studies to evaluate whether Allegheny County's $24 million property reassessment was accurate, finding significant flaws including overassessment of affordable neighborhoods and underassessment of expensive homes, contrary to the county assessor's own positive findings.","keywords":["assessment","properties","reassessment"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":["Excel"],"full_text":"When its ancient, inaccurate property assessments sparked several lawsuits and a judge ordered things put right, Allegheny County coughed up $24 million to an Ohio firm to reassess the nearly 600,000 real estate parcels in Pittsburgh and its suburbs. Simply put, a ratio study compares assessed values with the actual prices properties sell for in the most recent transactions, to see how well assessors are predicting the market. But after the results came in, homeowner hordes practically stormed the ap- peals office, protesting that their new val- ues were off by a mile. Were they right, or just complaining? Did taxpayers get their money's worth from the reassessment? Ratio study The Pittsburgh Tribune-Review set out to answer those questions this spring. To do so, we used computer-assisted report- ing and a standard analysis tool of the as- sessment industry, called a ratio study. Simply put, a ratio study compares assessed values with the actual prices prop- erties sell for in the most recent transac- tions, to see how well assessors are predict- ing the market. Ratio studies involve thou- sands of calculations, property by prop- erty, which means they are an ideal job for your whirring desktop assistant. The county assessor's office conducted its own ratio study, using old and filtered sales data provided by the contractor, Sa- bre Systems and Service. Big surprise - it found a few isolated problems, but over- all, good work. Gold star sticker, guys. Flaws in assessment But the Trib, using industry standards the county didn't follow, found serious flaws in the reassessment. In the city of Pittsburgh, one of every five properties sold for more than double, or less than half, the new as- sessed value in 2000. Throughout the county, the Trib found the least expensive neighborhoods were frequently overassessed, while pricier homes were routinely assessed at hundreds of thousands of dollars less than what they sold for. To do a ratio study, you need a database of property tax rolls. The information must include: 1. The assessed value of each property. 2. The date of the property's most recent sale, and the price. 3. Information about the sale to deter- mine whether it was an \"arm's length\" trans- action on the open market. Weeding out The last is crucial. Many properties sell at prices that don't reflect their true value. An elderly mother sells her home to her son for $1, sometimes called a \"love and affec- tion\" sale. An owner fails to pay real estate taxes, and the property is seized and sold at a sheriff's sale at a bargain basement price. A seller includes an 18-foot fishing boat in the deal. Sales like this can't be used to establish the market value of homes in a neighbor- hood, SO they have to be weeded out in a ratio study. The computerized tax rolls we used contained a code describing the type of sale; we used this to select only county-qualified arm's length transactions of $10,000 or more for our study. Because the new assessments are supposed to reflect a property's value in 2000, the Trib looked at sales from 2000. (The assessors' own study used sales from 1996-99, the very sales Sabre used to come up with its values in the first place.) In the end, we narrowed the tax rolls down to an Excel spreadsheet of 16,502 properties that sold last year in Allegheny County, including 4,338 in Pittsburgh. The newspaper used two common mea- sures of accuracy in ratio studies: median sales ratio and coefficient of dispersion. Finding the median Median sales ratio shows if properties are typically underassessed, overassessed or as- sessed just right. A property's sales ratio is its assessed value divided by its sale price, typically expressed as a percentage. Less than 100 percent means assessors lowballed the value compared to what it sold for; more than 100 means the opposite. The Trib cal- culated sales ratios for every property in a school district (or Pittsburgh ward) and found the median, to find out whether homes were more likely to be over- or un- derassessed in that neighborhood. There are 43 school districts in Allegh- eny County. Schools get the bulk of prop- erty tax revenue, so they made ideal \"neigh- borhoods\" on which to base our measure- ments - small enough to isolate problem areas, big enough to have sufficient sales for statistically sound findings. To speed up find- ing the medians in each of these neighbor- hoods, I used an Excel Subtotals trick. The newspaper used two common measures of accuracy in ratio studies: median sales ratio and coefficient of dispersion. Experts I talked to from the Chicago-based International Association of Assessing Offic- ers said a good rule of thumb for using MSR is anything below 95 or above 105 is a problem. Our study found wealthy neighborhoods as low as 88, and three poor city wards above 120. Coefficient of dispersion Coefficient of dispersion is a much more difficult concept. Basically, it de- scribes how many percentage points each property's sales ratio is off the median sales ratio. In a non-cookie-cutter neighbor- hood, the national standard COD is 15 or less for houses, or 20 or less for a mix of residential and commercial properties. COD measures consistency, which makes it an important companion to the MSR when discussing assessment accu- racy. The city of Pittsburgh provides a perfect example. Some got the shaft Pittsburgh's MSR was a perfect 100, mean- ing properties were no more likely to be un- derassessed than overassessed. But its COD was 24, meaning the average property was either underassessed or overassessed by 24 percent. Translation: A lot of owners were getting big breaks. A lot were getting the shaft. There are formulas for calculating COD, but they are complex, and I didn't want to strip a brain gear or melt my hard drive. Luckily, I got help from an expert. Peter Davis, a Kansas Department of Revenue of- ficial and the guy who helped write the lat- est ratio study standards for the IAAO, let me use a free Visual Basic demo program the department has posted on its website: (www.ink.org/public/kdor/pud/pvdratio.html) (It comes bundled with instructions.) The program, which Davis invites any other interested reporter to use, calculates COD from an Excel spreadsheet of assessments and sale prices. It also figures a confidence interval, use- ful for neighborhoods with CODs close to the line. Davis also patiently provided much valu- able advice on constructing the study fairly. For example, before calculating COD it is standard industry practice to \"trim\" off properties with the most extreme sales ra- tios. Unlike MSR, COD can be affected sub- stantially by just a few bad assessments. The IAAO-approved standard method of deal- ing with this discards outlying ratios more than 1.5 times beyond the range of the middle 50 percent of the sample. It took some time on the spreadsheet, but it was worth it to keep the results as watertight as possible. To read the Trib's main story on its ratio study, go to http://www.triblive.com/ tribinfo/assess.html Q & A by Mark Houser You have an Excel sheet with thousands of homes and their sales ratios, grouped by their neighborhood. You want the median sales ratio for each neighborhood. How can you do this quickly and easily? Trick Microsoft. You can use the Subtotal function, even though it's not built to do medians. Go ahead and use Data/ Subtotals to find the sum of sales ratios at each change in neighborhood. Then, highlight the column with the subtotals, and pick Edit/Replace. Replace \"SUBTOTAL (9,\" with \"MEDIAN(\". Hoo-hah! Faked you out, Bill Gates. Once we had the numbers, we fleshed them out with details. We talked with homeowners on one street where homes sold for a quarter of a million and were assessed at half that. We talked the neighbor a few blocks away whose $45,000 house was taxed as if it were worth $63,100. We got a county overseer to admit that his assessors were murmuring privately about \"so many mis- takes it's unbelievable.\" Sabre Systems never responded to our re- peated calls and e-mails. Some county offi- cials criticized the Trib's methods. Others waved Trib clips as evidence of a major break- down. Since our analysis appeared, several other ratio studies have been done - one by the county controller, another by a consultant for county council, yet another by our cross- town competition. All of them have found serious accuracy problems with the new as- sessed values, contradicting the findings of our county executive. Meanwhile, 89,000 homeowners have appealed their new values, far more than the county ever expected. The vast majority are getting reductions. Allegheny County now expects to pay $20 million in refunds, more than double what it budgeted. And several county elected officials are calling for Sabre Systems to refund some of that $24 million. Mark Houser can be reached by e-mail at mhouser@tribweb.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-poorly-run-program","slug":"2001-septemberoctober-poorly-run-program","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-poorly-run-program.html","year":2001,"month":"September/October","monthNum":9,"headline":"Poorly run program","kicker":"HOUSING","author_name":"Steve Bivens","author_title":"Investigative Reporter","affiliation":"KTRK-TV","summary":"An investigation by KTRK's 13 Undercover unit, confirmed by a $300,000 audit, reveals widespread mismanagement in Houston's Section 8 housing program, including excessive rents, inadequate employee competency, and landlords collecting payments for properties they don't own.","keywords":["housing","corruption","investigation"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Excel","CD-ROM","Internet"],"full_text":"The Houston Housing Authority's Sec- tion 8 program is in crisis. KTRK's 13 Undercover unit first found the problems, and now a $300,000 audit paid for by the housing authority confirms what they found. That audit recommends major changes, finding that half the authority's employees couldn't do sixth-grade math. Section 8 is supposed to provide safe, decent, and affordable housing to needy families by using tax dollars to subsidize rents. It's a popular program in Houston, with more than 12,000 families partici- pating and a six-year waiting list. The gov- ernment spends millions of dollars this way in Houston every year. But 13 Undercover found many short- comings in how that money is spent, in- cluding questionable rents, shoddy living conditions and a landlord who collected nearly $10,000 in government rent checks for a house she didn't own. The start The investigation started with a tip about a Section 8 slumlord. The tipster described unlivable conditions, an uncaring landlord, and an unresponsive Housing Authority. We decided to request some Section 8 data. The Housing Authority wanted to e-mail the data. We agreed, but soon ran into a snag. We knew Section 8 administrators didn't want to release their client's names. We didn't need names but needed addresses. So, under the Texas Public Information Act, we asked for more than we needed - an elec- tronic record of all clients' names, addresses and rent amounts. As we expected, the Hous- ing Authority stuck to keeping the client's names private. Neither side wanted to take it to the Texas Attorney General. Putting it into a usable form was a challenge. The Housing Authority wanted to e-mail the data. We agreed, but soon ran into a snag. The file was too big to pass through our firewall. So to downsize it, the authority broke it into parts. We cut and pasted it together. The database came in Microsoft Excel format, contain- ing a little more than 12,000 records. We paid a minimal amount. It took a few simple sorts to get our in- vestigation rolling. We first sorted by amount of rent. That gave us the highest rents and the lowest. Just by looking at a map and knowing the city, we had a good sense of what rents were out of line. That's how we found the Housing Authority pay- ing $1,000 a month for a house when nearby properties were renting for about $350. Getting answers How did some rents get that far out of line? We asked for documentation of how the Housing Authority calculated rents. When we got the Rent Reasonableness Sur- veys, we got another story. The authority supposedly compares a property with three others that are similar in value, size and lo- cation. But most of the time, properties were compared to just one other house. To compare values, we went online to our County Appraisal District (www.hcad.org). At that site, you just punch in an address, and get an appraised value. Then we checked the loca- tions on a map. By going though the top 20 highest Section 8 rents this way, we found one place compared to another worth five times as much. We also found houses compared to properties on the other side of town, and one house compared to a house that did not exist. When we went to that $1,000-a-month house, we found what the data couldn't show. The place was a dump. We learned the Housing Authority kept simple inspection records in databases by year. Those records contained client number, ad- dress, date of inspection and results. We asked for those databases for 1999 and 2000. Both contained about 30,000 records. Instead of trying to squeeze those through the network, we got the Housing Authority to provide those to us on CD-ROM. Once we got the tables, we combined the two for analysis. Again, we started looking at this data by doing a few simple sorts. But we did en- counter a few difficulties. These records were just plain unkempt. Many records were in- complete, and the data entry was very in- consistent. There were numerous misspell- ings, spaces in places where they didn't be- long, and some cells contained info clearly meant for other cells. Manually fixing the problems was an option, but that would have taken a monumental effort. We decided to sort first and see what happened. Sorted by address, we noticed some properties had failed repeated inspections. We first sorted by address, That gave us a complete inspection history of every prop- erty for the last two years. The inspection records classify results as pass, fail, incon- clusive or vacant. Sorted by address, we no- ticed some properties had failed repeated in- spections. In some cases, properties had failed seven inspections in a row, but the gov- ernment checks kept flowing. Still sorted by address, we found some houses that had multiple vacant or inconclu- sive inspections in a row. Some Section 8 cli- ents had moved out months ago, and yet the inspectors kept paying visits to vacant houses. That's how we found the fraud. One house we visited with repeated vacant inspections had a new resident. And she was plenty mad at Section 8 for continuing to show up for inspections when she wasn't on Section 8. She had bought the house nearly a year before. We asked the Housing Authority for hardcopies of the corresponding Section 8 rent checks. It turned out the Section 8 landlord who used to own the house had been fraudu- lently collecting rent for two years after she sold it. We also dumped all three tables into Microsoft Access on a laptop and linked the tables by client number. That proved useful when we went out to check out houses. We had rent amount and inspection history avail- able when we needed it for interviews and for asking questions of Section 8 administrators. Remember the tipster? We never did check out her house, but we did get it in- spected and fixed up. After our stories, the Housing Authority inspected all the prop- erties in the program. About 25 percent of them failed. Those that failed received just days to fix the problems. Several inspectors were either disciplined or let go. Stephen Bivens can be reached by e- mail at Stephen.D.Bivens@abc.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-water-guzzlers","slug":"2001-septemberoctober-water-guzzlers","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-water-guzzlers.html","year":2001,"month":"September/October","monthNum":9,"headline":"Water guzzlers","kicker":"DROUGHT","author_name":"Bob King and Chris Barker","author_title":"Reporter","affiliation":"The Palm Beach Post","summary":"A South Florida newspaper investigated whether the largest water users in drought-stricken Palm Beach County were conserving, facing significant obstacles from fragmented utility systems and uncooperative records departments before ultimately uncovering the water consumption habits of the county's most profligate households.","keywords":["drought","conservation","investigation"],"topics":["Data & Databases","Investigative Methods","Health & Environment"],"technologies":["9-track tape","Excel"],"full_text":"For the past year, South Florida has been sweltering through its worst drought in history, triggering restrictions on all sorts of water uses - lawn sprinkling, car washing, farm irrigation, you name it. But we wondered: Are our biggest water users really conserving? And is anybody enforc- ing the rules? For one story, we decided to look at the habits of our largest, most water-using homes and trust us, Palm Beach County has a lot. We were aided by Florida's strong public records law, which requires govern- ment-owned water utilities to open their billing records to the public. For some cities and towns, we reluctantly gave up the search because we were running out of time to get the story in the newspaper. But local government here is divided among the county and 37 cities, towns and villages; some run their own utili- ties, and some are part of regional wa- ter systems. So the search for data proved much longer and more complicated than we expected. The reporting and writing took nearly two months. First step First, we asked the utilities serving the county and 15 of the largest or wealthiest cities for a list of the single-family homes that used the most water in the year 2000. It would have been much simpler to ask for the top consumers for a single month; however, we asked for a year because we wanted the most consistently profligate water users, not people who might have had a bad leak for a week or two. Some utilities, such as Palm Beach County and the city of Boca Raton, had no problem giving us what we needed; in some cases, the turnaround time was a day or less. Other utilities claimed that their 1970s-era computer systems couldn't pro- duce the data without writing a program, which would cost money. West Palm Beach, for instance, wanted to charge us $600 for programming time, We also got the runaround when we asked for West Palm Beach's complete water billing data- base on nine-track tape, which would have allowed us to run our own searches. For some cities and towns, we reluc- tantly gave up the search because we were running out of time to get the story in the newspaper. We wanted West Palm Beach, though - its utility provides water to the town of Palm Beach, the island home of Donald Trump and other money-makers. Gotcha Finally our computer assisted reporting coordinator, Christine Stapleton, discov- ered that West Palm Beach periodically produces a report of its top 300 water ac- counts when it's considering rate changes. Since that computer program already ex- ists, the city could simply run it for us at much less cost. One snag, though: Some water users had as many as seven separate water ac- counts for a single home. That meant we had to keep filing public records requests to get the additional account numbers for the top water users, then get the water use for each account. It also turned out that the wealthy often like to give fictitious addresses to different parts of their sprawling estates. So we spent hours driving around the island and examining prop- erty records to match the addresses on the wa- ter bills to actual locations. We entered the top water customers for each utility into an Excel spreadsheet, list- ing the water supplier, the customer, the customer's address and the amount of water used in 2000. Many utilities use dif- ferent units of measurement - thousands of gallons, hundreds of cubic feet, etc. - so we created a column to translate all numbers into gallons. We finally wound up with data from seven utility systems representing a major- ity of the county's households. The top 10 list was no surprise: All homes in the town of Palm Beach, including one owned by Revlon boss Ron Perelman. The top user, Arby's franchiser Nelson Peltz, used more than 2 million gallons a month, enough for 80,000 five-minute showers. The story intrigued the mayor of West Palm Beach, who said he would explore the idea of charging a \"guzzler fee\" to encour- age the heaviest water users to conserve. As a followup, we examined another of South Florida's major water users: our hundreds of golf courses. Meanwhile, Peltz turned out to be a friend of Rush Limbaugh, who denounced the story on the air the next day as a classic example of liberal class-envy. Limbaugh also announced that he was asking his wife to flush the toilet continually in hopes of mak- ing the list next year. So stay tuned. More stories As a followup, we examined another of South Florida's major water users: our hundreds of golf courses. But what was supposed to be a modest explainer story became more interesting when we discovered that several golf courses used a billion gallons more last year than they were legally allowed to - and nobody was penalizing them. At first glance, this should have been an easy story to pursue: A state agency, the South Florida Water Management District, issues water permits to the golf courses, and it collects reports about their water use. But the reporting was far from simple. For one thing, district managers claimed they had no easily retrievable database list- ing the golf courses, their allowable water use and their actual pumping. Creating such a database would involve \"generating a record,\" which would allow the district to charge us hundreds of dollars for programming time. So reporter Chris Barker spent hours sit- ting in the district's offices, poring over the courses' permit files and typing the num- bers into Excel. His persistence paid off when one dis- trict employee took pity on him and gener- ated an electronic report containing some of the information he needed. But then she stopped helping us, and the district's lawyers said all our requests had to go through them. Dirty data Also, the numbers themselves were a complete mess. Golf courses routinely added or dropped digits in their pumpage reports; some failed to submit reports for entire quar- ters, with no penalty from the district. So Barker had to spend more hours question- ing the employees who worked with the numbers about what they meant. The district also employed a convoluted form of water-accounting that lumped together water pumped from wells, water pumped from ponds, water pumped from wells into ponds, and recycled sewage commingled with the other water sources. We used the most conser- vative numbers, which knocked the most dra- matic over-consumers off our Top 10 list. We still wound up with 13 Palm Beach County golf courses, including Donald Trump's, that used more than a total of 1 billion extra gallons in 2000 - a fact that surprised people at the water district. A few days after the story ran, the district's board ordered an audit of its entire water permitting system. Some environmen- talists told us the story pointed to a dirty secret about Florida's water management system: The state really has no idea how much water people are using. Bob King can be reached by e-mail at bob_king@pbpost.com. Chris Barker can be reached by e-mail at chris_barker@pbpost.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-start-the-car","slug":"2001-septemberoctober-start-the-car","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-start-the-car.html","year":2001,"month":"September/October","monthNum":9,"headline":"Start the CAR","kicker":"LOCAL GOVERNMENT","author_name":"Paula Lavigne Sullivan","author_title":"Reporter","affiliation":"The News Tribune","summary":"A journalist at The News Tribune describes her journey learning computer-assisted reporting (CAR) while covering county government, starting with property records and navigating technical challenges like data formats and field definitions to demonstrate CAR's value to her newsroom.","keywords":["reporting","data","journalist"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Access","9-track tape","CD-ROM"],"full_text":"About a year ago, my idea of mapping was pulling out an atlas and a ruler. I was still grinding my teeth over simple Access func- tions and wondered with every mouse click, \"Am I missing something?\" It was about this time that I got a new beat at The News Tribune (Tacoma, Wash.) covering county government and all related county agencies, from law enforcement to solid waste. The 2000 Census was also mine to shepherd. CAR started Although I was interested in taking off with my new beat, I knew that my primary goal was to get computer-assisted reporting off the ground at the Trib. I was a beginner, learning by failing and failing again, but I knew what CAR could do and believed our newsroom could ben- efit if more reporters realized its potential for their beats. So here was my challenge: I had to get better myself. I had to show the editors why they should invest in CAR. And I had to get the attention of other reporters. was a beginner, learning by failing and failing again, but I knew what CAR could do and believed our newsroom could benefit if more reporters realized its potential for their beats. Through determination, the support of a few key people and a beat that has been quite conducive to CAR, I was able to do all three. Today we might not have a newsroom of re- porters who are proficient in CAR, but at least they're interested and trying. For starters, I think the key for me was getting a good beat that worked well for ex- perimenting with CAR projects. The county was a good geography to tackle. It was small enough to digest, but large enough to show patterns and diversity among its cities. And county government - at least in most Washington counties - covers a wide range of services including law enforce- ment, planning, elections, utilities and so on. Voting records, building permits, criminal histories and property assess- ments were all part of the deal. Nine-track I started with property records. I got my first lesson in negotiating for the data with an agency that hadn't received such a request from a newspaper before. It took a few weeks and we got the data on nine-track tapes. After dusting off the newspaper's nine-track tape player and toy- ing around with it for a while, I soon wised up and sent the tapes to MaryJo Sylwester at NICAR to put them on CD. But the data from the CDs didn't correspond with the record layout. We assumed there was a prob- lem with the tapes, so we went back to the assessor's office and got the data on cartridges instead. We found the source of the problem when Sylwester looked closer at the hand- written record layout and found packed fields we hadn't noticed before. When I got them back on CD the second time, everything worked well. But that didn't mean my problems were over. Most of the fields were in acronyms that I didn't understand. So before I started making assumptions, I took the layout to the assessor's office, sat down with members of the records staff and went over each field one by one. Technicians helpful By that time, I also had developed a rela- tionship with the technicians in the assessor's office, especially with records manager Tami Kohl, who was patient enough to answer my numerous questions. For someone still learn- ing the ropes, that type of relationship helped me get support for using the data and an understanding of which findings were im- portant. We now have a system to get full updates on an annual basis and smaller updates throughout the year. Property records were good for showing the value of CAR to the newsroom. Not only could we use them for projects, but we could also use them for daily reporting, even break- ing news. The records had addresses, owner names, values, taxes, exemptions and sales history. Taxes and values were hot topics for readers as well, and editors appreciated that. I did one simple story that showed the disparity between the market value and the taxable value of homes in some parts of the county. I also used the database to find the owners of homes that had been contaminated by methamphetamine labs. In doing the meth story, the Tacoma- Pierce County Health Department gave me a short list of properties that had been con- taminated by meth pollution. I typed that list into a table in Access. I matched it up, by parcel number, to the data from the assessor's office and got the owners' names. Then I ran the owner's names through our online Courtlink criminal conviction service to see which owners were not also party to the meth incident that led to the contamination. (Be- cause, hey, if you're the one who contami- nated the house it's your house- then there's little pity for you when you've got to pay to get it detoxified.) Matching values with geography My next step was trying to match prop- erty values with geography, which is when I started with the ArcView mapping software. This was another time when the county beat helped, because county government is re- sponsible for planning and zoning and had developed a stellar GIS staff. They had a trea- sure trove of maps and a staff with a good deal of expertise - again a value for someone still learning. Although it helped to work with local of- ficials, the negotiations weren't always easy. For most agencies, this was the first time they were getting such requests from a newspa- per. I ran into a few typical roadblocks, in- cluding officials saying they couldn't do it, they didn't have time to do it, if they did it our computers couldn't handle it, and, oh, by the way, it'll cost at least $1,000. I knew we had to have a better way. Even if I convinced reporters that the county had all this data they could use on their beats, I risked having them go to the agency and get frustrated, or give up, at the first denial. I set up an informal meeting with our executive editor David Zeeck, my team leader, myself, the county's communications director and attorneys from both sides. Even if I convinced reporters that the county had all this data they could use on their beats, I risked having them go to the agency and get frustrated, or give up, at the first denial. We talked about how our reporters would be making more requests for elec- tronic data in the future and how it might be better for both sides if all county de- partments had a similar policy for han- dling such media requests. We haven't had a chance yet to test these county officials on their willingness to cooperate, but we did get our ArcView shapefiles at no cost after facing an initial charge of about $100 each. I used Microsoft Access for both matches. For the assessed value vs. market value story, I used one database from the assessor's office of recent sales. I merely listed properties by parcel number and calculated the difference between the two values. (I also had an ArcView shapefile of all the parcels in the county. I matched the parcel numbers from the assessor data to the parcel numbers on the ArcView data and plotted them on a map of the county to see if this was a phenomenon centered in a specific area. Turns out it wasn't, but it was worth checking.) Since then, reporters have started talk- ing to their editors about CAR projects and a few have come to me for help. I be- lieve we'll see more of them tackling CAR projects in the near future. Software: Microsoft Access, MapInfo Data: ASCII text files of sex offender information; names and addresses public schools; the locations and operators of licensed daycare centers and family-care homes. The Center for Sex Offender Management has more information on sex offender issues (www.csom.org). The center, cosponsored by the U.S. Department of Justice's Office of Justice Programs, was established in 1997. The site includes a reference library and related links. in their newsrooms, I'd recommend getting the support of at least one senior editor. We were fortunate to have editors who gave us the hardware, software and blind faith that we were doing something productive. I'd also recommend that you trick at least one other reporter into helping you. I realized, through the help of my col- league Dave Wickert, that no matter how much singular determination you have, it's always important to have someone to counter your assumptions and share your frustration when a simple Access function turns into a fiasco. Paula Lavigne Sullivan can be reached by e-mail at Paula.Sullivan@mail.tribnet.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-prepared-success","slug":"2001-septemberoctober-prepared-success","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-prepared-success.html","year":2001,"month":"September/October","monthNum":9,"headline":"Prepared success","kicker":"CENSUS","author_name":"Colleen O'Dea","author_title":"CAR person","affiliation":"Daily Record","summary":"A smaller newspaper's approach to efficiently covering Census data releases through advance planning, dummy databases, and pre-written story templates, allowing them to compete with larger publications despite limited resources.","keywords":["Census","planning","databases"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["ArcView","Access"],"full_text":"The date of the release of the first Cen- sus data was like election night in a for- eign country: There was the deadline pres- sure to get stories, charts and our first ArcView maps in the paper, and the re- sults arrived in a format that was unknown previously (New Jersey came out first) and we couldn't be sure we'd even get it, never mind be able to work with it. Even more SO than for the big boys, good planning is crucial to success at smaller papers. That made the release of the second set of local numbers - the demographic pro- files - a cakewalk by comparison, since we had a nearly weeklong embargo and had the experiences of other states to model. Advance planning key Yet in both cases, advanced planning in the extreme was the key to getting solid, complete data packages that stood up to the competition in the next day's paper despite our smaller size and much smaller resources. Even more so than for the big boys, good planning is crucial to success at smaller papers - we have a circulation around 50,000, a staff of about 15 news reporters and a part-time CAR person (me). I used everything I learned at a couple of Census seminars and from the Census e-mail lists to do as much work as pos- sible in advance. That's what really made both Census experiences relatively smooth. And these same steps are the ones I continue to use in planning for the SF1 release. First, we examined the data to be re- leased and determined likely stories and graphics, leaving room for deviation when the numbers said something unexpected - as they did when the Census apparently missed counting hundreds of people in two of our towns. We wrote detailed story budgets and produced dummy charts and maps that we refined in advance so little editing was needed on deadline. Dummy databases Most importantly, I set up dummy da- tabases in Access in advance. I brought in all the old data against which I'd be com- paring. For my 2000 files, I used specs from the Census Web site and from Gannett News Service - they've been in- valuable to many Gannett papers. Then I wrote queries that would pull out only the towns and counties we'd want and show the changes we wanted, with all columns ordered the way it would eventually ap- pear in the paper. Thanks, Paul Overberg, for this advice! I knew I'd become a geek when I found myself singing and dancing with delight at my desk when my queries pro- duced perfect tables in seconds on re- districting day. We also did as much advance report- ing work as possible. Because I knew I wouldn't have the time to write on the date of the PL94 release but I wanted at least one byline in the package, I wrote largely in advance a story on the change in the population of our coverage area over the century - few papers still seem to have done the story looking at population changes over the 1900s, but I thought it was perfect. Since we pretty well knew the trends in advance, I had to do little more than plug in numbers that day. On deadline, we produced a four-page special section that covered the major trends and provided several types of chart and maps. We followed up with profiles of neighborhoods and dominant racial groups in our region and stories about growing diversity and growing segregation. We were a little more lax with our preparations for the demographic profile data, in part because I was busy on other stories and didn't have as much time to devote to the preparations and in part be- cause we let the length of the embargo lull us into a false sense of security. The planning was there, as were the old data and the 2000 data shells. But I didn't have time to write all the queries in ad- vance, which meant I had to spend more time querying data and had less for report- ing when the data was released in May. It got done, but I had to work much of a really nice spring weekend on it. I knew I'd become a geek when I found myself singing and dancing with delight at my desk when my queries produced perfect tables in seconds on redistricting day. The main problem we ran into was the discovery of a missing 10-year boy's body the day before the embargo was lifted. That bumped us below the fold and took all our front page space for a chart and photo. It also took some of our inside space, so we lost a chart, a map and a sidebar. But over the course of the three days, we got just about everything in, again, covering all our bases. Covering readership helped For the release of the demographic data, and we suspect everything else coming out, we've built more flexibility into our plan- ning. It's pretty easy to know how your population is growing or shrinking and whether it is diversifying if you cover your readership area well. But the demographic specifics - families with kids, senior citi- zens, age brackets - are harder to predict. When it turned out our senior popula- tion had grown by nearly 25 percent, we did a quick story explaining the trend in part, more senior citizens complexes be- ing built by towns to fulfill a state court mandate to provide affordable housing in wealthy Morris County. It keeps getting harder to do a good job with these numbers. For the SF1 release, we've again exam- ined the data and planned our stories but there's so much of it, I hope we're cover- ing all the major bases. There's a lot of data I know we won't need - the number of 28-year olds by race is at least one, I'm guessing. Still, I can only hope the com- petition doesn't find some tremendously interesting story by analyzing some in- nocuous factoid I'm planning to ignore. I'm also checking the Web sites of pa- pers in states with earlier release dates than New Jersey for ideas. Cross-referencing data Once again, I'm pulling comparable data off the 1990 STF1 CD and creating dummy files. I've read a lot about the in- ability of Access to work with the huge files being released - New Jersey's is 1.7 gigabytes - and ways to work around it. I've chosen to break the data into several (four is what I'm aiming for) smaller da- tabases. I'm cross referencing so I know what data is where. And I'll be writing as many queries as I can in advance. I'm also preparing my list of contacts and plan to make as many phone calls in advance as possible for comments on the expected trend stories we've planned - reaching out for gay couples is the obvi- ous one because doing such a story is al- most a sure thing and finding couples on deadline could be difficult. I'd really like to be able to take time off on the summer weekend the SF1 data is released. With my luck, I'll get the ad- vance work done and it will rain. Colleen O'Dea can be reached by e- mail at codcod@worldnet.att.net","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-septemberoctober-bits-bytes-and-barks","slug":"2001-septemberoctober-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2001-septemberoctober-bits-bytes-and-barks.html","year":2001,"month":"September/October","monthNum":9,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR announce two new staff positions: Jeff Porter as the first full-time director of the IRE and NICAR database library and Aron Pilhofer as director of the Campaign Finance Information Center. The organizations are also promoting their upcoming national conference in Philadelphia scheduled for October 11-14.","keywords":["staffing","journalism","database"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["spreadsheet","SPSS","SAS","Internet","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nStaffing News:\n\nTwo journalists took over staff positions at IRE and NICAR this summer. Jeff Porter is the first-ever full-time director of the IRE and NICAR database library.\n\nHe's worked more than 20 years in the newspaper industry. As a high school senior, Porter began developing black-and-white photographs in the darkroom of the Batesville Daily Guard, his small Arkansas hometown newspaper. After finishing his degree at Ouachita Baptist University in Arkadelphia, Ark., he became a reporter and editor, and taught journalism at Lyon College in Batesville. He became managing editor of the Batesville paper in 1988. It was in that position that he became interested in computer-assisted reporting.\n\nIn 1996, that interest led Porter to work for the Arkansas Democrat-Gazette in Little Rock, where he served as a computer-assisted reporting specialist, with data analysis, reporting and intranet management duties.\n\nPorter is also an instructor at the University of Missouri School of Journalism and helps conduct boot camps on computer-assisted reporting for print and broadcast journalists.\n\nHe can be contacted by e-mail at jeff@ire.org. Aron Pilhofer is the new director of the Campaign Finance Information Center. Before joining IRE, he was the computer-assisted reporting coordinator for The News Journal in Wilmington, Del., where he helped lead the paper's census coverage among other projects. Before that, he covered state politics and the New Jersey Legislature from the Gannett state bureau in Trenton.\n\nPilhofer replaces Ron Nixon, who is now the training director for IRE and NICAR. Pilhofer can be contacted by e-mail at aron@ire.org\n\nNICAR National Conference\n\nMake plans now to attend the NICAR national conference, to be held Oct. 11-14 in Philadelphia.\n\nConference Highlights include:\n\n- More than 50 panels that touch on every beat: Census, law enforcement, transportation, education, local government, health and more.\n\n- Hands-on classes: Spreadsheets, databases, mapping, SPSS, and SAS.\n\n- Internet training: Important Web sites, search techniques no reporter can do without, backgrounding people and businesses online.\n\nFor more information, call IRE and NICAR at 573-882-2042 or visit our conference Web site at www.ire.org/training/philly/. The Web site includes hotel and registration information.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-airport-insecurity","slug":"2001-novemberdecember-airport-insecurity","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-airport-insecurity.html","year":2001,"month":"November/December","monthNum":11,"headline":"Airport Insecurity","kicker":"SEPTEMBER I I","author_name":"Jeff Porter","author_title":"","affiliation":"IRE/NICAR","summary":"Following the 9/11 attacks, news organizations relied on CAR (Computer Assisted Reporting) and FAA enforcement databases to investigate airport security vulnerabilities, but the FAA quickly restricted access to the data it had previously made public.","keywords":["data","security","journalism"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["dBase","FTP"],"full_text":"Most Uplink readers are true believers, knowing that CAR is a necessity, not a simple luxury, for serious news organizations. That point certainly came home when newsrooms struggled with data after terrorism struck in the U.S. Sept. 11. In the aftermath of hijackers seizing four airlines and causing thousands of deaths, reporters began digging into the database of Federal Aviation Administration enforcement actions to find stories about the security of their local airports. The database includes lapses and violations of every level - from the passengers who innocently carries a knife to more serious security violations by airlines and airports. Do the data prove conclusively that airport security should have prevented the attacks? No. Do the data provide potential starting points for reporters covering the security status of American air travel? Absolutely. The IRE and NICAR data library, which provides the data to journalists, was deluged with telephone calls and data orders. During the weeks after the attacks, 119 news organizations purchased the data. IRE set up delivery via its FTP server. The database was on the library's regular schedule to be updated in October, so IRE pushed to update it even sooner. During the week of the attacks, a series of text files was obtained from the FAA, downloaded from the agency's public FTP server, and turned into a series of dBase IV files so journalists could use the data in almost any database program. This turned out to be a good decision. After stories started appearing and reporters started asking questions, the FAA turned off the data spigot within a week after the attacks. The files could no longer be downloaded. It seems that the data was suddenly deemed 'sensitive' by the agency that, since 1997, has provided copies of its Enforcement Information System database. Both the library and the NICAR-Le-mail discussion list agonized about trying to find answers about the complicated enforcement data. There are so many decisions to make on how to use the from deciding which types of violations are important to which airports or airlines to compare. But for example purposes, let's follow the path of a fictional news organization and how it reached the decisions it reached.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-novemberdecember-dangerous-drivers","slug":"2001-novemberdecember-dangerous-drivers","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-dangerous-drivers.html","year":2001,"month":"November/December","monthNum":11,"headline":"Dangerous drivers","kicker":"DATABASES","author_name":"David S. Fallis","author_title":"","affiliation":"Washington Post","summary":"Investigation into Montgomery County's lenient handling of repeat offenders in drunken driving cases, uncovering systemic prosecutorial lapses.","keywords":["drunk driving","court","leniency"],"topics":["Crime & Justice","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"The initial idea was simple: Find out how drunken drivers fared in court. The Post knew that Montgomery County, a District suburb of about 850,000 people, had a reputation among defense lawyers, police, prosecutors, traffic safety advocates, and even drunken drivers as one of the most lenient in Maryland. Ultimately, by layering paper records, electronic records and information gleaned in interviews, the Post was able to reveal a complete picture of the courts at work, finding that drunken drivers often have the upper hand. Those who had killed people in alcohol-related collisions often didn't spend a day behind bars. Drivers with arrests in the double digits often got lighter sentences with each new conviction. Prosecutors sometimes didn't know drivers' full records, leaving those who had killed others to be sentenced as first-time offenders. In examining who was responsible, the Post found prosecutors ignored repeat offender laws and expedited cases through generous plea bargains. Judges were unaware of mandatory jail laws for habitual drunken drivers and failed to distinguish between second-offenders and those with more than 10 arrests. The Post also found that key lawmakers, some of them attorneys who defend drunken drivers, were causing the state to lose millions in federal highway funds every year because of their refusal to crack down on their clients. The reporting began with a request for district court officials to provide us electronic court data, which they refused to do. This led to a decision to sample cases in Montgomery County.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2001-novemberdecember-database-tools","slug":"2001-novemberdecember-database-tools","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-database-tools.html","year":2001,"month":"November/December","monthNum":11,"headline":"Database tools","kicker":"DATA INPUT","author_name":"Ron Nixon","author_title":"Training director","affiliation":"IRE","summary":"This article explains practical database tools and features that journalists can use to efficiently enter and manage large volumes of paper documents, including forms, input masks, validation rules, and lookup wizards to streamline data collection.","keywords":["database","journalists","tools"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Access","Excel"],"full_text":"You've head the excuse from public information officers a thousand times. \"We only have it on paper.\" But this time, they're telling the truth. All the documents you need are there and the agency is willing to give it to you. But what can you do with thousands of sheets of paper? How do you analyze it? You could give up and pull a few documents and do a largely anecdotal story. Or you could do what hundreds of reporters have done: build your own databases. There have been numerous articles in Uplink showing reporters how to think about building a database - that is, think about what questions you want to ask the database, coming up with a structure and plan for inputting the data and establishing standards for data input. This article focuses on tools within database managers like Access and spreadsheets like Excel that will make data entry easier once you've come up with a plan for building the database.\n\nBuilding blocks\n\nForms-Entering data into a database can be a tedious exercise. And even the most loyal techno-junkie will get eyestrain trying to enter data into columns and rows. A form simplifies that process by allowing you to enter data into, well, a form. This removes lots of stress, especially when you're creating a database that has lots of fields.\nInput mask-These are useful functions in most databases and spreadsheets that allow you to specify exactly how you want data entered. Say you want all dates to be set up in the YYMMDD format. Using an input mask you can set your data field that way and no one can enter the dates any other way.\nValidation rules-Validation rules are similar to input masks. But unlike input masks, they allow you to limit the range of data entered into a field. Whereas the input mask will let you set the format of the date in the example above, validation rules allow you to limit the dates entered between two dates. For example you could only allow dates to be entered between 1990/01/01 and 2001/01/01.\nLook-Up wizard-The look-up wizard is a useful tool that can be used to cut down on entering repeated information like codes or city names. With the wizard, you can create a drop down menu and select the name or code you want. This cuts down on typing and mistakes from multiple data entries.\nEntering data into a database can be a tedious exercise A form simplifies that process.\nAuto number fields-Auto number fields are just that, a field that contains the number for each row. These are useful if you have data that does not have an obvious key field.\n\nOne more\n\nIndexes-If your database is going to include thousands of records, indexing the records might make sense. Indexes help speed up the searches and allow you to locate records quicker. Be warned however, that indexes themselves take up disk space and it may not be very beneficial to use indexes on smaller databases.\n\nBuilding you own database can be a daunting task and even most CAR-jockeys would rather not, if they don't have too. But the tools listed above can make the task a lot easier. So the next time an agency says \"We only have it on paper,\" don't despair; tools with database mangers and some spreadsheets can make the task a little less daunting.\n\nRon Nixon can be reached by e-mail at ron@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-tipsheets","slug":"2001-novemberdecember-tipsheets","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-tipsheets.html","year":2001,"month":"November/December","monthNum":11,"headline":"Tipsheets","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The IRE Resource Center offers tipsheets providing guidance on building databases for investigative reporting, including cost-effective strategies and best practices from experienced journalists.","keywords":["database","journalism","tipsheets"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"More information on building a database is available from following tipsheets in the IRE Resource Center:\n\nTipsheet #806. Handout includes \"Ten Commandments of Building Your Own Database\" and advice from Jeffrey Meitrodt, Carolyn Tuft, and David Washburn. Also available on audiotape by calling (303) 649-1811 ($11).\n\nTipsheet #948. \"Frugal and Proud of Frank Krummer explains how to stretch a CAR budget by sharing data with organizations and building databases from Web sites or NICAR sources.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-tracking-rail-accidents","slug":"2001-novemberdecember-tracking-rail-accidents","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-tracking-rail-accidents.html","year":2001,"month":"November/December","monthNum":11,"headline":"Tracking Rail Accidents","kicker":"TRANSPORTATION","author_name":"Russ Clemings","author_title":"","affiliation":"Fresno Bee","summary":"A journalist discovers that while computer-assisted reporting on railroad crossing accidents is easily accessible, the most dangerous crossings aren't always the best stories, leading him to investigate neglected crossings in California's San Joaquin Valley by visiting them in person.","keywords":["railroad","data","investigation"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["FoxPro","Access","Excel","Paradox","DBF"],"full_text":"Accident rates at railroad crossings are a mainstay of computer-assisted reporting, and for good reason - the data is freely available and easy to work with, and the subject is one of near-universal interest. But sometimes the most obvious story - the one that identifies the most dangerous crossings or lines - isn't the best story. After all, no matter how many safety measures are undertaken, there will always be a \"most dangerous\" crossing, and many times it will have earned that distinction simply because it's the busiest in terms of train or vehicle traffic or both.\n\nDeaths at rail crossings, as it turned out, are relatively rare, at least when compared to ordinary traffic accidents, homicides and other violent causes.\n\nMillions of dollars in federal, state and local funding has been spent in the past three decades to address the most serious crossing hazards, and by most accounts, progress has been made. Accidents and fatalities declined by two-thirds between 1976 and 1987, and since then have remained roughly level despite generally increased traffic.\n\nNot every crossing that needs attention has received it, however, and those that were neglected became the basis for a Fresno Bee package on May 6, 2001.\n\nReports downloaded\n\nFirst, the data: Since 1975, the Federal Railroad Administration has kept a database of rail crossing accidents. Railroads file reports, and an FRA contractor enters them into a computer, and the agency makes them available over the Internet at http://safetydata.fra.dot.gov. The same site also provides links to other FRA databases, including its national rail crossing inventory and its records on deaths and injuries among railway workers. The site allows users to query the data directly or download custom extracts in DBF (FoxPro), Access, Excel, Paradox and ACSII formats. I downloaded 25 years of accident data for California in a couple of hours one afternoon last winter and easily loaded them into FoxPro. Then I started doing queries and found well, not a whole lot, actually. Deaths at rail crossings, as it turned out, are relatively rare, at least when compared to ordinary traffic accidents, homicides and other violent causes. For the period from 1996 to 2000, the number of rail crossing fatalities in our six-county circulation area came to the not exactly eye-popping total of nine.\n\nBeat the street\n\nIt was time to get out from behind the computer for a while and see if I could find a story that the data wasn't showing me.\n\nTwo major rail lines - the Union Pacific and the Burlington Northern Santa Fe - pass through most of the cities and towns of the San Joaquin Valley to link northern and southern California. I resolved to visit every one of their crossings in the heart of our region, Fresno and Madera counties. Maybe, I thought, an idea would strike me after I'd inspected a hundred or so crossings in person.\n\nIt didn't take quite that long. After about an hour of looking at crossings on the first of the lines, the BNSF, I was driving in an industrial area south of downtown Fresno. The street I was on paralleled the tracks; another ran parallel on the opposite side. Two other streets - California and Hamilton avenues - crossed both streets and the tracks within a few blocks of each other.\n\nUnlike every other crossing I had seen until then, these two did not have crossing gates, only lights. Vehicles ranging from tamale carts to tractor-trailers skittered over the tracks. Suddenly the lights came on, a bell sounded, and a dark blue double-decked Amtrak California train lumbered into view from the north, blasting its horn. Some of the vehicles ignored the lights until the train was practically upon them. I found two more such crossings in a poor residential neighborhood just north of downtown, and another on the UP line in Selma, a town of 20,000 about 15 miles southeast of Fresno. Later, when I got back to the office and looked in the FRA data, I saw that there had been accidents at each of those crossings. The California Avenue crossing had been the site of eight accidents in 15 years. None was fatal, probably owing to the fact that trains move slowly through that congested area.\n\nFatals galore\n\nThe Selma crossing on North Street was another story. In one six-month period, there had been three accidents, one fatal, and five years earlier there was a double fatal. The story that developed from these discoveries focused on the failure of government agencies to address these hazards. When I interviewed Selma officials, they admitted that they had never applied for money from the federally funded account (called \"Section 130\") that pays for crossing gates. But when I interviewed the state official in charge of distributing those funds, he told me that the city should have applied. With the poor safety record of that crossing, the application would be likely to get immediate approval and funding.\n\nA county official suggested that I also look at a separate state-funded account that pays for grade separations - bridges and tunnels - to eliminate dangerous and congested crossings.\n\nAnemic fund\n\nThat fund is notoriously anemic. The waiting list for appropriations is measured in years. But when I started reading the fund's annual reports, published by the state Public Utilities Commission and posted on its Web site, I came across a curious reference to two Fresno projects that had been stalled. Funds for them, the report said, would be taken back and spent elsewhere.\n\nCity officials somewhat sheepishly admitted in subsequent interviews that, yes, they had received authorization from the fund for bridges at two of the city's most-traveled rail crossings, but had to forfeit the money because they couldn't come up with matching funds.\n\nThe city's budget documents showed that millions of dollars in discretionary transportation funds were passing through its treasury each year. But instead of setting it aside for the rail projects, the city had been spending much of it on things like new curbs and sidewalks - even tree-trimming - in a handful of residential neighborhoods.\n\nIt may not have been the story that I set out to do. But for the families of victims like Ray Sartini Jr., who died when his car was struck by a train at one of those two Fresno crossings, or Jose Patino, the victim in the most recent Selma fatality, the fact that their accidents were relative rarities made their tragedies no less compelling.\n\nRuss Clemings can be reached by e-mail at Clemings@cris.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-airport-performance","slug":"2001-novemberdecember-airport-performance","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-airport-performance.html","year":2001,"month":"November/December","monthNum":11,"headline":"Airport performance","kicker":"TRANSPORTATION","author_name":"Jeff Blyskal","author_title":"","affiliation":"Consumer Reports","summary":"Consumer Reports built a custom database to rate America's airports from scratch, revealing that travelers in major metro areas increasingly have multiple airport options including low-fare carrier hubs that can significantly improve their travel experience. The article outlines ten tips for journalists conducting original research, starting with defining standards of excellence and measurable metrics for airport performance.","keywords":["airports","database","competition"],"topics":["Data & Databases","Investigative Methods","Public Safety"],"technologies":["Excel","Nexis"],"full_text":"Airline ratings articles are a dime a dozen, in large part because of the easy availability of government airline performance databases and regular customer satisfaction surveys spoon-fed by J.D. Power & Associates.\n\nNot so for airports. Most airports are run by municipal authorities not geared for competition. Consequently, there are no airport performance compendiums a reporter can use to bang out a quick and comprehensive ratings. So when Consumer Reports set out on \"Rating America's airports\" (October, 2000), we knew we'd have to build the underlying database from the ground up.\n\nFinding trends\n\nWhy rate airports, anyway? Preliminary reporting revealed that big airports dominated by major full-fare carriers were beginning to get their first competition from smaller secondary airfields, which had become metro market beachheads for low-fare Southwest Airlines. Nor was this phenomenon isolated; it was a trend taking shape in metropolitan areas from Boston to L.A., Miami to Chicago, New York to San Francisco.\n\nAir travelers resigned to suffering long lines, long gate treks, and long delays at their same old airport may be surprised to learn that on some routes they can choose from a dozen different airports, if they take into account all their options at the departure, destination, and connecting cities. The choice, our ratings ultimately found, could make the difference between a tolerable, hellish or even a pleasant travel experience.\n\nThe benefits of building your own database are many: You get custom results that serve your specific goals. The drudgework of digging up all those numbers gives you greater depth of understanding and insight into your subject. Your original research is not dependent on available studies. Finally, you get exclusive data that your competitors will be hard-pressed to replicate.\n\nHere are 10 tips for do-it-yourselfers based on our four-month airport project:\n\n1. Define your standard of excellence. When Consumer Reports develops ratings, we aim to get to the heart of what consumers should be getting for their money. Airports are not a desired end per se, like a resort hotel; they're only a means to an end. But let's not be shy. Airports are an obstacle course between ground transport and what you're really there for: High-speed air travel.\n\nKnowing this, we took our bold empirical stand: A good airport is one that gets travelers up and down from the sky as quickly, safely, and inexpensively as possible, with minimum hassle.\n\nThe benefits of building your own database are many: You get custom results that serve your specific goals.\n\n2. Define your measures. How do you measure how well an airport meets this standard of quality? A Nexis search turned up few ideas or tools, and Web research of airport and airline sites revealed spotty, incomplete information.\n\nSo we used some common sense and our own good and bad experiences getting through airports for guidance. What determines how quickly you get on your way? Flight delays are one good measure. What constitutes hassle? How about a panicked search for an empty parking space as departure nears; rental car shuttles creeping through traffic-choked terminal roads; assignment to a Mad magazinesque Gate A1-Yea! Short walk! - that's really at the end of a three-mile-long concourse.\n\nThe airports\n\n3. Define the universe. Which airports? Drawing on the Annual Worldwide Airport Traffic Report published by the Airports Council International-North America trade group (www.aci-na.org) we zeroed in on the largest U.S. airports in the nation's most populous metro areas. We also included smaller alternative airports with jet service in these regions if they were within 50 miles of a major aerodrome.\n\n4. Organize! Because database builders have to deal with lots of individual bits of handpicked data, you've got to computerize those bits as soon as you get them. We plugged our data daily into an Excel 6.0 master spreadsheet, which allowed us to add, subtract, move, and modify data fields and records according to what the concurrent reporting for the article was telling us. The spreadsheet served three important organizing functions: 1) a behind-the-scenes storehouse of all information that might go into our final ratings; 2) an easy calculator for converting raw numbers into more meaningful comparative indexes; and 3) a project tracking tool that told which data were in and which were still missing.\n\nBecause database builders have to deal with lots of individual bits of handpicked data, you've got to computerize those bits as soon as you get them.\n\n5. Use outside vendors. Price is a major motivator for switching airports, and we used it as a key ratings measure. We hired GRA Inc., a Philadelphia airline consulting firm, to extract real airfare data from U.S. Department of Transportation nine-track tapes and give it to us in a medium and format we could use easily: Excel files on CD-ROM.\n\nThe little-known DOT source database we tapped-Origin and Destination Databank 1b-offers an amazing wealth of information: A 10 percent sample of the 40 million airline tickets actually sold for travel within the U.S. each quarter. We asked GRA to compile an abundance of very specific airfare data, which we ultimately distilled to a \"low fare\" score indicating which airport in each metro area had the lowest average restricted coach fares to their most popular destinations. This rating was based on more than 17 million tickets sold during the 12 months ended September 30, 1999.\n\n6. Get more data than you need. As with all reporting, you don't always know which information will wind up being meaningful. So it's always better to have data and not need them than to need them and not have them.\n\nNot important\n\nFor example, we gathered what we thought would be important numbers from the Federal Aviation Administration showing each airport's rate of runway incursions-planes in unauthorized proximity to other aircraft, people, or ground equipment. Incursions were a hot topic at the time, having risen 71 percent since 1993.\n\nBut interviews with safety experts and examination of the data showed the risk of actual collision was vanishingly small. There were just 328 incursions out of 68 million takeoffs and landings in 1999. And no airport we examined stood out as having a significantly worse incursion rate than others. So our final ratings did not include this measure. No sense alarming readers with meaningless comparisons.\n\n7. Use part of an existing database. We put together on-time flight arrival and departure performance data from a year's worth of monthly Air Travel Consumer Reports published by DOT and available online at www.dot.gov/airconsumer/index1.htm. The ATCR is ostensibly designed to rate airline performance, but some of the data series are also organized by airport, which we used to calculate the percentage of flights that arrived or departed 15 minutes or more late at each airport the previous year.\n\nSite visits\n\n8. Collect data in-person. To find out how far travelers must lug their carry-ons to the farthest gates, I traveled to 45 airports and walked more than 35 miles of concourses with a wheeled surveyor's measuring stick. Of course, this tour of U.S. airports was also used to get information on food prices and payphone charges, insight into terminal layout and design, digital video footage, anecdotes, and a firsthand understanding and feel for each airport.\n\n9. Use a questionnaire. To count all the payphones, ATMs, restaurants, and other services, we developed a detailed questionnaire that we faxed or e-mailed to some 75 airport authorities.\n\n10. Normalize and score. The raw numbers by themselves are not always meaningful. Sure, there are 40 restaurants to choose from, but that may not be a sufficient number if they're spread over seven terminals and you're prisoner at the end of one long concourse waiting to hear when your late flight will finally begin boarding. So instead of comparing the absolute number of locations where you can buy prepared food in each airport, we calculated the ratio of food outlets per terminal/concourse area. Similarly, we calculated the number of pay phones per 1,000 daily passengers, ATMs per terminal/concourse area, and available parking spaces per daily passenger whose travel itinerary originated at that airport.\n\nFinally, Finance Editor Louis Richman, Statistics Director Michael Saccucci, and I worked together to translate all of the comparisons into the familiar Consumer Reports ratings symbols (known around our Yonkers, N.Y. headquarters as \"blobs\") to help readers make fast and easy comparisons.\n\nJeff Blyskal can be reached by email at blysje@consumer.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-radioactive-contamination","slug":"2001-novemberdecember-radioactive-contamination","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-radioactive-contamination.html","year":2001,"month":"November/December","monthNum":11,"headline":"Radioactive contamination","kicker":"ENVIRONMENT","author_name":"Pete Eisler","author_title":"","affiliation":"USA Today","summary":"A journalism investigation into secret U.S. government nuclear weapons contractors revealed how over 300 private companies processed radioactive materials in the 1940s-50s while health risks to workers and communities were concealed. Computer-assisted reporting techniques, particularly spreadsheet organization of declassified documents, proved essential for managing and analyzing over 100,000 pages of records scattered across federal archives.","keywords":["radioactive","contractors","investigation"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["Excel"],"full_text":"The \"Poisoned workers & poisoned places\" series wasn't planned as a CAR project. With no databases to draw from, this story was supposed to be pure shoe-leather, an old-fashioned document dig through dusty archives spread across the country. It took all of a month before we turned to the computer. The series revealed that the U.S. government secretly hired more than 300 private companies during the 1940s and '50s to process thousands of tons of radioactive and toxic material for use in nuclear weapons. From small machine shops to big chemical plants, commercial manufacturing facilities across the nation were quietly converted to the risky job of handling uranium, thorium, polonium, beryllium and other dangerous material.\n\nThe newspaper showed that the government regularly documented severe health risks for employees at the contracting sites, but those studies were kept secret, leaving thousands of workers unaware of their increased odds for cancer and other illnesses. The contracting operations also contaminated the air, soil and water in dozens of communities, pumping out radioactive and toxic waste with little regard for the safety of neighboring residents.\n\nThe role that CAR played in this story illustrates the assistance computers can provide on a big project, even when there's no real database analysis involved. We used the computer less as a reporting tool than an organizational aid. But it nevertheless proved invaluable.\n\nVirtually all of the documentation for the series came from a review of more than 100,000 pages of declassified federal records on the nuclear weapons program. They were stored in a half-dozen or so libraries and federal archives across the country, from Atlanta to Las Vegas, Chicago to Washington. Most had never been reviewed since their release by the government under a declassification program that has dumped millions of pages of files from the weapons program into the public domain over the past decade. All of them were paper records, many of them so faded that they were barely legible. There was no index. Only a tiny fraction had been organized into a few searchable databases.\n\nTrack and retrieve\n\nOur goal was to go through all of these records, identify every company that secretly worked for the U.S. nuclear weapons program, document the type of work each firm did, and assess its impact on workers' health and the environment. With so many records to search, organizing our material in a way that would allow us to track and retrieve specific information and findings became critical. Some of the documents we found related to a single company; some included information that was pertinent to many of the contractors we were studying. Others didn't mention any companies at all, but contained general data on the health and environmental risks associated with particular operations that were being done at some of the facilities we were tracking.\n\nUsing Excel\n\nSo this was far more complicated than simply assigning a manila folder to each company and filling it up with whatever pertinent documents we happened to run across.\n\nWe decided to build an Excel spreadsheet that we could use to track the records and relate them to the applicable contractors. We set up a field for each company we identified, with additional fields for the firm's address, the type of work it did, the dates of its operation, the current use of the property, etc. We also included notes fields to list any document we found that might be relevant to the operations of each company, as well as the location of that document in our paper files. For example, the field for the Simonds Saw and Steel Co., a steel mill in Lockport, N.Y., that cut uranium metal rods used to produce nuclear weapons fuel, would include references to documents in our \"Health Studies\" file, our \"Uranium Processes\" file as well as our file on the company itself.\n\nGlorified index\n\nWe never actually crunched any data from the spreadsheet, other than a few basic queries to identify the number of firms in a particular state. We just used it as a glorified index. But that proved critical, because we were piecing together company histories from dozens of different types of documents. We might find a contract that named a particular contractor but didn't say what sort of work the company was doing. Another record might tell us what the work was, but it wouldn't include any information on whether the government had studied radiation doses to workers at the site. Another might offer information on typical radiation levels for workers in that type of operation, but it wouldn't list the company specifically as a place doing that work. In the end, it all had to be pieced together like a maddening puzzle. The spreadsheet simply helped us keep track of all the pieces - and where they fit together - in a much more efficient way. When it came time to run our list of weapons contractors in the paper, we already had all the information we needed in the database. We simply stripped away the notes fields, wiped out whatever coding we'd entered, and we had all the information we needed in a handy format for our graphics people. And even that wasn't the end of it. After the series ran, when readers began calling to ask where they could get more information on contracting sites in their area, I could call up the database and tell them exactly what records they needed to request from the government. It was only a few weeks before I got a call from an official at the Department of Energy: He wanted a copy of the database.\n\nPete Eisler can be reached by e-mail at peisler@usatoday.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-deciding-boundaries","slug":"2001-novemberdecember-deciding-boundaries","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-deciding-boundaries.html","year":2001,"month":"November/December","monthNum":11,"headline":"Deciding boundaries","kicker":"CENSUS","author_name":"John Perry","author_title":"","affiliation":"The Daily Oklahoman","summary":"An Oklahoma newspaper analyzes 2000 census data to explore racial segregation patterns, finding that school district boundaries may be a primary driver of residential segregation, with families choosing neighborhoods based on school quality rather than examining segregation itself.","keywords":["segregation","census","schools"],"topics":["Demographics & Census","Investigative Methods","Education"],"technologies":["ArcView","Two Theme Analyst"],"full_text":"I'll admit up front that our approach to the racial segregation story probably had as much to do with ego as with anything else. With the release of the first detailed race data from the 2000 census in March, we knew just about every newspaper would be writing stories about what the data said about racial segregation. Also, in 1995 we had written a series based on 1980 and 1990 census data showing that Oklahoma City was slowly becoming more racially integrated, but more segregated by income class.\n\nThe map showed an island of white dots in the center, each representing 50 more minority kids, surrounded by a sea of darker dots, each representing 50 more white kids.\n\nMost of our readers won't see the census stories in other newspapers, and probably no more than a handful have even vague memories of what we wrote six years ago. But still, we wanted to do something at least a little different from what we expected to see in other newspapers and from what we had already done.\n\nNo change in data\n\nWhen we looked at the redistricting data released in March, we found that racial segregation hadn't changed since 1990 anyway. The dissimilarity index for blacks and whites was 58.9 in 1990 and 58.2 in 2000. For American Indians, Oklahoma's largest minority group, the index went from 35.9 in 1990 to 34.2 in 2000.\n\nWe also rationalized that there is limited benefit to readers from just describing segregation's existence. We wanted to look beyond that to a possible cause. And the No. 1 suspect was the Oklahoma City school district.\n\nAfter a federal judge ordered busing to desegregate Oklahoma City schools in the 1970s, the district's attendance dropped from more than 70,000 to 35,000 in just a few years. The district's students are now mostly minority and poor, even despite a flurry of inner-city redevelopment in the last decade.\n\nThis suggested that families with children were deciding where to live based on school district boundaries. If that were true, we expected that families with children would more likely reside in racially homogeneous neighborhoods. That should make children more segregated than the general population. The data tended to support this theory.\n\nCalculating block groups\n\nThe actual data crunching for the story wasn't that unusual. For our earlier series on segregation, we had used tract-level data to calculate the dissimilarity index. But when we visited inner-city neighborhoods, we found that a single tract can include separate and unique neighborhoods. So we decided to calculate segregation indexes based on block groups this time around.\n\nWhen we calculated the dissimilarity index - most often described as the percent of a population that would have to move to evenly distribute them throughout an area - we found a small difference between segregation among children and among the total population. But the difference went in the predicted direction every time.\n\nWe also used ArcView 3.2 and the free add-on, Two Theme Analyst, to estimate from block-level data the population changes by school district. The under-18 population inside the Oklahoma City district boundaries grew by 2,610. But that resulted from the loss of 8,309 white (non-Hispanic white alone, in census speak) kids and a gain of 10,919 minority kids. Nearby suburban districts saw similar demographic shifts. Only more distant suburban districts had increases in white kid populations.\n\nMap helps pinpoint\n\nThe most powerful proof turned out to be a metro-area map that used dots to show where minority kid populations and white kid populations had grown. The map showed an island of white dots in the center, each representing 50 more minority kids, surrounded by a sea of darker dots, each representing 50 more white kids.\n\nThe story stayed conservative and said only that, \"Child population shifts in the Oklahoma City area suggest that school district boundaries may be a factor in residential segregation.\"\n\nGiven more time, we would have talked with people who recently moved into or out of different school districts. If the qualitative data from the interviews supported what we saw in the quantitative census data, we would have been more confident saying this is a cause of segregation.\n\nBut this is an issue we hope to revisit next year when we get the long-form census data that will include income and occupation information. And something we noticed makes us eager to do so. Even in the suburban districts where the number of white kids grew, the minority populations grew faster. This suggests that concerns about class, as much or more than about race, may be driving families from the inner city.\n\nJohn Perry can be reached by e-mail at jperry@oklahoman.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-pricing-homes","slug":"2001-novemberdecember-pricing-homes","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-pricing-homes.html","year":2001,"month":"November/December","monthNum":11,"headline":"Pricing homes","kicker":"REAL ESTATE","author_name":"Steve Orr","author_title":"","affiliation":"Democrat and Chronicle","summary":"Journalists built a custom database from scratch to track mortgage fraud by compiling public property records manually, highlighting both the control and risks of creating data systems independently rather than relying on pre-compiled datasets.","keywords":["database","mortgage","investigation"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Excel","Access"],"full_text":"The data we work with usually are already compiled when we get it. With luck, everything is in nice, neat columns, and is seemingly perfect. We check carefully for outliers and other anomalies, but if everything seems to add up, we'll accept it as accurate and go from there. It's a whole different story when you build a database from the ground up, by accumulating and keypunching your own data. On the one hand, it's reassuring to know you have total control over quality. On the other hand, though, it's disconcerting to realize that one misread digit or misdirected keystroke can turn your data into garbage.\n\nWe can track individual buyers or sellers, specific lenders, particular subdivisions - pretty much everything we need to.\n\nThe database in question was created to help with our ongoing coverage of an apparent mortgage-fraud ring led by Robert A. Amico and several family members, all of them homebuilders. New suburban homes were being sold and mortgaged for prices that seemed far too high, and many of them were quickly falling into foreclosure.\n\nJumbled piles\n\nAfter initial investigation yielded huge, jumbled piles of deeds and mortgages, it became clear we had to systematically track every facet of the process - acquisition, construction financing, initial sale and subsequent resales, purchase-price mortgages, second mortgages, foreclosures and so on.\n\nFirst we created an Excel spreadsheet, which was fine for basic information, but we needed a database to look for relationships and play with the numbers, so we made one.\n\nThe hardest part, as noted above, was acquiring the data. We knew the names of the builders, their companies and their associates, so we were able to identify properties they had built or purchased. Some information was obtained online from the unusually robust Web site of the local county clerk (www.clerk.co.monroe.ny.us), but because of the way the Web operation stores its records, we could not use it for most of our work. That later was accomplished the old-fashioned way - by standing at the counter in the clerk's office, wading through records on public-access computers and on paper, writing down the key information and later typing it into the database. Among the data we compiled were record-accession numbers used in the clerk's office; once we had these in hand, we could call up most of the records online from our office and print out copies of deeds and mortgages. (We found this feature particularly helpful as our lawyers asked for documentation as they vetted stories.)\n\nNot recommended\n\nFinding and comprehending all the data was not easy, and we would not recommend this approach to any reporter who does not possess a thorough understanding of how local property records are prepared and stored. Even though our team included someone with that knowledge, we found our data acquisition was a nightmare of shifting addresses and tax parcel numbers, of sales back and forth between interlocking companies and family members, of building loans for houses that were never built, and so on. Even though it became easier to find data after we had the first few subdivisions under our belt, the process took several months to complete. Truth be told, a few of those errors of transcription or entry were made and have subsequently been found, and there are a few data gaps that no degree of digging has been able to fill.\n\nThe Access database itself was relatively straightforward in its construction. For ease of use, it was broken into three tables, and each suspect property was given a unique ID number. That number and the address and subdivision became the basis of the first table. By the time we finished, this one had 237 records - one for each home the accused folks had been involved with.\n\nA second table was created that included a record for each property sale involving one of the homes. The key information here was the buyer and seller names and the sale price, along with a code to note what type of sale it was. (These codes, and similar ones in the mortgages table, proved the most confusing part of the database, as so many permutations were possible.) After the land purchase and the various resales and intra-family transfers were tracked down and entered, this one reached 581 records.\n\nMortgages\n\nThe third table tracks mortgages plus second mortgages that were taken out by some home-buyers/co-conspirators. Fields were created to record what happened to each loan - whether they were paid off or, as has happened at least 131 times so far, the borrower defaulted, prompting foreclosure proceedings. This table, which is still being amended as new foreclosures are filed, tracks 626 loans.\n\nTracking made easy\n\nThis arrangement allowed us to track the amount spent to build the homes in question, the amount for which they were sold (in many cases to straw buyers or family members), the amount borrowed, the number of foreclosures. We can track individual buyers or sellers, specific lenders, particular subdivisions - pretty much everything we need to. The database proved invaluable in many stories, including those done after three family members and a handful of associates were indicted by a federal grand jury. Nine months after the indictment, we continue to check the database for tidbits to lard our follow-up stories, and continue to update it with new foreclosures.\n\nSteve Orr can by reached by e-mail at sorr@DemocratandChronicle.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-family-offense","slug":"2001-novemberdecember-family-offense","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-family-offense.html","year":2001,"month":"November/December","monthNum":11,"headline":"Family offense","kicker":"COURTS","author_name":"Susan L. Oppat","author_title":"Reporter","affiliation":"The Ann Arbor News","summary":"A journalist with two decades of crime reporting created a comprehensive database of 435 child sexual assault cases to expose the reality that perpetrators are rarely strangers and that these crimes occur across all family types and socioeconomic levels.","keywords":["investigation","database","abuse"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Access"],"full_text":"I've spent most of my 21 years as a reporter working cops and courts, crime and punishment. It's been painfully clear that when it comes to the littlest victims, the children, there's plenty of crime, and not much punishment.\n\nLargely, punishment relies on already-traumatized children having the courage to tell strangers they were sexually assaulted, often at the hands of their own family members. But even when they do come forward, children often aren't believed.\n\nAnd, of course, there are the children who are too young to report. They are too young to speak - but not too young to be victimized.\n\nGoals\n\nI wanted to prove what I knew from experience that it happens, that the offender is only rarely a stranger, that it happens even in the best families, that the children are boys and girls, and every age.\n\nI also wanted to speak with conviction about what happens to the people who sexually abuse children.\n\nSelf-trained on Microsoft Access, I created a database with 35 different categories of information, the analysis of which would be the basis for all my reporting. I included:\n\nThe name of the suspect, age, gender, address, ZIP, previous criminal record, if known.\nThe name of the child, age, gender, address, ZIP code.\nThe location of the assault, date(s) of assault.\nThe relationship, if available, between the defendant and the victim.\nDetails on the nature of the assault, if available.\nWhich police agency investigated.\nThe exact charges, and how many filed.\nThe name of the judge, prosecutor, defense attorney.\nWhether the outcome was achieved by jury trial, bench trial, or plea agreement.\nThe charges on which the defendant was convicted, if any.\nThe sentence, and whether the defendant was sentenced to jail, prison or probation.\n\nI took a laptop computer to the county felony court, and spent more than a week combing through 435 cases of sexual assault on children under the age of 17 filed in the previous 6½ years. I had to limit myself to that time period because that's how long the cases have been listed on a computer database, and child sex assaults can be separated from all other sex assaults by an identifier on the case number.\n\nI was not charged a fee to sit in the file room and read through the hard-copy files. Court records here are listed in a computer database, but the information contained in the files is not. So it all had to be done by hand. I was a rookie with databases, but two things became immediately apparent:\n\nConsistency of input data is paramount. You must use a limited range of entries to get useful data. For example: Always refer to the police agency by acronym, or always spell it out. If you sometimes enter the acronym, and sometimes spell it out, your data won't be compiled accurately, and you'll be doing a lot of arithmetic by hand, always the quickest way to inaccurate data and ruined credibility.\nThe other thing seems like a no-brainer but it's crucial. Back up your data. Often. At LEAST daily, if not more. Keep it on diskettes AND the hard drive.\nYou REALLY don't want to re-enter all that data.\n\nTime to analyze\n\nThen it was time to analyze the data, over a period of days: How many of the defendants were men, how many women? Where did most of the defendants live in the county? What was the average age of the children? How many were related to the defendant, or at least knew the defendant? How many, really, were strangers? How many of the children were assaulted in their own homes? How many defendants were charged with the most serious offenses? How many of those pleaded down to a reduced charge? Which judge handed out the heaviest sentences? The lightest? How many of the defendants were children themselves? How many charges each defendant faced? Since there were only 207 defendants in 435 cases, it's obvious most were repeat offenders.\n\nSearching for reasons\n\nOn to the reporting: I talked to social researchers about why most of the offenses occurred in the area of the county with the lowest income. To prison officials and counselors about how they (don't) provide therapy to offenders while they're in prison, and their prognosis for change. To prosecutors and defense attorneys about why the overwhelming number of cases were pleaded down and most offenders serve only probation.\n\nI spoke to social experts about why children rape other children. To police about how hard it is to help children tell strangers their deepest, darkest secret while the offender is watching from just 15 feet away across the courtroom. To prosecutors and police about why many cases are reported, but never prosecuted.\n\nI interviewed the mother of a pair of young boys repeatedly molested by a trusted neighbor.\n\nAnd I spoke to police and counselors about what parents should really be looking for. It's not strangers you have to worry about. In 435 cases in 6½ years, only three strangers were prosecuted for sex assaults on children.\n\nThe result was a three-day series with nine or 10 stories, and additional graphics, including a ZIP code map indicating where the assaults occurred.\n\nSusan L. Oppat can be reached by e-mail at soppat@aa-news.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-dragging-justice","slug":"2001-novemberdecember-dragging-justice","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-dragging-justice.html","year":2001,"month":"November/December","monthNum":11,"headline":"Dragging justice","kicker":"COURTS","author_name":"Polly Saltonstall","author_title":"Freelance writer","affiliation":"Standard Times","summary":"A journalism investigation into New Bedford's drug court system revealed inconsistent sentencing patterns and high recidivism rates, with similar offenders receiving vastly different sentences despite tough state drug laws.","keywords":["sentencing","drugs","courts"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Excel","Access"],"full_text":"For years New Bedford, Mass., residents questioned why police seemed unable to make inroads on the city's escalating drug woes. They complained that dealers once arrested would show up back on the streets selling again within weeks. My editors asked me to take a look at how drug cases fared in the court system and to determine if the courts were part of the problem.\n\nThe resulting series \"Busted Justice: Drugs, the Law and Bristol County\" did just that. We found that despite tough state drug laws designed to minimize disparities, the courts were churning out an unpredictable, crazy-quilt pattern of sentences. In some cases, defendants who appeared to be low-level dealers or drug couriers were sentenced to as much time, sometimes more, than people with long records. Some defendants with previous drug convictions avoided jail completely, while others charged with their first offense ended up behind bars for years.\n\nRecidivism high\n\nAnd as a glut of cases bogged down the court system, delaying trials in some cases for as much as two years, recidivism among defendants was high. Many defendants were charged with new drug crimes while awaiting trial for previous arrests. But many of these people facing multiple cases were able to lump them together and serve one sentence for all charges. Most of the most serious cases such as drug trafficking where prison terms are mandatory upon conviction - were either dismissed or reduced, as prosecutors used the threat of the mandatory sentences to extract guilty pleas on lesser charges for much less jail time.\n\nDoing this as a CAR project was a big deal for the Standard Times, since no one on the staff knew anything about data analysis. I knew a little Excel but that was it. We had limited computer capacity and an extremely limited budget.\n\nI started my data collection with a computer printout of all the people arrested in 1999 on drug-related charges (cops told me they could not provide this on disk and I would not have known how to handle it anyway if they had). A newsclerk typed that list into Excel, and I then took it on my laptop computer up to district court where I looked through paper files for matches with my arrests.\n\nWe found that despite tough state drug laws designed to minimize disparities, the courts were churning out an unpredictable, crazy-quilt pattern of sentences.\n\nThe court's computerized files included minimal information, such as names and dispositions. But I wanted to know more about the process, such as whether the case was resolved through a trial and whether charges had been reduced or dismissed as a result of plea bargains. So I spent three weeks going through more than 2,500 files (all the court's cases for one year) looking for matches. For each case, I noted: the arresting officer, the date of arrest, the defendant's name, address and birthdate, the applicant's place of employment and birth, the judge, the prosecutor, the original charge and whether that charge was reduced as a result of a plea bargain, and the type and date of disposition.\n\nComplex database\n\nThe big problems began when I tried to analyze my database, which ended up being about 2,200 lines with 30 fields. All I knew was Excel and it quickly became apparent that my questions were too complex for that program. I called the folks at IRE who told me to use Access. But neither I nor anyone in my newsroom knew how to use Access. So I signed up for the IRE Bootcamp and took my database with me to Missouri. At the bootcamp, Tom McGinty showed me how to import my database into Access, but then realized I had lumped too much data into individual cells. For example, I had put complete sentencing information in one column. This included whether a defendant had been sentenced to jail time and how much, as well as length of probation. I had to separate the data in separate fields in order to answer such questions as what percentage of people found guilty had been sentenced to jail and then to figure their average jail time.\n\nCleaning the data and crunching the numbers took another two months because I was still learning the program as I went, and also because I became sidetracked on another story involving potential police corruption and/or misconduct. My queries included finding the percentage of convictions, percentage of guilty findings where charges were reduced as a result of plea, type of drugs involved, average ages of defendants, number of dealers versus users, and the length of time between arrests and sentencing.\n\nStartling finding\n\nOne startling finding was that very few people charged with dealing drugs in a school zone, a charge that carries a minimum of two years in jail under Massachusetts law, were actually sentenced on that charge. Prosecutors told me this was because the school zone law was flawed. Still these same prosecutors adamantly opposed a legislative proposal to loosen that particular mandatory sentence.\n\nThe next wrinkle was finding a way to compare my data with other states and/or regions. Although I obtained national drug conviction statistics from the Department of Justice, I was reluctant to compare them with the local data or rely too heavily on those comparisons without knowing whether I was comparing apples with apples.\n\nStatewide sentencing data was based on final dispositions and final charges. It did not consider the impact of plea bargains and to what degree charges were reduced as a result of pleas. The most conclusive information came from records obtained from the prosecutors in a neighboring county that showed the local prosecutor was making more deals than his other colleagues.\n\nHitting the pavement\n\nOnce I had a set of raw numbers and conclusions, with some help from two other reporters, John Doherty and Aaron Nicodamus, I hit the pavement and began reporting. This entailed interviews with defense attorneys, prosecutors, judges, police, policy makers, including members of a state task force convened to analyze state sentencing trends.\n\nAll I knew was Excel and it quickly became apparent that my questions were too complex for that program.\n\nIn an attempt to give the numbers faces, I used a public records request to obtain criminal background reports on almost 40 defendants and interviewed many of these same defendants or their attorneys - for those in prison that proved time consuming because corrections officials required inmates to write me a letter agreeing to interviews and not all wanted to talk. Massachusetts charges a $25 fee, set by state law, for an individual's criminal record. I found the information provided by the state corrections system did not always correspond to information in the local prosecutor's files, but that's another story. I sat through dozens of court trials and arraignments. Some of my best information came from chance interviews in the courtroom hallway. Once word got around about my project, many court regulars made a point of pulling me aside and giving me their two cents. This resulted in some good tips about cases that illustrated my findings. It also confirmed for me that the results of my data analysis were on target.\n\nInternet sources included the Bureau of Justice Statistics at http://www.ojp.usdoj.gov/bjs/. This site had good reports on drug arrests and convictions. The White House Office of National Drug Control Policy at http://www.whitehousedrugpolicy.gov/ This site had good links to other drug-related sites.\n\nLessons learned\n\nSince court computer records in Massachusetts are limited, when I began my investigation I was forced to make my own database. Being a complete novice with computer-assisted reporting, I know now I went about this the wrong way. Instead of thinking up front about how I might want to analyze my data, and hence how to organize it, I just jumped in with collecting information and inputting it into Excel. This meant that I had to go back into my database after the fact and spend days cleaning it up. This included simplifying columns by doing things like parsing out addresses and charges.\n\nLaptop problem\n\nAlso, I started using the newspaper's old Dell laptop. This broke down part way through the data collection and since the newspaper did not have another one, I was forced to use my own Macintosh Powerbook. This meant running Access through Virtual Windows, which added a layer of complications to my data analysis. Were I to do something like this again, I would think through equipment questions up front.\n\nPolly Saltonstall, formerly of the Standard Times, can be reached by e-mail at PSalty@aol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-affordable-car","slug":"2001-novemberdecember-affordable-car","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-affordable-car.html","year":2001,"month":"November/December","monthNum":11,"headline":"Affordable CAR","kicker":"SOFTWARE","author_name":"Aron Pilhofer","author_title":"Director","affiliation":"News Journal","summary":"A journalist discovers open source software like MySQL, Linux, and Apache as cost-effective alternatives to expensive commercial databases and applications for computer-assisted reporting projects. The article explains what open source software is and how it enabled the newsroom to build powerful data analysis tools at zero cost.","keywords":["database","open-source","newsroom"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["MySQL","Access","SQL Server","Oracle","Linux","Apache","PHP","CD-ROM","Internet"],"full_text":"I was well into a major campaign finance project last year when I realized I had a problem. I needed to analyze a database of two million records, and I knew that was a job I could not coax Microsoft Access into handling.\n\nI needed a high-octane database manager that wouldn't break the News Journal's modest CAR budget. That ruled out SQL Server, Oracle and every other commercial database application I could think of.\n\nAfter a few days of searching for a workaround, a colleague suggested I look into something called open source software. I had heard the term before, but assumed it meant just another flavor of low-end freeware or shareware.\n\nI needed a high-octane database manager that wouldn't break the News Journal's modest CAR budget.\n\nI was skeptical, but even more desperate, so I gave it a shot. An hour after downloading and installing a database application called MySQL, I knew how misplaced my concern was. MySQL was not only the answer to my specific problem, but I was able to find other open source applications that allowed me to extend my newsroom's CAR program well beyond what I thought was possible -- or affordable.\n\nBy the time I left the News Journal in June, reporters could search campaign finance records or voting records using our internal intranet, or, those with more advanced CAR skills could connect to any database in our library using Access as a graphical user interface. The tools we used were all open source: Linux, Apache, PHP and MySQL. The cost to the organization was zero.\n\nNo matter the size of your newsroom or budget, open source software may be worth a look for you too.\n\nWhat is it?\n\nOpen source software is often confused with freeware or shareware, but it is neither. It is a radical departure from traditional methods of developing and distributing software.\n\nIn practice, open source means just what it implies: The source code - or lines of instructions that tell the computer what to do - is available for anyone to read, modify and redistribute. But there is a catch: Everyone must play by the same rules. Anything based on an open source application must itself be open sourced.\n\nThe open source license ensures that enhancements, fixes or other changes are freely available to everyone and offers incentive for collaboration. Linux, probably the best-known open source project, involves thousands of programmers worldwide. (For more information, start with the Open Source Initiative web site, www.opensource.org).\n\nThe approach has been remarkably effective, much to the chagrin of corporations like Microsoft (which just launched a public relations blitz to discredit open source software). Linux has steadily increased in popularity, and was the operating system of choice on about 30 percent of all servers sold in the U.S. last year.\n\nApache is the Web server application running about 60 percent of all Web sites on the Internet - ironically, even a few owned by Microsoft. And MySQL is becoming one of the most popular database servers for mid- to large-scale Web sites, including Yahoo Financial, slashdot.org and NASA.\n\nOpen source: the good\n\nCost: Open source software is almost always free for non-commercial uses and usually can be downloaded straight off the Internet ready to install or purchased on CD-ROM for about the price of the media and shipping.\n\nQuality: Because so many developers are working on a project at one time, serious programming bugs are rare and are usually fixed almost as quickly as they are found. Enhancements are added remarkably quickly as well. At present, for example, Linux is the first and only operating system that supports Intel's newest generation microprocessors.\n\nNo matter the size of your newsroom or budget, open source software may be worth a look for you too.\n\nCross-platform compatibility: Open source developers have no incentive to \"lock\" users into a particular operating system or platform in fact, just the opposite. MySQL, for example, runs on 11 different hardware/operating system configurations, including Windows and Macintosh. That makes porting data or code from one platform to another virtually painless.\n\nSpeed: Open source software does not suffer from performance-killing \"feature bloat\" that plagues so much proprietary software. Enhancements are added only when needed, and with a premium placed on performance.\n\nOpen source: the bad\n\nDocumentation: Great programmers rarely make great technical writers, and it shows. The larger projects (including those mentioned here) all are extremely well documented, but that is not true across the board. Patience is a must.\n\nLearning curve: Much of the software useful for CAR is designed for the enterprise-level applications, and not targeted at desktop users or novices. Do not expect to always find a user-friendly interface, a manual free of highly technical compuspeak or point-and-click simplicity. Did I mention patience?\n\nSupport: Official support is nonexistent. The software is usually distributed as-is, without warranty or bundled support. However, there is almost always an active community of users, who are happy to help solve problems that crop up. There are also third-party vendors that offer support packages at a price.\n\nThe \"L\" word: Linux is the project that gave real momentum to the open source movement, and remains the operating system of choice for most developers. Porting applications to Windows or Macintosh is a recent phenomenon. Most ports I have tried run great in Windows, although they are sometimes more stable and faster under Linux. Bottom line: You may have to learn to love the penguin.\n\nWhat's out there?\n\nPlenty, and I am finding more every day. The best repository of open source applications is sourceforge.net. Here are a few I have found useful over the past year:\n\nMySQL: One of the fastest database managers and servers available, period. It is optimized to serve data for Web applications, but MySQL can function as a heavy-duty database manager for desktop use. No built-in GUI, but Access (or any ODBC-compliant database manager) can be used as one. Easy to install on Windows or Linux and great documentation. Web site: www.mysql.com\n\nPostgreSQL: Another high-end database server. It isn't as fast as MySQL but it does support many more features, including views and subqueries. It is not nearly as easy to install under Windows, however, and there is no Macintosh version I am aware of. Web site: www.postgresql.org\n\nMapserver: An open source application to build Web applications using GIS, similar to ArcIMS, but without the hefty price tag. Runs under Windows as well as Linux. Check the Gallery section of the Web site for examples of Mapserver in action. Web site: mapserver.gis.umn.edu\n\nPHP: A scripting language similar to Active Server Pages, but faster and much more flexible. PHP scripts will run unchanged on just about any platform, and has built-in support for nearly every database manager on the market, including Oracle and SQL Server. Web site: www.php.net\n\nFour more\n\nPython: A simple, yet powerful, programming language that is ideal for novice programmers or those who need programs that run on many different platforms. Python applications will run under Linux, Windows, the Macintosh OS, OS/2 and a number of other operating systems. Web site: www.python.org\n\nApache: The popular Web server application in the world, and with good reason. It is fast, secure and runs on 28 different operating system/hardware combinations. Configuration can be tricky, however. Web site: httpd.apache.org\n\nStarOffice: Sun Microsystems recently open sourced this integrated office suite, and it is available for download. The current version 5.2 is so-so, but Sun is promising big things in the upcoming 6.0 release. Web site: www.sun.com/products/staroffice/\n\nGnumeric: A spreadsheet application that rivals Excel feature-for-feature. Gnumeric is Linux-only right now. Web site: www.gnumeric.org\n\nAron Pilhofer can be reached by e-mail at aron@ire.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-danger-below","slug":"2001-novemberdecember-danger-below","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-danger-below.html","year":2001,"month":"November/December","monthNum":11,"headline":"Danger below","kicker":"ENVIRONMENT","author_name":"MaryJo Sylwester","author_title":"environment writer","affiliation":"Kansas City Star","summary":"Journalists used computer-assisted reporting to uncover that thousands of underground storage tanks continue to leak and pollute water and soil despite federal regulations and billions in prevention spending, with leak detection systems failing to work effectively across multiple states.","keywords":["tanks","regulations","leaks"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Access","DBF"],"full_text":"The world of underground fuel storage tanks has largely gone unnoticed by the media and environmental groups in the two decades since Congress first authorized environmental regulations for tanks.\n\nAt the same time, vast changes in the industry brought about by these new regulations have caused numerous problems that should be brought to the public's attention. Computer-assisted reporting can make that investigation a little easier.\n\nWith the help of CAR, Kansas City Star environment writer Michael Mansur and I uncovered the fact that thousands of leaking underground storage tanks continue to pollute water and soil, despite the new regulations and billions of dollars spent on prevention.\n\nThrough the data work and searching paper records we found leak detection was not working.\n\nOur story, published April 22, 2001, in The Star, came a month before a General Accounting Office report that called for congressional review of these problems.\n\nWe found that the regulations and problems vary from state to state, and as a result, you'll find a slightly different story in each state. But here are some guidelines to get you started.\n\nThe laws\n\nCongress authorized the first environmental regulations for underground storage tanks (UST) in 1984, requiring owners to register their tanks with the state. In 1988, Congress passed a series of laws that took effect gradually over the next decade. By December 1998, all tanks were required to have leak detection, corrosion protection and overfill protection. The law also banned the use of bare steel tanks and required owners to prove financial responsibility in case of a leak.\n\nMore information about the federal laws is available on the Environmental Protection Agency Web site, www.epa.gov/swerust11.\n\nState laws\n\nStates were required to either adopt the federal laws or enact their own, more stringent, regulations. Idaho is the only state that has not done this. Of the other states, most enacted a mirror of the federal regulations. A few, such as Florida and California, have more stringent laws. For example, Florida also requires tanks to be in an outer container, called secondary containment. Others ban fuel delivery to tanks that don't meet the state regulations.\n\nWe wanted to determine whether the new regulations were effective. We discovered that it was difficult to assess the effectiveness of the newest regulations (overspill and corrosion protection), but a much more important piece of the environmental laws - leak detection - had been in place since 1993.\n\nThrough the data work and searching paper records we found leak detection was not working. Even those leaks discovered after all tanks were required to be fully \"upgraded\" in late 1998 were primarily found when the owner took the tanks out of the ground, sometimes months or years after the leak first started.\n\nDatabases\n\nTo uncover this, we started with the Missouri and Kansas databases of tank registrations and leaks. Generally this information is kept in separate tables and possibly separate databases.\n\nAll states should have a database of tank/facility registrations. It may include both underground and aboveground tanks, depending on how your state program is set up. Most states don't have regulations for aboveground tanks, but they collect information from those who volunteer it. Leaks from aboveground tanks must be reported, however.\n\nWe obtained these databases from the storage tank units of the state environmental agencies. Both states required payment for copying the data, but it was less than $50, and they were prompt in responding. In both cases we received the information either in database format (dbf) tables or an Access database.\n\nThe first request was a little tricky in both states because we found neither agency was accustomed to giving out this information. At first, Kansas gave us their \"public\" version that doesn't include all of the information. Missouri, on the other hand, gave us everything the first couple times, then suddenly decided the comments field in the leaking tanks data wasn't public.\n\nMany states offer \"public\" version on their Web sites that would be worth checking out before making a request. Links to state Web sites are available at www.epa.gov/swerust1/states/stateurl.htm\n\nThe full databases could consist of multiple files, such as:\n\nOne listing each facility/site that has underground and/or aboveground tanks.\nOne listing each tank located at those facilities.\nOwner information.\nA leaking sites database.\nEnforcement actions data.\nInspections records.\n\nFacility/tanks\n\nThe facility portion will identify the particular gas station and its location. It might include geographical locators such as latitudes and longitudes that could be used to map the data. If so, be sure to also ask what datum they used to determine the \"lats\" and \"longs.\"\n\nThe tank portion will provide details about each tank: whether it is steel or fiberglass, when it was installed, the type of leak detection used, whether it meets the regulations, and many other details. The data may also include those tanks that have been \"closed\" or removed from the ground, as well as a date when it came out of service. This can be useful for understanding whether a site might be contaminated because of old tanks used prior to the regulations.\n\nThe Kansas data kept a separate record on each tank for each year. As a result, you could easily track changes in the tank status or compliance with the laws over several years. Missouri changed the information in the same record whenever the tank status changed; as a result, we lost a lot of historical perspective.\n\nWe found that about 30 percent of the active tanks in Missouri had not been inspected during the last three years.\n\nOwner information might be included with the facility or tank data, but it might also be kept separately. This should include the name of the company, a contact person, phone numbers and addresses.\n\nState oversight\n\nWe also obtained enforcement and inspection data for Missouri. The state only provided data on \"closed\" enforcement cases and said the open cases were exempt from the open records law. Since most of the tank laws had just gone into effect a year or two earlier, most of the enforcement cases pertinent to our story were still open.\n\nThe inspections data was useful to study the aspect of state oversight.\n\nBy doing a subquery between the facility table and the inspection table, I created a list of all active sites (those still pumping gas) that had NOT been inspected between 1998-2000. Then I created a list of facility ID's and the date of its last inspection. To get the last inspection date from the list of inspections (there might be multiple for each site), I did a subquery to find the \"maximum,\" or most recent inspection date.\n\nFrom that I created a table listing the name, address, city, owner, region, whether it had been upgraded and date of last inspection. We found that about 30 percent of the active tanks in Missouri had not been inspected during the last three years, the maximum time-span recommended for adequate state oversight. The leak database should list one record for each facility that has reported contamination from a leaky tank. Sometimes it's called the \"remediation\" or LUST data.\n\nThe term LUST is the one spot of humor in an otherwise serious industry. Instead of what you might think, LUST stands for Leaking Underground Storage Tank.\n\nThe Kansas data made it easy to determine how many leaks were discovered by leak detection because they had a field identifying that information in the LUST data. In Missouri, we pulled that information from paper records and created a new field for it in the database. In both states, leak detection found less than 5 percent of the leaks that occurred after December 1998.\n\nTank status\n\nThe most difficult part of the analysis was determining the status of the tanks at the time the leak occurred. We wanted to know if the tanks had been either replaced or \"upgraded\" to meet the new laws prior to the leak, or if the contamination occurred before the regulations took effect but didn't get reported until later. Missouri and Kansas didn't directly document this information in the data. Generally, we found it was available indirectly either in other parts of the database or in paper records.\n\nWithin the database, I used the tank installation and \"out of service\" dates for the tanks and compared those to the date the leak was discovered. I used paper records to fill gaps in the database. Both Missouri and Kansas were terrible about consistently filling in fields. The Kansas database could be a gold mine for a reporter because they have fields to track every little thing, but the field agents responsible for entering the information do so haphazardly.\n\nEPA data\n\nThis is one of those occasions where the federal level collects very little data. States submit two reports to the EPA each year. These include the number of leaks, active tanks, closed tanks, sites that started cleanup, sites that finished clean-up and number of emergency responses made by the agency. These numbers are then compiled into a list that is posted on the EPA's Web site in PDFs.\n\nThe paper reports include other details that provide a good overview of the state program and what things they are working on. You should be able to obtain the reports from either the state agency or the regional EPA office in your area. (In Missouri, their records department kept all of the EPA reports in one file that you could request).\n\nI put the numbers for end-of-the-year reports 1995-2000 from the PDFs into Excel and did some basic calculations.\n\nMaryJo Sylwester can be reached by e-mail at Imsylwester@publicintegrity.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-more-tank-stories","slug":"2001-novemberdecember-more-tank-stories","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-more-tank-stories.html","year":2001,"month":"November/December","monthNum":11,"headline":"More tank stories","kicker":"ENVIRONMENT","author_name":"Maryjo Sylwester","author_title":"","affiliation":"Center For Public Integrity","summary":"The article outlines multiple investigative angles on underground storage tank regulations, including insurance fund depletion, inspection oversight, and equipment failures, while offering practical advice for reporters tackling this complex beat.","keywords":["tanks","leaks","regulations"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":[],"full_text":"More tank stories\n\nThe Kansas City Star story covered the big picture - the effectiveness of the regulations - but there are many smaller pieces to this puzzle. You might find a story here that is easier and less time-consuming.\n\nInsurance funds: Most states have a gas-tax-funded insurance fund that helps clean up contamination from leaking tanks. Tank owners pay premiums to be covered in case their tanks leak, but the bulk of the cleanup funding comes from a gas tax. Originally, the states planned these funds as temporary measures to ensure sites were cleaned up, with the hope that private insurance companies would take over after the new regulations stemmed the bulk of the leaks. Since the leaks are continuing, a few state funds are starting to run out of money.\n\nInspections: Focus on the state oversight aspect by studying the inspection system. How many sites do the inspectors visit annually? What's the workload like for the typical inspector? What do the inspectors generally find in terms of compliance with the laws?\n\nEquipment: We found that some tank owners did not properly use leak-detection and corrosion-protection systems. Some incorrectly programmed the computer for the leak detection, ignored alarms indicating a leak, or turned off the electricity necessary for corrosion protection. There is also growing evidence nationally that the interior lining in steel tanks is sloughing off, leaving the tank susceptible to corrosion.\n\nAdvice\n\nThis area can be a quagmire. But it's worth wading into. Here are my recommendations, if you're going to dive in:\n\nSpend as much time as possible in advance getting to know the lingo and the laws of underground storage tanks before starting the reporting. I found people in this industry are particularly harsh to reporters who don't know what they're asking about. Once I could speak the lingo, industry officials were frank and forthcoming, as if I was a colleague. Look for brochures and newsletters geared toward tank owners.\n\nRequest a handful of paper files for leaking sites and read those before you get too far in your investigation. By going through even just a few documents, I really got a better sense of how the system operated and what all the lingo meant. I could also clearly see how the database was constructed and identify the pitfalls.\n\nBe prepared to spend a good deal of time on this story, even if you pick out one of the smaller stories. Simply getting to know the lingo will take you a few weeks. This is a good story for picking at slowly while you're working on other stories.\n\nMaryjo Sylwester can be reached by e-mail at imsylwester@publicintegrity.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-a-pet-project","slug":"2001-novemberdecember-a-pet-project","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-a-pet-project.html","year":2001,"month":"November/December","monthNum":11,"headline":"A pet project","kicker":"FIRESTONE","author_name":"Sean Mussenden","author_title":"Reporter","affiliation":"Capital News Service","summary":"Two young reporters discovered that the National Highway Traffic Safety Administration had miscalculated Firestone tire-related deaths by including two pet deaths in their official count, exposing a serious flaw in a dataset being used by major news organizations nationwide.","keywords":["verification","dataset","safety"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["Excel"],"full_text":"Never completely trust government data. Sounds like a no-brainer for veteran CAR disciples. But it took a bit of simple spreadsheet work and a chance discovery to hammer that point home for two young reporters. My colleague Mark K. Matthews and I, both working for a student wire service run by the University of Maryland's College of Journalism, broke a national story last fall that embarrassed federal safety regulators investigating Firestone tire failures.\n\nWe disclosed that the National Highway Traffic Safety Administration had miscalculated the number of human deaths linked to the tires with dead pets. Granted it was only two animals - a German shepherd and a Persian cat out of 103 total deaths reported to the agency as of Sep. 15, 2000.\n\nMajor problem\n\nBut the error exposed a serious flaw in a dataset that every major newspaper, wire service and network news program in the country was using to supplement their blanket Firestone coverage.\n\nMy first venture to the NHTSA Web site began as a quest to localize the national Firestone story. Stories by the University of Maryland's wire service, Capital News Service, are largely run by newspapers in Maryland.\n\nIn September, my editor in the Maryland state capital bureau of Capital News Service, Adrianne Flynn, wanted to find the number of Maryland accidents linked to Firestone tires.\n\nFirst step\n\nLike many journalists across the country, I downloaded NHTSA data from the Web into an Excel spreadsheet. I filtered for accidents reported from Maryland and did some quick analysis, mostly looking at the type of accidents (lots of rollovers), vehicles (usually Fords), and injuries (no fatals). After double and triple checking my work, I wrote a short sidebar for our Maryland clients to run along side the national stories.\n\nTwo weeks later, when NHTSA updated their data and added more records, it seemed natural to again trot out the Maryland data for a story. Again, I downloaded the spreadsheet, and did some quick analysis.\n\nBut this time there was a fatal listed for Maryland, a surprise given that most of the fatal accidents linked to Firestone tread separation occurred in warmer Southern climes.\n\nWe needed to verify that it was correct with the hard copy of the accident record given to NHTSA. We also needed more information, specifically names and contact information, to begin fleshing out the story (We would learn later that NHTSA redacted all names.)\n\nD.C. help\n\nFrom the spreadsheet, I pulled the individual record number and called Matthews, stationed in the Capital News Service Washington, D.C. bureau. He went to the records office, and, after a predictable hassle, obtained a copy.\n\nLike the spreadsheet, the hard copy showed two fatals. But the short narrative said nothing of people dying, just two animals. We knew we had a story, if we could confirm in the few hours before deadline that NHTSA was including these two pet deaths in their oft-repeated number of deaths linked to the tires.\n\nAfter a bit of pressure - Matthews in person at NHTSA and I over the phone - we finally got a spokesman to confirm the error. He said the agency would remove the two pets from their totals and comb through their records in search of any more errors.\n\nSome of our regular clients picked up the story, and several national newspapers, nightly news programs, and wire services ran stories about our discovery.\n\nReuters, in their version, had the good sense to call People for the Ethical Treatment of Animals for comment on the pets removal, something I did two days later in a follow-up I wrote about NHTSA naming the animals on their Web site. Predictably, PETA was outraged that the pets were removed from the list.\n\nLessons\n\nI suppose this story taught us two things:\n\n1. Always take great pains to verify the accuracy of a dataset before reporting anything. For us, that meant looking at the hard copy of the accident record. If we had not, and simply printed that two fatals had been reported from Maryland, our story would have been incorrect.\n\n2. It does not take a months-long database project to write a good story using CAR skills. Just using basic techniques in day-to-day reporting, like filtering in Excel for example, can lead to a decent story.\n\nSean Mussenden can be reached at smussenden@orlandosentinel.com.\n\nMark K. Matthews can be reached at mmthews@wam.umd.edu.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-uplink-story-ideas","slug":"2001-novemberdecember-uplink-story-ideas","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-uplink-story-ideas.html","year":2001,"month":"November/December","monthNum":11,"headline":"Uplink story ideas","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE/NICAR organization is soliciting story ideas and technical insights from journalists, particularly those involving computer-assisted reporting techniques, innovative data work, or underexplored beats.","keywords":["journalism","data","stories"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Have you or one of your colleagues recently published a story using CAR that has not been done before or involved particularly difficult data work?\n\nDo you know of a technical problem (or its solution) that others may like to hear about?\n\nOr is there some issue or beat that we haven't covered?\n\nIf you have a story idea, we'd like to hear from you. Please contact associate editors Amy Sherrill or Mike Sherry either by e-mail at amys@ire.org or mikes@nicar.org or by phone at (573) 884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2001-novemberdecember-bits-bytes-and-barks","slug":"2001-novemberdecember-bits-bytes-and-barks","url":"https://dwillis.github.io/uplink/article/2001-novemberdecember-bits-bytes-and-barks.html","year":2001,"month":"November/December","monthNum":11,"headline":"Bits, Bytes and Barks","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE announced the postponement of its Philadelphia national conference to March 2002 due to 9/11 attacks, while promoting a January 2002 DC regional conference on topics including terrorism and the U.S. Census, and updated its NASA Air Safety Reporting System database for journalists.","keywords":["conference","journalism","database"],"topics":["Journalism & Newsroom","Public Safety","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Bits, Bytes and Barks\n\nNICAR national conference\n\nBecause of the Sept. 11 terrorist attacks on the East Coast, the IRE Board postponed the Philadelphia conference until March 14-17, 2002. Much of the program will be unchanged and the conference will remain in Philadelphia (See details below). However, we are adding new sessions to discuss coverage of the attacks. The conference site is now the Double Tree Hotel (Broad Street at Locust), in the heart of Philly's theater district. For hotel reservations call: (215)-893-1600. Please ask for the Investigative Reporters and Editors room block. Reservation cut-off is Friday, Feb. 8. Room Cost is $129 single/double, $139 triple/quad. For more information call IRE at (573) 882-2042 or go to www.ire.org/training/philly/\n\nDC regional conference\n\nPlan to attend one of the most practical training events in Washington, D.C. The 2002 regional conference, to be held on January 26, will address hot topics like the U.S. Census, terrorism, aviation, health, and education. You can also learn about important Web sites and crucial Internet search techniques. There will also be an optional CAR day on January 27 at the nearby Medill School of Journalism Washington bureau. The conference hosts are The National Press Club, Investigative Reporters and Editors, and the National Institute for Computer-Assisted Reporting. For more information on the conference, including registration information, go to www.ire.org/training/dc00\n\nData update\n\nThe database library recently updated the NASA Air Safety Reporting System database. It is current to February 2001. The ASRS database consists of anonymous reports about aviation safety. Anyone is eligible to file an ASRS report, including air traffic controllers, pilots, flight attendants and passengers. The database has information on more than 98,000 events back to 1988. The cost of the database is: $65 for small news organizations, $130 for medium news organizations and $155 for large news organizations. For more information about the ASRS database or other NICAR databases, go to www.ire.org/datalibrary/databases. To order, call (573) 884-7711 or (573) 884-7332.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-new-uplink","slug":"2002-february-new-uplink","url":"https://dwillis.github.io/uplink/article/2002-february-new-uplink.html","year":2002,"month":"February","monthNum":2,"headline":"New Uplink","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"NICAR","summary":"Uplink newsletter announces a redesign aimed at improving content delivery, with this edition focusing on security and terrorism coverage following September 11th events.","keywords":["redesign","security","newsletter"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"This edition of Uplink is the result of rethinking and redesigning, with the goal of delivering the most useful information we can. The design changes were well under way when the events of Sept. 11 occurred, so in addition to a new look, Uplink devotes most of this edition to the coverage of security and terrorism.\n\nAs always, we seek your input, so please drop us a line and let us know if the design meets with your approval, if you have suggestions, which articles appeal to you the most, and, of course, if you have suggestions for future articles. We welcome stories from our subscribers. E-mail your thoughts to Managing Editor Jeff Porter at jeff@nicar.org, or mail him at 138 Neff Annex, Missouri School of Journalism, Columbia, MO 65211.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-data-updates-trucks-dams-osha-sba","slug":"2002-february-data-updates-trucks-dams-osha-sba","url":"https://dwillis.github.io/uplink/article/2002-february-data-updates-trucks-dams-osha-sba.html","year":2002,"month":"February","monthNum":2,"headline":"Data updates: Trucks, Dams, OSHA, SBA","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library has updated several public databases including records on commercial trucks, U.S. dams, workplace safety inspections, and disaster loans approved after September 11th attacks.","keywords":["databases","disasters","safety"],"topics":["Data & Databases"],"technologies":[],"full_text":"The IRE and NICAR Database Library has recently updated the following databases:\n\nTrucks\n\nThe U.S. Department of Transportation's Truck Census File, which contains records on each company that has commercial interstate vehicles weighing more than 10,000 pounds. This group includes buses, semis and shippers of hazardous materials. Current as of July 2001.\n\nDams\n\nThe National Inventory of Dams includes U.S. dams. The fields cover the construction date, the last inspection date and whether an emergency action plan is in place in case the dam fails. Current as of mid-October.\n\nOSHA\n\nThe OSHA workplace safety data contains detailed information about workplace inspections, injuries, hazardous-substance accidents and violations for companies in the U.S. and its territories. Current as of March 2001.\n\nSBA\n\nThe Small Business Administration disaster loan database includes loans that have been approved - although not necessarily fully distributed - for businesses and individuals in the wake of emergencies. The update includes more than 700 loans related to the Sept. 11 terrorist attacks. Current as of Oct. 31, 2001.\n\nFor more information about the database library's collection, including purchase prices, go to www.ire.org/datalibrary/databases/. To order data, contact NICAR at 573-884-7711 or download an order form at www.ire.org/datalibrary/orderform/orderform.pdf","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-flunking-tests","slug":"2002-february-flunking-tests","url":"https://dwillis.github.io/uplink/article/2002-february-flunking-tests.html","year":2002,"month":"February","monthNum":2,"headline":"Flunking tests","kicker":"TEACHER QUALITY","author_name":"Becky Beaupre","author_title":"Reporter","affiliation":"Chicago Sun-Times","summary":"An investigation revealed that over 180,000 Illinois students were taught by teachers who had flunked at least one competency test. The analysis exposed significant gaps in teacher quality, especially in low-income and high-minority schools.","keywords":["teacher","test","Illinois"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Teachers who had flunked at least one test of teacher competence taught more than 180,000 Illinois schoolchildren last year. Some of these teachers failed exams - including a Basic Skills test written at an eighth- or ninth-grade level - five or 10 times before passing. Some never passed. But before the Chicago Sun-Times started poring over teacher test score data earlier this year, nobody but the individual test takers knew it - not even the state agency that kept the data. Our quest to investigate teacher quality in Illinois began with a Freedom of Information Act request for data on teacher certification. In the end, we made some startling discoveries - such as the existence of a special education teacher who had failed 24 of 25 tests. (In Illinois, teachers must pass a Basic Skills test - which contains sections on reading, grammar, math and writing - and a subject matter test in the area in which they plan to teach.) We learned that the largest proportion of flunkers were concentrated in the neediest schools: those with the poorest and lowest-achieving students, and those with the highest percentage of minorities. We also identified 635 full-time teachers who had never passed the Basic Skills test required for a regular Illinois teaching certificate. But the road to these findings was bumpy. It took us five months to figure out what data existed and convince the state to give it to us. Nobody we interviewed was able to answer questions on all the data. And nobody had ever conducted the analysis we were attempting. We would find someone who could explain one aspect of the database to us, but who had almost no knowledge of the rest. What's more, outdated records and duplicates had to be identified and corrected. We were also under a unique deadline: I was solely responsible for the analysis and was expecting my first child at the end of the summer. Our biggest hurdle was pulling together the information we needed into a usable database. The data resided in several different departments at the state Board of Education. One file contained the names of every person who had taken a test since 1988, which tests they took, when they took them and whether they passed or failed. Raw score data was available in a separate file for tests taken since July 1999. Another database listed the certificates held by each Illinois teacher. A fourth file contained information on each teacher's assignment—where and what they were teaching. A fifth file—one we already had on hand from a previous project—gave us important information on each public school in Illinois, including breakdowns by race and income. Another file from a previous FOIA provided us with the most recent student scores on state exams. This file, which contained records for each Illinois student (minus names and Social Security numbers), allowed us to examine the correlation between teacher performance on certification tests and student achievement.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-february-poor-airport-security","slug":"2002-february-poor-airport-security","url":"https://dwillis.github.io/uplink/article/2002-february-poor-airport-security.html","year":2002,"month":"February","monthNum":2,"headline":"Poor airport security","kicker":"SPOTLIGHT: TERRORISM","author_name":"Matt Carroll","author_title":"","affiliation":"Boston Globe","summary":"Following the September 11 attacks, the investigation focused on how hijackers bypassed security at Logan International Airport, exposing the Massachusetts Port Authority as a patronage haven with lax security measures.","keywords":["airport","security","terrorism"],"topics":["Public Safety","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"How were the Sept. 11 hijackers able to skip through security at Logan International Airport to commandeer not one, but two airplanes? That was one of the key questions Boston Globe reporters sought to answer following the horrific events that ended with the four planes crashed, the World Trade Center in ruins, and thousands dead. This much was known about Logan already: The Massachusetts Port Authority, the public agency that runs the airport, was a patronage haven, filled with former state officials, relatives and cronies. Articles in the days following the crashes hammered that point home. The suspicion was that security at Logan was weak. There was anecdotal proof—passengers who walked onto planes while accidentally carrying along knives, scissors and corkscrews—but no hard numbers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-february-cnns-flight-risk","slug":"2002-february-cnns-flight-risk","url":"https://dwillis.github.io/uplink/article/2002-february-cnns-flight-risk.html","year":2002,"month":"February","monthNum":2,"headline":"CNN's 'Flight Risk'","kicker":"SPOTLIGHT: TERRORISM","author_name":"Denise Prodigo-Herrmann","author_title":"Database consultant","affiliation":"CNN","summary":"Following 9/11, journalists analyzed FAA enforcement data to examine security violations at major U.S. airports and airlines, focusing on weapons detected over a 10-year period to identify potential gaps in airport security screening.","keywords":["security","airports","weapons"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["Access"],"full_text":"In the weeks following 9-11, journalists around the country were scrambling to make sense of the Federal Aviation Administration Enforcement Information System (EIS) database.\n\nUnder the guise of \"national security and safety\" the Federal Aviation Administration refused to comment about any information relating to the enforcement database, making it difficult to decipher the meanings of some codes within the database. The NICAR listserv was flooded with information, ideas, concerns and strategies relating to the data and was a helpful resource during the entire analysis process.\n\nBecause CNN was interested in security issues relating to the top 25 airports and the top 10 airlines in the country, I obtained a list of the top 25 airports based on passenger load from Airports Council International and a list of the top 10 airlines from the U.S. Department of Transportation. My analysis focused mainly on the FAA EIS database, but reports about airport security and safety from the General Accounting Office also were used. While the enforcement data was said to date back to 1962, the earlier years of data did not seem complete. The final CNN analysis covered a 10-year period from 1991-2000.\n\nWithin the FAA database, the main table included all incidents relating to enforcement, such as flight operations, training, drug testing, hazardous materials and security. Using Microsoft Access, security incidents were isolated using cat_code = 20 in the main table. The security table detailed the type of security violations.\n\nWhen linked with the main table, the database started to tell a story. For example: On April 19, 1991, at Anchorage International Airport, concourse B, a male passenger, whose identity is expunged in this database, had a firearm in his possession. The firearm was detected in the X-ray machine, and the firearm was unloaded.\n\nFor the CNN analysis, it would be impossible to provide that level of detail on deadline for 25 airports and 10 airlines over a decade.\n\nAs with all information collected from databases, the data must be confirmed by other sources, such as police records, to accurately report the story. So, I decided to summarize the database by counting specific weapons detected at airports over a 10-year period. My theory was, if airports like Newark and Logan could have terrorists board planes with weapons, they likely would have a low number of weapons detected at their airports as compared to equally large U.S. airports. If few weapons are detected as compared to other airports, then screeners aren't doing their jobs.\n\nFocus on weapons\n\nWith more than 300 security codes and no FAA confirmation, it was difficult to ascertain the meanings of some of these codes. After much debate and research, I decided to focus on all security breaches related specifically to weapons. Weapons were defined using the following security descriptions: bludgeon, explosive, firearm, firearm hand gun, firearm - long gun, incendiary, knife, other weapon, stun gun and tear gas/chemical.\n\nGrouping these descriptions into one category showed the number of weapons detected at each airport by year with further breakdowns of the number of weapons.\n\nSecurity descriptions allegedly relating to tests conducted by the FAA, where FAA employees dress in plain clothes and try to get items through security check points, were not used as the meanings of the code descriptions were unclear. Some reports, for example, stated that the code \"Fail Detect HG\" meant failure to detect a handgun while others reported that it meant failure to detect a hand grenade. Without FAA confirmation of the meanings, I decided to take a more cautious approach to my analysis.\n\nStories and graphics\n\nThe six-week project produced nine stories written by CNN.com reporter Mike Fish and a number of detailed graphics designed by the CNN.com staff. The graphics showed:\n\nThe number of incidents and violations at the top 25 airports and the top 10 airlines over a decade. For each incident, there could be numerous violations. The numbers were broken down year by year for each airport and airline in separate charts.\nThe number of weapons found at the top 25 airports over a 10-year period. These numbers also were broken down year by year for each airport in separate charts.\nThe dollar amount paid in fines by the top 10 airlines from 1998 - 2000, according to the FAA quarterly enforcement reports.\nThe annual turnover rate of security screeners at 18 airports as reported by a GAO aviation security report from June 2000. The percentages of security violations for the same period also were reported from the enforcement database.\n\nDenise Prodigo-Herrmann can be reached by e-mail at dmph@bellsouth.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-in-depth-online","slug":"2002-february-in-depth-online","url":"https://dwillis.github.io/uplink/article/2002-february-in-depth-online.html","year":2002,"month":"February","monthNum":2,"headline":"In-depth online","kicker":"SPOTLIGHT: TERRORISM","author_name":"Mike Fish","author_title":"","affiliation":"CNN.com","summary":"A CNN.com investigation reveals that private security firms hired by airlines to staff airport checkpoints operate with inadequate staffing due to low wages, leading to high turnover rates that correlate with lower security breach detection at major airports, raising concerns about aviation safety.","keywords":["security","airports","turnover"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Before the events of Sept. 11, air travelers might have thought security personnel at airport checkpoints worked for the government. The airlines, maybe. Or they may have guessed the local airport authority.\n\nThe truth is, under Federal Aviation Administration guidelines, the airlines are responsible for airport security - but they bid the job out to private security firms. So, how does the system grade out and should travelers feel secure? And, if not, what are the holes in the system and why have they gone uncorrected?\n\nArmed with a laundry list of questions, CNN.com embarked on a computer-assisted reporting project that produced the four-part series \"Flight Risk\" (www.cnn.com/SPECIALS/2001/trade.center/flight.risk/index.html). The Web site contracted with Denise Prodigo-Herrmann, former Atlanta Journal-Constitution database editor, to clean up the data and oversee its analysis. We also performed extensive searches of government and aviation Web sites, which proved helpful in providing context and leading us to earlier critical reports submitted to Congress.\n\nA 2000 report from the General Accounting Office was of particular interest, revealing that the starting salary for security screeners at 14 of the nation's 18 largest airports was $6 or less an hour (www.gao.gov click on Airport Security under GAO Reports). The federal minimum wage is $5.15 an hour.\n\nSaddled with low wages, entire airport security teams - considered one of the most crucial jobs in assuring passenger safety - routinely turn over at least once a year.\n\nWatchdogs had identified this extraordinary turnover as a dangerous shortcoming, though offering little or no statistical support. In a bid to do so, we took the GAO turnover rates for the individual airports and compared them with security breaches reported by the FAA during the same 12-month period ending April 1999.\n\nBreaking security\n\nOur analysis revealed that at major airports experiencing a more than 200 percent annual turnover rate - like Lambert in St. Louis, Hartsfield in Atlanta and O'Hare in Chicago - screeners detected a smaller percentage of security breaches. And at airports with lower turnover, screeners uncovered more security violations.\n\nThe CNN.com computer-assisted analysis of FAA enforcement data from 1991 to 2000 also found:\n\nThe top 10 carriers incurred $13.13 million in FAA fines since 1998, fines which critics say have done little to fix security shortcomings.\nLogan International Airport in Boston, Dulles in Virginia and Newark International - the originating points for the Sept. 11 terrorist hijackings - have some of the lowest number of weapons caught by the airline screeners as a percentage of all reported security violations at each airport.\nAmerican Airlines, one of the carriers whose planes were hijacked on Sept. 11, was cited for the most security incidents in the decade. Delta Air Lines was second, and United Airlines, the other carrier whose planes were hijacked, had the third-highest number of incidents. Combined, American and United airlines have paid more than $5 million in FAA fines during the past three years.\n\nOur six-week project produced nine stories under the sub-heads: The System, Previous Warnings, Comparing U.S. to Europe and Solutions. Kevin Drew edited the project, while designer Emily Chen worked with Ann Hoevel developing graphics, charts and images.\n\nBecause CNN.com isn't hamstrung by space limitations of a daily newspaper, we were able to effectively support our stories with extensive graphics and charts. We were able to take the top 10 airlines and break down their security breaches on a year-by-year basis over the last decade. The same was done with the nation's top 25 airports, where we further detailed the weapons violations at each of the airports.\n\nCritics of the security system had accused the FAA of perpetuating a system of levying fines when breaches are found rather than seeking harsher corrective measures for the airlines. We chose to profile one of the leading security providers, Atlanta-based Argenbright, while also examining the fines assessed by the FAA against the airlines over the past three years (www.faa.gov/agc/enforcement).\n\nDifferent approach\n\nSeveral media organizations also were analyzing FAA data in the wake of Sept. 11, but Prodigo-Herrmann followed a slightly different approach for CNN.com.\n\nWhile the data came with detailed record layouts, some data within specific fields were hard to decipher, specifically security descriptions. Despite numerous requests from NICAR, the FAA refused to unveil details about the data. The NICAR listserv was flooded with commentary from journalists around the country on the complexity of the security code definitions. Prodigo-Herrmann focused her analysis on weapons or violations actually detected by screeners as opposed to an analysis of weapons that screeners missed in FAA tests of the system.\n\nMike Fish can be reached by e-mail at Mike.Fish@turner.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-links","slug":"2002-february-links","url":"https://dwillis.github.io/uplink/article/2002-february-links.html","year":2002,"month":"February","monthNum":2,"headline":"Links","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"CNN","summary":"CNN's \"Flight Risk\" project, a collaborative investigative report by Mike Fish with database analysis, design, and graphics from a team of specialists, examines aviation-related issues.","keywords":["aviation","investigation","CNN"],"topics":["Investigative Methods","Public Safety"],"technologies":[],"full_text":"The CNN.com original project \"Flight Risk\" was reported by Mike Fish. Denise Prodigo-Herrmann conducted the database analysis. Kevin Drew was the editor. Emily Chen was the designer and worked with Ann Hoevel to develop graphics, charts and images. Meriah Doty also developed images. Robert Felker was the illustrator. Charles Rawls was the web editor. www.cnn.com/SPECIALS/2001/trade.center/flight.risk/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-access-after-attacks","slug":"2002-february-access-after-attacks","url":"https://dwillis.github.io/uplink/article/2002-february-access-after-attacks.html","year":2002,"month":"February","monthNum":2,"headline":"Access after attacks","kicker":"SPOTLIGHT: TERRORISM","author_name":"Jennifer LaFleur","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"Following the September 11 attacks, federal agencies have begun removing public databases and records from their websites citing security concerns, despite no explicit White House directive to do so. Attorney General John Ashcroft's memorandum backing these actions has raised concerns about restrictions on public access to government information protected under the Freedom of Information Act.","keywords":["security","transparency","government"],"topics":["Government & Politics","Data & Databases","Public Safety"],"technologies":["World Wide Web"],"full_text":"Just as federal agencies were beginning to develop useful Web sites, where more and more data was being made available, many agencies are now swiftly pulling databases from their sites.\n\nSince the Sept. 11 attacks, several federal agencies have removed public records and documents from their Web sites because of security concerns. In addition, some states have proposed rules that may limit access to public records.\n\nMuch of the information federal agencies handle is covered by the federal Freedom of Information Act, a law enacted in 1966 that requires government agencies to disclose records. Since then, Congress has amended the law to encourage, but not require, agencies to make commonly requested information available on their Web sites. Many responded to that call, but some are now beginning to rethink what they put online. Among them:\n\nThe Bureau of Transportation Statistics has discontinued public access to the National Transportation Atlas Databases and the North American Transportation Atlas. Environmental groups have used information from those sites to assess the impact of transportation proposals.\n\nThe U.S. Geological Survey has removed several reports relating to water quality.\n\nThe Office of Pipeline Safety has removed access to the pipeline mapping system, making the information available only to pipeline operators and government officials.\n\nThe Federal Aviation Administration has removed a link to enforcement data that includes security violations at the nation's airports. Several news organizations used the information for stories before the agency removed it from its site.\n\nThe Environmental Protection Agency has removed risk-management-plan records for chemical plants and other facilities that store and use chemicals.\n\nThe state of New Jersey has removed chemical information from its Web site.\n\nChilling message\n\nIn each case, the agencies cited the need to take precautions in the face of the terrorist threat. But it appears they are acting on their own. A White House spokesman said that it has issued no request for agencies to remove materials from their sites. But they have done it with the backing of U.S. Attorney General John Ashcroft. On Oct. 12, Ashcroft issued a memorandum to the heads of all departments and agencies in which he told agencies to carefully consider \"disclosure determinations\" under the FOIA.\n\nThe memo also states: \"When you carefully consider FOIA requests and decide to withhold records in whole or in part, you can be assured that the Department of Justice will defend your decisions unless they lack a sound legal basis or present an unwarranted risk of adverse impact on the ability of other agencies to protect other important records.\"\n\nThe memo \"sends a chilling message to the entire federal bureaucracy that they better think long and hard before they fulfill an FOIA request or they will be in trouble,\" said Charles Davis, executive director of the Freedom of Information Center, based at the Missouri School of Journalism in Columbia. \"If they want to deny it, the Justice Department is right behind them. In addition, it gives very little credit to the positive values of freedom of information.\"\n\nSecurity also was a concern for the FAA when the agency removed a file of enforcement records on Sept. 14 at the request of the agency's general counsel, according to FAA spokesman Paul Turk. But Turk noted that limited access to enforcement information is still available on the Web site; it lacks some of the details that were in the database.\n\nThose details are what Cleveland Plain Dealer reporter Beth Marchak says are so important. She used FAA enforcement data to examine the 1996 ValuJet crash in the Florida Everglades.\n\n\"You can't make intelligent decisions about safety unless you can look at these records,\" Marchak said. \"In the past, I've used the data to look at how FAA handles enforcement cases. They were settling cases for pennies on the dollar and a lot of cases were just being resolved with paper. People need to know that.\"\n\nNew security restriction\n\nAnother database routinely used by reporters to tell their readers and viewers that might be at risk from old, rarely inspected dams was the Army Corps of Engineers National Inventory of Dams. Now, when you go to crunch.tec.army.mil/nid/webpages/nid.cfm and click on MAP DISPLAY, you get a screen that says:\n\nPublic access to the National Inventory of Dams is currently restricted. Please contact the NID Pagemaster if you have any questions. Thank you.\n\nThe most recent data on dams is still available online at a Stanford University data query site and from NICAR's data library.\n\nThose with concerns about access to government information don't discount the terrorist threat. But it requires balance with the public's right to know, says Gary Bass, executive director of OMB Watch, a nonprofit organization concerned with freedom of information issues.\n\n\"We live in an open society. Everything we put out carries a risk. Are we not going to announce where the next football game is?\" Bass said. \"The flip of that is we end up living in a closed system.\"\n\nOMB Watch has come under criticism itself recently for continuing to make risk-management-plan data available. Bass said such information is necessary for citizens and communities to know their risks.\n\n\"They could identify the names of these chemical companies, but if you go to the yellow pages you can do the same thing,\" he said.\n\nThe records also provide information about what effects an accident would have. \"It will say something like the plume would go 4.3 miles affecting X number of people, two child-care centers and a school,\" Bass said.\n\n\"If I were the parent of a kid in one of those child-care centers, I would want to know that. Doesn't OMB Watch have a responsibility to provide that information to the parents. If we don't do that, aren't we guilty of not doing everything we can to protect them?\"\n\nFor almost 14 years, Paul Orum, director of the Working Group on Community Right-to-Know, has used public records to try to make chemical plants safer for citizens and to make communities aware of their risks.\n\n\"We have pushed for many years for improved site security and hazard reduction at chemical plants,\" Orum said.\n\n\"There has been a great deal of complacency that has accompanied traditional secrecy in the industry. Environment groups and public interest groups have relied on the right to know to push government and industry to make changes as far as hazards.\"\n\nHaphazard actions\n\nBass also argues that much of the information was removed randomly without a clear directive. When he's asked agencies for the reasons behind their actions, he's gotten vague responses.\n\nAn e-mail response to Bass from the Department of Transportation's information services department said: \"Due to the attacks on September 11th, BTS and all other government agencies have had to re-evaluate the content available through our Web pages.\"\n\n\"We need to have a reasoned debate about this,\" Bass said. \"We shouldn't be acting precipitously to take things down.\n\nBass was particularly concerned about the phrase \"all other government agencies.\" He argues that much of these actions have been done without a clear plan.\n\n\"If there is a reasonable argument with reasonable benchmarks. I think we want to hear that and act accordingly. But until then I think in an open society public right to know has to prevail.\"\n\n\"Bin Laden doesn't need a click of a mouse to get this information,\" Bass said.\n\nPress for information\n\nIn the current atmosphere, reporters may have to be more diligent in their requests for public information.\n\nAgencies can't just deny a record without citing a specific exemption to FOIA or a state public records law, said Rebecca Daugherty, director of the Freedom of Information Service Center of the Reporters Committee for Freedom of the Press.\n\nAnd agencies can't simply invoke national security, Daugherty notes. \"If you're going to invoke national security, the records have to be classified. If it has been public before, they can't now reclassify it.\"\n\nDaugherty also said that reporters should appeal any denials they get and that organizations such as the Reporters Committee can help reporters write appeal letters. She also suggested that reporters ask for expedited review for their requests dealing with Sept. 11 and terrorism. Doing so puts those requests at the front of the line.\n\n\"They are probably going to be successful when they do because of the widespread interest,\" Daugherty said.\n\nDaugherty also emphasized the importance of covering public records issues at this time.\n\n\"I don't think the public has any idea how much information is being withheld from them,\" she said. \"Journalists need to challenge closures and write about them.\"\n\nJennifer LaFleur can be reached by e-mail at jlafleur@post-dispatch.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-links-2","slug":"2002-february-links-2","url":"https://dwillis.github.io/uplink/article/2002-february-links-2.html","year":2002,"month":"February","monthNum":2,"headline":"Links","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"CNN","summary":"CNN's investigative project \"Flight Risk\" examines aviation-related issues through reporting by Mike Fish, supported by database analysis, editorial oversight, and multimedia design elements from a specialized team.","keywords":["aviation","investigation","CNN"],"topics":["Investigative Methods","Public Safety"],"technologies":[],"full_text":"The CNN.com original project \"Flight Risk\" was reported by Mike Fish. Denise Prodigo-Herrmann conducted the database analysis. Kevin Drew was the editor. Emily Chen was the designer and worked with Ann Hoevel to develop graphics, charts and images. Meriah Doty also developed images. Robert Felker was the illustrator. Charles Rawls was the web editor. www.cnn.com/SPECIALS/2001/trade.center/flight.risk/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-putting-together-the-data-puzzle","slug":"2002-february-putting-together-the-data-puzzle","url":"https://dwillis.github.io/uplink/article/2002-february-putting-together-the-data-puzzle.html","year":2002,"month":"February","monthNum":2,"headline":"Putting together the data puzzle","kicker":"SPOTLIGHT: TERRORISM","author_name":"John Perry","author_title":"Data journalist","affiliation":"The Oklahoman","summary":"A journalist uses data analysis techniques to decode the FAA's cryptic airport security violation database, discovering it contains both security failures and successes, with different violation codes corresponding to passenger violations versus FAA inspector tests.","keywords":["database","security","FAA"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":[],"full_text":"Putting together the data puzzle\n\nBy John Perry, The Oklahoman\n\nThe Federal Aviation Administration probably deserves an award for creativity in thwarting public access to information. The agency's Enforcement Information System database, which includes information about airport security violations, is publicly available. But the FAA releases only sketchy documentation.\n\nAmong the information you can't get: an explanation of the security violation codes. Explaining what those codes mean could endanger airport security, the FAA argues.\n\nAfter the Sept. 11 terrorist attack, that left newspaper data-crunchers scratching their heads, trying to make sense enough from the enforcement data to evaluate security at their local airports. What exactly is the difference, for example, between security violation code C130, cryptically described in the documentation as \"FAIL DETECT - GENERAL\" and violation code M140, described as \"DETECT - X-RAY.\" And more importantly, which records represent security failures.\n\nData revelation\n\nThe enforcement data is split into four interconnected tables. For those of us looking at security violations, two tables are of primary interest: the main table that contains one record with general information for each case and a secondary table that usually includes several records with security violation codes for each case number.\n\nAnother table connects cases to the specific regulation violated, and another gives the action taken by the FAA in response to the violation.\n\nThe first thing I did with the files was to run a series of cross tabulations to see which codes in the violator name, violation source, operation type, certification type and employer fields matched with each other and with the security violation codes.\n\nThe first revelation, at least for me, was that the data included not just security lapses by airlines and airports, but also violations by members of the general public - the data recorded not only security lapses, but also security successes.\n\nWith only a few exceptions, when the operator type was either passenger or non-passenger, the certification type was \"NONE,\" the violator name was expunged, the violation source was \"LOCAL/ST GOVT\" and there was an entry in the sex field.\n\nSecurity codes that matched with the passenger and non-passenger operator type records tended to come from one group of codes, all beginning with an M, that seemed to include things people would get in trouble for: \"FIREARM,\" \"EXPLOSIVE,\" \"DETECT - X-RAY,\" \"WEAPON LOADED,\" \"FAILURE TO SUBMIT TO SCREENING.\"\n\nCrosstabs looking at records where the operation type was \"AIR CARRIER\" and the violation source was \"SURVEILLANCE\" or \"SPECIAL SURVEILLANCE/INSPECTION\" tended to match with security violation codes such as \"FAIL DETECT - HAND GRENADE.\" These were apparently the incidents where FAA inspectors succeeded in sneaking things past screening stations.\n\nBut just to confuse things, in almost every case where the operator type was passenger or non-passenger, there was also the name of an airline in the employer field. So were these airline employees?\n\nThe FAA wasn't explaining. But the description of the field in the documentation said it included \"violator employer if applicable, or responsible air operator.\" Since airlines are responsible for passenger screening, the employer field entries made sense.\n\nFilling in blanks\n\nTo confirm which records recorded security lapses and which were security successes, I took a clue from the violation source code \"LOCAL/ST GOVT\" and tried to match the records where the operation type was passenger or non-passenger with our database of local crime incidents.\n\nOur crime incident database goes back to 1992 and is updated monthly by e-mail. The airport is a separate police reporting district to itself, so I pulled all the incident records from that reporting district and matched it with the FAA violation data by date.\n\nIn every case, the FAA violations matched with a crime incident record that seemed to match the security violation descriptions.\n\nOur cop reporter pulled the full narrative police reports for the matching incidents, and when we compare the incident narratives with the FAA security violation codes, the data started to make more sense.\n\nFor example, a March 26, 2000, security violation in the main FAA table connected with four records in the security code table with the following codes: \"DETECT - X-RAY,\" \"FIREARM - HANDGUN,\" \"WEAPON - LOADED\" and \"WEAPON - CONFISCATED.\"\n\nThe police narrative report said that the X-ray operator detected a handgun in a purse and called airport police. The purse was owned by a Fletcher, Okla., reserve police officer who had forgotten her handgun was in her purse when she came to meet her son-in-law. After a discussion with the FBI, she was released, but the police kept her gun.\n\nThe most entertaining incident, however, was coded only as \"MISC OTHER\" in the FAA data. When asked at the ticket counter if anyone had given him anything to carry on the plane, one man replied, \"Just the bomb.\"\n\nThat quip got him a visit with the police and the FBI. He was released to catch a later flight.\n\nJohn Perry can be reached by e-mail at jperry@oklahoman.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-tipsheets","slug":"2002-february-tipsheets","url":"https://dwillis.github.io/uplink/article/2002-february-tipsheets.html","year":2002,"month":"February","monthNum":2,"headline":"Tipsheets","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE","summary":"The IRE Resource Center offers multiple tipsheets providing guidance for journalists investigating airline security and aviation safety, including information on relevant agencies, crash documentation, and computer-assisted reporting techniques.","keywords":["aviation","security","reporting"],"topics":["Public Safety","Investigative Methods","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"More information on airline security is available from the following tipsheets in the IRE Resource Center:\n\nTipsheet #1210. Includes a list of agency Web sites, such as the National Transportation Safety Board and the Federal Aviation Administration. It also explains what each site can offer reporters investigating aviation safety.\n\nTipsheet #729. Covers how to use aviation safety documents when an aircraft crashes in your hometown, including information on what documents are obtainable and how to get them. documents. Audio tape is available through Gaylor MultiMedia, Inc., 888-310-3100, for $6, ask for tape #63.\n\nTipsheet #613. Explains how to use CAR to cover the aviation community. It includes a sample story from The Seattle Times, sources for covering aviation and sample documents from the Federal Aviation Administration. Audio tape available through Gaylor MultiMedia, Inc. 615-361-3611, for $6, ask for tape #14.\n\nTo order, call the Resource Center at 573-882-3364 or go to www.ire.org/resourcecenter/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-stories","slug":"2002-february-stories","url":"https://dwillis.github.io/uplink/article/2002-february-stories.html","year":2002,"month":"February","monthNum":2,"headline":"Stories","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"The IRE Resource Center offers investigative journalism stories exposing housing code violations and slumlord practices, including reports on how landlords evade enforcement and how systemic legal loopholes enable chronic problem properties across major cities.","keywords":["housing","enforcement","landlords"],"topics":["Investigative Methods","Business & Economy","Government & Politics"],"technologies":[],"full_text":"Other stories on housing problems available from the IRE Resource Center:\n\nStory#17226 New Times Los Angeles reported that \"slum properties can be shuffled among a small cadre of co-conspirators, keeping law enforcement and housing activists at bay,\" revealing \"how a slumlord managed to defy the best efforts of housing enforcement officials and his own tenants to bring a substandard building up to code, even when he was slapped with a $455,000 judgment.\"\n\nStory#15948 The St. Paul Pioneer Press published a three-part report on \"chronic problem properties\" and the effect they have on the Twin Cities. \"At the heart of the issue,\" the newspaper reported, \"is a badly broken system, rife with legal loopholes, spotty enforcement and too few penalties for lawbreakers.\" The newspaper found that \"more than 6,400 properties in St. Paul and another 22,000 in Minneapolis have failed to comply with housing codes for two years or longer,\" inspectors are unwilling to crack down on the offenders and \"a small group of landlords account for a disproportionate number of complaints.\"\n\nTo order, call the Resource Center at 573-882-3364 or go to www.ire.org/resourcecenter/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-security-problems","slug":"2002-february-security-problems","url":"https://dwillis.github.io/uplink/article/2002-february-security-problems.html","year":2002,"month":"February","monthNum":2,"headline":"Security problems","kicker":"SPOTLIGHT: TERRORISM","author_name":"Paul D'Ambrosio","author_title":"","affiliation":"Asbury Park Press","summary":"A journalist describes using CAR techniques and FAA security databases to report on security lapses at Newark Airport following 9/11, despite severe technological constraints and communication challenges. By focusing on specific security violation codes in the FAA database, the reporter was able to quickly identify critical security failures that became relevant to the investigation.","keywords":["security","database","airport"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":[],"full_text":"As news organizations scrambled to produce special editions Sept. 11, CAR writers and editors were faced with reporting under some of the most difficult circumstances possible.\n\nWhile the unfolding news changed from minute to minute, communication overloads at the Asbury Park (N.J) Press and perhaps other news outlets around New York City put a crimp in our usual tools of the trade. Telephone connections were sporadic and, worse, our Internet service was down more than it was up. Our on-site, online servers were overwhelmed with perhaps 10 times as many page views as normal.\n\nEven the day after, it took more than three hours to download the latest FAA security data from NICAR - a 25-meg database that would normally slide through our pipeline within minutes.\n\nThe delay pushed the first story about weak security at Newark International Airport right up to our 8 p.m. deadline. I had never used the security data before, but was familiar with FAA terminology and mindset, having written several stories on the administration in the late 1990s.\n\nThere were two key tasks: use the database to elicit reaction from airline officials, and find real people to tell their tales of security lapses.\n\nZeroing in with data\n\nIn the database, I focused on the SECURITY table, one of four in the FAA data. Pressed for time, I focused exclusively on Newark airport, the departure airport for United flight 93 the flight that crashed in a Pennsylvania field.\n\nAfter quickly scanning the security code violations at Newark (coded by its common call letters, EWR) I zeroed in only on two security issues: mock weapons making their way through checkpoints, and unauthorized personnel entering air operation areas, or AOAs. The latter are supposedly secure sites to prevent those bent on destruction from tampering with the aircraft itself.\n\nBoth areas were a critical concern since at that point, no one knew how the hijackers got their box cutters or knives on board.\n\nI used security codes C111, and C130 to C139, which focused on security officers failing to detect weapons during undercover tests. A131 and A132 also gave me counts on unauthorized access to AOAs. The FAA cited United Airlines four times in 1999 for screening violations, but only issued warning letters.\n\nBlackout limitations\n\nUnited and the FAA had imposed a media blackout, so no more information about the lapses could be learned that night. The Port Authority of New York and New Jersey, which owns Newark Airport, provided some background on how security in the terminals worked.\n\nThe data had limitations. There was no way to know how many times airport security was tested, so no rate could be developed to compare one facility with another. But, recent Inspector General and General Accounting Office reports confirmed security was a major problem at airports around the county, which added depth to the story.\n\nWe also found a local resident who told of walking onto two empty planes just a month before the hijackings. She contacted me after reading our first story. She said she was appalled at the lack of security, which was so lax that an airline worker even helped her get onto one aircraft.\n\nHer experience led to a second story the next day about people entering the sterile air operations areas. The data was again helpful in that area since AOA violations were well noted.\n\nPaul D'Ambrosio can be reached by e-mail at pmd@app.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-tipsheets-2","slug":"2002-february-tipsheets-2","url":"https://dwillis.github.io/uplink/article/2002-february-tipsheets-2.html","year":2002,"month":"February","monthNum":2,"headline":"Tipsheets","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE","summary":"The IRE Resource Center provides multiple tipsheets on airline security and aviation safety, offering guidance on accessing government agencies, crash investigation documents, and using computer-assisted reporting to cover aviation issues.","keywords":["aviation","security","reporting"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["World Wide Web"],"full_text":"More information on airline security is available from the following tipsheets in the IRE Resource Center:\n\nTipsheet #1210. Includes a list of agency Web sites, such as the National Transportation Safety Board and the Federal Aviation Administration. It also explains what each site can offer reporters investigating aviation safety.\n\nTipsheet #729. Covers how to use aviation safety documents when an aircraft crashes in your hometown, including information on what documents are obtainable and how to get them. documents. Audio tape is available through Gaylor MultiMedia, Inc., 888-310-3100, for $6, ask for tape #63.\n\nTipsheet #613. Explains how to use CAR to cover the aviation community. It includes a sample story from The Seattle Times, sources for covering aviation and sample documents from the Federal Aviation Administration. Audio tape available through Gaylor MultiMedia, Inc. 615-361-3611, for $6, ask for tape #14.\n\nTo order, call the Resource Center at 573-882-3364 or go to www.ire.org/resourcecenter/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-comparing-foot-traffic","slug":"2002-february-comparing-foot-traffic","url":"https://dwillis.github.io/uplink/article/2002-february-comparing-foot-traffic.html","year":2002,"month":"February","monthNum":2,"headline":"Comparing foot traffic","kicker":"SPOTLIGHT: TERRORISM","author_name":"James E. Wilkerson","author_title":"","affiliation":"The Morning Call","summary":"A CAR reporter investigated security lapses at Lehigh Valley International Airport by analyzing FAA enforcement data, then used Department of Transportation enplanement data to identify comparable airports for context.","keywords":["airport","security","data"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["Excel","Internet"],"full_text":"Comparing foot traffic\n\nLike other CAR reporters across the country, I turned to the FAA enforcement data in the wake of the Sept. 11 terrorist attacks to see what I could find about our local airport.\n\nGoing back five years, I found that the Lehigh Valley International Airport had seen relatively little action. That smelled right, considering that it's a relatively small airport in a quiet corner of Pennsylvania.\n\nHowever, two incidents stood out. In two separate tests conducted within three months of each other in 1999, FAA inspectors sneaked contraband through a passenger-security checkpoint. The entries for the smuggled objects were obscure, but others on the NICAR list had already decoded them as a fake bomb and a fake hand grenade.\n\nThose incidents alone made for an interesting story, but before I started writing I had to put them into context by looking at similar lapses at other airports.\n\nThose incidents alone made for an interesting story, but before I started writing I had to put them into context by looking at similar lapses at other airports. Because the Lehigh Valley International Airport is pretty small, I didn't think it fair to compare it to larger nearby airports in Philadelphia, New York and New Jersey. But I didn't have a clue about how to choose similar airports.\n\nWe all know how helpful the FAA has been in working with reporters lately, so needless to say, my call to flaks there for guidance went unanswered.\n\nEnplanement data\n\nI turned again to the NICAR mailing list, where some discussion of airport enplanement data caught my interest. I sent out a message asking for help and I soon had 1999 Department of Transportation enplanement data for all airports in the United States showing the number of passengers boarding planes and passing through.\n\nI used Excel to sort the airports by the enplanement numbers. I eyeballed where our airport fell as compared to others and decided that a spread of 20,000 enplanements below and above our airport would be fair for a rough comparison. Others used different approaches.\n\nJeff Thomas of The Gazette (Colorado Springs, Colo.) chose airports with enplanements from 50 to 150 percent of his local airport, giving him a range of 37 airports, plenty for a fair comparison. Ken Ward Jr. of the Charleston (W.Va.) Gazette, chose a 25 percent range above and below his airport. He ran the results by an airport administrator, who confirmed that he had a good set of airports comparable to his. David Washburn, of The San Diego Union-Tribune, grabbed the largest 30 airports, of which his ranked 28th.\n\nComparing airports\n\nMy method left me with 20 airports. I used the Internet to look at information on each and eliminated a handful that obviously served a very different purpose than our airport, such as one in Nevada that appeared to be used exclusively by tourists into the Grand Canyon area.\n\nFinally, I did the shoe-leather reporting necessary to fill in the gaps. I went out to the local airport to talk to personnel and get a better feel for exactly where the incidents occurred. I also called on security experts to give me some perspective about how the FAA conducts the tests.\n\nOn Sept. 21, we ran a front-page story that came as a shock to many readers, not to mention other reporters and editors, who were stunned that FAA inspectors were able to smuggle contraband into airports on such a regular basis.\n\nJames Wilkerson can be reached by e-mail at james.wilkerson@mcall.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-stories-2","slug":"2002-february-stories-2","url":"https://dwillis.github.io/uplink/article/2002-february-stories-2.html","year":2002,"month":"February","monthNum":2,"headline":"Stories","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE Resource Center highlights investigative journalism stories exposing systemic failures in housing enforcement, including how slumlords exploit legal loopholes and evade penalties while thousands of properties remain substandard.","keywords":["slumlords","enforcement","housing"],"topics":["Business & Economy","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"Other stories on housing problems available from the IRE Resource Center:\n\nStory#17226 New Times Los Angeles reported that \"slum properties can be shuffled among a small cadre of co-conspirators, keeping law enforcement and housing activists at bay,\" revealing \"how a slumlord managed to defy the best efforts of housing enforcement officials and his own tenants to bring a substandard building up to code, even when he was slapped with a $455,000 judgment.\"\n\nStory#15948 The St. Paul Pioneer Press published a three-part report on \"chronic problem properties\" and the effect they have on the Twin Cities. \"At the heart of the issue,\" the newspaper reported, \"is a badly broken system, rife with legal loopholes, spotty enforcement and too few penalties for lawbreakers.\" The newspaper found that \"more than 6,400 properties in St. Paul and another 22,000 in Minneapolis have failed to comply with housing codes for two years or longer,\" inspectors are unwilling to crack down on the offenders and \"a small group of landlords account for a disproportionate number of complaints.\"\n\nTo order, call the Resource Center at 573-882-3364 or go to www.ire.org/resourcecenter/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-tracking-a-suspect","slug":"2002-february-tracking-a-suspect","url":"https://dwillis.github.io/uplink/article/2002-february-tracking-a-suspect.html","year":2002,"month":"February","monthNum":2,"headline":"Tracking a suspect","kicker":"SPOTLIGHT: TERRORISM","author_name":"Phil Williams","author_title":"","affiliation":"WTVF-TV","summary":"A journalist uses computer-assisted reporting techniques, including databases and online searches, to investigate a potential terrorist suspect with connections to Nashville and discover security lapses at the airport.","keywords":["investigation","database","security"],"topics":["Investigative Methods","Public Safety","Technology"],"technologies":["Windows","Google","AutoTrack","IRE-L","anywho.com","landings.com"],"full_text":"Tracking a suspect\n\nIt was a phone call that brought an amusing reminder of the value of computer-assisted reporting.\n\nFor days, my photographer and I had been chasing down leads and reporting on a possible terrorist suspect who had operated in and out of the Nashville area. So, when the cell phone rang, we thought it might be someone with new information.\n\nInstead, it was an investigator with a top law enforcement agency who had a question for us: Where were we getting our information?\n\nIn fact, in the days following the Sept. 11th terrorist attack, I would be repeatedly reminded of how many doors - or, should I say, Windows-have been opened by these CAR techniques.\n\nRight after the attack, our first quick-turn investigation focused on the company in charge of security at Nashville International Airport. Argenbright Security also held contracts at the airports in Boston, Washington and Newark used by the terrorists.\n\nA quick Google search pulled up years of news stories documenting the company's long history of security lapses. Combined with information that Tennessee regulators had caught the company using a security guard with a criminal history, we had our first story on the company's lackluster reputation.\n\nQuestions about Argenbright's background aside, that story still didn't answer the question about how well or poorly the company had actually performed at our airport.\n\nSlicing the data\n\nTo get at that question, I downloaded the Federal Aviation Administration's enforcement database from NICAR and immediately sliced out only the data dealing with security violations at Nashville International Airport.\n\nIn analyzing the results and comparing notes with other NICARians, it became clear that there was lots of information that could easily be misinterpreted. That was especially true, given the FAA's refusal to help interpret the data.\n\nFor example, I pulled out the raw number of \"security violations\" at the airport. But many were cases where individuals were caught with weapons and other contraband. In fact, those violations might be an indicator of good security.\n\nSo, I decided to focus on a narrow snapshot - violations where security personnel had failed to detect weapons. The result was a story that the security forces had repeatedly flunked tests involving handguns, hand grenades and dynamite bombs.\n\nFinding people\n\nOver the coming days, we would use AutoTrack to background individuals whose names emerged as possible suspects.\n\nBut the most interesting development came from a post to the IRE-L listserv by an international colleague. The post was a list of 370 individuals with possible terrorist ties that Finnish banking authorities had inadvertently released.\n\nOne of those names, Khaled Alzeedi, had a Tennessee address and phone number, as well as an e-mail address for a company at zidiaviation.com.\n\nA quick search of anywho.com linked the address to an aircraft sales company in nearby Clarksville, Tenn. The phone number belonged to a Hilton Suites in downtown Nashville.\n\nFrom computer-assisted reporting to plain gumshoe work, we discovered that the Saudi pilot had come to Tennessee in August to buy two Tampico aircraft that would be used to train pilots. Hotel workers recalled that he had claimed to be an Iranian diplomat, and restaurant employees insisted he had been back since the Sept. 11 attacks.\n\nKeep searching\n\nWith each new development, I returned to the computer to conduct new searches to see where those threads might lead.\n\nFor example, a search of aircraft registrations at landings.com confirmed that two Tampicos had been registered to Zidi Aviation in mid-August. Then, with those tail numbers, I went to the Web site for the aircraft sales company and discovered the sale information for the two aircraft - and photos of the actual planes!\n\nAnother Google search for Zidi Aviation led to the discovery that the company had been an authorized distributor for flight simulation software manufactured by Jeppesen, a Boeing subsidiary. A Jeppesen attorney refused to comment on the relationship with Zidi, saying she couldn't discuss \"anything related to terrorism.\"\n\nI also came across a reference for another Zidi Aviation e-mail address at zidinet.com. But the Web page for zidinet.com was nothing more than white type on a black screen, with no links. Our Webmaster suggested that I try a bit of simple hacking and add a \"/images\" to the URL. Then - Voila! - that led to a prototype Web page that showed pilots training to fly jumbo jetliners.\n\nLocal police followed our stories, but they reported back to me that they had checked with the local FBI office, and agents insisted they were not looking for Alzeedi. That had other media virtually ignoring our reporting.\n\nSo, it came as a relief when The Associated Press, following up on our stories, got confirmation from an assistant FBI director that the agency was indeed searching for Alzeedi.\n\nPhil Williams can be reached by e-mail at pwilliams@newschannel5.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-tipsheet","slug":"2002-february-tipsheet","url":"https://dwillis.github.io/uplink/article/2002-february-tipsheet.html","year":2002,"month":"February","monthNum":2,"headline":"Tipsheet","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE","summary":"This article provides journalism resources including a tipsheet with contacts for investigating terrorists and fringe groups, plus a story examining the threat of terrorist use of nuclear weapons and radioactive materials.","keywords":["terrorism","investigation","nuclear"],"topics":["Investigative Methods","Public Safety"],"technologies":["World Wide Web"],"full_text":"Tipsheet\n\nMore information on terrorists is available from the following tipsheet in the IRE Resource Center:\n\nTipsheet #1168. From a 2000 New York conference panel, this tipsheet consists of a list of contact names and organizations to help reporters investigate terrorists and fringe groups.\n\nStory\n\nAnother story on terrorists is available from the IRE Resource Center:\n\nStory #16709. \"Will Terrorists Go Nuclear?\" American Spectator. After the 1993 World Trade Center bombing, the National Security Council began investigating terrorist use of nuclear weapons. This story looks at the threat radioactive waste materials pose to the public.\n\nTo order, call the Resource Center at 573-882-3364 or go to www.ire.org/resourcecenter","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-flipping-a-table","slug":"2002-february-flipping-a-table","url":"https://dwillis.github.io/uplink/article/2002-february-flipping-a-table.html","year":2002,"month":"February","monthNum":2,"headline":"Flipping a table","kicker":"From the Lists","author_name":"","author_title":"","affiliation":"","summary":"A quick tip explaining how to transpose rows and columns in an Excel spreadsheet using Paste Special.","keywords":["Excel","spreadsheets","tips"],"topics":["Data & Databases","Technology"],"technologies":["Excel"],"full_text":"Q: How do I flip an Excel table so that the information in rows goes in columns and columns rows?\n\nA: Select the cells you want to flip and hit copy. Open a new workbook or sheet. Then go to PASTE SPECIAL and down near the bottom click on transpose. Then hit OK.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-ranking-with-excel","slug":"2002-february-ranking-with-excel","url":"https://dwillis.github.io/uplink/article/2002-february-ranking-with-excel.html","year":2002,"month":"February","monthNum":2,"headline":"Ranking with Excel","kicker":"From the Lists","author_name":"","author_title":"","affiliation":"","summary":"An Excel user asks how to highlight top-ranked election candidates using conditional formatting, and receives instructions for using RANK function results with bold and colored formatting for positions 9 or below.","keywords":["Excel","ranking","formatting"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"Ranking with Excel\n\nQ: I'm working with election results. We had 27 people contending for nine council positions; choose nine. I'm using the RANK function in Excel to tell me how each candidate ranked within each ward/precinct. Is there a way to write an expression that would boldface any rank/number 9 or below?\n\nA: Simply highlight the entire column and select FORMAT > CONDITIONAL FORMATTING Select CELL VALUE IS and LESS THAN OR EQUAL TO and type in the number 9 now select FORMAT and BOLD and RED and voila everything that is less than or equal to 9 should be red and bold.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-updating-with-access","slug":"2002-february-updating-with-access","url":"https://dwillis.github.io/uplink/article/2002-february-updating-with-access.html","year":2002,"month":"February","monthNum":2,"headline":"Updating with Access","kicker":"From the Lists","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"A NICAR-L discussion post explains how to use the Trim function in an Access update query to remove leading and trailing spaces from database fields.","keywords":["Access","Trim","spaces"],"topics":["Data & Databases","Technology"],"technologies":["Access"],"full_text":"Updating with Access\n\nQ: Is there a way to trim leading and trailing spaces from an Access field by doing an update query or some other function? I tried updating NewField with the expression: Trim([OldField]) and all it does is remove the equal sign and insert the text \"trim([OldField])\" in the field I'm updating.\n\nA: I just used this one and it worked fine, trimming excess spaces at the beginning or end but not in the middle:\n\nUPDATE [Text] SET [Text].newtext=Trim([oldtext]);\n\nText is the table. Oldtext and Newtext are the two fields.\n\nTo subscribe the NICAR-L discussion list, follow the instructions at www.ire.org/membership/subscribe/nicar-l.html.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-detailing-identities","slug":"2002-february-detailing-identities","url":"https://dwillis.github.io/uplink/article/2002-february-detailing-identities.html","year":2002,"month":"February","monthNum":2,"headline":"Detailing identities","kicker":"SPOTLIGHT: TERRORISM","author_name":"Richard J. Dalton Jr.","author_title":"Computer-assisted reporting specialist","affiliation":"Newsday","summary":"Newsday reporters created a comprehensive database of September 11th victims by collecting names from multiple sources, then used computer-assisted reporting and data analysis tools to reveal the geographic and demographic reach of the tragedy across the United States.","keywords":["victims","database","analysis"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Nexis","AutoTrack","Access","ArcView"],"full_text":"Just hours after two planes crashed into the World Trade Center, while all of us were still absorbing the immensity of the tragedy, Newsday reporters began writing profiles of the victims for the next day's paper. It has become the newspaper's forte, after winning a Pulitzer Prize for its coverage of the 1996 crash of TWA Flight 800.\n\nOver the next several weeks, reporters gathered names of victims from many different sources: Web sites of companies who lost employees, press releases, relatives, airlines, the medical examiner's office, wire services, death notices, the police and New York State Supreme Court, which, at the request of relatives, issues death certificates for victims.\n\nReporters then turned to Nexis and AutoTrack to track down the victims' hometowns, ages and phone numbers.\n\nA month after the disaster, we had collected the full names and hometowns of nearly 2,650 victims. The database, now even larger, is available online at www.newsday.com.\n\nProfiling victims\n\nThe database proved useful for editors to keep track of which profiles had been written. But it also enabled three of Newsday's computer-assisted reporting specialists - Robert Fresco, Jo Craven McGinty and myself - to write a story showing just how far-reaching the acts of terrorism were.\n\nAnalyzing the database in Microsoft Access, Newsday's enterprise team wrote a story revealing that victims came from at least 41 states and the District of Columbia. Using ArcView, we created a map of the United States, highlighting how many victims came from each state. Another map showed the metropolitan area, with various sized dots indicating how many victims perished from each community.\n\nThe maps were powerful images, with all but nine states highlighted on the U.S. map and with dots small and large all over the place on the metropolitan area map. The map clearly showed a cluster of victims within easy commuting distance of Manhattan. Most of Long Island's victims, for instance, came from Nassau and Western Suffolk counties, each within an hour or less commute by train. (That's considered an easy commute in New York.)\n\nWe showed that the terrorists took the lives of people from more than 770 cities and towns - from Portland, Maine, to Honolulu, Hawaii.\n\nWe also produced a chart showing the number of victims in each age group, by decade. More than a third of the victims were in their 30s, and nearly half hadn't yet celebrated their 40th birthday.\n\nFiguring percentages\n\nWe then determined the gender, based on the first name of the victim. The Census Bureau provides a database of male first names and female first names at www.census.gov/ftp/pub/genealogy/names/, including how frequently each name occurs for men and women. It's a little quirky. For instance, both the male and female databases list \"James.\" But, only one-hundredth of 1 percent of women were listed as having the name \"James,\" compared with 3.3 percent of men.\n\nOf course, some names are both male and female, such as \"Jamie\" - 0.07 percent of men and 0.15 percent of women. When a name was listed in both the male and female databases, we used the frequencies to determine whether it was male, female or both. So, as expected, \"James\" turned out to be a male name while \"Jamie\" could be either. When calculating the percentage of male and female victims, we used only the names that were either male or female, excluding those that could be either.\n\nWe then supplemented the Census database with another database of names we found at www.incompetech.com/named/. The database is searchable online, and the Webmaster e-mailed us a link to download it.\n\nUsing these lookup tables, Newsday estimated that three-quarters of the victims were men. (OK, that's a lot of work for something seemed obvious given the profiles we had been doing. But it gave us an approximate percentage.)\n\nWe, of course, revealed the shortcomings of the analysis. Our database was skewed toward mostly male victims of the New York City police and fire departments, which released the names of virtually all their victims. Many of the companies that lost employees have not released lists.\n\nRichard J. Dalton Jr. can be reached by e-mail at rdalton@newsday.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-storytipsheet-available-from-the-ire-resource-center","slug":"2002-february-storytipsheet-available-from-the-ire-resource-center","url":"https://dwillis.github.io/uplink/article/2002-february-storytipsheet-available-from-the-ire-resource-center.html","year":2002,"month":"February","monthNum":2,"headline":"Story/Tipsheet Available from the IRE Resource Center:","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"This journalism resource guide features two stories: Michael Paterniti's account of the Swissair Flight 111 crash exploring the emotional impact on families and rescue workers, and Bruce Shapiro's personal narrative about surviving a violent stabbing attack.","keywords":["crash","crime","survival"],"topics":["Crime & Justice","Journalism & Newsroom"],"technologies":[],"full_text":"Story #16854. Michael Paterniti of Esquire tells the story of the crash of Swissair Flight 111. He tries to capture the feeling of the crash site and the emotions of the family members and rescue workers.\n\nTipsheet #538. On Aug. 7, 1994, Bruce Shapiro was among seven people stabbed and seriously wounded in a coffee bar a few blocks from his house. In \"One Violent Crime,\" Shapiro shares the experience of being a victim.\n\nTo order, call the Resource Center at 573-882-3364 or go to www.ire.org/resourcecenter/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-war-windfall","slug":"2002-february-war-windfall","url":"https://dwillis.github.io/uplink/article/2002-february-war-windfall.html","year":2002,"month":"February","monthNum":2,"headline":"War windfall","kicker":"SPOTLIGHT: TERRORISM","author_name":"David Wethe & Michael Whiteley","author_title":"","affiliation":"Dallas Business Journal","summary":"After 9/11, a Dallas-Fort Worth newsroom used NICAR's federal contractor database to identify local defense companies that might benefit from increased military spending, finding over 6,500 contracts worth $7.7 billion in the region.","keywords":["contractors","database","defense"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["FoxPro"],"full_text":"Within hours of the Sept. 11 terrorist attacks, many started to wonder what - if anything - the nation would do to respond.\n\nTexans wondered if their loved ones would get the call to go to war. And the tech companies, machine shops and aviation subcontractors of the Dallas-Fort Worth area - businesses that had accounted for a dime of every dollar spent on Desert Storm - wondered if they could balance the losses to the airline industry with some profit from a new war.\n\nWe decided to use NICAR's database of federal contractors to identify and interview some of the companies on the early call lists for the Department of Defense or the Defense Logistics Agency.\n\nWe slated the story for a quick turnaround - to appear in next week's issue while the idea was still fresh on our readers' minds.\n\nOn Sept. 14, we ordered the Texas share of the database, which covered companies and their federal contract obligations for 2000. It arrived the following Monday morning, giving us time to write a story for the Sept. 21 paper.\n\nDefense contracts\n\nUsing Visual FoxPro 6.0, we ran a series of relatively simple queries that split out the defense contracts, finding contractors based in the Dallas-Fort Worth area, summing their contract obligations and sorting in descending order.\n\nWe queried NICAR's data into successive tables, culminating with \"metrodef.dbf.\" The values \"DD\" and \"DT\" in the field \"prtagncy\" identified contracts issued by defense agencies, including the DOD, the DLA, the Air Force and the Army.\n\nWe queried the table \"Fedprc.dbf\" with: SELECT FROM fedprc; WHERE Fedprc.prtagncy = \"DD\"; OR (Fedprc.prtagncy = \"DT\") ORDER BY Fedprc.contractor; INTO TABLE :\\defense\\defquery.dbf\n\nWe renamed defquery.db to \"Texdef.dbf\" and queried it to isolate contractors' home cities using the query: SELECT DISTINCT Texdef.contcity; FROM texdef; ORDER BY Texdef.contcity; INTO TABLE city.dbf\n\nWe edited the results to include only cities in Dallas County, Tarrant County and the surrounding counties in the Dallas-Fort Worth area.\n\nWe then ran the new table, \"city.dbf\" against our \"Texdef\" table using: SELECT FROM texdef INNER JOIN city ; ON Texdef.contcity = City.contcity; INTO TABLE metrodef.dbf\n\nBig dollar amounts\n\nThe query yielded a table of Dallas-Fort Worth area defense contractors - or at least contractors who did defense work in the Dallas-Fort Worth area last year - with a total of 6,507 contracts worth $7.7 billion. By comparison, our crunch of Texdef produced 19,614 contracts worth $12.8 billion for 2000. That meant the Dallas-Fort Worth area companies handled contracts accounting for 60 cents of every dollar spent in Texas.\n\nThe next step was to identify the major players, such as Lockheed Martin Corp., and to sort out the smaller shops that might depend on a DOD call for survival and might add a human element to our story.\n\nWe isolated the names of the contractors and the dollar amount of each of their contracts using: SELECT Metrodef.contnmif1, Metrodef.dollaramt; FROM metrodef; ORDER BY Metrodef.dollaramt; INTO TABLE companytot.dbf\n\nAnd we tallied their combined contract values, in descending order, with: SELECT companytot.contnmcif1, SUM (dollaramt) FROM companytot; GROUP BY companytot.contnmcif1; ORDER BY 2 desc; INTO TABLE Metrosum.dbf\n\nWith the help of staff writers Hala Habal and Margaret Allen, we divided the companies and called a healthy mix of those who'd done a brisk business with DOD and DLA in 2000 and those who'd handled smaller contracts.\n\nEarly on, we found some irony. Luminator Holding L.L.C. of Plano was expecting heavy losses from a division that makes instrument lights for jet airliners. But those losses were somewhat offset by an emergency order for search lights phoned in from a military base overseas within days of the attack.\n\nIn Garland, we found beefed-up security at a company identified as the sole manufacturer of U.S.-certified bombs. In Fort Worth, a company that makes bomb fuses and parts for the Bradley fighting vehicles had received the call to speed up orders.\n\nAlso in Garland, a startup quickly received word from the Pentagon to be ready for delivery of its new Thermal Angel, a device that can be used for warming IV fluids in the battlefield.\n\nQuick, easy story\n\nThis story was done in a hurry. It was a quick and easy CAR story that can be done in just about any newsroom that has a database manager.\n\nAlthough no one really knew what kind of war we were talking about, we felt confident many companies in our database would get the call. And one of them offered some news - the possibility that the nation might not be ready for another war.\n\n\"I don't think their inventory was that great,\" said Kyle Blake, vice president of AGH in Fort Worth. \"After Desert Storm, the Department of Defense pretty much turned the spigot off. It got behind and never caught up again.\"\n\nMichael Whiteley can be reached by e-mail at mwhiteley@bizjournals.com\n\nDavid Wethe can be reached by e-mail at dwethe@bizjournals.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-data","slug":"2002-february-data","url":"https://dwillis.github.io/uplink/article/2002-february-data.html","year":2002,"month":"February","monthNum":2,"headline":"Data","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR data library provides comprehensive FAA and NASA databases for investigative journalists to research aviation safety, enforcement actions, maintenance issues, accidents, aircraft ownership, pilot credentials, and anonymous safety reports.","keywords":["aviation","databases","FAA"],"topics":["Data & Databases","Public Safety","Investigative Methods"],"technologies":[],"full_text":"The IRE and NICAR data library offers these databases that can pinpoint information:\n\nFAA Enforcements A database of FAA enforcement actions against airlines, pilots, mechanics, and designees.\n\nFAA Service Difficulty Reports: A database of maintenance incidents collected by the FAA for the purpose of tracking repair problems with commercial, private, and military aircraft, and aircraft componentry.\n\nFAA Accidents and Incidents: A database of mainly U.S. flights where there was an accident or an incident, including crashes, collisions, deaths, injuries, major mechanical problems or costly damages.\n\nFAA Aircraft Registry: A listing of all aircraft and aircraft owners registered in the United States.\n\nFAA Airmen Directory: The FAA Airmen Directory is a listing of pilots and other airmen, including the type of certificate(s) they hold and their ratings.\n\nNASA Air Safety Reporting System: A database of anonymous reports of airplane safety submitted by pilots, flight attendants, air traffic controllers and passengers.\n\nFor more information about the database library's collection, including purchase prices, go to www.ire.org/datalibrary/databases/. To order data, contact NICAR at 573-884-7711 or download an order form at www.ire.org/datalibrary/orderform/orderform.pdf","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-meeting-the-data-challenge","slug":"2002-february-meeting-the-data-challenge","url":"https://dwillis.github.io/uplink/article/2002-february-meeting-the-data-challenge.html","year":2002,"month":"February","monthNum":2,"headline":"Meeting the data challenge","kicker":"SPOTLIGHT: TERRORISM","author_name":"Justin Mayo and David Heath","author_title":"","affiliation":"The Seattle Times","summary":"The Seattle Times investigated airport security at Seattle-Tacoma International Airport following the 9/11 attacks by analyzing FAA enforcement data, conducting interviews, and requesting incident reports to provide context on security violations and system vulnerabilities.","keywords":["security","airport","violations"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"In just a couple of hours, 19 hijackers shattered public confidence in airport security. Even officials at the Federal Aviation Administration, who make the rules and monitor airport security, were so unsure of the system's ability to prevent further hijackings that they kept airlines grounded for three days.\n\nSo it was natural for newspapers to ask how safe airports are.\n\nThe Seattle Times began looking for information on Seattle-Tacoma International Airport security immediately. The data available through the FAA's Enforcement Information System had serious limitations. What's more, the agency was of little help and in some cases even hostile in our efforts to understand the data.\n\nTo supplement this data, we also requested all security incident reports at the airport for two years. We also interviewed employees and managers at airport screening companies, experts at the General Accounting Office and Transportation Department's Inspector General's office and former FAA security officials.\n\nFinding context\n\nThe database lists each individual violation of FAA security rules, including the name of the offender, the name of the airport, the date, a description of the violation and what actions were taken for the past 20 years. FAA hires inspectors to test screeners and those violations were listed as \"failure to detect\" various types of weapons.\n\nWe first used the database for a Sept. 12 story, culling general statistics on the number of security violations and proposed penalties for airlines from the database.\n\nBut it was apparent that there was much more local detail, especially in the descriptions of security violations committed by airports and airlines. The problem was how to interpret it and put it in context without the benefit of knowledgeable sources willing to talk.\n\nFinding a focus\n\nThe mechanics of analyzing the FAA enforcement data are straightforward.\n\nTo find security violations, you link the Main table, which contains basic data on the violator and when and where the violation occurred, to the Security table, which has distinct violations for each incident. Then, link to the Actions table to find the FAA's response, including civil penalties, warning letters, certificate revocation, etc.\n\nEarly on you have to decide how you're going to define a \"security violation.\" One incident from the Main table could contain several security violations or actions. We decided to focus on incidents, not violations. That meant that if a loaded gun were found in a checked bag, which is against FAA rules, it was counted only once even though the agency may have issued several citations for failing to declare the weapon, illegally storing ammunition and improperly packing a gun.\n\nThe main challenge, however, was deciding which security violations to focus on. There are about 250 distinct security codes that airports, airlines, passengers or mechanics could be cited for violating. Most were cryptic, such as failure to detect \"encap weapon.\" We were unable to decipher that description before our deadline.\n\nTo narrow our scope we looked at weapons violations.\n\nThe enforcement data can be divided into two broad categories of weapons violations: those where the security system failed and those where it worked. In the Security table the \"failure to detect\" codes show where airline screeners let FAA inspectors slip test weapons through undetected. These failures are detailed by date, airline, type of test weapon and in many cases even the specific concourse where the violation occurred. We used these failures to document problems at the Seattle-Tacoma airport and to show how many test objects - pipe bombs, hand grenades, pistols the screening system let through.\n\nFinding answers\n\nWe decided against comparing Seattle's results to other airports because at that time we had no way tell how frequently the FAA tested each airport. We were concerned that FAA inspectors in some jurisdictions might be more diligent than in others, making comparisons impossible.\n\nWe knew from the book Flying Blind, Flying Safe by former DOT Inspector General Mary Schiavo that Sea-Tac screeners had detected explosive devices only 12 percent of the time in tests - one of the worst performances in the nation.\n\nThe data show the security system working when real weapons are discovered, usually because a passenger fails to declare a gun in a checked bag or has one in their carry-on. Guns were frequently discovered in checked bags at Sea-Tac, even though few checked bags are put through X-ray machines or bomb-detection devices. We used these violations to calculate the number of firearms found at major airports and computed a rate based on domestic enplaned passengers. Sea-Tac ranked second in the country in 1999.\n\nSupplementing the enforcement data, local police records and other sources showed that the FAA seriously underreports incidents involving weapons. Port of Seattle records showed gun violations, mostly minor, occurring almost every day at the airport - everything from a passenger failing to disclose that a weapon has been stored in a checked suitcase to trying to walk through the metal detector with a concealed handgun strapped to an ankle holster.\n\nFinding sources\n\nBesides reporting the data, we interviewed a 38-year-old screener, Teuila Tuitele, who said that the training she received inadequately prepared her for the job. She had some classroom training during her first week and passed a multiple-choice test asking how she should handle irate passengers or what to do if she detected a weapon on the X-ray machine. She was shown and tested on how to identify suspicious bomb-like shapes. She said screeners needed only an 85 percent detection rate to get assigned a job.\n\nIt is not uncommon for a screener to sit at the X-ray machine for two or even three hours without a break. \"Sitting there more than 30 minutes you fall asleep,\" Tuitele said. \"It's like watching the TV.\"\n\nJustin Mayo can be reached by e-mail at jmayo@seattletimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-the-stories-mentioned-in-this-edition-of-uplink","slug":"2002-february-the-stories-mentioned-in-this-edition-of-uplink","url":"https://dwillis.github.io/uplink/article/2002-february-the-stories-mentioned-in-this-edition-of-uplink.html","year":2002,"month":"February","monthNum":2,"headline":"The stories mentioned in this edition of Uplink","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE","summary":"This brief announcement informs readers that stories from the Uplink newsletter will be available through the IRE Resource Center, with instructions on how to search and order them.","keywords":["IRE","stories","resource"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"The stories mentioned in this edition of Uplink will soon be available from the IRE Resource Center. Search the story database at www.ire.org/resourcecenter for order numbers. Call 573-882-3364 to order these and other investigative stories.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-data-2","slug":"2002-february-data-2","url":"https://dwillis.github.io/uplink/article/2002-february-data-2.html","year":2002,"month":"February","monthNum":2,"headline":"Data","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR data library provides comprehensive FAA and NASA databases for investigating aviation safety, enforcement actions, maintenance issues, and pilot credentials.","keywords":["aviation","databases","FAA"],"topics":["Public Safety","Data & Databases"],"technologies":[],"full_text":"The IRE and NICAR data library offers these databases that can pinpoint information:\n\nFAA Enforcements A database of FAA enforcement actions against airlines, pilots, mechanics, and designees.\n\nFAA Service Difficulty Reports: A database of maintenance incidents collected by the FAA for the purpose of tracking repair problems with commercial, private, and military aircraft, and aircraft componentry.\n\nFAA Accidents and Incidents: A database of mainly U.S. flights where there was an accident or an incident, including crashes, collisions, deaths, injuries, major mechanical problems or costly damages.\n\nFAA Aircraft Registry: A listing of all aircraft and aircraft owners registered in the United States.\n\nFAA Airmen Directory: The FAA Airmen Directory is a listing of pilots and other airmen, including the type of certificate(s) they hold and their ratings.\n\nNASA Air Safety Reporting System: A database of anonymous reports of airplane safety submitted by pilots, flight attendants, air traffic controllers and passengers.\n\nFor more information about the database library's collection, including purchase prices, go to www.ire.org/datalibrary/databases/. To order data, contact NICAR at 573-884-7711 or download an order form at www.ire.org/datalibrary/orderform/orderform.pdf","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-doing-cross-tabs-can-create-quick-comparisons","slug":"2002-february-doing-cross-tabs-can-create-quick-comparisons","url":"https://dwillis.github.io/uplink/article/2002-february-doing-cross-tabs-can-create-quick-comparisons.html","year":2002,"month":"February","monthNum":2,"headline":"Doing cross tabs can create quick comparisons","kicker":"Tech tips","author_name":"Jeff Porter","author_title":"Managing Editor","affiliation":"IRE/NICAR","summary":"Cross tabs and pivot tables are powerful data analysis tools that allow journalists to quickly reorganize and compare large datasets by rearranging fields into rows and columns. The article demonstrates how to use SQL queries in Visual FoxPro to create cross-tab comparisons of SBA disaster loan data by state and loan status.","keywords":["spreadsheet","database","comparison"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Excel","FoxPro","SQL"],"full_text":"Doing cross tabs can create quick comparisons\n\nBy Jeff Porter, IRE/NICAR\n\nCall it an art form - rearranging data in a useful way, comparing fields across columns instead of down long rows. The spreadsheet program Excel calls it a pivot table. Database managers often call it a cross tab. For Visual FoxPro users, it's a fast, quick step to compare hundreds of thousands of records instantly.\n\nTake, for example, Small Business Administration disaster loan data. The loans themselves are the primary form of federal assistance for nonfarm, private-sector disaster losses. The loans help homeowners, renters, businesses of all sizes and nonprofit organizations finance their rebuilding. The loans, which are repaid to the Treasury, are provided at low-interest rates and have tailored repayment schedules befitting each borrower's financial capability.\n\nTo create a cross-tab comparison, the rules are simple.\n\nYou want three fields. Ideally, the first has the most potential variables. Using the SBA disaster loan data, we'll compare states and a few other geographic areas, such as territories.\n\nIn the second field, you want to limit the number of variables, technically, to 255. In practical terms, you want just a few variables. In the SBA disaster loan database, there's a field called CHGOFF, noting whether a business disaster loan is deemed uncollectible. It's either a YES or NO - just two possible variables and perfect for a cross-tab.\n\nThe third field is the data field, a total count or sum, for example, categorized by the two other fields. For this example, the count will be categorized as the status of each loan in each state.\n\nFor Visual FoxPro 5.0, here's the SQL to first list the borrowers' states, whether the loan was charged off, and the total number for each category by state. The query limits the type of loan to just businesses - coded as a capital B in the data not individuals, because the SBA won't tell the status of an individual's loan. In this example, the number of loans is limited to those since the beginning of 1990 to businesses: select borstate, chgoff, count (*) from disaster; where apprdate > {12/31/89) and type = 'B', group by 1, 2 The results would look like this:\n\nTHE Query\n\nBorstate Chgoff Ltd\n\nNO 1\nAE NO 1\nAK NO 68\nAL NO 97\nAL YES 2\nAR NO 216\nAZ NO 10\nCA NO 150\nCO NO 25\nCT NO 20\nDC NO 37\nDE NO 4\nFL NO 829\nFL YES 4\nGA NO 161\n\nHelpful, but only slightly. Now, one could export results to Excel and create a pivot table to show the CHGOFF variables across the top of the spreadsheet and the states down the side, but Visual FoxPro can do it quicker. Just use one line of SQL in the software's command window: do _genxtab)\n\nWith this short command, you've created a new table called Xtab, created in the software's default directory. It should, though, be already open, so simply type\n\nbrowse\n\nAnd you should see this:\n\nHE Xtab\nBorstate No Yes\n 28 4\nAA 2\nAE 8\nAK 695 29\nAL 1416 117\nAP 4\nAR 642 25\nAS 179 41\nAZ 340 38\nCA 36787 4752\n\nBetter - at least one can read across and see in a glance how a particular state is doing. Note here, for example, that in California, 36,787 loans aren't in trouble, while 4,752 are categorized as uncollectible.\n\nWhat's that as percentage? Again, one could export over to Excel; instead, though, one could run one last, quick query to show the result. For this example, the number of total loans is limited to 101 or more by state or territory, to leave out areas with a tiny number of loans that might skew a comparison: select borstate, no, yes, yes/(no+yes) pct from xtab; where no+yes > 100; order by 4 desc\n\nThe query creates a new field - the number of loans charged off, divided by the total number - and names it PCT, short for percentage. Also added: The last line tells FoxPro to sort the records by highest percentage to lowest, finding the states with the highest percentage of loan default. The answer:\n\nQuery\nBorstate No Yes Pct\nAS 179 41 0.1864\nRI 881 18 0.1538\nSC 1770 303 0.1462\nGA 2007 289 0.1259\nPR 5349 732 0.1204\nFL 10345 1394 0.1187\nCA 36787 4752 0.1144\nHI 1666 207 0.1105\nMO 1922 221 0.1031\nIL 2074 237 0.1026\nNH 132 15 0.1020\nAZ 340 38 0.1005\nLA 2367 251 0.0959\nKS 642 61 0.0868\nNM 139 13 0.0855\nSD 718 67 0.0854\nIN 410 38 0.0848\n\nIn the first record, AS stands for American Samoa, a U.S. territory with 19 percent of the loans defaulting during the time period. Those California numbers show up at No. 7 in this calculation, with 11 percent.\n\nStates, of course, make up a simple example. Reporters can filter and cross-tab by city, zip code, even type of business to compare and calculate, filtering before or after running the cross-tab command. The SBA disaster loan data table contains 731,352 records, far beyond Excel's pivot table capability.\n\nOf course, even a well executed cross-tab doesn't relieve the reporter of reading the fine print about the data. The table lists the state of the borrower, not necessarily the location of the damaged property. The majority of the most recent loans, including those in the days after the Sept. 11 attacks, haven't had time to run into serious trouble.\n\nJeff Porter can be reached by e-mail jeff@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-a-problem-company","slug":"2002-february-a-problem-company","url":"https://dwillis.github.io/uplink/article/2002-february-a-problem-company.html","year":2002,"month":"February","monthNum":2,"headline":"A problem company","kicker":"HOUSING","author_name":"R.G. Dunlop and Mark Schaver","author_title":"","affiliation":"The Courier-Journal","summary":"The Courier-Journal used computer-assisted reporting and traditional journalism to investigate code enforcement in Jefferson County, Kentucky, uncovering how a sprawling apartment company operated with minimal regulatory oversight due to inadequate inspector resources and incomplete government databases.","keywords":["enforcement","database","properties"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Access"],"full_text":"Code enforcement seems like a ripe subject for computer-assisted reporting, and that is where reporters and editors at The Courier-Journal (Louisville, Ky.) turned when they first considered a housing project in mid-2001.\n\nWhat they found, however, was a government database that contained potentially key information but needed lots of legwork before it could be unlocked.\n\nWhat followed was an example of how traditional reporting and database analysis supported and strengthened each other.\n\nBased on preliminary, anecdotal evidence, the newspaper decided to focus on a controversial apartment-management company, River City Properties, Inc., whose sprawling, low-rent complexes were located mostly in Jefferson County, outside the city limits of Louisville.\n\nThe county was of particular interest because it contained nearly as many rental-housing units as did the city, but had less than one third as many housing code-enforcement inspectors. And the Jefferson County Health Department, which is responsible for code enforcement in the county, was reportedly strapped for resources.\n\nRental units in the county were spread across 323 square miles, an area more than five times larger than the city itself and a logistical challenge for a mere handful of inspectors to navigate. And housing stock in the county, while generally much newer than that in the city, was aging to the point that decline was beginning to surface in some neighborhoods. Without inspectors' vigilance, we thought, landlords could take liberties.\n\nThe first hurdle was obtaining the database, which was maintained not by the local health department, but by the state Cabinet for Health Services, of which local health departments are a part.\n\nSketchy database\n\nAfter a series of telephone calls, and written requests under Kentucky's Open-Records law, we got the database in Access format, making it easy to work with. Unfortunately, however, it provided only sketchy information for four years' worth of code-enforcement inspections about 7,000 total - that were available.\n\nEach complaint had a unique control number and was logged by date and by address. But, landlords' names and corporate affiliations, if any, were listed only occasionally.\n\nBecause of the missing ownership information, the health department couldn't assemble an overall inspection history for a particular property owner or manager, or even for a specific address. So how could we? And unless we knew the history of enforcement and the results attained, how could we judge the effectiveness of the health department's code-enforcement program?\n\nIf there was a solution, we decided, it lay in the street addresses contained in the database. By assembling the addresses of every property managed by River City Properties, we could begin to unlock its code-violation history as shown in the database.\n\nTo determine River City's apartment holdings, a reporter searched tax records and reviewed about 60 personal-injury and other lawsuits filed over a 20-year period against the company, its president, Garey Higdon, or the individual apartment complexes managed by River City Properties and owned at least partially by Higdon. Interviews with code-enforcement officials, attorneys, realtors and representatives of various advocacy groups revealed still more about River City's real-estate empire.\n\nMissing information\n\nBut neither the lawsuits, the local tax records, nor the sources disclosed the crucial piece of information necessary to link the properties to the health department's database. That missing link was a comprehensive, up-to-date list of addresses for all the apartments in a particular complex.\n\nWithout those, complaints in the database couldn't be tied to a common source - in this case, a single property manager. We would not be able to determine River City's violation history or the health department's handling of it.\n\nFinding the addresses was time consuming. River City wouldn't help - Higdon refused to be interviewed and his attorney declined to provide the names or locations of properties managed by River City. The telephone book and the city directory provided only the address of an apartment complex's main address as determined by local planning and zoning officials, and did not account for addresses among the various buildings in a complex.\n\nSome of the company's complexes were crisscrossed by as many as eight or 10 streets. To determine conclusively how many complaints each complex generated would require knowing the street names and numbers for each building in each complex. To get this, a reporter went to each apartment complex associated with River City Properties - more than 40 in all.\n\nSome streets, and some buildings, were unmarked or incompletely identified with names or numerals. The sketchy and often outdated public records that were available sometimes conflicted with visual observations. But eventually, by checking and rechecking, the newspaper was able to compile an address list with which reporters and editors felt comfortable.\n\nCleaning the data\n\nUltimately, the decision was made to focus on 17 complexes managed by River City Properties, Inc., and owned at least in part by Higdon. Other properties owned by Higdon but sold long ago were eliminated, as was another set of apartments currently or formerly owned by one of his brothers.\n\nThe newspaper constructed its own Access database of River City-managed properties, including tables for owners, street addresses, lawsuits and tax assessments. The addresses in that database were then matched against the county health department's database to determine how many housing-code complaints River City Properties' complexes had generated.\n\nThis couldn't be done with a simple query, however.\n\nThe county health department's database was dirty, with many streets misspelled or misnamed. So we created a new column in the database to put corrected street names, while preserving the uncorrected ones in case there were questions later. This made it easier to pull out the right properties and enabled us to calculate counts on complaints for each complex.\n\nBecause there were so many addresses, we decided that the simplest and surest approach for finding River City's properties would be to go through the database record by record, looking for the ones that matched the addresses the reporter had compiled.\n\nWe created a separate table of just River City complaints by deleting non-River City complaints from a database copy. That gave us two tables: one for all complaints, and one for just River City's.\n\nAlthough this was a crude way to do it, it was the surest and most straightforward. Having a separate table of just River City properties made it easier to work with. We were confident these were the right properties, because we had examined each record individually, leaving out any that didn't clearly belong to River City.\n\nThe results were still striking: more than 1,200 complaints filed with the health department during the four-year period were tied to River City's complexes. And, the vast majority of complaints were anything but frivolous. Nearly 70 percent resulted in either a violation or an inspector taking some other sort of action, including a follow-up visit to the site of the problem.\n\nHerculean task\n\nThe newspaper also devoted many hours to pursuing a database maintained by the City of Louisville, which has a separate housing inspection office. Although only a few River City properties were in the city, we also wanted to look at the company's violation history within the city. Obtaining city's records was complicated by the fact that the city's database was on a mainframe using decades-old CA Datacom/DB software that no one still employed by the city fully understood.\n\nThe city's computer staff ran previously created programs to generate reports from the database, but no one understood how the dozens of tables that made up the database were linked together and nobody was skilled at programming it. When the city had questions it couldn't answer, it called on a former city employee who now worked as a contractor.\n\nUnder Kentucky's Open Records law, agencies are required to provide electronic records only in their native format or as ASCII text. The city initially told us that the database would cost us $100,000 to obtain, because we would have to have purchase the original software to use it.\n\nWe knew this was nonsense, but we were afraid that even if we pursued a long records battle and obtained the database for a minimal cost, there was the chance we wouldn't be able to make sense of it once it was in our hands. Even if we had all the data, we had no idea of the relationships between all the tables, and it could have been a Herculean task to figure it out.\n\nThe Courier-Journal decided the best solution would be to pay the $600 the city said it would cost to hire the former employee to write a program that would pull from the database just the records and fields we wanted in a single, flat file. As an added bonus, we got the records in Access format, which wasn't a requirement of the law.\n\nThis database showed River City Properties had a large number of complaints at its city complexes as well, although that finding ultimately played only a small part in our series.\n\nStories and results\n\nRiver City officials had long argued that complaints were not a fair barometer by which to judge the condition of its properties. Large and numerous apartment complexes naturally generate more complaints, company officials contended.\n\nThe newspaper found, however, that River City's 25 percent share of all complaints filed with the county health department during the period far exceeded the company's share of the rental housing market in the county, as determined by census data. The company's share was less than 4 percent.\n\nAfter the newspaper published a three-part series about River City Properties and the health department in October, the company publicly pledged to do a better job and the health department admitted that it was underfunded and understaffed. Officials said the issues may be addressed when city and county governments merge in 2003.\n\nR.G. Dunlop can be reached by e-mail at rdunlop@courier-journal.com\n\nMark Schaver can be reached by e-mail at mschaver@courier-journal.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-stories-3","slug":"2002-february-stories-3","url":"https://dwillis.github.io/uplink/article/2002-february-stories-3.html","year":2002,"month":"February","monthNum":2,"headline":"Stories","kicker":"readme.txt","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"The IRE Resource Center catalogs investigative reports on housing violations and slumlord practices, including stories about property owners evading enforcement and widespread code compliance failures in major cities.","keywords":["housing","enforcement","landlords"],"topics":["Investigative Methods","Business & Economy"],"technologies":[],"full_text":"Other stories on housing problems available from the IRE Resource Center:\n\nStory#17226 New Times Los Angeles reported that \"slum properties can be shuffled among a small cadre of co-conspirators, keeping law enforcement and housing activists at bay,\" revealing \"how a slumlord managed to defy the best efforts of housing enforcement officials and his own tenants to bring a substandard building up to code, even when he was slapped with a $455,000 judgment.\"\n\nStory#15948 The St. Paul Pioneer Press published a three-part report on \"chronic problem properties\" and the effect they have on the Twin Cities. \"At the heart of the issue,\" the newspaper reported, \"is a badly broken system, rife with legal loopholes, spotty enforcement and too few penalties for lawbreakers.\" The newspaper found that \"more than 6,400 properties in St. Paul and another 22,000 in Minneapolis have failed to comply with housing codes for two years or longer,\" inspectors are unwilling to crack down on the offenders and \"a small group of landlords account for a disproportionate number of complaints.\"\n\nTo order, call the Resource Center at 573-882-3364 or go to www.ire.org/resourcecenter/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-lessons-from-disaster","slug":"2002-february-lessons-from-disaster","url":"https://dwillis.github.io/uplink/article/2002-february-lessons-from-disaster.html","year":2002,"month":"February","monthNum":2,"headline":"Lessons from disaster","kicker":"SPOTLIGHT: TERRORISM","author_name":"Jeff Porter","author_title":"","affiliation":"IRE/NICAR","summary":"Journalists should prepare disaster coverage in advance by building databases and understanding data limitations, as exemplified by how newsrooms used FAA enforcement data after 9/11 to expose systemic airport security failures.","keywords":["preparation","databases","security"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["World Wide Web","FTP"],"full_text":"Be prepared, say the Boy Scouts. In the last few months, more and more journalists are saying it, too.\n\nIndeed, one of the consistent lessons from almost any disaster is to keep a warehouse of data, preparing it for instant use for the next catastrophe. The advantages are twofold: There's no wait for the data, and reporters, without the pressure of disaster, can begin understanding the best uses and pitfalls of the data.\n\nCase in point: FAA enforcement data, which many newsrooms used to focus on security breaches in airports after terrorist attacks killed thousands on U.S. soil.\n\nThe Web searches can be excellent starting points, especially on deadline, but only in a limited way.\n\n\"It was obvious on Sept. 11 that airport security had failed dreadfully,\" says Ron Campbell with the Orange County Register. \"The FAA database showed that the failure was systemic. It showed many, many failures at every major airport nationwide for several years. It showed that the system worked on Sept. 11 exactly the way it was designed to work - badly.\"\n\nThe database, though, has its own limitations. It's often cryptic and open to interpretation (see numerous accounts of the use of the data in this edition of Uplink). While the data included the cases in which FAA inspectors penetrated airport security during surveillance, it didn't include those times when airport screeners caught them. The database, after all, is about enforcement actions. If the airport screeners did their job, no enforcement action is required.\n\n\"As valuable as the FAA database was, it has huge gaps,\" Campbell says. \"We don't know how many times security caught the FAA trying to penetrate security, so there is no way to determine which airports had the worst security. We can't even compute a nationwide security breach rate, since the FAA won't say on a national level how often they tried to breach security but were stopped by alert guards.\"\n\nCiting vague security concerns, the FAA refused to answer even rudimentary questions about the data.\n\nYears before the terrorist attacks, Elizabeth Marchak of The Plain Dealer (Cleveland, Ohio) pushed preparation for aviation disasters. Never, she wrote in a conference handout that's available from the IRE Resource Center, use data on deadline you've never seen before.\n\nBy their very nature, databases are incomplete, according to Marchak, and it takes some experience to fully understand those limitations.\n\nOnline aviation data\n\nThere are a few shortcuts, thanks to the World Wide Web, that allow reporters to look up simple information about aircraft, pilots, accidents and incidents. So part of preparing for disaster coverage is conscientiously bookmarking Web sites. Numerous IRE tipsheets show Web addresses that would help. Again, though, the searchable online databases are incomplete and limited. It pays to read the fine print even on a government Web site.\n\nThe FAA Flight Standards Service Web site promotes itself as providing safety-related information, and in fact, it's a place worth bookmarking. Reporters can look up airworthiness directives and recent service difficulty reports, search for pilot schools, and link to search pages for accidents and incidents, both from the FAA and the NTSB.\n\nAnother FAA Web presence: the FAA Civil Aviation Registry, where a reporter can look up ownership of an aircraft, by registration number or serial number.\n\nA non-government but useful Web site comes from Aviation Research Group/US Inc. You can sign up to receive preliminary accident information via e-mail for free. According to the company's Web site, several times a day it downloads the FAA's preliminary accident files, then sends out automatic notifications to e-mail subscribers. You can choose categories and filters, too. For example, you can choose to receive only notifications of all multi-engine airplanes, or just single and multi-engine jets, or just helicopters.\n\nLike the databases residing on a reporter's desktop or a newsroom's network, though, Web-based or e-mailed data have limitations, too. The FAA's lookup information on service difficulty reports, for example, only go back to 1995. The actual database of SDRs, though, goes back to 1974. One can look up ownership of particular aircraft through the FAA Civil Registry Web search, but it's problematic to summarize and count how many airplanes of a certain type are owned by a particular company.\n\nPreparation is key\n\nThe Web searches can be excellent starting points, especially on deadline, but only in a limited way. They don't take the place of actually obtaining and understanding the data.\n\nAnd that should take place not in the heat of a disaster, but beforehand, when authorities are less harried and, in the case of the FAA, might be more open.\n\n\"The FAA database taught me a lesson for the future,\" Campbell says. \"When getting government databases, always find out how the data is compiled. In the case of the FAA database, we know that airport security guards, airport security coordinators and FAA inspectors all must have filled out reports. There is undoubtedly a manual telling these folks how to fill out reports, explaining for example when to charge multiple violations for a single offense. But we don't have that manual, and it's extremely unlikely post-Sept. 11 that the FAA will give it up.\"\n\nJeff Porter can be reached by e-mail at jeff@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-links-3","slug":"2002-february-links-3","url":"https://dwillis.github.io/uplink/article/2002-february-links-3.html","year":2002,"month":"February","monthNum":2,"headline":"Links","kicker":"readme.txt","author_name":"","author_title":"Reporter","affiliation":"CNN","summary":"CNN's investigative project 'Flight Risk' reports on aviation security issues, featuring database analysis and multimedia elements developed by a specialized team of journalists, editors, and designers.","keywords":["aviation","investigation","security"],"topics":["Investigative Methods","Public Safety"],"technologies":[],"full_text":"The CNN.com original project \"Flight Risk\" was reported by Mike Fish. Denise Prodigo-Herrmann conducted the database analysis. Kevin Drew was the editor. Emily Chen was the designer and worked with Ann Hoevel to develop graphics, charts and images. Meriah Doty also developed images. Robert Felker was the illustrator. Charles Rawls was the web editor. www.cnn.com/SPECIALS/2001/trade.center/flight.risk/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-fighting-the-airport","slug":"2002-february-fighting-the-airport","url":"https://dwillis.github.io/uplink/article/2002-february-fighting-the-airport.html","year":2002,"month":"February","monthNum":2,"headline":"Fighting the airport","kicker":"SPOTLIGHT: TERRORISM","author_name":"Alan Judd and Maurice Tamman","author_title":"","affiliation":"The Atlanta Journal-Constitution","summary":"After 9/11, Atlanta airport officials resisted journalists' reporting on security vulnerabilities, restricting press access to terminals and criticizing coverage as unpatriotic fear-mongering. Reporters had to purchase airline tickets to access restricted areas and report on the airport's operations during the crisis.","keywords":["security","airport","journalism"],"topics":["Public Safety","Investigative Methods","Journalism & Newsroom"],"technologies":["Internet"],"full_text":"Fighting the airport\n\nBy Alan Judd and Maurice Tamman, The Atlanta Journal-Constitution\n\nBen DeCosta runs the world's busiest airport, Atlanta's Hartsfield International. So on Sept. 26, when The Atlanta Journal-Constitution reported on Hartsfield's history of security breaches, DeCosta was not happy.\n\nIn DeCosta's view, the story, based on the Federal Aviation Administration's database of security violations, did nothing more than confirm the worst fears of air travelers days after terrorists hijacked four planes to attack New York and Washington.\n\nWhat's more, he said, \"they have no right to have their worst fears confirmed.\"\n\nDeCosta's comment typified the official reaction to unsettling post-Sept. 11 revelations about the safety of the nation's air travel system. After the Journal-Constitution and many other newspapers wrote stories off the database of violations, the FAA quickly pulled it from its Web site.\n\n(This shouldn't have been surprising; a few years ago, after several stories appeared on the alarming rate at which airport security screeners failed to detect fake bombs and other weapons planted by the FAA, the government declared the information to be \"security sensitive\" and restricted public access.)\n\nAviation focus changed\n\nBefore Sept. 11, reporting on commercial aviation often focused on matters of convenience: flight delays and limits on carry-on baggage.\n\nAfter Sept. 11, when news organizations began examining the far more important issues of safety and security, the new scrutiny made the people who run the air travel system more than a little uncomfortable.\n\nIn Atlanta, DeCosta virtually accused a reporter of being unpatriotic for reporting on the security breaches.\n\nNoting that passenger loads on airliners had dropped by 40 percent the week after air travel resumed, DeCosta said: \"What the public needs is more than your newspaper gives them. You have a special responsibility not to exaggerate and exacerbate an already bad situation. Get people to focus, instead of scaring the hell out of them.\"\n\nNot that officials were going to make our reporting on anything at the airport easy.\n\nBarricades to reporters\n\nReporting about the airport required access to concourses and terminals, but airport officials would not allow the press access beyond the security gates leading to the terminal trains and gates.\n\nIn the week after the attack, we attempted to document how little business was being done at what we routinely describe as the world's busiest airport. That required getting behind the security checkpoint.\n\nWe solved the problem by purchasing unrestricted full fare tickets. The reporters, however, had to be careful. The reporters needed to order the ticket online for a flight within a few hours of the period they would be working behind the lines.\n\nThe reporters did not check in, but presented a printout of the Web page ticket confirmation page. In one case, the ticket cost $987.\n\nAfter that, it was through the metal detectors, past the security guards and out to the terminals.\n\nIt's quite remarkable that reporters were considered such a security risk that they were not allowed out to the terminals. Meanwhile, any shlub with a home computer and a printer could re-create a confirmation Web page, print it out and have complete, unfettered access to the terminals.\n\nAfter passing in and out of the terminals at will, the reporters had to get in line at a ticket counter, before their flight was scheduled to depart, and get a refund.\n\nAirline employees\n\nWith passengers jittery about flight in the days that followed the reopening of airports, it was clear airline employees were worried. While airport officials were useless, airline employees were polite, friendly, even plaintive.\n\nIn one case, a Delta employee behind one counter pleaded with a reporter not to cancel his round trip ticket to Orlando.\n\nHe asked several questions about why the reporter was not flying.\n\nDo you know flying is safe? Do you know there are no delays? Yes it was expensive, but why not go?\n\nThen he squinted suspiciously at the black-haired, bearded reporter. He waited a few seconds, before pressing a key on his keyboard. A receipt printed out.\n\n\"You will receive your credit within 30 days,\" he said.\n\nMaurice Tamman can be reached by e-mail at mtamman@ajc.com.\n\nAlan Judd can be reached by e-mail at ajudd@ajc.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-cheap-easy-solution","slug":"2002-february-cheap-easy-solution","url":"https://dwillis.github.io/uplink/article/2002-february-cheap-easy-solution.html","year":2002,"month":"February","monthNum":2,"headline":"Cheap, easy solution","kicker":"INTRANET","author_name":"Andy Crain","author_title":"","affiliation":"Sarasota Herald-Tribune","summary":"A journalist describes how to build a functional intranet system using free ColdFusion Express software, Microsoft Access databases, and basic hardware, demonstrating that professional web applications don't require expensive enterprise software.","keywords":["intranet","ColdFusion","budget"],"topics":["Technology","Journalism & Newsroom","Investigative Methods"],"technologies":["ColdFusion","Access","HTML","SQL Server","IIS","ASP","MySQL","Apache","PHP","Paradox","dBase","FoxPro","Excel","Lotus Approach"],"full_text":"Your boss asks you to build \"one of those intranet things.\" It's a top priority and must be finished quickly, your boss says, patting you on the back. And oh yeah, there's no money in the budget for this.\n\nImpossible? Actually, no.\n\nDeveloping an intranet, one that's hooked up to databases and full of all sorts of dynamic features, is well within the capabilities of even the most overworked CAR grunt. All it takes is a Web server and ColdFusion Express, the free, trimmed-down version of the popular and expensive Web application server.\n\nYou'll have the support of an easy-to-learn language, great documentation (the kind that comes with commercially distributed software, which is ColdFusion's main enterprise) and a huge ColdFusion programming community. Best of all, you can get away with all this without spending a penny.\n\nAt the (Sarasota) Herald-Tribune, I created such a system without a single software purchase. Reporters and editors query research databases, search in-house and AP stylebooks, share sources, request and assign photos, participate in discussion forums, and more.\n\nBig hits, low cost\n\nIn an average week, the site responds to just under 5,000 page requests and sends 150 MB of data to over 160 distinct addresses, using free software and Microsoft Access databases. Even the hardware's low-budget: an old PC running Windows 95 with 32 MB of RAM.\n\nThe popular intranet model is to build \"dynamic\" Web pages that change and present different views depending on information stored in a database.\n\nDoing this typically requires a Web server, which sends out pages formatted in HTML in response to user requests, for example the clicking of a link to a new page; a database server, which stores the always-changing information that is presented in these pages; and some sort of \"glue\" to link everything together.\n\nYou could hook up an enterprise database server, such as Microsoft's SQL Server, with Microsoft's IIS Web server and use ASP to build dynamic, database-driven pages. All well and good, if you have the cash to pay for it.\n\nOr you could try an open-source solution. A combination of a MySQL database server, an Apache Web server, and PHP to build the pages and link everything together packs just as much punch, but with a much more appealing price tag: $0. The downside is a much steeper learning curve and cumbersome, sometimes spotty documentation. Although this solution has a lot going for it, if you're pressed for time, it's probably not the right one.\n\nEnter ColdFusion Express, which combines all the benefits of a commercial solution with one of the more appealing features of open-source software (it's free).\n\nColdFusion Express Server (download it at www.macromedia.com/software/coldfusion/trial/cf_server_express.html) is distributed as a teaser for Macromedia's ColdFusion Server. Presumably, the company hopes that you'll ultimately purchase their flagship product, which goes for between $1,300 and $5,000.\n\nColdFusion limitations\n\nAs a result, ColdFusion Express lacks some of the features of the full-blown product. For example, there's a cap on the number of simultaneous connections to the server. ColdFusion Express supports a maximum of five.\n\nThis isn't as bad as it sounds. Connections last mere milliseconds, so dozens, even hundreds, of users can be clicking around via ColdFusion Express at the same time.\n\nAlso, there is limited native driver support for databases.\n\nIf your data is in Oracle, IBM's DB2, Informix, SQL Server, or some other high-end database, you'll need the cash-money version of ColdFusion. ColdFusion Express works only with desktop databases and has native drivers for Microsoft Access, Paradox, dBase, FoxPro, Excel or Lotus Approach.\n\nWith drivers for Access and other database systems, ColdFusion Express becomes a virtual database server, running queries itself and returning results. You don't even need Access on the server; ColdFusion reads and processes mdb files directly. Many times, ColdFusion will process some complex queries faster than Access.\n\nAnyone familiar with HTML will feel at home with ColdFusion, which is completely tag-based. A ColdFusion application is nothing more than a typical HTML document with ColdFusion tags inserted in among the HTML tags. Requests for ColdFusion pages are sent first to the ColdFusion Express Server, which will process the tags, query databases and whatever else it's told to do, and then return the results to the Web server as HTML.\n\nAndy Crain can be reached by e-mail at andy.crain@heraldtribune.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-february-links-4","slug":"2002-february-links-4","url":"https://dwillis.github.io/uplink/article/2002-february-links-4.html","year":2002,"month":"February","monthNum":2,"headline":"Links","kicker":"readme.txt","author_name":"","author_title":"Reporter","affiliation":"CNN","summary":"CNN's investigative project 'Flight Risk' examined aviation security issues, combining reporting by Mike Fish with database analysis and multimedia elements created by a team of editors, designers, and illustrators.","keywords":["aviation","investigation","security"],"topics":["Investigative Methods","Public Safety","Technology"],"technologies":["World Wide Web"],"full_text":"The CNN.com original project \"Flight Risk\" was reported by Mike Fish. Denise Prodigo-Herrmann conducted the database analysis. Kevin Drew was the editor. Emily Chen was the designer and worked with Ann Hoevel to develop graphics, charts and images. Meriah Doty also developed images. Robert Felker was the illustrator. Charles Rawls was the web editor. www.cnn.com/SPECIALS/2001/trade.center/flight.risk/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-march-april-bits-and-bytes","slug":"2002-march-april-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2002-march-april-bits-and-bytes.html","year":2002,"month":"March-April","monthNum":3,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"This article outlines recent updates to the IRE and NICAR Database Library, covering databases for medical device failures, home mortgages, truck accidents, and fatal car accidents.","keywords":["databases","NICAR","transportation"],"topics":["Data & Databases","Investigative Methods"],"technologies":["SAS","dBase"],"full_text":"Data updates: The IRE and NICAR Database Library has recently updated the following databases: MAUDE. IRE and NICAR in early March updated its copy of the Manufacturer and User Facility Device Experience database, from the U.S. Food and Drug Administration, just as doctors in Baltimore and officials with another federal agency criticized a manufacturer for a faulty medical device. The MAUDE data includes information about problems that occurred with medical devices. Current as of 2001. Home mortgages. This database includes a record of mortgage applications, including demographic information about loan applicants, including race, sex and income; the purpose of the loan; whether the buyer intends to live in the home; the type of loan; and the outcome of the loan application. The dataset also includes geographical information on applicants, such as Census tract, metropolitan statistical area, state and county. Current as of 2000. Trucks. The truck accidents database consists of accidents that involve commercial vehicles that weigh more than 10,000 pounds, as reported by state transportation agencies to the U.S. Department of Transportation. The vehicles included in this database are buses, semis, moving trucks, rental trucks and hazardous-material carriers. Current as of 2000. Fatal accidents. The latest updated Fatal Accident Reporting System database consists of three relational tables, containing data on automobile accidents on public U.S. roads that resulted in the death of one or more people within 30 days of the accident. In the past, the NICAR data files were available in four relational tables. For the current update, NICAR used SAS software to replicate the data tables used by the Department of Transportation and translated those tables into DBF format, which can be opened by almost any database manager software. In the latest data update, SAS translates the official DOT codes to plain English in the data tables. Covers 1975-2000.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-building-it-from-scratch-wtc-victim-database","slug":"2002-march-april-building-it-from-scratch-wtc-victim-database","url":"https://dwillis.github.io/uplink/article/2002-march-april-building-it-from-scratch-wtc-victim-database.html","year":2002,"month":"March-April","monthNum":3,"headline":"Building it from scratch: WTC victim database","kicker":"TERRORISM","author_name":"Paul Overberg and Anthony DeBarros","author_title":"Database Editor","affiliation":"USA Today","summary":"USA Today recounts how their data editors manually constructed a database of September 11 victims to verify death toll figures and tell personal stories.","keywords":["9/11","WTC","victims"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Some of the best databases are not the ones you buy or acquire through FOIA but the ones you build yourself, one fact at a time. That's what USA Today found when it set out to chronicle the thousands of lives lost in the terrorist attacks of Sept. 11. The result was several stories that effectively challenged official reports of the death toll, as well as a major enterprise story that chronicled the last moments inside the World Trade Center. On Sept. 12, USA Today's four database editors huddled to catch their breath and draw up a CAR-based plan.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-mining-workplace-data","slug":"2002-march-april-mining-workplace-data","url":"https://dwillis.github.io/uplink/article/2002-march-april-mining-workplace-data.html","year":2002,"month":"March-April","monthNum":3,"headline":"Mining workplace data","kicker":"SPOTLIGHT: WORKPLACE","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"An exploration of various workplace data sources, including OSHA logs, pension filings, and workers' compensation records, used by journalists to investigate employer practices.","keywords":["workplace","pensions","OSHA"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":["spreadsheet","10K Wizard"],"full_text":"A quick scan of your daily paper or network news program shows that workplace woes are big news: Enron, once a little-known energy company, now is a household word. Workers - and not just at Enron - are fretting about the health of their employer- backed retirement savings plans. In this issue of Uplink, we focus on workplace data. Stephanie Armour of USA Today recounts how she used computer-assisted-reporting's most basic tool - the spreadsheet - to build her own database of cases of immigrant maids being abused by their employers. In another article, Armour shares some reporting ideas and data sources that you can use to scrutinize employers and suspect workplace practices. Over the years, many journalists have plunged into the massive enforcement database kept by the federal Occupational Safety and Health Administration, only to come up gasping for air. Ron Nixon, the national trainer for IRE & NICAR in this issue offers some guidance for journalists who want to stay afloat while working with the OSHA data. After the fall of Enron flattened out its employee retirement savings plans, heavy in company stock, St. Paul Pioneer Press business reporter Jim McCartney turned to the web for the local angle. McCartney wanted to see whether the retirement savings plans offered by the 15 biggest publicly-traded companies in the Twin Cities were loaded with company stock. So he turned to the online 10k Wizard and looked for 11K annual reports the companies submitted to the U.S. Securities Exchange Commission that provide detailed financial information about employer 401(k) retirements savings plans. After looking over the documents and punching the numbers into his calculator, McCartney wrote a Feb. 2 story saying two-thirds of the top companies had 401(k) plans in which company stock accounted for at least 40 percent of the total assets. Pension plan administrators every year submit detailed information about the plans using Form 5500, available with other documents on the Department of Labor's Web site. Foreign/immigrant workers. Data from the Immigration and Naturalization Service can provide some clues. The INS data, sold by the NICAR database library, specifies the sex, age, marital status, country of birth, country of last permanent residency, port of entry and ZIP Code of intended residence for immigrants. State workers compensation data, used by journalists investigating bogus injury claims and injury patterns at problem work sites, can be put to other uses. Last year, reporters at the Washington Post dug through ten years of Maryland Workers Compensation Commission data as they probed the Prince George's County Police Department.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-following-the-riverboat-pilots","slug":"2002-march-april-following-the-riverboat-pilots","url":"https://dwillis.github.io/uplink/article/2002-march-april-following-the-riverboat-pilots.html","year":2002,"month":"March-April","monthNum":3,"headline":"Following the riverboat pilots","kicker":"SPOTLIGHT: WORKPLACE","author_name":"Jeffrey Meitrodt","author_title":"","affiliation":"The New Orleans Times-Picayune","summary":"An investigation by The Times-Picayune reveals high salaries, nepotism, and lack of accountability among Louisiana's riverboat pilots through spreadsheet analysis of resumes and accident reports.","keywords":["nepotism","river pilots","Louisiana"],"topics":["Investigative Methods","Government & Politics","Public Safety"],"technologies":["Excel"],"full_text":"Teal Grue was the poster boy for our lead story about what's wrong with Louisiana's archaic system of selecting and overseeing Mississippi River pilots. Grue dropped out of high school after 10th grade, had a DUI conviction, and tested positive for marijuana before becoming an elite river pilot. We reviewed resumes and typed them into an Excel spreadsheet after The Times-Picayune decided to investigate what some believe is the ultimate good-old-boys club in Louisiana. Louisiana pilots earned about $321,000 each last year, and we documented that widespread nepotism makes it virtually impossible for non-relatives to join. Of the 100 people elected to become river pilots in recent years, 85 are related to other pilots. We also analyzed accident reports and pilot misconduct records, which are judged by the pilots themselves. The records contained several bombshells. First, we discovered that two pilots were allowed to keep their jobs after testing positive for cocaine use in 2000. Second, we found that one group racked up nearly 100 accidents yet failed to discipline a single pilot in a decade.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-how-data-can-correct-government-stats","slug":"2002-march-april-how-data-can-correct-government-stats","url":"https://dwillis.github.io/uplink/article/2002-march-april-how-data-can-correct-government-stats.html","year":2002,"month":"March-April","monthNum":3,"headline":"How data can correct government stats","kicker":"TERRORISM","author_name":"Mark Fazlollah and Peter Nicholas","author_title":"","affiliation":"Philadelphia Inquirer","summary":"The Philadelphia Inquirer used TRAC and PACER data to prove the Department of Justice was inflating terrorism conviction statistics with non-terrorist criminal cases.","keywords":["DOJ","terrorism","TRAC"],"topics":["Investigative Methods","Crime & Justice","Government & Politics"],"technologies":["PACER"],"full_text":"There was quick action when the Philadelphia Inquirer reported that the Department of Justice had dramatically overstated its record of convicting terrorists, with conservative and liberal congressmen calling for an investigation. The December story, which documented a five-year pattern of inflated terrorism statistics, went unchallenged by the Justice Department. Our basic data came from Syracuse University's Transactional Records Access Clearinghouse (TRAC). We paired the justice information with federal court data offered through the Public Access to Court Electronic Records (PACER). The pairing is painfully time consuming and best suited to small samplings of cases. PACER allowed us to select all cases filed on a specific date in any federal court district. For each federal district, we entered the date of each Justice Department's terrorism case. Usually, there were no more than a dozen criminal cases filed on a single day. Of those, there usually would be only one with the specific criminal charge matching the Justice Department data. In one supposed case of terrorism in Seattle, a pilot sent a phony report that another pilot was planning a hijacking due to a personal feud. The agent told us this was his 'only real case of terrorism' in three years. Now Congress wants to know how pervasive the problem is.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-readmetxt","slug":"2002-march-april-readmetxt","url":"https://dwillis.github.io/uplink/article/2002-march-april-readmetxt.html","year":2002,"month":"March-April","monthNum":3,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"This article provides information about various resources available through IRE and NICAR, including investigative stories, databases, training materials, and tipsheets for journalists.","keywords":["resources","journalists","training"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Stories\n\nThe stories mentioned in this edition of Uplink will soon be available from the IRE Resource Center. Search the story database at www.ire.org/resourcecenter for order numbers. Call 573-882-3364 to order these and other investigative stories.\n\nDatabases\n\nTo view the list of databases available at the IRE and NICAR Data Library, view the Web site www.ire.org/datalibrary. From this page, you can also download an order form. The library's telephone number is 573-884-7711.\n\nTech tips\n\nHave you found a technical solution that saves you time? Share it as an Uplink Tech Tip, a story of 500-1,000 words showing a technique that you'd be willing to share. Please contact Managing Editor Jeff Porter at jeff@nicar.org or by telephone at 573-882-1982.\n\nTraining material\n\nFor a calendar of IRE and NICAR training events, visit the Web page www.ire.org/training/otr.html.\n\nTipsheets\n\nThe IRE Resource Center offers a searchable database of tipsheets for journalists at www.ire.org/resourcecenter. Many are downloadable, or call 573-882-3364 to order tipsheets.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-march-april-from-the-lists","slug":"2002-march-april-from-the-lists","url":"https://dwillis.github.io/uplink/article/2002-march-april-from-the-lists.html","year":2002,"month":"March-April","monthNum":3,"headline":"From the Lists","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"A compilation of technical advice from NICAR listservs, covering file transfers, database management in Access, and mapping techniques.","keywords":["Access","Excel","listserv"],"topics":["Data & Databases","Technology","Investigative Methods"],"technologies":["FTP","SPSS","Excel","Access","ArcView"],"full_text":"Listserv messages you may have missed between members of NICAR-L. Topics include the benefits of using FTP software over browser downloads for speed and batching files; converting SPSS data to Excel using 'File/Save As'; copying import specifications between Access databases using the 'Get External Data' function and checking the 'Import/Export Specs' option; creating dot-density maps in ArcView by aggregating point data into census tracts; using the instr(), Left(), and Mid() functions in Access to split name fields; and calculating weighted averages for school test scores. Another user tip addresses disk space errors in Access by changing the location of TEMP files in the Windows Environment settings.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-handy-internet-resources-for-reporters","slug":"2002-march-april-handy-internet-resources-for-reporters","url":"https://dwillis.github.io/uplink/article/2002-march-april-handy-internet-resources-for-reporters.html","year":2002,"month":"March-April","monthNum":3,"headline":"Handy Internet resources for reporters","kicker":"ONLINE","author_name":"John Tedesco","author_title":"","affiliation":"San Antonio Express-News","summary":"A guide to the most useful websites for investigative journalists, covering everything from charity tax filings to federal court records and expert finders.","keywords":["Internet","research","journalism"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"What kinds of Internet sites are being bookmarked these days by web-savvy reporters? The right URL address can lead to aviation accident reports, campaign contribution databases, people finders, tax records for charities and prison inmate tracking systems. Sites discussed include Liszt for listservs, Google Groups, Profnet for experts, PubMed for health studies, PACER for court records, Martindale for lawyers, GuideStar for charities, OSHA for worker safety, and FedStats for federal figures. Some of the best sites are maintained by reporters and researchers, such as the Journalist's Toolbox, Power Reporting, and The Reporter's Desktop.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-automating-parsing","slug":"2002-march-april-automating-parsing","url":"https://dwillis.github.io/uplink/article/2002-march-april-automating-parsing.html","year":2002,"month":"March-April","monthNum":3,"headline":"Automating parsing","kicker":"Tech tips","author_name":"Ron Nixon","author_title":"","affiliation":"IRE and NICAR","summary":"Ron Nixon reviews various software tools like Splitter and NameChop that automate the tedious process of parsing and cleaning dirty name and address data.","keywords":["parsing","software","data cleaning"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Access","Splitter","NameChop","DupeDetect","McXtract","ParseRat","Potters Wheel"],"full_text":"A few years back while I was a reporter at the Roanoke (Va.) Times, I came across a problem in a database that confounded me for days: names spelled hundreds of different ways. After searching the Web I found a little program called Splitter made for Microsoft Access databases. The program not only splits names no matter how they are entered, it also guesses the gender. One of my favorites programs, NameChop, is available for free from Semaphore Corp.'s Website. In addition to NameChop, Semaphore also gives away DupeDetect, a program that lets you flag duplicate records. Another program that works fairly well at parsing files is McXtract. A parsing program that had gotten attention on the IRE listserv for parsing is ParseRat, which costs about $50 and parses addresses as well. Programmers in the open source movement have also developed a parsing and data-cleaning program called Potters Wheel.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-money-in-and-from-the-bank","slug":"2002-march-april-money-in-and-from-the-bank","url":"https://dwillis.github.io/uplink/article/2002-march-april-money-in-and-from-the-bank.html","year":2002,"month":"March-April","monthNum":3,"headline":"Money in - and from - the bank","kicker":"CAMPAIGN FINANCE","author_name":"Aron Pilhofer","author_title":"","affiliation":"IRE and NICAR","summary":"The Wilmington News Journal investigated a surge in campaign donations from MBNA executives, linking the spending to efforts to pass bankruptcy reform legislation.","keywords":["MBNA","campaign finance","bankruptcy"],"topics":["Government & Politics","Business & Economy","Data & Databases"],"technologies":[],"full_text":"The News Journal in Wilmington, Del., discovered a sudden upsurge in campaign spending by MBNA, one of the nation's largest issuers of credit cards. MBNA executives were the single largest identifiable group of contributors to then-candidate George W. Bush's presidential campaign, giving more than $250,000. MBNA ended up as the most generous campaign contributor among finance/credit companies in 2000, giving more than $3.5 million. The News Journal uncovered memos that showed how top MBNA executives were influencing employees to contribute to candidates the company favored. Campaign-finance reform advocates said it was more than coincidental that the company's interest in national politics came at a time when an important piece of bankruptcy reform legislation was pending in Congress. The project involved compiling data from several campaign finance sources, including raw FEC data and paper filings.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-students-reveal-gifts-to-legislators","slug":"2002-march-april-students-reveal-gifts-to-legislators","url":"https://dwillis.github.io/uplink/article/2002-march-april-students-reveal-gifts-to-legislators.html","year":2002,"month":"March-April","monthNum":3,"headline":"Students reveal gifts to legislators","kicker":"CAMPAIGN FINANCE","author_name":"Jeff C. South","author_title":"Professor","affiliation":"Virginia Commonwealth University","summary":"VCU journalism students built a database of gifts given to Virginia legislators, highlighting expensive trips and potential conflicts of interest.","keywords":["ethics","legislature","gifts"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":["Excel"],"full_text":"My Legislative Reporting students at Virginia Commonwealth University created and analyzed a relatively small database: a list of all gifts received last year by state legislators, the governor and other top officials. The dataset contained fewer than 1,000 records, totaling about $185,000 in gifts, taken from the conflict-of-interest forms. We highlighted the biggest gifts - including trips to hunt caribou in the Arctic Circle and quail in Georgia. Many gifts have a cachet that campaign donations don't. Access is the goal of most big gifts and campaign contributions. Constituents may view certain gifts more suspiciously than campaign contributions. The class entered the Schedule E information into a spreadsheet using Microsoft Excel. We then analyzed the data with Excel's PivotTable Report feature. This allowed us to see how much each donor had given in gifts and how much each lawmaker had received. One student decided to write about gifts given to Gov. Jim Gilmore, who reported 337 gifts worth almost $55,500.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-march-april-mining-workplace-data-continued","slug":"2002-march-april-mining-workplace-data-continued","url":"https://dwillis.github.io/uplink/article/2002-march-april-mining-workplace-data-continued.html","year":2002,"month":"March-April","monthNum":3,"headline":"Mining workplace data (continued)","kicker":"workplace","author_name":"David Herzog","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists can use public databases like 10-K Wizard, FreeERISA, and Department of Labor records to investigate pension plans and retirement savings, with examples of reporters uncovering excessive company stock concentrations in major corporations' 401(k) plans.","keywords":["pensions","databases","investigation"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["10k Wizard","FreeERISA","9-track tape"],"full_text":"& NICAR in this issue offers some guidance for journalists who want to stay afloat while working with the OSHA data.\n\nThese articles just touch on what you do. There are plenty of other angles that data-savvy reporters can follow.\n\nPensions\n\nAfter the fall of Enron flattened out its employee retirement savings plans, heavy in company stock, St. Paul Pioneer Press business reporter Jim McCartney turned to the web for the local angle.\n\nMcCartney wanted to see whether the retirement savings plans offered by the 15 biggest publicly-traded companies in the Twin Cities were loaded with company stock. So he turned to the online 10k Wizard and looked for 11K annual reports the companies submitted to the U.S. Securities Exchange Commission that provide detailed financial information about employer 401(k) retirements savings plans. Not all plans are covered; only those in which the employer matches employee contributions in company stock, and grants the employees the right to buy more shares. McCartney said he found the data on 10k Wizard, a subscription service, up to date.\n\nMcCartney obtained data about other pension plans at FreeERISA, a site that offers no-cost access to pension plan data collected by the Department of Labor's Pension and Welfare Benefits Administration under the Employee Retirement Income Security Act. McCartney said that the data for some companies was outdated by as much as three years.\n\nAfter looking over the documents and punching the numbers into his calculator, McCartney wrote a Feb. 2 story saying two-thirds of the top companies had 401(k) plans in which company stock accounted for at least 40 percent of the total assets. McCartney found two companies based in the Twin Cities - Target and Best - with company stock concentrations higher than Enron's.\n\nComfortable with heavy-duty number crunching? You can obtain the entire ERISA database from the U.S. Department of Labor, and compare some 750,000 pension plans across the country. Pension plan administrators every year submit detailed information about the plans using Form 5500, available with other documents on the Department of Labor's Web site. In the coming months, IRE and NICAR will be selling the pension data in an easy-to-use format.\n\nFive years ago, News 12 Long Island reporter Mark Lagerkvist obtained the massive database (a whopping 6gb on 27 9-track tapes!) from the department via a federal Freedom of Information Act request. After carving out smaller data sets and analyzing them, Lagerkvist broadcast \"Stolen Dreams,\" a special report on how employers plundered pension plans.\n\nForeign/immigrant workers\n\nSoon after the Sept. 11 attacks, federal agents began sweeping the United States for suspected terrorists, focusing on Middle Eastern men. Foreign visitors, legally in the United States with temporary work visas, were questioned. So were legal immigrants.\n\nSo where are these workers? Data from the Immigration and Naturalization Service, part of the U.S. Justice Department, can provide some clues. The INS data, sold by the NICAR database library, specifies the sex, age, marital status, country of birth, country of last permanent residency, port of entry and ZIP Code of intended residence for immigrants.\n\nWith some rudimentary queries in your database manager, you'd be able to find out what national groups live in your area, and see which occupations are the most prevalent for the groups.\n\nWorkers compensation\n\nState workers compensation data, used by journalists investigating bogus injury claims and injury patterns at problem work sites, can be put to other uses.\n\nLast year, reporters at the Washington Post dug through ten years of Maryland Workers Compensation Commission data as they probed the Prince George's County Police Department. [For details, see \"Detailing Questionable Police Shootings Required Extensive Search of Documents\" in the March-April 2002 issue of The IRE Journal.] While examining fatal shootings by police, the reporters mined the data for cases where police officers filed disability claims because of the stress and emotional trauma related to the shootings. One field in the database covering 700,000 unique claims sometimes contained a detailed narrative of the shootings. After identifying the cases, the reporters obtained the original case documents on file with the commission.\n\nPost reporters used the data again when they were looking at out-of-control police dogs. This time, the reporters queried for cases where police filed disability claims because they were bitten by department canines.\n\nOther journalists have put workers compensation data to use in more traditional ways. In 1997, Mary Fricker and Bob Norberg, reporters at the Santa Rosa Press Democrat analyzed workers comp data and unmasked claims that fraud was behind one third of the disputed cases.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-march-april-upcoming-car-training","slug":"2002-march-april-upcoming-car-training","url":"https://dwillis.github.io/uplink/article/2002-march-april-upcoming-car-training.html","year":2002,"month":"March-April","monthNum":3,"headline":"Upcoming CAR Training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering multiple journalism training workshops in spring and summer 2002, including advanced statistics, GIS mapping, religious census data, and CAR boot camps for reporters seeking to improve their computer-assisted reporting and data analysis skills.","keywords":["training","statistics","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["SPSS","ArcView","spreadsheet"],"full_text":"Advanced Statistics Workshop, June 9-13 - Chapel Hill, N.C CAR guru and Knight Chair in Journalism Professor Philip Meyer once again will present an advanced statistics workshop with IRE and NICAR. The workshop strengthens the skills of reporters who want to move beyond basic computer-assisted reporting and use social research methods and statistical analysis in their work, including understanding and using regressions, indexes, sampling and polling. Participants are introduced to SPSS, a statistical software package. Other instructors include Sarah Cohen of The Washington Post and IRE Executive Director Brant Houston.\n\nMapping the Census Seminar, May 3-5 - Columbia, Mo. IRE and NICAR will offer this intensive hands-on training using ArcView GIS software, with a focus on mapping Census data to uncover stories. We will look at noteworthy stories that have used mapping. The sessions will include address geocoding, thematic mapping, overlaying and spatial analysis. Trainers include Andy Lehren of Dateline NBC, Jennifer LaFleur of the St. Louis Post-Dispatch and David Herzog of NICAR and the Missouri School of Journalism. Participants are encouraged to bring their own data to work on during open lab time.\n\nUsing the Religious Census, April 10-11, 2002 - Washington, D.C. Religion Newswriters Association Workshop at National Press Club: Reporters will learn how to use data from Glenmary Research Institute's religious census, which will be released in May or early June. The Census is done every 10 years by the Institute, a religious-based research organization. The Census Bureau often refers reporters to this organization since they are barred from asking religious questions. The religious census contains information relating to the number of churches and total church membership by denomination by state and county.\n\nCAR Boot Camps\n\nMarch 24-29 and May 19-24 - Columbia, Mo. IRE and NICAR are offering two CAR boot camps in Missouri this spring with special tracks. The six-day boot camps each open with special sessions on campaign finance and each offer four fellowships for journalists at small- to medium-sized news organizations who are interested in improving their coverage of elections. The May boot camp also will have special sessions on using Census data.\nAug. 4-9 - Columbia, Mo. A special track for broadcast journalists will be featured at this Boot Camp. Learn how to acquire electronic information, use spreadsheets and databases to analyze the information and to translate that information into high-impact stories. In addition, NICAR provides follow-up help when participants return to their news organizations.\n\nFor registration and fellowship information, go to www.ire.org/training/otr.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-mayjune-bits-and-bytes","slug":"2002-mayjune-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2002-mayjune-bits-and-bytes.html","year":2002,"month":"May/June","monthNum":5,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library recently added new datasets regarding truck inspections and federal grants, while updating existing databases on boating accidents, AIDS cases, and aircraft maintenance.","keywords":["databases","trucking","boating"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"The IRE and NICAR Database Library completed two additions of its government database collection: the Department of Transportation's database of truck inspections and data files from the Census Bureau's Federal Assistance Award Data System (FAADS). The truck inspections database includes seven relational tables, covering millions of inspections and violations across the United States. The library offers the data nationwide or by state. The FAADS files include tables for each fiscal year, covering grants and other financial assistance, including the recipient and location, plus a description of the program. FAADS files, too, are offered both nationwide or by state. The U.S. Recreational Boat Accident Database includes 7,740 accidents in 2000, obtained in April from the U.S. Coast Guard. The database consists of four relational tables containing data on each accident. Details for each accident include when and where the accident occurred, its causes and details of deaths or injuries. Those purchasing the database from the Database Library will also receive data files covering 1969 through 1999. The AIDS Public Information Data Set contains information about AIDS cases reported to state and local health departments since 1981. These agencies share this information with the Centers for Disease Control. The data set includes each AIDS patient and summary information by state, metropolitan area, mode of exposure to HIV, sex, race/ethnicity, age, vital status and case definition category. Service Difficulty Reports consist of maintenance incidents collected by the Federal Aviation Administration for the purpose of tracking repair problems with private, commercial and military aircraft and aircraft components. The data is reported by tail number and aircraft serial number, so it is possible to trace the maintenance history of a particular airplane with this database. To find out more about these or other databases, visit the Web page www.ire.org/datalibrary/ or call 573-884-7711. To suggest articles for Uplink, send your e-mail to uplink@nicar.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-mayjune-databases-highlight-tired-truckers","slug":"2002-mayjune-databases-highlight-tired-truckers","url":"https://dwillis.github.io/uplink/article/2002-mayjune-databases-highlight-tired-truckers.html","year":2002,"month":"May/June","monthNum":5,"headline":"Databases highlight tired truckers","kicker":"TRANSPORTATION","author_name":"Greg Reeves","author_title":"","affiliation":"The Kansas City Star","summary":"Reporters at The Kansas City Star utilized multiple databases to investigate and report on the fatal consequences of long-haul truck driver fatigue.","keywords":["trucking","fatigue","transportation"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":[],"full_text":"Investigating fatigue among over-the-road truckers last year was frustrating work: we had to analyze not one, but several databases, all of them missing crashes and key pieces of information. But when we were done, special projects reporter Judy Thomas and I were able to piece together enough solid information to show the fatal consequences of truckers driving tired. In \"Dead Tired,\" a three-day series that ran in The Kansas City Star starting Dec. 16, we reported that the long hours truckers log on the road contributed to hundreds of deaths every year.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-mayjune-a-conference-report","slug":"2002-mayjune-a-conference-report","url":"https://dwillis.github.io/uplink/article/2002-mayjune-a-conference-report.html","year":2002,"month":"May/June","monthNum":5,"headline":"A conference report","kicker":"SPOTLIGHT: NICAR IN PHILADELPHIA","author_name":"David Herzog","author_title":"Missouri School of Journalism and NICAR","affiliation":"NICAR","summary":"Over 400 journalists attended the annual IRE and NICAR conference in Philadelphia to participate in hands-on classes and presentations focused on data analysis skills.","keywords":["NICAR","conference","training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet"],"full_text":"More than 400 journalists passionate about computer-assisted reporting in March gathered in Philadelphia to trade tips, brush up on their skills and network. With newsrooms across the country slashing training budgets, many of the journalists attending IRE and NICAR's annual computer-assisted reporting conference paid their own way, and invested in their own careers. Journalists packed 55 hands-on classes, learning everything from basic spreadsheet skills to statistical analysis and advanced programming tricks. And they had the chance to sample from more than five dozen presentations, panels and roundtable discussions aimed at broadcast, print and online journalists. John Allen Paulos, the Temple University mathematician and author of \"A Mathematician Reads the Newspaper,\" kicked off the four-day conference on March 14 with a spoken tour of his book, outlining the numeric pitfalls that journalists have fallen into.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-mayjune-car-projects-garner-top-honors","slug":"2002-mayjune-car-projects-garner-top-honors","url":"https://dwillis.github.io/uplink/article/2002-mayjune-car-projects-garner-top-honors.html","year":2002,"month":"May/June","monthNum":5,"headline":"CAR projects garner top honors","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Washington Post and a University of Missouri student received top investigative reporting honors for data-driven projects on child protection failures and storm water permitting.","keywords":["awards","investigative","journalism"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":[],"full_text":"Sari Horwitz, Scott Higham and Sarah Cohen of The Washington Post won a prestigious Investigative Reporters and Editors medal for exploring the deaths of children in the District of Columbia. For the same series, the Post team won the 2002 Pulitzer Prize for investigative reporting. A student IRE Award went to University of Missouri School of Journalism graduate Mary Jo Sylvester, now at the Center for Public Integrity. Both Sylvester and Post database editor Cohen have worked with IRE and NICAR. Cohen is a former training director for IRE and NICAR. Sylvester, as a graduate student, served as IRE and NICAR data library administrator. She's now the database editor for the Center for Public Integrity. The Post team discovered that 229 children died during a seven-year period after their dangerous family situations came to the attention of the district's child protection system. Despite strict confidentiality laws, the team pieced together records for 180 of those deaths, compiled them into a database and found that one in five – mostly infants and toddlers – lost their lives after government workers failed to take key preventive action or placed the children in unsafe homes or institutions. \"The project, which has resulted in wide-ranging reforms, answers the highest call of investigative journalism,\" the IRE Award judges said in granting the medal. Sylvester won the student award for \"A Flood of Problems.\" Her package for the Columbia Missourian exposed problems with a local storm water permitting system, the consequences for residents and why readers should care. The May-June 2002 edition of The IRE Journal includes a complete listing of winners. The annual awards of IRE – a 4,200 member professional organization – recognize outstanding investigative work in print, broadcast, online media and for work furthering freedom of information. The awards will be presented during a June 1 luncheon at the IRE Annual Conference in San Francisco.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-mayjune-databases-add-depth-to-pool-stories","slug":"2002-mayjune-databases-add-depth-to-pool-stories","url":"https://dwillis.github.io/uplink/article/2002-mayjune-databases-add-depth-to-pool-stories.html","year":2002,"month":"May/June","monthNum":5,"headline":"Databases add depth to pool stories","kicker":"CONSUMER REPORTING","author_name":"Mike Sherry","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists can use U.S. Consumer Product Safety Commission databases to provide detailed insights into swimming pool accidents and other consumer product related injuries.","keywords":["consumer safety","CPSC","swimming pools"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["Access"],"full_text":"It's a good bet your news organization will soon be doing the quick-hit pool opening story. Why not add depth to an otherwise shallow story? You can do that – and many other stories – with U.S. Consumer Product Safety Commission (CPSC) death and injury data. The data should be manageable for anybody with a basic knowledge of software such as Microsoft Access, and you can get a decade's worth for free. Along with swimming pools, the data include incidents involving thousands of other consumer products – from air conditioners to vacuum cleaners. Many of your readers, viewers or listeners might not realize how dangerous swimming can be. Of the more than 44,000 deaths included in CPSC data running from January 1990 to October 2001, more than 5,300 records cited swimming pools as the main product involved with the death. Other oft-cited products include bathtubs and showers and stairs and steps. Swimming deaths increase when slides and other related products are taken into account. The death database is just one of four CPSC provides. Also available are the National Electronic Injury Surveillance System (NEISS), Injury/Potential Injury Incident File (IPII), and In-depth Investigations (INDP). Like the death data, the narrative fields are one of the best features of these three datasets. Additionally, the IPII and INDP tables include the city and state where the incident occurred. The incident and NEISS tables can be linked with the investigation table through a unique identifier known as the task number. When dealing with the age fields, keep in mind that CPSC uses a three-digit convention for children under the age of 2. For example, a child that is 23 months old is listed in the data as \"223.\" Each of the four databases includes multiple product fields. These fields just tell which products were involved with the injury or death.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-mayjune-readmetxt","slug":"2002-mayjune-readmetxt","url":"https://dwillis.github.io/uplink/article/2002-mayjune-readmetxt.html","year":2002,"month":"May/June","monthNum":5,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article warns journalists about limitations in SF3 census data, including imputation methods for missing responses and significant error margins that are larger for smaller geographic areas.","keywords":["census","imputation","uncertainty"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"Cautionary notes about SF3 data\n\n1. This data comes from the census long form, which about one household in six fills out. If the school enrollment question isn't answered despite the best efforts of census-takers, the Census Bureau will use rules it has tested to fill in the database based on known characteristics of the household and its neighborhood. To see what percentage of any area you're studying has this imputed data for this question, see table P110.\n\n2. Because this data comes from a survey, each number has a little cloud of uncertainty around it, just like an opinion poll. We usually call these \"error margins.\" The smaller the geographical area you're studying, the bigger the cloud is relative to the totals. Make sure you're not overstating what the data says by understanding that error margin. Use the SF3 documentation to compute the applicable margin, or get help from a local statistics professor.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-mayjune-resources-for-consumer-related-stories","slug":"2002-mayjune-resources-for-consumer-related-stories","url":"https://dwillis.github.io/uplink/article/2002-mayjune-resources-for-consumer-related-stories.html","year":2002,"month":"May/June","monthNum":5,"headline":"Resources for consumer-related stories","kicker":"CONSUMER REPORTING","author_name":"Ian Demsky","author_title":"","affiliation":"IRE and NICAR","summary":"This article highlights investigative resources available through IRE and NICAR, including databases on medical device failures, physician malpractice, and vehicle recalls.","keywords":["medical devices","malpractice","vehicle recalls"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["MAUDE","National Practitioner Data Bank"],"full_text":"IRE and NICAR offer several databases, stories and tipsheets for reporters investigating consumer-related issues. The Manufacturer and User Facility Device Experience (MAUDE) database from the Food and Drug Administration contains information about problems that occurred with medical devices. The problems include injuries, deaths or product malfunctions. The data include detailed narratives about the device failures. Specific devices can be searched to determine how often they're reported to have failed. Patient-specific and hospital-specific information has been removed for privacy reasons. The National Practitioner Data Bank from the Department of Health and Human Services has information about malpractice payments, licensure, clinical privileges, professional society membership and adverse action reports concerning physicians, dentists and other licensed health-care professionals. Practitioner- and hospital-specific information is excluded from the file. Despite that, the Hartford Courant used the data and a variety of public records to examine several doctors who continued to practice medicine despite being sued for malpractice. The vehicle recalls and complaints database from the National Highway Transportation Safety Administration contains information about vehicles that have been recalled by manufacturers and consumer complaints about them dating back to 1966. About 75 percent of recalls are initiated by the manufacturers, because of problems – identified or perceived – with a vehicle. For the rest, NTSHA contacts the manufacturer after receiving complaints, although the agency itself never issues recalls on its own.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-mayjune-readmetxt-2","slug":"2002-mayjune-readmetxt-2","url":"https://dwillis.github.io/uplink/article/2002-mayjune-readmetxt-2.html","year":2002,"month":"May/June","monthNum":5,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article explains important limitations of SF3 census data, including that it comes from a sample survey with imputed values and statistical margins of error that are larger for smaller geographic areas.","keywords":["census","data","uncertainty"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"Cautionary notes about SF3 data\n\n1. This data comes from the census long form, which about one household in six fills out. If the school enrollment question isn't answered despite the best efforts of census-takers, the Census Bureau will use rules it has tested to fill in the database based on known characteristics of the household and its neighborhood. To see what percentage of any area you're studying has this imputed data for this question, see table P110.\n\n2. Because this data comes from a survey, each number has a little cloud of uncertainty around it, just like an opinion poll. We usually call these \"error margins.\" The smaller the geographical area you're studying, the bigger the cloud is relative to the totals. Make sure you're not overstating what the data says by understanding that error margin. Use the SF3 documentation to compute the applicable margin, or get help from a local statistics professor.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-mayjune-using-census-data-for-non-census-stories","slug":"2002-mayjune-using-census-data-for-non-census-stories","url":"https://dwillis.github.io/uplink/article/2002-mayjune-using-census-data-for-non-census-stories.html","year":2002,"month":"May/June","monthNum":5,"headline":"Using census data for non-census stories","kicker":"SPOTLIGHT: NICAR IN PHILADELPHIA","author_name":"Paul Overberg","author_title":"","affiliation":"USA Today","summary":"Paul Overberg explains how journalists can use the Census 2000 Summary File 3 to research school enrollment trends, private school popularity, and neighborhood demographics.","keywords":["census","education","demographics"],"topics":["Demographics & Census","Education","Investigative Methods"],"technologies":[],"full_text":"If you're looking for local school stories, it's easy to be put off by complex census tables. But this data will be published this summer for areas as big as states and as small as block groups in Census 2000 Summary File 3 data files. Usually census tracts, areas that average 1,500 homes, are the smallest standard areas worth analyzing. The school enrollment data list is a good example of how census data can yield lots of non-census stories. Instead of looking at the data for stories, bring your knowledge of a subject – like your community's schools – to this data. See if this data can back up or knock down your hunches and ideas. Why do I need census enrollment data when I get it each year from the school district? No school superintendent has all the data on private schools or home schooling. School systems collect data by school, not neighborhood. Census data from 1990 and 2000 lets you see broad patterns comparable around the state and nation. If you subtract an area's population age 18 and over and ages 3 to 5 from the sum of the two \"not enrolled\" categories, you get kids ages 5 to 18 who aren't in any school. You can calculate the public-private ratio for each neighborhood and map it to find school systems in trouble. Look at enrollment for graduate programs to see trends in adult education. You can also identify college town student concentrations or explore how magnet and charter schools are changing neighborhoods. For deeper analysis, compare private school enrollment percentages with neighborhood poverty rates to find unexpected patterns. This data comes from the census long form, which about one household in six fills out. Because this data comes from a survey, each number has an error margin that should be understood to avoid overstating what the data says.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-mayjune-readmetxt-3","slug":"2002-mayjune-readmetxt-3","url":"https://dwillis.github.io/uplink/article/2002-mayjune-readmetxt-3.html","year":2002,"month":"May/June","monthNum":5,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article warns that SF3 census data on school enrollment comes from a sample survey with imputed values and significant error margins, particularly for smaller geographic areas, requiring careful interpretation.","keywords":["census","uncertainty","imputation"],"topics":["Data & Databases","Demographics & Census"],"technologies":[],"full_text":"Cautionary notes about SF3 data\n\n1. This data comes from the census long form, which about one household in six fills out. If the school enrollment question isn't answered despite the best efforts of census-takers, the Census Bureau will use rules it has tested to fill in the database based on known characteristics of the household and its neighborhood. To see what percentage of any area you're studying has this imputed data for this question, see table P110.\n\n2. Because this data comes from a survey, each number has a little cloud of uncertainty around it, just like an opinion poll. We usually call these \"error margins.\" The smaller the geographical area you're studying, the bigger the cloud is relative to the totals. Make sure you're not overstating what the data says by understanding that error margin. Use the SF3 documentation to compute the applicable margin, or get help from a local statistics professor.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-mayjune-intranet-tools-and-preparing-for-disaster","slug":"2002-mayjune-intranet-tools-and-preparing-for-disaster","url":"https://dwillis.github.io/uplink/article/2002-mayjune-intranet-tools-and-preparing-for-disaster.html","year":2002,"month":"May/June","monthNum":5,"headline":"Intranet tools and preparing for disaster","kicker":"SPOTLIGHT: NICAR IN PHILADELPHIA","author_name":"Tom Torok","author_title":"Computer-Assisted Reporting Developer","affiliation":"The New York Times","summary":"Tom Torok describes how The New York Times used an intranet to manage information after September 11 and provides guidelines for building newsroom tools that allow for decentralized updates.","keywords":["intranet","breaking news","management"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Atex","World Wide Web","HTML","spreadsheet"],"full_text":"The New York Times taxed traditional reporting tools to the limit shortly after the first plane slammed into the World Trade Center.\n\nWithin 24 hours, more than 600 files of story notes accumulated in the newsroom's Atex computer system metro queue. There were more related files in the foreign, national and business queues. Searching for any snippet of information became increasingly difficult and slow as torrents of information poured in.\n\nSome reporters tried other means of organization. One started a spreadsheet to track suspects that soon grew to monstrous proportions. The spreadsheet was difficult to read and almost impossible to print. Attempts to share the spreadsheet over the network compounded the unwieldiness.\n\nA terrorism team was assembled and editors had trouble tracking more than 80 reporters and their assignments, phone numbers and e-mail addresses.\n\nI didn't know it at the time, but I had an intranet ready for a story of the magnitude of Sept. 11.\n\nManaging the information\n\nA web application to search the accumulating notes was ready the next morning.\n\nA virtual online newsroom began to take shape by the second afternoon, with applications that allowed lightning-fast searches of accumulated notes, suspects, reporters, phone numbers – anything that involved Sept. 11. Reporters and editors could add or update information whenever they wanted, from wherever they wanted.\n\nThe intranet within an intranet wasn't always the perfect tool – no such tool exists for what had happened — but it did help significantly in managing information.\n\nIn all, about 20 pages were created to deal with the coverage. But with the exception of one of those 20 pages, I either already had similar pages that could be modified to suit the task or had a program ready to write the code needed.\n\nEvery \"borrowed\" Web page needed tinkering – some needed extensive rewrites – but anything I had on hand was better than starting from scratch. And what I had on hand was a virtual newsroom waiting for a project.\n\nMany of the pages were written for coverage of the 2000 national party conventions, where they were never used. The Sept. 11 pages were better received than the convention pages. In the first two months, the newsroom was calling up those related pages between 2,200 and 2,600 times a day.\n\nPages for preparation\n\nRather than enumerate which pages the Times used for Sept. 11 coverage, let's concentrate on pages that you should have ready for the next breaking story or project, be it big or small. To start, you want to prepare four pages to display information:\n\n1. A news-brief or announcements page on which information important to the project can be posted.\n\n2. A whereabouts page that will allow anyone involved in the project to find reporters, editors and assignments quickly.\n\n3. A text-search page to search collected text or HTML documents related to the project.\n\n4. An events page to track events related the project, such as press conferences or meetings.\n\nBefore you create a monster, ask yourself some questions about these pages.\n\nWho will keep them current? Who can add breaking information? Who can delete information? Who can edit embarrassing typos? If all answers point to you, you're heading for trouble before you start.\n\nTake a look around the newsroom. A good look. Is there anyone who can edit a Web page while you're sleeping, on vacation or on deadline? Probably not.\n\nAs important as you are to your newsroom's computer-assisted reporting efforts, you are not indispensable. In fact, you will become a bottleneck if your intranet cannot be brought up to date without you. Talented writers and news gatherers surround you. Why not marshal their talents to help? Probably all but two people in your newsroom can type into a Web input box. (The other two can't do anything with computers. Accept that and get on with life.)\n\nTo bring more hands into the process, try following a few guidelines that will help keep your intranet current:\n\n• Guideline 1. Do not post anything on your intranet unless there is some mechanism (or persons in addition to yourself) in place to keep the information current.\n\n• Guideline 2. Whenever possible, construct information pages so they are fully and easily editable by any persons you choose at anytime they choose.\n\nWe're taking it as a given that no one else in your newsroom knows HTML, so we've got to add a few more guidelines to make 1 and 2 practical:\n\n• Guideline 3. Use databases to drive virtually all of your content that is not archived text.\n\n• Guideline 4. Use interactive templates to display the contents of the databases.\n\nWith these guidelines in mind, you'll need one more page in your basic package to allow selected users to upload new text files for your text search page. You'll also need at least three more pages to make the announcements, whereabouts and events pages fully editable on the Web. The interactive templates for the latter tasks are complicated, tedious to build and difficult to debug. In that light, consider another guideline:\n\n• Guideline 5. Use one or more programs to create the code to","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-july-august-bits-and-bytes","slug":"2002-july-august-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2002-july-august-bits-and-bytes.html","year":2002,"month":"July-August","monthNum":7,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Investigative Reporters and Editors will hold a regional conference in Savannah, Georgia on October 19-20, featuring computer-assisted reporting training and investigative journalism sessions, along with updates to several databases including federal spending, contracts, and crime reports.","keywords":["training","databases","reporting"],"topics":["Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Training in Savannah\n\nInvestigative Reporters and Editors will offer computer-assisted reporting training during its regional conference in Savannah, Ga., Oct. 19-20. The hands-on CAR workshop will be 9 a.m. to noon Sunday, Oct. 20. There will be a limited number of registration slots open for the basic skills workshop. The investigative reporting sessions on Saturday will spotlight nut-and-bolts reporting skills and hot issues in the South.\n\nFor updated information on speakers and the conference schedule, go to www.ire.org/training/savannah02/\n\nData updates:\n\nThe IRE and NICAR Database Library recently updated the following databases:\n\nFederal spending\n\nThe Consolidated Federal Funds Report database, current as of fiscal year 2001, is a valuable tool for exploring federal expenditures in detail. Using the data, collected by the U.S. Census Bureau, journalists can look at federal assistance and overlay that information with demographics and housing about counties and cities. The database shows how much money the federal government has spent. It covers federal expenditures or obligations for the following categories: grants, salaries and wages, procurement contracts, direct payments to individuals, other direct payments, direct loans, guaranteed or insured loans, and insurance.\n\nFederal contracts\n\nThe federal contracts database provides details about more than 560,000 contracts, including the dollar amount, contracting agency and the type of product or service involved. The database contains information about each federal contract worth more than $25,000. For a few ideas about using the database see page 9.\n\nUniform Crime Reports\n\nThe FBI Uniform Crime Reports database is now current through 2000. Most of the data consists of the \"index\" crimes: murder, non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny/theft, motor-vehicle theft and arson.\n\nIn addition, the database provides details about property crimes and includes the dollar values of the crimes. Additional data show age, sex and race information of people arrested.\n\nThe Supplemental Homicide Report table provides more detailed information on homicides, including demographic information about the victim and offender.\n\nTo order databases, contact NICAR at 573-884-7711, or go to www.ire.org/datalibrary/databases/ for more information.\n\nmuch money the federal government has spent. It covers federal expenditures or obligations for the following categories: grants, salaries and wages, procurement contracts, direct payments to individuals, other direct payments, direct loans, guaranteed or insured loans, and insurance.\n\nFederal contracts\n\nThe federal contracts database provides details about more than 560,000 contracts, including the dollar amount, contracting agency and the type of product or service involved. The database contains information about each federal contract worth more than $25,000. For a few ideas about using the database see page 9.\n\nUniform Crime Reports\n\nThe FBI Uniform Crime Reports database is now current through 2000. Most of the data consists of the \"index\" crimes: murder, non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny/theft, motor-vehicle theft and arson.\n\nIn addition, the database provides details about property crimes and includes the dollar values of the crimes. Additional data show age, sex and race information of people arrested.\n\nThe Supplemental Homicide Report table provides more detailed information on homicides, including demographic information about the victim and offender.\n\nTo order databases, contact NICAR at 573-884-7711, or go to www.ire.org/datalibrary/databases/ for more information.\n\nmuch money the federal government has spent. It covers federal expenditures or obligations for the following categories: grants, salaries and wages, procurement contracts, direct payments to individuals, other direct payments, direct loans, guaranteed or insured loans, and insurance.\n\nFederal contracts\n\nThe federal contracts database provides details about more than 560,000 contracts, including the dollar amount, contracting agency and the type of product or service involved. The database contains information about each federal contract worth more than $25,000. For a few ideas about using the database see page 9.\n\nUniform Crime Reports\n\nThe FBI Uniform Crime Reports database is now current through 2000. Most of the data consists of the \"index\" crimes: murder, non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny/theft, motor-vehicle theft and arson.\n\nIn addition, the database provides details about property crimes and includes the dollar values of the crimes. Additional data show age, sex and race information of people arrested.\n\nThe Supplemental Homicide Report table provides more detailed information on homicides, including demographic information about the victim and offender.\n\nTo order databases, contact NICAR at 573-884-7711, or go to www.ire.org/datalibrary/databases/ for more information.\n\nmuch money the federal government has spent. It covers federal expenditures or obligations for the following categories: grants, salaries and wages, procurement contracts, direct payments to individuals, other direct payments, direct loans, guaranteed or insured loans, and insurance.\n\nFederal contracts\n\nThe federal contracts database provides details about more than 560,000 contracts, including the dollar amount, contracting agency and the type of product or service involved. The database contains information about each federal contract worth more than $25,000. For a few ideas about using the database see page 9.\n\nUniform Crime Reports\n\nThe FBI Uniform Crime Reports database is now current through 2000. Most of the data consists of the \"index\" crimes: murder, non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny/theft, motor-vehicle theft and arson.\n\nIn addition, the database provides details about property crimes and includes the dollar values of the crimes. Additional data show age, sex and race information of people arrested.\n\nThe Supplemental Homicide Report table provides more detailed information on homicides, including demographic information about the victim and offender.\n\nTo order databases, contact NICAR at 573-884-7711, or go to www.ire.org/datalibrary/databases/ for more information.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-july-august-ritalin-roulette-state-data-pinpoints-drug-use-by-zip-code","slug":"2002-july-august-ritalin-roulette-state-data-pinpoints-drug-use-by-zip-code","url":"https://dwillis.github.io/uplink/article/2002-july-august-ritalin-roulette-state-data-pinpoints-drug-use-by-zip-code.html","year":2002,"month":"July-August","monthNum":7,"headline":"RITALIN ROULETTE State data pinpoints drug use by ZIP code","kicker":"","author_name":"Mark Skertic","author_title":"","affiliation":"Chicago Sun-Times","summary":"A Chicago journalist describes obtaining prescription-monitoring data from Illinois to report on ADHD drug use in children, discovering that state records showing where patients live—rather than just where prescriptions are filled—provided more accurate geographic data than previously available DEA point-of-sale information.","keywords":["ADHD","prescriptions","data"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"The story sounded so simple when my editor gave me the assignment: Do something on the amount of Ritalin and similar drugs used in the Chicago area to treat children for attention deficit/hyperactivity disorder (ADHD).\n\nIt turned out to be a months-long effort that involved the first time an Illinois newspaper using data from the state's prescription-monitoring program. Of course, when I got the assignment I didn't realize Illinois even collected the data.\n\nSeventeen other states have similar programs - California, Hawaii, Idaho, Indiana, Kentucky, Massachusetts, Michigan, Nevada, New Mexico, New York, Oklahoma, Pennsylvania, Rhode Island, Texas, Utah, Washington and West Virginia.\n\nEach state's program is slightly different from the others, but the overall intent is the same: to collect information about the prescription of controlled substances. Ritalin, Adderall and other ADHD drugs are Schedule II narcotics, so details about their use is collected under these programs. For details on the programs, see the DEA's Web site at www.deadiversion.usdoj.gov/pubs/program/rx_monitor/faqs.htm\n\nIn Illinois, every time a pharmacist fills a prescription for Ritalin, the state gets a record of that transaction electronically. The record includes the name and address of the person who received the drug, his or her birth date, and amount of the drug prescribed, which doctor wrote the script and where it was filled.\n\nThe discovery that Illinois collected data on where people who get prescriptions filled live was a potential gold mine.\n\nData pinpoints drug use\n\nThe Drug Enforcement Administration encourages states to do prescription monitoring because it's helpful in tracking doctor shopping - instances where the same patient visits several physicians in hopes of collecting multiple prescriptions. It's also a good way to see if some doctors are prescribing unusually large amount of certain drugs.\n\nFor reporters it's an excellent way of seeing exactly where some drugs are being used.\n\nMany stories have been done using DEA point-of-sale data, which shows how much Ritalin and other Schedule II narcotics are dispensed by pharmacies. The best use of this data was in 2001 when The (Cleveland) Plain Dealer used it to measure Ritalin usage for every county nationally.\n\nBut that data has severe limitations when you want to measure usage by ZIP code, as we did in Chicago. In many areas, people don't get their prescription filled in the same ZIP code where they live. In Chicago, for example, point-of-sale data shows an unusually high Ritalin-usage rate in the downtown Loop area. But that's only because thousands of people come to work, get their children's prescriptions filled at pharmacies in the parents' office buildings, and then they take it home to the suburbs in the evening.\n\nThe discovery that Illinois collected data on where people who get prescriptions filled live was a potential gold mine. All we had to do was get our hands on the data.\n\nNegotiations and threats\n\nIllinois' Department of Human Services initially told the Chicago Sun-Times it would cost $6,000 to provide information on where people were being prescribed Ritalin. When I called back to ask why the charge was so outrageously high, I was told that someone was going to have to write a program to extract that information from the prescription-monitoring database. (It was at this point that we realized Illinois even had a prescription-monitoring program).\n\nFine, we told them. Just send us the entire database and we'll extract what we need. That, the state told us, was impossible.\n\nWe insisted they were mistaken, and after several more telephone calls, a few letters and promises of legal action, the state agreed with us - to a point. Based on our negotiations with them, the department agreed to give us portions of the data we were after.\n\nNames of patients and doctors weren't provided, nor were pharmacies. The state also refused to provide data for all counties in Illinois, arguing that in some counties one person had received a particular drug - and we could theoretically track down that patient, thus violating his right to privacy.\n\nIn the end, we agreed to request data only from the ZIP codes in the six counties in northeast Illinois - the region where the vast majority of the state lives and where the Sun-Times sells most of its newspapers. A few weeks later, a CD packed with data arrived in my mailbox.\n\nFalse steps and analysis\n\nThere were a few problems. The data didn't include dates of birth, only patient ages (another concession we agreed to because of the state's fears that patients could be identified). It looked like there were a lot of babies receiving unusually large doses of some drugs.\n\nA call to the state explained the problem. Anytime a record lacked a DOB, the programmers inserted a zero. That made analyzing the amount of drugs given infants impossible.\n\nAt about this point the project was sidelined by Sept. 11 coverage. Ultimately, we asked the state to re-run the data, fix the age field (those missing DOBs were now labeled with 999) and update it with more numbers collected since our last request. Soon after, we had a CD with 641,735 records. Each line was 98 characters long.\n\nThe data imported into Paradox (version 9.0) without a problem. The record layout the state provided made parsing the lines of data simple. But that didn't make it easy to read.\n\nThe data had no drug names, only National Drug Code numbers. The number tells you exactly what the drug is and how strong a dose was prescribed. For example, 9042769 is a 10-milligram dose of methylphenidate hydrochloride (the generic form of Ritalin) in tablet form.\n\nThe state's data had 457 different NDC numbers. That doesn't mean there were 457 different drugs. In many cases, the same drug has multiple NDC numbers. For example, the NDC number 7811749 also indicates the patient received 10-milligram methylphenidate hydrochloride tablets.\n\nTo make sense of it all, we downloaded the NDC dictionary from the DEA (PC and Mac versions are at www.deadiversion.usdoj.gov/arcos/ndc/ndc_dic.htm.) By joining this data to the state's, we had a database that showed at a glance what drugs were prescribed and at what dosage.\n\nFinally, we needed to know how many people lived in each ZIP code and how many fell into different age groups. We used Census ZIP Code Tabulation Area population. It's available from American Factfinder on the Census Bureau Web site www.census.gov.\n\nWith this new, enhanced database - it now included the state's data, drug names and population figures - we did some relatively simple calculations. Looking at Ritalin, for example, we added up all the milligrams of Ritalin prescribed to people under age 18 in each ZIP code, then divided by the number of people under age 18 living there.\n\nThis ratio helped us learn where the largest proportions of certain populations were receiving the drug.\n\nMapping the concentration\n\nUsing ArcView 3.2, we opened ZIP code shape files for Illinois, and linked our state data to that on the \"ZIP Code\" fields. Then we mapped the ratio.\n\nEven someone with only a passing familiarity with the Chicago area could see the patterns: ZIP codes in the north and far west suburbs - traditionally some of the area's wealthiest - were the same places were large concentrations of the under age 18 population received these drugs.\n\nMost of the city, as well as the south suburbs, had relatively low usage rates.\n\nWe ran a lot of those ratios - for specific drugs and also for all forms of methylphenidate. We also used it to examine usage of ADHD drugs such as Adderall and Dexedrine. Generally, the results were consistent - wealthier areas have larger concentrations of the drugs being used.\n\nAt this point a lot more traditional reporting began, with calls to doctors and parent groups, as well as visits to schools and meetings of support groups for parents with ADHD children.\n\nAnd while many of them had anecdotal observations about where usage was high or low, we had concrete numbers that helped focus the story on ADHD in a way that hadn't been done locally before.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-july-august-covering-money-votes-and-power","slug":"2002-july-august-covering-money-votes-and-power","url":"https://dwillis.github.io/uplink/article/2002-july-august-covering-money-votes-and-power.html","year":2002,"month":"July-August","monthNum":7,"headline":"Covering money, votes and power","kicker":"SPOTLIGHT: CAMPAIGN FINANCE","author_name":"Aron Pilhofer","author_title":"","affiliation":"Uplink","summary":"As the Bipartisan Campaign Reform Act takes effect in November 2002, this journalism newsletter provides resources and reporting guidance on covering campaign finance, including data sources, analytical tools, and expert tipsheets for understanding the overhaul of federal campaign finance regulations.","keywords":["campaign","finance","reporting"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["GIS","FoxPro","Internet","World Wide Web"],"full_text":"Campaign finance has been big news of late, whether it's the gushing generosity of Enron executives, the major parties' insatiable appetite for soft money or the rise of millionaire candidates funding $100 million campaigns.\n\nAnd this fall comes the main event: On Nov. 6, the Bipartisan Campaign Reform Act - the first major overhaul of the federal campaign finance system in a quarter century - will kick in.\n\nAlthough the new law only applies to federal candidates and committees, it will affect all levels - county, state and local. For example, many observers predict that state and local committees will be the new conduits for the hundreds of millions in soft money the national parties will be prohibited from raising under the new law.\n\nBut first things first. Election 2002 is an especially crucial midterm election, with a closely divided electorate and a Congress reflecting that divide. Both parties believe they can win a majority in the House and Senate, and the national party committees are raising record amounts of hard and soft money.\n\nIn short, there will not be a dearth of storylines heading into the fall campaign season. And with that in mind, this issue of Uplink is intended to help add depth to your reporting on campaign finance.\n\nDavid Gulliver of The Virginian-Pilot covers the trend of independently wealthy candidates.\nDavid Herzog of IRE/NICAR shows how some journalists have used GIS software in their reporting to show visually, for example, the pockets of power.\nHank Shaw, formerly of The (Fredericksburg, Va.) Free Lance-Star writes about ways campaign finance data can be used to understand more about the interests backing a particular candidate, including the little used expenditure data.\nGriff Palmer of the San Jose Mercury News demonstrates how to use FoxPro's scripting capabilities to automate some of the more laborious tasks, such as downloading and importing campaign finance data.\nAnd I discuss the wealth of data to be found at the Federal Election Commission's new electronic filing Web site, and how easy it is to download, import and analyze it.\n\nThis is just a fraction of what's available through IRE, the Campaign Finance Information Center and other organizations.\n\nFor data, tipsheets and other information, visit the CFIC Web site at www.campaignfinance.org. Our Resource Center includes tipsheets written by some of the best in the business.\n\nTracker, our quarterly newsletter, is chock full of practical information and advice from reporters who know the issue inside and out. And of course, the NICAR Database Library has up-to-date data for every election cycle since 1991-92 in an easy-to-import format.\n\nThe FEC (www.fec.gov) has posted a number of easy-to-understand documents and reports on its Web site that will help reporters understand the complexities of the system. Links to your state elections offices can be found on the CFIC site (www.campaignfinance.org/states/index.html).\n\nFor data on the Web, Center for Responsive Politics (www.opensecrets.org) has one of the best searchable databases of federal data. The National Institute on Money in State Politics (www.followthemoney.org) offers the same sorts of lookups, but for state candidates and committees.\n\nTo learn more about how the law will change when the reform law takes effect, visit the Campaign Finance Institute (www.cfinst.org), which has done an outstanding synopsis and analysis of the new law. Common Cause (www.commoncause.org) and Public Citizen (www.publiccitizen.org) also have written extensively on the subject.\n\nLastly, there is one trend that reporters at all levels should be aware of: the rise of political nonprofits, sometimes called Section 527 organizations. The groups, along with state and local party committees, are expected to be the loophole allowing soft money back into the federal system. They also are becoming more common in local and state elections. For an extensive history, download the Common Cause report \"Under the Radar,\" available on its Web site. Public Citizen also has done three studies examining the problems of disclosure and policing of 527s, which ranges from poor to nonexistent. Those reports also are available on Public Citizen Web site.\n\nThe campaign finance issue will be big news for some time to come, and covering elections and the issues surrounding them is only going to get more complex as the parties and candidates must get more creative in finding ways to fund their elections. The resources above are a good start.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-data-access-year-after-attacks-files-still-guarded","slug":"2002-september-october-data-access-year-after-attacks-files-still-guarded","url":"https://dwillis.github.io/uplink/article/2002-september-october-data-access-year-after-attacks-files-still-guarded.html","year":2002,"month":"September - October","monthNum":9,"headline":"DATA ACCESS Year after attacks files still guarded","kicker":"DATA ACCESS","author_name":"Jennifer LaFleur","author_title":"","affiliation":"Reporters Committee for Freedom of the Press","summary":"Federal agencies removal of information from web sites post-9/11 hampers journalists' access, with new legislation potentially restricting more data.","keywords":["data access","post-9/11","government restriction"],"topics":["Data & Databases","Government & Politics","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"After Congress passed the 1996 amendments to the Freedom of Information Act, government agencies began putting more data and documents on their Web sites. But that trend reversed after Sept. 11, 2001, when several federal agencies moved to take down maps, databases, and entire Web sites from the public domain citing security reasons. Journalists no longer could get to the data they had used before in news stories. Instead of the usual reports and information on Web sites, many users instead found messages saying that sites were down or information was unavailable. The U.S. Army's Redstone Arsenal posted a message about portions of their homepage being temporarily blocked. Some information has been returned to government Web sites, but much continues to be unavailable, with additional information either being removed or not updated. Tracking closures, OMB Watch has tracked federal agencies' removal of information, seeing a dramatic immediate shutdown and not much reversal. Databases online earlier like federal agency enforcement data have been withheld citing security concerns, limiting information that had earlier been publicly accessible, highlighted with examples like environmental, pipeline data, and more, creating challenges for both journalists and the public in accessing formerly open records.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-september-october-databases-add-depth-to-school-stories","slug":"2002-september-october-databases-add-depth-to-school-stories","url":"https://dwillis.github.io/uplink/article/2002-september-october-databases-add-depth-to-school-stories.html","year":2002,"month":"September - October","monthNum":9,"headline":"Databases add depth to school stories","kicker":"SPOTLIGHT: EDUCATION","author_name":"Holly Hacker","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists are encouraged to use databases to add depth to school reporting, with the No Child Left Behind Act increasing the need for detailed education stories.","keywords":["education reporting","databases","No Child Left Behind"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"School's back in session, so as students tackle reading and math, journalists can get busy reporting. Whether it's teacher quality or test scores, budgets or bus rides, the education beat offers lots of issues worth exploring with computer-assisted reporting. Nowadays, journalists need to be even more insightful and aggressive in their education reporting. The federal government and all 50 states have put public schools under growing pressure to help kids succeed in the classroom. Most notably is the No Child Left Behind Act signed into law by President Bush this year. It requires states to set academic standards for children at each grade level and test children to see if they're meeting those standards. Some groups of children tend to perform better than others, but in the past those disparities rarely came to light. Now schools must break down test results by race, English proficiency, socioeconomic status, and other variables. Schools and districts must publish report cards with test scores, graduation rates, teacher qualifications, and other data. So for education reporters and CAR junkies, these mandates make for a veritable data-fest with lots of numbers, statistics, and trends to scrutinize.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-september-october-data-shows-cronyism-in-schools","slug":"2002-september-october-data-shows-cronyism-in-schools","url":"https://dwillis.github.io/uplink/article/2002-september-october-data-shows-cronyism-in-schools.html","year":2002,"month":"September - October","monthNum":9,"headline":"Data shows cronyism in schools","kicker":"SPOTLIGHT: EDUCATION","author_name":"Jason Grotto","author_title":"","affiliation":"The Miami Herald","summary":"An investigation reveals how taxpayer dollars benefit lobbyists and contractors instead of students in Miami-Dade County Public Schools.","keywords":["school spending","lobbyists","taxpayer dollars"],"topics":["Government & Politics","Data & Databases","Education"],"technologies":[],"full_text":"The principal of Little River Elementary School wears sneakers to work. Armed with a walkie-talkie and clipboard, she patrols one of the poorest schools in Miami-Dade County, monitoring classrooms, comforting children, and assisting teachers. She also prepares grant applications and solicits corporate donations to buttress the meager resources available for her school, which scores at the bottom on Florida's Comprehensive Assessment Test. The cigar-chomping lobbyist wears expensive suits to work. Using political connections with school board members and a powerful union chief, he scored nearly $300 million in school business for his clients, including a questionable contract that landed him a $4 million payday. Investigative reporting revealed how taxpayer dollars often benefit lobbyists, contractors, and former school board members rather than the district's 365,000 children. The Herald reviewed a decade's worth of district spending helping explain why morale is low and how politics pervades the school system. A database was created pulling in ten years of paper campaign contributions, forming a new database from school district lobbyist registration forms, and obtaining a purchasing database from the school district revealing influential connections and large financial payouts, clouding the goal of school funding for bettering student education.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-september-october-bits-and-bytes-new-and-updated-from-the-database-library","slug":"2002-september-october-bits-and-bytes-new-and-updated-from-the-database-library","url":"https://dwillis.github.io/uplink/article/2002-september-october-bits-and-bytes-new-and-updated-from-the-database-library.html","year":2002,"month":"September - October","monthNum":9,"headline":"Bits & Bytes New and updated from the Database Library","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library has added the Consumer Product Safety Commission Injury and Death database to its collection, offering detailed information about consumer product hazards and safety issues from 1990-2001.","keywords":["safety","products","database"],"topics":["Data & Databases","Investigative Methods","Health & Environment"],"technologies":[],"full_text":"Consumer product safety (new)\n\nWhen you mix everyday life and consumer products, the results can be hazardous to your health. For example, from 1990 through 2001 more than 300 people died after they tripped over a consumer product. Find information like this and detailed information about many consumer products in the newest addition to the IRE and NICAR Database Library's government database collection: the Consumer Product Safety Commission Injury and Death database:\n\nFormer NICAR data analyst Mike Sherry used the CPSC data to investigate elevator safety. His story, \"Missouri Program to Check Elevators Fails to Deliver,\" ran in the July 7 St. Louis Post-Dispatch.\n\nThe nationwide data covering 1990-2001 includes information about potential injuries, deaths and investigations related to consumer products. Some of the products include children's toys, bicycles, swimming pools, ATVs (three- and four-wheelers), sports equipment, hair dryers, smoke detectors, playpens, hobby items, lawn mowers and playground equipment.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-nicar-update-changes-with-uplink-database-library","slug":"2002-september-october-nicar-update-changes-with-uplink-database-library","url":"https://dwillis.github.io/uplink/article/2002-september-october-nicar-update-changes-with-uplink-database-library.html","year":2002,"month":"September - October","monthNum":9,"headline":"NICAR UPDATE Changes with Uplink, Database Library","kicker":"NICAR UPDATE","author_name":"David Herzog","author_title":"Managing Editor","affiliation":"NICAR","summary":"Uplink newsletter announces a redesign with new features like mapping coverage while retaining popular sections, and introduces coordination with The IRE Journal plus four new contributing editors from the University of Missouri.","keywords":["redesign","journalism","database"],"topics":["Journalism & Newsroom","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"If you've been reading Uplink for a while, you've probably noticed some changes over the past year. We redesigned the newsletter and added new features to keep you current with the latest goings-on in computer-assisted reporting. For instance, we now highlight some of the best uses of analytic mapping software in \"Mapping it Out.\" But we haven't thrown out those longstanding features, such as \"Tech Tips,\" that our readers have found invaluable.\n\nLikewise, you'll see some more changes in the issues ahead. One is that we'll coordinate Uplink more with The IRE Journal. That means by 2003 we'll have Uplink and the IRE Journal on the same release schedule. And we'll tie Uplink articles more closely to ones that run in The IRE Journal. So if we have an IRE Journal article about an investigative story, the corresponding issue of Uplink may have a related article that dissects the database work that went into the story.\n\nWe're also moving \"Bits & Bytes\" off Uplink's cover and onto page 2 to make room for one more front-page story. Some of these articles you'll see on the cover will take a look at issues that affect journalists doing computer-assisted reporting. This edition's story about restrictions to data a year after the terrorist attacks of Sept. 11 is but one example.\n\nUplink has four new contributing editors who also are working as University of Missouri School of Journalism graduate research assistants in the NICAR Database Library. They'll write regularly for Uplink and help develop story ideas. The contributing editors are:\n\nMegan Christensen, who is a veteran of the database library, in her second year working on her master's degree and has interned at the Detroit Free Press.\nJaimi Dowdell, another veteran of the database library who just started working on her master's degree this fall.\nHolly Hacker, who just left the St. Louis Post-Dispatch to work on her master's degree. She has reported for the Ventura County (Calif.) Star, the Antioch (Calif.) Ledger-Dispatch and the Gazette Newspapers in Montgomery County, Md.\nStephanie Kang, another newcomer to the database library, has begun working on her master's degree and had interned at the Los Angeles Times.\n\nIn addition, Jeff Porter and I are swapping Uplink duties so Jeff can focus more of his attention on the work of the database library, where business has picked up considerably during the past few months. With this issue, Jeff becomes assistant managing editor and I become managing editor.\n\nThanks for reading Uplink and feel free to e-mail us at uplink@nicar.org if you have any suggestions.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-census-workshops-scheduled-for-reporters-editors","slug":"2002-september-october-census-workshops-scheduled-for-reporters-editors","url":"https://dwillis.github.io/uplink/article/2002-september-october-census-workshops-scheduled-for-reporters-editors.html","year":2002,"month":"September - October","monthNum":9,"headline":"Census workshops scheduled for reporters, editors","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering three workshops led by Paul Overberg and Steve Doig to teach journalists how to analyze Census data and develop stories using resources like the American Community Survey and Public Use Microdata.","keywords":["Census","workshops","journalists"],"topics":["Demographics & Census","Journalism & Newsroom"],"technologies":[],"full_text":"IRE and NICAR have scheduled three workshops for journalists to learn how to use Census data.\n\nPaul Overberg of USA Today and Steve Doig of Arizona State University will lead these sessions Oct. 5-6, at USA Today headquarters in McLean, Va.; Nov. 16-17 in Tempe, Ariz.; and Jan. 11-12, 2003 in Columbia, Mo.\n\nThe workshops will include story ideas and tips for using the latest and upcoming data releases, including the American Community Survey and Public Use Microdata.\n\nVisit www.2000census.org for more information and to register.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-building-a-database-for-tracking-state-money","slug":"2002-september-october-building-a-database-for-tracking-state-money","url":"https://dwillis.github.io/uplink/article/2002-september-october-building-a-database-for-tracking-state-money.html","year":2002,"month":"September - October","monthNum":9,"headline":"Building a database for tracking state money","kicker":"CAMPAIGN FINANCE","author_name":"MaryJo Sylwester","author_title":"","affiliation":"Center for Public Integrity","summary":"The Center for Public Integrity conducted a yearlong investigation of state political party finances by converting nearly 400,000 paper and electronic records into standardized databases, revealing that 46 percent of state party money in the 2000 election came from national soft money. The article outlines practical lessons learned in organizing, cleaning, and consolidating inconsistent financial data across multiple states into usable tables for analysis.","keywords":["database","finance","investigation"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":["Access","Excel","Word"],"full_text":"State Secrets, an investigation of contributions and expenditures reported by state political party committees, started with a pile of paper reports that stood more than 15 feet high. That doesn't count thousands of records obtained directly in electronic format.\n\nAll told, there were nearly 400,000 records - and no easy way to put them together. The paper forms differed from state to state. Even the content in a single state didn't always remain consistent.\n\nAs a result, compressing all of that into one table of contributions and another of expenditures - and cleaning them provided valuable lessons in how to build an effective database.\n\nThe Center for Public Integrity, in conjunction with the Center for Responsive Politics and the National Institute on Money in State Politics, published the findings of the yearlong investigation on June 25. (See the July-Aug. 2002 Uplink for more on political reporting.)\n\nOur analysis showed that 46 percent of the money state political parties collected in the 2000 election was soft money from national party coffers. These transfers confirmed a commonly held perception that state parties are used to channel soft money and influence presidential and congressional elections.\n\nPaper to data\n\nEven the most organized person would be taxed by the collection of hundreds of reports and the data entry. Planning how you're going to tackle the paper pile is crucial no matter the database size. If you don't set it up properly, you'll have bigger problems later.\n\nThere were some things that made life easier with State Secrets:\n\nI created an Access database where our staff members tracked our FOIA requests for each state, including phone numbers, Web sites, and notes. They also had a Word file for each state for more extensive notes and e-mails. I could run a query to quickly see what was missing.\nI pored over the campaign finance forms listed the various features of each in an Excel worksheet. This showed me which fields were necessary for data entry and where the limitations were.\nWe hired a data entry firm. This was, by far, the best move we made.\nWe broke the work into manageable chunks. In this case, it was on a state-by-state basis. Once all the data entry, basic cleanup and checking was done on each of the states, we appended the files together. Some of the cleanup (like standardizing names) was better left until after we assembled the full table, however.\nWe ran summary queries to compare the total expenditures to the summaries listed on paper reports. We found missing pages, duplicate records and even bad math on the part of the political parties that we would not have found without these queries.\n\nIn hindsight, I should have created ID numbers for each of the paper reports. The data entry house could have typed those IDs into the data to indicate which paper report each item came from. And the forms could have been stored in numerical order for easier access.\n\nInstead, we used the name or date of the report (such as \"Fourth Quarter report\" or \"Pre-Primary\"), but there was so much inconsistency we had to spend several days standardizing the report names in order to do the summary queries mentioned above.\n\nI also would have assigned a code to each committee to be typed in addition to the name. We applied a code only after the information was typed a difficult process because the committee names were never consistent.\n\nThe scrubbing phase\n\nBefore embarking on data cleanup you should answer two questions: What standards do you want to meet? And how dirty is the data? (See page 16 for details.)\n\nOn the expenditure data, we decided it would be an ineffective use of our time to clean up all recipient names, but consultant names should be standardized so we could focus on those. On the contribution data, we scrubbed and scrubbed to make sure the contributor names were as clean as we could get them. We set a very high standard because we wanted to be positive that we identified the true top donors.\n\nFiguring out the dirtiness of the data is the harder part. I'd recommend running queries, and making notes about the variations you find. Are there big chunks with consistent patterns that could be fixed using an update query? If you can pinpoint places to do mass updates first, that will save work later. Eventually, you'll have to do line-by-line cleanup no matter what, so do whatever you can to minimize that.\n\nAll of the suggestions I've listed here are beneficial no matter how small or large the database. Consistency and advanced planning will make your job much easier in the long run.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-plotting-school-data","slug":"2002-september-october-plotting-school-data","url":"https://dwillis.github.io/uplink/article/2002-september-october-plotting-school-data.html","year":2002,"month":"September - October","monthNum":9,"headline":"Plotting school data","kicker":"MAPPING","author_name":"","author_title":"","affiliation":"","summary":"Journalists are using GIS mapping and analysis to uncover trends in education reporting, from tracking student enrollment shifts across school districts to analyzing whether charter schools reflect their surrounding communities.","keywords":["GIS","education","mapping"],"topics":["Education","Technology","Investigative Methods"],"technologies":["Excel","ArcView"],"full_text":"Public education agencies collect huge amounts of information about students, teachers and schools. And most of that data can easily be tied into a geographic location: a school building point, an attendance zone or even a census tract. So there are plenty of opportunities for journalists to use GIS analysis in education reporting. Here are some ways that journalists have been turning to GIS to help uncover trends in education.\n\nShifting attendance\n\nJohn Welbes, an education reporter for the St. Paul Pioneer Press, heard during a school board meeting last year that some Minnesota school districts had been losing students under the state's open enrollment system. That meant those schools also lost state aid money tied to enrollment.\n\nWelbes obtained an Excel spreadsheet from the state that contained one row for every public school student. One column contained the identification number of the school that the student attended. Another contained the identification number of the school district in which the student resided.\n\nUsing that file, Janet Roberts, the newspaper's computer-assisted reporting coordinator, summarized the data by loss or gain for the region's school districts. She imported the table to ArcView and mapped the changes in the Minneapolis-St. Paul metropolitan area.\n\nThe map showed that center city districts lost students; so did some in the suburbs.\n\nCharter schools\n\nAs part of its \"Cheating the Classroom\" stories, reporters at The Miami Herald wanted to see whether charter schools in Miami-Dade County were anything like their surrounding neighborhoods.\n\nThe Herald had a data file with information about each school, including the school's street address. Using the geocoding function in ArcView, database editor Jason Grotto created a point map layer that showed the location of all the schools. Then he joined a table of school demographic information and another of neighborhood demographic data from Census 2000.\n\nGrotto then selected nearby non-charter public schools in another map layer and found that many of the charter schools' student demographics were different from those in adjacent public schools. In addition, the GIS analysis showed that the charter schools failed to reflect the neighborhoods they were supposed to serve.\n\nFor instance, according to its contract with the Miami-Dade schools, Miami Shores/Barry University charter school is required to serve students within a 4-mile radius, and its population should reflect the community. But the mapping showed the charter was the only public middle school in that 4 mile radius where fewer than 25 percent of the students received free or reduced lunch. It also was the only school with a majority white student population.\n\nSchool week\n\nThe Denver Post used ArcView 3.2 for a story about Colorado school districts that switched to four-day weeks. The map showed that the switch to a shorter school week mostly was happening in the rural parts of the state, where administrators were trying to save money.\n\nMonte Whaley, one of the newspaper's education reporters, entered the information about the district schedules from the Colorado Department of Education into a spreadsheet. CAR editor Jeff Roberts then mapped the results using a school districts shapefile.\n\nWould you be willing to share a mapping example with fellow journalists?","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-uncovering-back-door-vouchers","slug":"2002-september-october-uncovering-back-door-vouchers","url":"https://dwillis.github.io/uplink/article/2002-september-october-uncovering-back-door-vouchers.html","year":2002,"month":"September - October","monthNum":9,"headline":"Uncovering \"back-door vouchers\"","kicker":"SPOTLIGHT: EDUCATION","author_name":"James E. Wilkerson","author_title":"Reporter","affiliation":"The Morning Call","summary":"Journalists investigating Pennsylvania's scholarship tax-credit program discovered that millions in state tax dollars were being funneled to private religious schools with minimal oversight, using computer-assisted reporting and database creation to expose what critics called 'back-door vouchers.'","keywords":["vouchers","investigation","database"],"topics":["Education","Data & Databases","Government & Politics"],"technologies":["Access","UltraEdit","Word","Excel","Internet","Nexis","GuideStar"],"full_text":"Uncovering \"back-door vouchers\"\n\nBy James E. Wilkerson, The (Allentown, Pa.) Morning Call\n\nLate last year, the superintendent of schools for the Roman Catholic Diocese of Allentown, Pa., told our education reporter Nancy Averett that the schools had received more than $600,000 in tuition donations from local businesses under a new state tax-credit program.\n\nShe already knew about the program, but was surprised by the amount going to one diocese. Dubbed \"back-door vouchers\" by critics, the scholarship tax credits were clearly funneling a lot of tax dollars to private, nonprofit schools. So Nancy came to me wondering if we could find out exactly how much.\n\nProgram administrators with the state Department of Community and Economic Development said they didn't know. In fact, there was a lot about the donations they didn't know. So we ended up having to build our own database, one that allowed us to show how millions of dollars in tax money was being funneled to religious schools with very little supervision or follow-up from the state.\n\nThe program, started in early 2001, gives businesses up to 90 percent state tax credit for money they donate to nonprofit organizations, which then distribute the money to students for tuition. Unlike full-blown vouchers, voted down in the state legislature three times in recent years, supporters emphasized that the tax credit took no money directly from public school funds.\n\nThat argument seemed a little flimsy to us, however. The program diverted money away from the state coffers, money that could have easily gone to education. And, we suspected, most of that money went to private religious schools. In past legislative sessions, the potential of public money aiding private religious schools had triggered vehement opposition.\n\nBackground research\n\nTo do the job right, we had to examine more than 100 organizations approved by the state to receive scholarship money.\n\nThat's where the computer-assisted reporting work started. Nancy was familiar with Access, so I created an easy-to-use database that we could both use whenever our schedules allowed. Using the Forms Wizard, I created a simple data entry form that included the organization's name and other basic information, and provided space for us to add research notes, Web sites and other information we found.\n\nThe state had lists of approved organizations on its Web site, including address, contact information and Web address. I copied those into a powerful text editor, UltraEdit, (available at www.ultraedit.com) and designed macros that parsed the five rows of data for each organization into a fixed-width text file that I later imported into Access. I added the fields from the table we had created in Access and we were ready to roll.\n\nWe relied almost exclusively on the Internet, Lexis-Nexis and our own newspaper archive for researching the organizations. Especially useful was GuideStar (www.guidestar.com), a database containing the financial disclosure forms (Form 990) that nonprofit organizations are supposed to file annually with the Internal Revenue Service. In some cases we were unable to find information about an organization elsewhere, but the 990s on GuideStar provided details about how the groups were distributing money.\n\nMerging data\n\nWithin three weeks or so we had finished the bulk of our research. By that time, the state had given us a list of contributing businesses and the donation amounts that went to the nonprofits.\n\nUnfortunately, the data was in two files, with the business name and address information in Word and the donation amounts in Excel. I converted the Word document into a text file and again used UltraEdit to convert the information into columns and rows. Once that was completed, I imported the fixed-width text and Excel files into Access and merged them into one table.\n\nFortunately, with few exceptions, the nonprofit names were the same in all of the documents I was using. That allowed me to join on the name field with little clean-up.\n\nWhen it came time to produce numbers, I had whittled the data down to two tables: Our research table, which included data about the organizations receiving money, and the business and donation table.\n\nTallying results\n\nI used SQL Server for my analysis, but the work could have just as easily been done in Access or any other database program. I joined the two tables on the organization name field and ran some simple Group By queries to look at the distribution of the donations.\n\nIn the end, we found that more than half of the $17 million in donations had gone to scholarship organizations that primarily benefited religious schools. Looking only at scholarship organizations, about 75 percent of donations had gone to religious schools.\n\nAfter running our findings by state officials and finding out that they had higher totals, we realized that we were missing some data. We put the story on hold until the year-end data became available in April. However, in the following weeks we kept plugging away at organization research and updated other data. When the final data became available we were ready to start writing.\n\nOur final story, which ran Aug. 4, showed not only that the lion's share of money was going to religious groups, but that the state was doing very little to track how the money was being used.\n\nThough the total contribution tally had climbed to about $30 million, the share of scholarship contributions going to religious organizations held steady, at about 75 percent, so our story did not change drastically during the delay.\n\nIn fact, interviews with administrators at some of top contribution-getters in the state showed that much of the money was being distributed to students already in private schools, undermining the state's claim that needy students in poorly performing public districts were being targeted.\n\nAlong with the story, I built a Web lookup tool that allows readers a look at contribution lists. The story and query are available at www.mcall.com/all-a1_5voucheraug04.story\n\nJames E. Wilkerson can be reached by e-mail at james.wilkerson@mcall.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-measuring-long-bus-rides-for-rural-children","slug":"2002-september-october-measuring-long-bus-rides-for-rural-children","url":"https://dwillis.github.io/uplink/article/2002-september-october-measuring-long-bus-rides-for-rural-children.html","year":2002,"month":"September - October","monthNum":9,"headline":"Measuring long bus rides for rural children","kicker":"SPOTLIGHT: EDUCATION","author_name":"Scott Finn and Eric Eyre","author_title":"","affiliation":"The Charleston Gazette","summary":"West Virginia journalists investigated the impact of school consolidations on student bus rides by constructing their own database of 1,569 bus runs after discovering the state had ignored a law requiring such a study. Their analysis found that two-thirds of elementary bus routes exceeded state guidelines, with some students spending over three hours daily commuting.","keywords":["consolidation","transportation","investigation"],"topics":["Investigative Methods","Data & Databases","Education"],"technologies":["Excel"],"full_text":"During the past decade, West Virginia education officials closed one of every five schools in a massive consolidation drive. State officials said the closings were necessary because of dwindling enrollments in rural areas.\n\nParents and consolidation opponents said rural children were forced to ride the bus four hours a day or longer because of the closings. State officials said most children rode the bus for only a short period of time. Each side relied on anecdotal evidence to back their claims.\n\nWe started with asking a simple question: \"How long are children's bus rides in West Virginia?\"\n\nWe thought that someone, somewhere within the educational bureaucracy would have attempted to answer that question. Instead, we discovered the state Department of Education had ignored a 1998 state law telling them to study the issue of lengthening bus rides.\n\nEven if education officials had followed the law, West Virginia's antiquated record-keeping system made it almost impossible to answer the question. In other states, school systems keep computerized records of bus routes and riders. But most of the school districts in 35 rural counties did not. In fact, bus drivers themselves kept most of the records by hand.\n\nSo we had to construct our own database of 1,569 bus runs in rural counties. We started our project intending to study all bus rides in the state, but we soon discovered that many urban districts also did not keep computerized busing records. We decided to focus on the state's rural counties, in part because that's where the longest bus rides occur, and to keep the data entry manageable.\n\nThe process, from FOIA requests to data entry to double-checking, took us eight months. But the resulting ability to write with authority made the effort worthwhile.\n\nIn \"The Long Haul,\" published in August, we found the number of children who rode the bus more than two hours a day doubled during the 1990s. Also, we found that two-thirds of bus routes carrying elementary children exceeded state guidelines for a \"reasonable\" bus ride - an hour per day or less. The average student at Clay Elementary, a typical rural school, spent 37 minutes on the bus each way, nine minutes longer than the average Los Angeles commuter.\n\nOur data helped us locate the best areas for conducting interviews and finding stories. In one county, elementary school students counted the 46 bolts on the roof of their bus to while away their hour-and-40 minute ride to school. Teachers at one high school slipped cups of coffee to 15-year-old students to help them stay awake in class. Parents of one 4-year-old dressed him in his sleep so he could enjoy a few more minutes of sleep before his one-and-a-half hour ride.\n\nGathering the data\n\nThrough the state Freedom of Information Act, we obtained records for bus runs for the state's 35 most rural counties. (Those counties were eligible for federal school construction funds because they were the most sparsely populated of the state's 55 counties.)\n\nAt first, several county school systems fought our efforts to obtain the bus routes. Some claimed it contained private information, so we asked them to redact any personal information, such as children's names. Other school systems wanted to charge us large sums for the printing. A couple of counties required us to drive up to three hours to their headquarters to obtain the records.\n\nWe created an Excel spreadsheet that included when each run started, when it stopped, and how much time children rode inbetween.\n\nThe bus runs represented the longest possible time a child might ride a bus. The run began when the first child stepped on the bus, and ended when the children were dropped off at an elementary, middle or high school. The bus run times did not include the time children stood waiting for the bus to arrive. These were ideal times, not taking into account bad weather or traffic delays.\n\nCalculating with Excel\n\nTo determine how long each bus run was we subtracted the start time from the drop-off time. We then used Excel's Count If function to count only the runs that exceeded state guidelines. We divided that by the total count to determine the percent of runs over the guidelines.\n\nThe average bus ride could not be tabulated for most counties because the schools don't track when individual children board their bus. Clay County is one of the few that tracks exactly when each child boards the bus and when they are dropped off. Because we had the data, we were able to determine that the average Clay Elementary student spent 37 minutes on the bus, more than the state guideline of 30 minutes. The average middle school student was on the bus for 34 minutes, the average high school student for 38.\n\nWe checked our results with transportation directors in all 35 counties, both through phone calls and by sending a copy of their county's data to them through the mail. We made corrections to the database based on their responses. In some cases, we spent half an hour going over a particularly complicated bus schedule with a transportation director.\n\nFor purposes of the stories, junior high schools were classified as middle schools. Buses for special education students, who often ride much longer than other students, were excluded from the database.\n\nWe also relied upon two surveys of county school transportation directors by the state Department of Education, one in 1992 and the most recent in 1996. The survey asked transportation directors to estimate how many children rode the bus for longer than state guidelines allow.\n\nThose surveys are imperfect, because they rely upon the impressions of the directors and not hard data. For example, one county's survey had exactly 600 students on the bus for one time increment, and exactly 700 for another increment, leading us to believe the times were estimates, not the detailed study state officials led us to believe. If anything, we believed they tended to underestimate the amount of time children spend on the bus, based on comparisons to our spreadsheet.\n\nSo far, the story has resulted in a promise from the state transportation director to computerize all bus schedules and complete its own study about the length of bus rides.\n\nWe are planning more stories this fall on the damaging effects school closings have on the state's youngest and poorest students.\n\nTo see the whole series, visit www.wvgazette.com/news/Closing/\n\nContact Scott Finn by e-mail at sfinn@wvgazette.com.\n\nContact Eric Eyre by e-mail at ericeyre@wvgazette.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-rating-cities-readiness-to-handle-attacks","slug":"2002-september-october-rating-cities-readiness-to-handle-attacks","url":"https://dwillis.github.io/uplink/article/2002-september-october-rating-cities-readiness-to-handle-attacks.html","year":2002,"month":"September - October","monthNum":9,"headline":"Rating cities' readiness to handle attacks","kicker":"HOMELAND SECURITY","author_name":"Mike Fish","author_title":"Reporter","affiliation":"CNN.com","summary":"CNN.com undertook a computer-assisted reporting project to assess the terrorism preparedness of major U.S. cities post-9/11, ultimately evaluating 30 cities based on population and police force size to inform the public while avoiding providing a blueprint for potential attacks.","keywords":["preparedness","cities","terrorism"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Just how prepared are major U.S. cities to face a terrorist attack? Can we assume places like New York and Washington, D.C are more up to speed than, say, Milwaukee or Columbus?\n\nThe questions were being posed by editors at CNN.com as another possible follow up to events of Sept. 11. Like several news organizations, we had completed a computer-assisted reporting project that focused on the airline industry and security at the country's busiest airports, and there was a desire to similarly rank, or at least gauge, the performance of major cities.\n\nTerrorism experts, who by all appearances had multiplied in number after Sept. 11, were insisting on two things: no city is immune from risk and no one is fully prepared to handle the fallout of an attack. So folks couldn't help but worry about their own safety as well as suddenly wondering if their city was schooled to deal with everything from biological to chemical threats.\n\nTo assess the preparedness of the largest cities, CNN.com undertook a computer-assisted reporting project last fall that produced \"How Prepared Is Your City?\" on the Web site, a 30-minute special report on \"Wolf Blitzer Reports\" and also a TV package by CNN Homeland Security correspondent Jeanne Meserve.\n\nThe early challenges were plenty. First, we were determined to produce a thoughtful package that would both enlighten and address people's concerns, yet at the same time wouldn't serve as a blueprint for would-be terrorists and criminals. We found this also to be the top concern of the experts who were interviewed or assisted us in evaluating the cities.\n\nThis ongoing dialogue with a team of experts proved invaluable, as they often had first-hand knowledge of the preparedness efforts and vulnerabilities within a number of major cities. We also were able to seek their input on the statistical data, federal grant money and such criteria that would best help in evaluating the work done by city officials leading up to Sept. 11.\n\nNarrowing the cities\n\nOK, but how many cities could we review in depth? We thought initially 50, until deciding it would be too overwhelming and time consuming a task. And when we considered 25, we struggled to justify eliminating important cities like Miami and Atlanta that barely missed making the cut.\n\nUltimately, we reviewed national crime statistics reports and developed our top 30 cities by the criteria that cities had to have a population of at least 350,000 and also employ at least 1,100 full-time police officers. Even then, to include as many cities as possible, Dallas/Fort Worth and San Francisco/San Jose were combined rather than evaluated individually. We made sure to present separate data on each city in our final report and, as it turned out, our final rankings for these cities would have been the same no matter which way we did it.\n\nThat bit of information would prove helpful in addressing challenges from Fort Worth and San Jose officials, who argued that their respective cities had their own individual issues and didn't care to be evaluated with their neighbors.\n\nOnce we were comfortable with our list of cities, we dug for every shred of relevant data and public record that was available. The analysis would focus on four crucial categories: transportation (access to and from emergency sites); hospital/medical; federal preparedness (an evaluation of funding and training programs); and emergency management.\n\nTrue preparedness is often deemed best achieved where the local, state and federal governments - as well as members of the private sector - work as partners. But we recognized a challenge in preparing for emergencies was also getting access to available resources.\n\nThe Centers for Disease Control and Prevention, we found, awarded $120 million to states during the past two years to upgrade public health's capacity to respond to bioterrorism. But only four cities received separate grants, including New York ($1.14 million), Chicago ($902,089), Los Angeles ($850,275) and Washington, D.C. ($570,317).\n\nFinding, analyzing and weighting the value of consistent, statistical data that could be applied, in our four categories, across the 30 selected cities, would be a key to making this study work.\n\nSurveying experts\n\nEarly in the project, however, it became apparent that statistics alone were not going to tell us the full story of a city's preparedness. Some of the information that would be valuable in evaluating cities, such as details about pharmaceutical stockpiles, is appropriately guarded.\n\nThe experts stressed the need to weigh intangibles, such as how the uniformed services work together as well as the level of commitment from the private sector to participate in training. So while accumulating the data in each of our four categories, CNN.com also provided local emergency management directors with a 10-question survey to evaluate staffing, training and such key questions as:\n\nWho does your city's emergency management director report to?\nA. Mayor's office/city or county manager\nB. Police department\nC. Fire department\nD. Other\n\nThe number of your full-time staff dedicated solely to emergency management?\nA. More than 10\nB. 5-to-10\nC. 1-to-5\nD. 0\n\nIn the current fiscal year, the amount received by your department for terrorism planning, training and equipment?\nA. More than $1 million\nB. $500,000-to-$1 million\nC. $100,000-to-$500,000\nD. Less than $100,000\n\nAll of the data accumulated by CNN.com was later shared with the local emergency management directors, who were offered the opportunity to comment and report any corrections. In most cases, it required multiple follow-up calls and faxes but it was crucial that officials confirm the accuracy of data that would be used in the city evaluations.\n\nEventually, we would provide our empirical data to six law enforcement, security/terrorism and emergency management experts asking them to rate the cities in each of the four categories, using a one-to-five scale (one being best prepared). The Disaster and Terrorism Assessment Survey given the experts was a 22-page document that included key potential vulnerabilities of each city, general city information such as the number of police officers per 10,000 residents and then specific data compiled within our four categories.\n\nSuch a scale is recognized by, among others, the National Institute for Computer-Assisted Reporting, as a common grading system within academia, the media and law enforcement. The U.S. Department of Justice provides the same scale for a self-assessment by states applying for domestic preparedness grants. The most comprehensive was hospital/medical, which included 18 items ranging from physicians and emergency outpatients per 10,000 residents to whether the state employed a public health veterinarian and the number CDC-sponsored \"disease detectives.\"\n\nThe final overall assessment, using Excel spreadsheet software, of the 30 cities was based on the simple average of the experts ranking, with CNN.com weighting the hospital/medical category 1.5 times because of its importance and the volume of relevant data. Cities were judged as being most prepared, well prepared, prepared or less prepared.\n\nWhile reviewing the data, we stressed the importance for the experts to rely on their own experience with the cities. So factored into the evaluation was their awareness that some cities face a greater risk because of such things as location, commerce, landmarks and hosting major events.\n\nIn the end, only New York, having earlier experienced attacks on the World Trade Center, was judged most prepared. We did a separate story looking at New York's performance and the lessons learned.\n\nThe major knock against New York came in the transportation category, where experts cited the potential for gridlock - a factor in getting responders to the scene as well as transporting the injured and ill to medical facilities.\n\nSeveral other large cities also fared poorly under transportation - notably Boston, Philadelphia and Detroit. The panel judged them overall less prepared, joined by Las Vegas, Milwaukee and New Orleans. CNN.com shared the rankings with city officials before publishing and produced a separate story detailing heightened preparedness as well as steps taken by the individual cities since Sept. 11. Our analysis also found:\n\nWhile cities are better prepared than prior to Sept. 11, significant improvement is still needed in public health, emergency response, evacuation planning and inter-agency communication.\n\nLas Vegas and New Orleans, two major tourist/convention cities, are located in states that do not employ public health veterinarians, who could be helpful in the early detection of a biological crisis. Nor did the cities have a CDC-sponsored disease detective.\n\nCities faring best are those like New York who routinely host major events or traditionally face natural disasters, such as Florida and California cities.\n\nThe two-month project produced eight stories for the Web site. Web site visitors could rank the cities on different variables and see how their cities compared to others.\n\nMike Fish can be contacted by e-mail at mike.fish@turner.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-faa-withholds-data","slug":"2002-september-october-faa-withholds-data","url":"https://dwillis.github.io/uplink/article/2002-september-october-faa-withholds-data.html","year":2002,"month":"September - October","monthNum":9,"headline":"FAA withholds data","kicker":"","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"The FAA has withheld enforcement database data from journalists since September 2001, citing security concerns and refusing to release information about security and hazardous material violations despite previously making the data publicly available. IRE has challenged the withholding through FOIA requests, arguing the public has a right to know about regulatory compliance and safety issues.","keywords":["FAA","transparency","security"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["FTP","CD-ROM","Internet"],"full_text":"FAA withholds data\n\nA year ago, right after the Sept. 11 terrorist attacks, IRE and NICAR provided copies of the Federal Aviation Administration enforcement database to well over 100 news organizations. Story after story based on the data provided glimpses of security problems in airports across the United States.\n\nIRE and NICAR had planned to update the database that October, with a routine download from the administration's FTP server. Three days after the attacks, though, the Database Library downloaded the database so journalists could have the most recent information available.\n\nThat was the last time the FAA data update was routine.\n\nShortly after the update, the administration pulled the data from its FTP server, citing security. The data has yet to resurface in its entirety.\n\nAfter the Database Library's March 29 Freedom of Information Act request, the FAA supplied a copy of the database on June 22 - minus any mention of security or hazardous material violations.\n\nThe database still lists actions in some categories, such as flight operation or maintenance problems. As for security or hazardous material violations, the FAA argues that it's none of the public's business.\n\nThe official response from Joseph Tintera, FAA's manager of the regulatory support division, says, \"sensitive security information has been withheld citing regulations on whether the information is \"detrimental to the safety of persons traveling in transportation.\"\n\nOn July 18, IRE filed a letter asking for that denial to be reconsidered.\n\nIRE argues that since the information resided on publicly accessible Internet servers from at least November 2000 until October 2001, the FAA had set a precedent. Even prior to that, the agency had provided the data on CD.\n\nAn additional argument: The request didn't seek records that reveal details of inspections, investigations and alleged violations and findings, but rather only general information, allowed to be disclosed.\n\nIndeed, even before the terrorist attacks, the FAA held back enforcement actions from public release for a year.\n\nAside from security violations, the FAA denying information about problems with hazardous materials is particularly troubling.\n\nEven before Sept. 11, journalists have used the enforcement database to look at hazardous material violations. In 1999, The (Cleveland) Plain Dealer published a series examining hazardous materials on airlines.\n\nIn July, IRE wrote to the agency: \"The public should have access to this information to be informed whether government-regulated entities have abided by federal regulations and how government agencies have applied regulations. Failing to provide information to the public concerning mishandling of hazardous materials disregards the public interest and can be detrimental to the safety of persons traveling in transportation.\"\n\nAt this writing, the agency has yet to respond.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-september-october-tech-tip-foxpro-program-cleans-standardizes-names","slug":"2002-september-october-tech-tip-foxpro-program-cleans-standardizes-names","url":"https://dwillis.github.io/uplink/article/2002-september-october-tech-tip-foxpro-program-cleans-standardizes-names.html","year":2002,"month":"September - October","monthNum":9,"headline":"Tech tip.. FoxPro program cleans, standardizes names","kicker":"Tech tip","author_name":"MaryJo Sylwester","author_title":"","affiliation":"NICAR","summary":"A journalist describes a method for crowdsourcing data cleanup by exporting chunks to Excel, having colleagues mark changes, then using Excel formulas to generate FoxPro code that automatically applies those changes to the main database.","keywords":["data","Excel","FoxPro"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["FoxPro","Access","Excel"],"full_text":"Because of the massive size of the State Secrets database (see page 3), we couldn't do cleanup in Access and I was the only person in our office with software powerful enough to handle the data (Visual FoxPro). So I came up with a way to have others help with the cleanup without having direct access.\n\nI exported chunks of the data to Excel files, and had the others put the changes in a new field. Then I wrote a FoxPro program using the Excel copy down functions to apply the changes to the main data file. For further information, read on, then download an Excel file and FoxPro program file that shows this in action (you can open the FoxPro file with NotePad) from www.nicar.org/techtips.html.\n\nFor example, here's a list of names that needed to be standardized. In this case the first name field needs to be updated to the combined name:\n\nID\tLAST\tFIRST\nVA556\tSmith\tJohn R\nVA557\tSmith\tJohn\nVA558\tSmith\tJohn & Jane\nCITY\tSTATE\tNEW FIRST\nReston\tVA\tJohn R & Jane\nReston\tVA\tJohn R & Jane\nReston\tVA\tJohn R & Jane\n\nThen I added two new columns, putting the following pieces of code in them and copying down the length of the data: REPLACE ALL NEWFIRST WITH FOR ID=\n\nSo in this example, the two pieces of code are in columns G and H. The data (as shown above) is in columns A through F. In the next blank column, you concatenate together various pieces to write the code for a simple program. Here's what it would look like for this example:\n\n=G2&\n\nThe result for the first line should look like this: REPLACE ALL NEWFIRST WITH \"John R & Jane\" for ID= \"VA556\"\n\nTo explain: The ampersand (&) is the tool to tie the pieces together. The first set of quotes provides a space between the word \"with\" and the \"John R & Jane.\" The next set of quotes (it's four double quotes) is how you tell the computer you want to include the quotes in your answer something that you need in this case to tell the computer that \"John R & Jane\" is a text string.\n\nWhen you copy down the concatenated formula, you get the appropriate ID numbers for each line without a lot of repetitive typing. Then copy just the field that has your answers and paste it into a PRG file in FoxPro. It will look something like this:\n\nREPLACE ALL NEWFIRST WITH \"John R & Jane\" for ID= \"VA556\"\nREPLACE ALL NEWFIRST WITH \"John R & Jane\" for ID= \"VA557\"\nREPLACE ALL NEWFIRST WITH \"John R & Jane\" for ID= \"VA558\"\n\nYou can use this same trick to apply changes to more than one record at a time by looking for keywords. For example, we used this for coding the expenditures. I created a list in Excel of possible keywords that we might find listed in the purpose field (most of the list came from a group-by query on the purpose field).\n\nWe had previously decided what code each keyword would fall under, so I assigned those in the Excel file. Here's what it looked like:\n\nMAIL\tA4\nSALARIES\tA2\nHOTEL\tA1\nAIRFARE\tA1\nOFFICE SUPPLIES\tA5\nPOLLING\tC4\nDIRECT MAIL\tC6\n\nThen, in the Excel file, I added the FoxPro commands, but this time using wildcards. Here's what the command looks like when completed:\n\nREPLACE ALL PURPCODE WITH \"A4\" FOR \"MAIL\" $ PURPOSE\n\nThe dollar sign symbol tells FoxPro to find the term \"MAIL\" anywhere in the field called PURPOSE.\n\nThe coding program ended up being more than 400 lines long because I had so many possible keywords. I kept the Excel file throughout the coding process, and that proved to be worthwhile when we decided to tweak codes.\n\nInstead of having to sort through the whole FoxPro program file, I simply went back to the Excel file, sorted or filtered it to find the codes I wanted (there might be anywhere from two to 50 lines for any given code) and made the changes. Then I re-copied the formula and pasted it into a new program file. It took maybe five minutes to make hundreds of changes.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-gis-data-carries-high-cost","slug":"2002-november-december-gis-data-carries-high-cost","url":"https://dwillis.github.io/uplink/article/2002-november-december-gis-data-carries-high-cost.html","year":2002,"month":"November December","monthNum":11,"headline":"GIS data carries high cost","kicker":"DATA ACCESS","author_name":"Jaimi Dowdell","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists across the country are facing significant barriers and high costs when attempting to obtain GIS data from government agencies, with some agencies charging thousands of dollars despite the data being publicly funded or required by public records laws.","keywords":["GIS","access","data"],"topics":["Investigative Methods","Technology","Government & Politics"],"technologies":["GIS","CD-ROM"],"full_text":"For three years, David Washburn of The San Diego Union-Tribune has been itching to get his hands on a file from SanGIS, an organization that handles San Diego County's GIS data. Although the GIS data is available to the public, neither Washburn nor his newspaper is ready to fork over $15,000 for one year of data.\n\n\"I think my county is ripping me off and the taxpayers off how they are handling GIS,\" Washburn said.\n\nWithin the past five years, use of GIS software and data has exploded in newsrooms. Many journalists using computer-assisted reporting have found the programs essential and use them in their daily reporting. While many government agencies have posted GIS data files on their Web sites for free download, other agencies haven't been as open.\n\nOther journalists are running into the same problems Washburn has experienced. The privatization of government functions and amendments to public records laws has hampered access to GIS data. He said that because SanGIS was initially funded by taxpayer money and backed by the county it should be more reasonable with its prices.\n\n\"I totally think they are breaking the law by doing this because it involves taxpayer money,\" said Washburn, an investigative reporter. SanGIS contends that because it is now self-sustaining, its prices are reasonable and reflect the time and effort it takes to digitize the county layers.\n\nJournalists are finding that, along with charging high costs for GIS data, the departments have basically cornered the market. While some private businesses such as real estate or law firms may be able to afford the data, most news organizations lack the money.\n\nLaws increase costs\n\nWhile the GIS costs in San Diego County are one extreme, journalists have encountered a spectrum of difficulties and costs when attempting to obtain GIS data from government agencies. The availability and cost of GIS data depends on the location, public records laws and who's reviewing the request for data.\n\nPart of the reason Dan Keating, database editor at The Washington Post, first began using GIS data was because it was so easily accessible from county GIS departments in Florida, where he had worked before. \"Both Miami-Dade and Broward counties gave me CDs with dozens and dozens of local GIS layers for practically nothing, no more than $50,\" Keating said.\n\nThat changed once he left The Miami Herald and began working in Washington, D.C. While costs for data from the District and Virginia counties were reasonable, he said, obtaining GIS data from agencies in Maryland proved to be a nightmare. \"The state has an exemption to its public records law for GIS, saying that it can be sold for market cost. So layers from a county can cost thousands of dollars,\" Keating said.\n\nMaurice Tamman, staff writer at The Atlanta Journal-Constitution, has had troubles dealing with GIS laws. While working on a story requiring parcel maps, Tamman requested GIS data from Forsyth County and was quoted a price of more than $10,000 for the maps he was seeking.\n\nA recently enacted Georgia law gives agencies the authority to charge for GIS data based upon the actual recovery and development time involved with the GIS. Tamman said the county follows this new law to the letter.\n\nMeanwhile, he was able to receive the same type of GIS data from a neighboring county absolutely free. \"It is interesting to me, because traditionally we view any data produced by the government as being fundamentally free,\" Tamman said. \"But GIS has been placed on a pedestal because it has some intrinsic market value.\"\n\nFighting some battles\n\nSome reporters have gone head-to-head with government agencies challenging fees based on outrageous standards. While many of these journalists bear the battle scars, some have the GIS data to show for it as well.\n\nPaula Sullivan, city hall reporter and computer-assisted reporting coordinator at The (Tacoma, Wash.) News Tribune, went around in circles trying to negotiate with Pierce County for some GIS data. The agency's going rate was $60 per shapefile, when all they had to do to produce the files was copy ones already created.\n\n\"State law does not allow them to do this, so we complained to the county executive and - instead of trying to negotiate a reasonable price - they gave it to us free just that once,\" Sullivan said.\n\nSince that initial dispute, she said they have received other files from the county. \"However, the department continues to charge the $60 rate to others and makes them sign all sorts of forms, including a data-sharing agreement that we have always refused to sign.\"\n\n\"GIS has been placed on a pedestal because it has some intrinsic market value.\"\n\nWhile some journalists have had to fight government agencies to lower costs for GIS data, others have found GIS departments to be friendly, helpful, and even willing to provide free data.\n\nAlthough Sullivan had to convince one Washington county to give her data, she said dealing with the city of Tacoma is a different story. This city posts many of its most-accessed shapefiles on its Web site, free to any user with an Internet connection.\n\nWith the recent explosion in the use of GIS, more state, local and federal agencies have been putting GIS data on their Web sites for free download.\n\nMore experiences\n\nJohn McCarthy, a reporter at Florida Today, had a good experience with one state agency.\n\nMcCarthy requested GIS layers of Florida's coastline to analyze how vulnerable those areas would be to hurricanes. Florida's Emergency Operations Center provided the data quickly. \"Their GIS specialist spent quite a bit of time on the phone with me working out bugs and was very cheerful about it,\" he said.\n\nOther journalists have had to give in just a little to receive the shapefiles and layers that they needed. Maricopa County, Ariz., was willing to give Enric Volante, CAR coordinator and database reporter at the Arizona Daily Star, GIS data for free, but only if he signed a liability waiver.\n\n\"First time in my many years of journalism that I ever signed a liability waiver in order to get a public record,\" Volante said. \"I also filed a formal protest, but they didn't seem to understand what all the fuss was about.\"\n\nJohn Hill, systems editor at The (Vancouver, Wash.) Columbian, has received just about all of the GIS data a newsroom could need from the Clark County GIS department. The newspaper paid $1,200 per year for two CDs that include school district boundaries and soil contours. \"While it does seem like a lot of money, when we have a GIS project going it's all money well spent,\" Hill said.\n\nWhile many journalists have been able to get their hands on the data they need, there are still some like Tamman and Washburn who have been completely shut off from the data because of the high prices GIS departments demand.\n\n\"It is certainly putting a roadblock to somebody trying to do their job,\" Tamman said. \"I couldn't do some of those stories I wanted to do. Whether it'll stop me from trying, though, that's another story.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-mapping-finds-bars-dui-wrecks-linked","slug":"2002-november-december-mapping-finds-bars-dui-wrecks-linked","url":"https://dwillis.github.io/uplink/article/2002-november-december-mapping-finds-bars-dui-wrecks-linked.html","year":2002,"month":"November December","monthNum":11,"headline":"Mapping finds bars, DUI wrecks linked","kicker":"SPOTLIGHT: CRIME & JUSTICE","author_name":"Rose Ciotta and Karl Stark","author_title":"","affiliation":"The Inquirer","summary":"An investigation into drunken driving deaths in the Philadelphia region used mapping and data analysis to identify dangerous road segments, overcoming Pennsylvania's restrictions on accident data by obtaining information through alternative sources and appeals.","keywords":["drunken","data","accidents"],"topics":["Investigative Methods","Public Safety","Technology"],"technologies":["ArcView","Spatial Analyst"],"full_text":"Drunken driving was old news. It had been written about and rallied against for years. Yet, this year, for the first time in nearly a decade, drunken driving became a more deadly crime than murder. After years of decline, drunken driving's death toll was inching up nationally and in New Jersey, and holding steady in Pennsylvania.\n\nThose facts gave us ample reason to investigate drunken driving in the Philadelphia region. But, we faced the challenge of finding a fresh approach. Mapping and density analysis gave us the tools to dig deeper and come up with new information for our series, \"Loaded for Trouble,\" which ran in June.\n\nFrom researchers we learned of cutting-edge work that found a link between the concentrations of liquor licenses and the locations of alcohol-related accidents.\n\nWe set out to test the connection in the Philadelphia region. Our first problem was getting the data. In Pennsylvania, accident locations are a secret. Attorneys for the state cite a law that protects the state from liability claims. In addition, Pennsylvania's Department of Transportation several years ago removed from its public data on accidents just about every field of interest, including the date of the accident, whether alcohol was involved and whether there were any injuries. State lawyers cited another law that requires the to withhold the identities of accident victims.\n\nIn appealing these rulings, The Inquirer succeeded in getting PennDOT to release data at the town level, which allowed an analysis of trends in accidents by town and county over a five-year period. That data, however, did not identify any roads. We still did not know the worst areas in the region for alcohol-related crashes.\n\nFor that, we had to rely on backdoor methods. Working through sources within the department, we were able to work around PennDOT's secrecy policy. We obtained data that located accidents on state roads in five-mile road segments. The table also included the total for a five-year period of alcohol-related accidents and fatalities. With that information, we were able to calculate the road segments with the highest per mile rate of alcohol-related accidents. That analysis gave us our first look at the worst roads for alcohol-related accidents. Once we had this data, we appealed to the state to release the latest year's data. By publication, we had road specific accident data for the most recent five-year period.\n\nCreating 'hot spots'\n\nUsing ArcView 8.1, we mapped the accident data and revealed for the first time roads with above-average accident rates. We focused the story on state Route 3, a 25-mile stretch that extends from Philadelphia into its western suburbs that included a segment with the highest accident rate at 22 per mile.\n\nWorking from paper records that we had accumulated from police and other sources, we found alcohol-related accidents that had occurred on Route 3 in which people had been killed. Through our reporting, we also located other accidents that had occurred on the very roads that police knew had high alcohol-related crash rates and where bars were concentrated. In some cases, they were located in towns where police made relatively few arrests for drunken driving. The tragedy of drunken driving was told through the stories of the victims who had been killed and injured.\n\nNext, we obtained liquor license data from the state's Liquor Control Board. In this case, using ArcView's Spatial Analyst extension worked much better than producing an ArcView map. Mapping the 4,000 establishments by plotting each location produced a map of little value. Likewise, a shade map of license counts by town was also misleading since it would have incorrectly given the impression that licenses were evenly distributed across a town when in reality they were generally clumped in older, established neighborhoods.\n\nWe wanted to generate a map showing the liquor license \"hot spots.\" The best tool turned out to be Spatial Analyst, using the actual locations of the point data to generate the \"hot spots\" in just the area of the map where the liquor licenses were concentrated, giving a much more accurate rendition.\n\nThe result was a map that displayed concentrations of liquor establishments from two to six or more per square mile. The more liquor establishments, the darker the ring.\n\nWhen layered with the accident data, we were able to identify the same link between crashes and liquor establishments that researchers had come up with in California. While crashes have many causes - road design, weather, speed - those involving alcohol happened most where bars and restaurants are concentrated.\n\nThis link was especially important in Pennsylvania. A new law makes it possible to sell liquor licenses within counties. Previously, a population-based quota had restricted licenses to the towns in which they were already located, resulting in a concentration of drinking establishments in older, more densely populated neighborhoods.\n\nOur analysis accounted for traffic volume. We found that some of the busiest roads did not have the highest accident rates. Parts of Interstate 95 had the most traffic in the region - but not nearly the highest rates of alcohol-related crashes. We also learned that using traffic count data was flawed because traffic counts are usually taken during the daytime, while alcohol-related crashes were more likely to occur between 8 p.m. and 4 a.m.\n\nBecause we were new to density analysis, we also consulted several experts including Paul J. Gruenewald of the Pacific Institute for Research and Evaluation (www.pire.org) who had done research in California studies.\n\nFor those of you who want to know more about how Spatial Analyst works, here's a brief summary from ESRI's \"Using ArcGIS Spatial Analyst:\" Spatial Analyst calculates density by spreading point values over a surface. A density value is calculated for each cell in the output raster. Points that fall within the search area are summed and then divided by the search area size to get each cell's density value. We elected to calculate density using the kernel calculations because points at the center of the raster cell's search area are weighted more heavily than those near the edge. The result is a smoother distribution of values.\n\nMapping put online\n\nFor online, Matthew Ericson, senior editor for Web technology, designed an interactive map using ESRI's ArcIMS Internet map server, which allowed readers to zoom in on the most dangerous roads in the region and see the total number of alcohol-related crashes, the number of alcohol-related fatalities and the per mile crash rate.\n\nThe Web site (www.philly.com/mld/inquirer/news/special_packages/dui) included a number of interactive tools that allowed our readers to go beyond what we printed in the newspaper and see how DUI affects their lives. A blood alcohol calculator lets readers enter their weight and gender to see how many drinks it takes to reach various levels of impairment.\n\nIn addition to the accident/liquor license analysis, the project included an analysis of police arrest data by town. Calculating a per 10,000 arrest rate and dividing the results into quintiles allowed us to compare police department performance. Readers could also look up their town in a searchable database to find detailed information on the number of alcohol-related crashes by town and by county, and arrests by town, by county, by arrest rate and per police officer. The project included the analysis of other data from courts in two states and Philadelphia and campaign contributions.\n\nContact Rose Ciotta by e-mail at ricotta@phillynews.com\nContact Karl Stark by e-mail at kstark@phillynews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-spotlight","slug":"2002-november-december-spotlight","url":"https://dwillis.github.io/uplink/article/2002-november-december-spotlight.html","year":2002,"month":"November December","monthNum":11,"headline":"SPOTLIGHT:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Computer-assisted reporting reveals critical flaws in criminal justice systems, including Texas judges failing to grant speedy trials resulting in suspect releases, racial disparities in Pittsburgh jury selection, and the link between liquor license locations and drunk driving incidents.","keywords":["justice","investigation","data"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"SPOTLIGHT:\n\nThis issue showcases some of the best uses of CAR to dig into crime and justice. Reporters discovered that the failure of some Texas judges to grant speedy trials led to the release of thousands of suspects, that there was racial inequity in Pittsburgh juries, and the correlation between drunken driving accidents and liquor license locations.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-data-boosts-crime-stories","slug":"2002-november-december-data-boosts-crime-stories","url":"https://dwillis.github.io/uplink/article/2002-november-december-data-boosts-crime-stories.html","year":2002,"month":"November December","monthNum":11,"headline":"Data boosts crime stories","kicker":"","author_name":"Megan Christensen","author_title":"","affiliation":"IRE and NICAR","summary":"Reporters have leveraged computer-assisted reporting and data analysis, using resources such as the FBI's Uniform Crime Reports, to enhance their coverage of national and local crime trends.","keywords":["crime","data analysis","reporting"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"For the resourceful journalist using computer-assisted reporting, there are as many great criminal justice stories waiting to be written as there are Law & Order reruns on cable. Stories about local and national crime trends often begin with the FBI's Uniform Crime Reports. The data is compiled from voluntary reports submitted by city, county, state and national law enforcement agencies. Roughly 95 percent of all police agencies report their data, which outlines crime totals and details homicides. The database also contains information about law enforcement agencies and officers, and demographics of people arrested for major crimes. A useful companion is the National Crime Victimization Survey, a Department of Justice report based on a household survey designed to estimate the incidence of crimes such as theft, robbery, assault and burglary. Journalists can use the data to estimate crimes that may not have been reported to law enforcement.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2002-november-december-bits-and-bytes","slug":"2002-november-december-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2002-november-december-bits-and-bytes.html","year":2002,"month":"November December","monthNum":11,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The Database Library has updated its collection of transportation databases, including the Fatal Accident Reporting System for vehicle accidents, truck crash data, and multiple aviation databases for aircraft registration and accident reporting. These resources allow journalists to investigate accidents, identify vehicle and aircraft owners, and access detailed incident information dating back several decades.","keywords":["accidents","databases","transportation"],"topics":["Data & Databases","Public Safety"],"technologies":["FARS","DOT Truck Accidents database","Aircraft Registration Database","FAA Service Difficulty Reports","NASA Air Safety Reporting System"],"full_text":"Recent updates from the Database Library\n\nVehicle accidents (updated)\n\nHow many people died in accidents on U.S. roadways last year? What are some of the more dangerous highways? Questions like these can be answered with the Fatal Accident Reporting System (FARS) database from the U.S. Department of Transportation. Records are available from 1975, when the DOT started tracking fatal accidents, through 2001. The database has three main tables to track by person, vehicle or accident. With this database, it's possible to find out many details, such as how fast a vehicle was traveling, weather and road conditions, and whether the driver had prior convictions for DUI or speeding.\n\nAnother database deals specifically with truck crashes, whether or not anyone died. The DOT Truck Accidents database has commercial carrier accidents as reported to state and local police from 1988 through 2001. The main table has details on the crash, such as number of injuries and vehicles involved, plus the name and location of the carrier. The database also has the truck's DOT census number, which makes it possible to link the crash database to truck inspection or truck census databases, also available from the Database library.\n\nAviation data (updated)\n\nThe Aircraft Registration Database has information on more than 320,000 planes, including those owned by businesses, individuals and government. Current through May 2002, the database allows journalists to find the owner of an aircraft by looking up the registration number (commonly called the \"N number\"). This comes in handy for covering plane crashes, especially if someone can identify the N number on the plane's tail. The data also provides the plane's serial number, which is useful when searching the FAA Service Difficulty Reports database, also available from NICAR, for reported mechanical problems.\n\nFind out about accidents and other incidents involving airplanes in the United States with the FAA Accidents and incidents database, from 1990 through August 2002. The database contains reports of crashes, collisions, deaths, injuries, major mechanical problems and other events, from small private planes to large commercial carriers. Another database available from NICAR has accidents and incidents from 1973 to 1989.\n\nThe Database Library offers other transportation databases, including vehicle recalls and complaints; accidents involving hazardous materials and the NASA Air Safety Reporting System.\n\nTo order or find out more about these databases, visit www.ire.org/datalibrary or call 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-aviation-data-updated","slug":"2002-november-december-aviation-data-updated","url":"https://dwillis.github.io/uplink/article/2002-november-december-aviation-data-updated.html","year":2002,"month":"November December","monthNum":11,"headline":"Aviation data (updated)","kicker":"NICAR UPDATE","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"NICAR provides journalists with aircraft registration data covering over 320,000 planes and FAA accident/incident databases spanning from 1973 to 2002, enabling reporters to investigate plane crashes and mechanical issues. The data library offers additional transportation databases including vehicle recalls and hazmat accidents.","keywords":["aircraft","accidents","databases"],"topics":["Public Safety","Data & Databases"],"technologies":[],"full_text":"Aviation data (updated)\n\nThe Aircraft Registration Database has information on more than 320,000 planes, including those owned by businesses, individuals and government. Current through May 2002, the database allows journalists to find the owner of an aircraft by looking up the registration number (commonly called the \"N number\"). This comes in handy for covering plane crashes, especially if someone can identify the N number on the plane's tail. The data also provides the plane's serial number, which is useful when searching the FAA Service Difficulty Reports database, also available from NICAR, for reported mechanical problems.\n\nFind out about accidents and other incidents involving airplanes in the United States with the FAA Accidents and Incidents database, from 1990 through August 2002. The database contains reports of crashes, collisions, deaths, injuries, major mechanical problems and other events, from small private planes to large commercial carriers. Another database available from NICAR has accidents and incidents from 1973 to 1989.\n\nThe Database Library offers other transportation databases, including vehicle recalls and complaints; accidents involving hazardous materials and the NASA Air Safety Reporting System.\n\nTo order or find out more about these databases, visit www.ire.org/datalibrary or call 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-another-member-of-the-family-moves-on","slug":"2002-november-december-another-member-of-the-family-moves-on","url":"https://dwillis.github.io/uplink/article/2002-november-december-another-member-of-the-family-moves-on.html","year":2002,"month":"November December","monthNum":11,"headline":"Another member of the family moves on","kicker":"","author_name":"David Herzog","author_title":"","affiliation":"IRE and NICAR","summary":"Aron Pilhofer, former director of IRE's Campaign Finance Information Center, has joined the Center for Public Integrity and will continue contributing to journalism through regular features on open source software tools.","keywords":["campaign","software","databases"],"topics":["Journalism & Newsroom","Technology","Government & Politics"],"technologies":["Python","Linux","MySQL","TUTOS","OpenOffice"],"full_text":"As some of you know, Aron Pilhofer, who directed IRE's Campaign Finance Information Center, has taken a job as database editor with the Center for Public Integrity in Washington, D.C.\n\nFor more than a year, Aron had been busy with IRE and NICAR projects. He revamped the CFIC Web site and launched its two online databases where IRE members can search for federal political campaign contributions and simply click on the contributor's name to see whether the contributor received federal contracts.\n\nAron revived Tracker, the center's quarterly publication that covers campaign finance reporting, and trained journalists across the country in how to analyze campaign finance data for news reporting.\n\nAron served as NICAR's in-house open source software guru, promoting the virtues of free software tools with all kinds of strange names - Python, Mandrake, MySQL, TUTOS. When Aron spoke at the annual CAR conference in Philadelphia this year about \"World-Class CAR on 99 Cents a Day,\" he wasn't kidding.\n\nStarting with the next issue of Uplink, Aron will contribute a regular feature focusing on open source software that journalists can use for their work. For starters, he'll write about his experience with OpenOffice, the open source alternative to Microsoft Office that's been garnering some interest on the NICAR-L listserv.\n\nBest wishes in your return to the newsroom, Aron.\n\nContact David Herzog by e-mail at dherzog@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-upcoming-car-census-training","slug":"2002-november-december-upcoming-car-census-training","url":"https://dwillis.github.io/uplink/article/2002-november-december-upcoming-car-census-training.html","year":2002,"month":"November December","monthNum":11,"headline":"Upcoming CAR, census training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering training workshops in computer-assisted reporting and Census 2000 data analysis for journalists, with multiple events scheduled in January and February across different locations.","keywords":["training","census","journalism"],"topics":["Journalism & Newsroom","Data & Databases","Investigative Methods"],"technologies":["spreadsheet"],"full_text":"IRE and NICAR has numerous training opportunities in the coming months for journalists who want to learn computer-assisted reporting and data analysis for reporting on Census 2000. For a complete list of training events visit www.ire.org/training/otr.html.\n\nHere are a few of the many training highlights:\n\nThe Jan. 5-10 computer-assisted reporting boot camp in Columbia, Mo., provides hands-on training in how to use spreadsheets and database managers to analyze data for high impact stories.\n\nThe Jan. 11-12 Census reporting workshop in Columbia, Mo., will offer a track for beginners and another for journalists with intermediate census reporting skills. Paul Overberg of USA Today and Steve Doig of Arizona State University will lead the classes. Hands-on sessions are included.\n\nA Feb. 21 census workshop at Arizona State University in Tempe is intended to help beginners learn how to find stories in the data that's already been released from Census 2000. It offers a good introduction for beat reporters, graphic artists or assignment editors who need a focused, concrete introduction to demographic data and computer-assisted reporting.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-covering-income-commuting-with-sf3","slug":"2002-november-december-covering-income-commuting-with-sf3","url":"https://dwillis.github.io/uplink/article/2002-november-december-covering-income-commuting-with-sf3.html","year":2002,"month":"November December","monthNum":11,"headline":"Covering income, commuting with SF3","kicker":"CENSUS 2000","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"The U.S. Census Bureau released Summary File 3 data in August and September 2000, containing detailed demographic information from the 2000 Census that journalists are using to investigate stories like gender wage gaps and changes in the Black middle class, though data limitations often prevent reporters from reaching definitive conclusions about causation.","keywords":["Census","data","journalists"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["Summary File 3","PDF"],"full_text":"The flood of Census data is over. For now.\n\nU.S. Census Bureau servers, like journalists across the U.S., worked overtime for two months as the Summary File 3 became available, state by state, in August and September.\n\nIRE's Census-L listserv buzzed with questions and stories. Journalists communicated by code, it seemed. Sample question on the Census-L: \"Are there numbers for median household income by race for 1990? I'm looking in STF3 and not seeing such a table. Anyone?\" Answer: \"Table P82.\"\n\nIncredibly, most Census-L subscribers understood that answer. It refers to a specific table released after the 1990 Census.\n\nThe SF3 data came through as a series of tables, derived from the Census Bureau's April 2000 long-form survey, sent to about one of six households. With that, the agency took the responses and crafted summaries, released in 76 data files for each state, plus a file containing geographic information.\n\nAnd while the data can - and will - keep reporters busy for months to come, there are stories aplenty to examine and learn from.\n\nThe gender gap\n\nThe Detroit News, for example, looked at the slow improvement in the \"gender gap\" - the difference between what men make and what women make in the same jobs.\n\nBrad Heath conducted data analysis for the newspaper. A big challenge, he said, was not to go too far.\n\n\"The biggest problem was that the data stopped just shy of giving me the smoking gun I was looking for,\" he said. \"The data the Census Bureau provided show inequality to be sure, but they can't get you far enough to show actual bias.\"\n\nStill, The Detroit News could report that while every state has made progress in closing the gender gap from 1989 to 1999, Michigan lagged. In that state, women who worked full time made 67 cents for every dollar earned by men, a figure that budged less than 5 cents in the 1990s. Nationwide, women who worked full time in 1999 earned 73 cents for each dollar men earned.\n\nLike almost any database, though, the Census data didn't answer the question: Why?\n\n\"The census made clear that women and men work at different jobs and have different educations, both of which clearly play a role in the earnings disparity,\" Heath said. \"But there wasn't enough there to come up with a solid answer to why Michigan ranks so low in pay for women. That was frustrating. And because the Census Bureau changed some of its job categories in the past decade, it wasn't really possible to show how well women are making inroads into higher-paying fields here.\"\n\nComplaints about comparing 1990 and 2000 data items were common headaches expressed on Census-L traffic for more than a year.\n\nRising black income\n\nDwight Worley of The Journal News of White Plains, N.Y., encountered challenges in reporting on the rise of the black middle class population in New York in general and The Journal News' circulation area in particular.\n\nOne worry: error margins. What might look like a big change or trend could turn out to be statistically insignificant, depending the error margins or how much of the data is imputed - that is, estimated - by the Census Bureau.\n\n\"So you have to be careful - especially when you're comparing data over several decades,\" Worley said. \"That's why we stuck with using state and county-level data, which have lower error margins than town or place level data.\"\n\nTo make those several-decades comparisons, Worley looked at income, education, employment, poverty and home ownership factors, from the SF3 in 2000 and from either Summary Files 3 or 4 from earlier years. He perused PDF files on the bureau's Web site for 1990 SF4 information. Going back to 1980 and 1970 was even more difficult. He finally went to government document libraries in New York City and Westchester County to go through books and microfiche images and just plugged in numbers by hand.\n\nThen there's the definition of a household vs. a family. Here's how the Census Bureau divides the two groups: A household includes all of the people who occupy a housing unit. A family includes a householder and others, living in the same household, who are related to the householder by birth, marriage or adoption. Households, then, may include more members than do families. And not all households contain families, since a household may be composed of a group of unrelated people or of one person living alone.\n\nThere are tables that break down the number of workers by family type and the aggregate income of each type, but not by race. Household income is broken down by race, though, and counts everyone - not just those who constitute a family.\n\nBut, Worley said, even that can leave questions. \"If there are, on average, four workers in a black household now when there used to be only two in 1970 then that could seriously alter the analysis of rising incomes. In that instance, household income is rising, but it could be because there are more workers in the house, not necessarily because individual blacks are earning more money.\"\n\nRace, ethnicity, money\n\nThe Miami Herald found a rise in black income in Florida. That newspaper used median income by race and ethnicity to make a comparison down to the level of places that have a population of at least 10,000.\n\nThe Census Bureau provided a 1990 table of those places and median income.\n\nSaid The Miami Herald's Tim Henderson: \"That was fine as far as it went, and showed us that there was a large increase in black income in Florida, but it wasn't reflected in our urban county (Miami-Dade) - only in our suburban county, Broward. There, black income soared and reached 81 percent of white non-Hispanic income, better than anywhere else in the state.\"\n\nOne of the problems of using only places and counties, he said, is that it could mask changes in areas that weren't marked by municipal boundaries.\n\n\"We could see this clearly when we mapped tracts by black income, screening for tracts with a significant black population [at least 10 percent],\" he said. \"A new regional area of black affluence stood out clearly at the western border of the two counties, and the ethnic makeup of those tracts made it clear that Jamaicans were playing a role in that new affluence. African-Americans also dominated in the adjoining Dade County tracts.\"\n\nThose points made a story about the region's ethnic chemistry a positive note in light of the recurrent criticism that the region has no black middle class to provide hope and support for the poverty-stricken inner cities, Henderson said.\n\nCommuting with CAR\n\nAny movement from the middle of a city means an increase in the number of commuters and the time they spend on the roads. Matt Waite followed commuters in Pasco County, Fla., for the St. Petersburg Times.\n\nEconomic demographics play a large part in the commuter game.\n\n\"Pasco County is made up of three distinct areas: west, central and least,\" he said. \"The east is the same as it was 40 years ago - rural. Central is the rapidly developing area of $150,000-plus homes. West is where all the retirees moved in the '60s and '70s, and they are all now dying off, leaving their $40,000 to $60,000 homes behind. The people moving in are 20- or 30-something, with kids.\"\n\nThose moves, of course, mean an interesting mix of commuters.\n\n\"You have a population of people buying low-cost homes who have to commute long distances, which is counterintuitive,\" Waite said. \"You expect more well-off people driving a ways to their downtown offices. But you don't expect to find an area with a $23,000 median household income driving an average of 30 minutes to work every day.\"\n\nAnd, like the gender gap or income stories, the data can't tell the whole story. In Waite's case, the big question is where the commuter is going. The data can only tell where the commuter lives.\n\nNow journalists are awaiting the arrival of still another data release - called Public Use Microdata Sample, or PUMS. Beginning with a release tentatively set for December, PUMS data is a sample of the actual responses. With that, journalists can go beyond the Census Bureau's cross-tabs and build their own summary tables.\n\nContact Jeff Porter by e-mail at jeff@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-access-forms-help-build-inspections-database","slug":"2002-november-december-access-forms-help-build-inspections-database","url":"https://dwillis.github.io/uplink/article/2002-november-december-access-forms-help-build-inspections-database.html","year":2002,"month":"November December","monthNum":11,"headline":"Access forms help build inspections database","kicker":"ELEVATOR SAFETY","author_name":"Mike Sherry","author_title":"reporter","affiliation":"St. Louis Post-Dispatch","summary":"A journalist used computer-assisted reporting to investigate elevator safety in Missouri, creating a database to analyze inspection violations that had never been systematically evaluated before, revealing serious hazards like broken phones and malfunctioning machine room doors.","keywords":["elevators","database","violations"],"topics":["Data & Databases","Investigative Methods","Public Safety"],"technologies":["Access"],"full_text":"Elevators have become as much a part of everyday life as cars.\n\nBut while journalists investigate many facets of vehicle safety, such as fatal accidents and faulty equipment, there is precious little reporting about elevators - even though they carry millions of people every year.\n\nSo, when I came across the Missouri Elevator Safety Board on the state Web site, it piqued my interest. If the state was worried enough about elevator safety to appoint a board to oversee it, then perhaps a story was there.\n\nThe result was a computer-assisted reporting story - written as part of my master's degree final project at the University of Missouri School of Journalism - that ran July 7 in the St. Louis Post-Dispatch. Given that the United States has an estimated 700,000 elevators, this is a story that can be replicated in just about any community.\n\nSome of the unique challenges of this story forced me to learn important CAR lessons that others can benefit from.\n\nDesigning databases\n\nUsing Access 97 and Access 2000, I grappled with database design for a relatively small database. It had many tables, but the largest one only had about 20,000 records. Everyone needs to think about database design when information must be entered from paper records.\n\nThe state Elevator Safety Unit had an extensive Access database that included a lot of data, such as inspection dates and elevator locations. But the unit was so understaffed that the staff had no time to enter all the violations uncovered by inspectors. The violations were detailed in color-coded folders (with a different color for every month) that contained more than 10,000 reports. I took a random sample of about 260 reports that had an average of about four violations. This gave me a good idea of the types of problems inspectors found.\n\nNext, I created an Access form for data entry using a Tech Tip in the April 2000 Uplink as a guide. (Uplink articles from 1990 to the present are available to IRE members at www.ire.org/resourcecenter.)\n\nThe problem, however, was that inspectors did not record violations consistently. Some inspectors referenced particular sections of the code book, some just jotted down notes, and some did both. I needed consistency, however, so I could use Access to group by and count the records to identify the most common violations.\n\nLearning lookups\n\nSo I created a lookup table, where I coded each type of violation with a number. For instance, machine room door problems got a code of \"3.\" I linked my data-entry form to that table, which gave me a drop down menu each time I needed to enter a particular violation.\n\nAfter running the queries, I found that many elevators had serious problems like broken phones. Machine room doors that didn't close and lock on their own were also hazards because someone could walk in and be hurt or damage equipment that could harm passengers. This was the first time anyone - including agency staff - evaluated the seriousness of the problems.\n\nSetting up the violation codes solved one quandary, but another remained: how to link the violations table with the activity table that the Elevator Safety Unit had created to keep track of inspection dates and other information.\n\nEach elevator had a unique state ID number, which I entered in my violations table. The problem, however, was that each elevator often appeared more than once in the activity table - mainly because the state entered a record when that elevator was registered, then created separate records for the subsequent inspections. Meanwhile, each one of my violation records included the equipment ID, so if an elevator had four violations, that ID would be in four records. I needed to modify the tables so I could avoid a many-to-many join.\n\nI needed a unique identifier to identify each inspection, so I created one in both tables. Then I ran update queries that combined the equipment ID with the inspection date.\n\nCreating a violation code table and unique IDs may seem like old hat to CAR veterans, but for people like me - who have not built a lot of databases - these were good skills to learn. They apply to any sort of story that requires building a database or guarding against many-to-many joins.\n\nFinding the story\n\nBut what about the story? I found an agency that was underfunded and understaffed. The database analysis showed that the state had inspected only about 40 percent of the elevators in Missouri, even though the program was created nearly a decade ago. Additionally, of those that were inspected, about 13 percent went more than year before they were reinspected - letting potentially harmful deficiencies persist.\n\nThe database analysis showed that the state had inspected only 40 percent of the elevators in Missouri.\n\nStates handle elevator inspections differently. Some make this a state function while others leave it up to municipalities. No matter who is doing it, you will probably find an agency that lacks the manpower to thoroughly inspect elevators annually. Getting a sense of the inspection budget and inspection staff is a starting point for this story.\n\nAdditionally, plan on reading lots of lawsuits - in both state and federal courts. It's a good bet that the enforcement agency lacks great information on accidents, but lawsuits against elevator companies are a treasure trove.\n\nContact Mike Sherry, a former data analyst in the IRE and NICAR database library and now a reporter for CQ Weekly by e-mail at MSherry@cq.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-mapping-the-census","slug":"2002-november-december-mapping-the-census","url":"https://dwillis.github.io/uplink/article/2002-november-december-mapping-the-census.html","year":2002,"month":"November December","monthNum":11,"headline":"Mapping the census","kicker":"MAPPING IT OUT","author_name":"","author_title":"","affiliation":"","summary":"Journalists across the U.S. are using GIS mapping technology with Census Bureau data to uncover demographic patterns, including language barriers, poverty pockets, and commuting disparities among different communities.","keywords":["census","mapping","demographics"],"topics":["Demographics & Census","Investigative Methods","Technology"],"technologies":["ArcView"],"full_text":"The U.S. Census Bureau's latest big data release - long-form questionnaire results reported down to the tract or block group - is keeping journalists busy as they report the latest economic, social and demographic trends. (See article on page 3) Many of the journalists have been using geographic information system (GIS) programs to uncover patterns and changes. Here are some recent examples from newsrooms across the country:\n\nLanguage barriers\n\nRon Campbell at The Orange County Register used ArcView 3.2 and Census long-form data to find areas with the highest concentrations of residents over 14 who did not speak English fluently. The map he generated (Map 1) displayed the tracts with the highest non-English speaking rates using darker shades.\n\nOne Santa Ana neighborhood - with 57 percent of its population over age 14 not speaking English - had the highest rate. Another reporter visited the neighborhood and found residents who routinely spoke Spanish at home, church and inside stores.\n\nA demographer quoted in the story noted that the neighborhood, like some others, is the point of arrival for new immigrants who find it easier to adapt by living among people who speak the same tongue.\n\nSome residents described the tension between generations: Parents wanted their children to speak Spanish, while their children wanted to learn English.\n\nRich and poor\n\nMapping helped Garry Lenton, the CAR coordinator at The (Harrisburg, Pa.) Patriot-News, find pockets of poverty in Cumberland County. The county has one of the highest median incomes in the Harrisburg region, yet it had the third-highest poverty rate increase among Pennsylvania's 67 counties.\n\nLenton wanted to learn the extent of poverty in the county's municipalities. After posting a request for advice to the NICAR-L listserv, Lenton settled on mapping the percentage of people meeting federal poverty guidelines by community and running the numbers of families in poverty by municipality within the newspaper's graphic. (Map 2)\n\nMapping by percent revealed that some of the sparsely populated communities, mainly in the rural western portion of the county, suffered the highest poverty rates. Reporters for the paper later found that a boom in the number of low-income housing units had helped fuel the increase.\n\nCommuting patterns\n\nJournalists have written plenty of stories showing how affluent commuters are willing to spend more time on the road so they can have big houses in the suburbs. Using GIS, journalists at the Fort Worth Star-Telegram discovered another pattern.\n\nCAR specialist Jeff Claassen analyzed long-form commuting pattern data by tract with ArcView 3.2 to identify areas where a higher percentage of commuters left before 7 a.m. for work. The newspaper found that people living in low-income inner city neighborhoods had some of the longest and earliest commutes (Map 3).\n\nThe Star-Telegram used its findings for a \"Your Commute\" page report dedicated to Census 2000 commuting statistics for the newspaper's area.\n\nWould you be willing to share a mapping example with fellow journalists? Send an electronic copy of the map along with details to David Herzog at dherzog@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-data-shows-judges-set-suspects-free","slug":"2002-november-december-data-shows-judges-set-suspects-free","url":"https://dwillis.github.io/uplink/article/2002-november-december-data-shows-judges-set-suspects-free.html","year":2002,"month":"November December","monthNum":11,"headline":"Data shows judges set suspects free","kicker":"SPOTLIGHT: CRIME & JUSTICE","author_name":"Joe Ellis and Brian Collister","author_title":"Reporter","affiliation":"KMOL San Antonio","summary":"Journalists in San Antonio investigated Bexar County's criminal justice system and discovered thousands of cases were dismissed due to lack of speedy trial, after fighting over a year for access to court records data that county officials initially tried to prevent them from obtaining.","keywords":["investigation","courts","dismissed"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Access"],"full_text":"Thousands of accused criminals in Bexar County, Texas, are off the hook because the county justice system couldn't get their cases to court fast enough.\n\nWe broke the story after four months of investigating, but fought more than a year for the data. No other news organization in San Antonio had investigated the Bexar County Court system with a database of its court records. And some county officials wanted to make sure we didn't either.\n\nThe battle began in December 2000. We asked for all Bexar County criminal court records to be provided electronically. We needed a complete database to see how judges run their courts.\n\nAt first, the county's Information Services Department quoted a $17,000 cost for our request, citing costly programming since there was no existing program for the job. After gathering advice from the NICAR-L listserv, researching the cost, and seeking help from the state's General Services Commission, we negotiated the cost down to about $2,000. But the county folks still dragged their feet.\n\nVia the Texas Open Records Law, we monitored the correspondence between county officials regarding our request and bothered them constantly. Agreed deadlines for providing the data were not met. So finally, we contacted the company that set up the county's computer records system and hired one of its employees to write and run the program for $1,000. Within weeks, we had a \"flat file\" we imported in to an Access database.\n\nQuerying the data\n\nOur first objective was to see how tough - or lenient - the criminal court judges were on crimes like DWI, drugs, assaults or sex offenses. However, when we looked at the \"COURT DISPOSITION\" field, we kept noticing cases \"dismissed for lack of speedy trial.\" That got our attention. So we isolated all the \"DSMD-SPEEDY TRIAL\" cases, and we counted them by the \"DISPOSITION YEAR\" field we created from the \"DISPOSITION DATE\" field in the data table.\n\nWe noticed a dramatic increase from the 607 cases dismissed for \"lack of a speedy trial\" between 1996 and 1998 to the 5,200 cases dismissed for the same reason from 1999 to 2001. We decided to focus on the latter three-year period and filtered out those 5,200 records and put them in a separate table.\n\nThrough further querying we calculated how long these cases were on file. Some were 15, 10, and five years old. But it seemed strange that many would be less than two years, or even one year old. Law experts agreed.\n\nWe then started looking through hundreds of individual case files, looking for answers, and people to help with our story.\n\nIn those files we found victims like Virginia, who had been beaten by her husband once and had a restraining order against him when he assaulted her the second time.\n\nShe didn't know, until we told her, that a criminal court judge had thrown out the case against her now ex-husband because he didn't get a speedy trial.\n\nA \"count\" query of the \"OFFENSE DESCRIPTION\" field of our data table, showed more than 400 other assault cases that didn't get to court fast enough.\n\nWe also found 734 DWI-related cases where the driver was never prosecuted, including the driver who hit Myrna Ellison.\n\n\"I assumed something was done,\" Ellison said.\n\nNothing was done, though. Even though it was the driver's second DWI arrest, Judge M'Liss Christian dismissed the case.\n\nChristian wouldn't agree to an interview, but when we caught up with her outside the courthouse, she claimed the misdemeanor cases she was tossing didn't have victims.\n\nThe judge obviously never met Myrna and Virginia.\n\nOverall, we found Judge Tim Johnson, a judge with 14 years on the bench, tossed out the most cases with 1,528 speedy trial dismissals. Fourth-year judge Al Alonso had 1,057 dismissals, followed by seventh-year judge Karen Crouch with 920, and Christian with 802 dismissals.\n\nNone of these judges would agree to an on-camera interview, but Johnson, sent a statement on behalf of all county court judges: \"The cases dismissed in Bexar County are dismissed under appropriate circumstances with the proper procedures and are justified both by statutory and case law.\"\n\nBut we found the courthouse crisis unique to Bexar County.\n\nWe checked with the Office of Court Administration in Austin and its records show: Harris (Houston), Dallas (Dallas), and Travis (Austin) County Courts combined report only 129 cases dismissed for \"lack of a speedy trial\" over the past three fiscal years. Bexar County courts report dismissing close to 4,200 cases on speedy trial grounds over the same period.\n\nTallying the cost\n\nCourthouse sources explained that the judges had created a \"speedy trial docket\" in 1999, an aggressive dismissal procedure. On the docket, judges choose cases they want dismissed and assign them to attorneys who help get rid of them, sometimes hundreds at a time.\n\nFor taxpayers, we found another concern.\n\nWe obtained electronic payment records from the Bexar County Auditor's Office. This data details how much money each court-appointed attorney made in each court, case by case, for the past five years. With the \"CASE NUMBER\" field as the unique identifier, we matched the payment records with the speedy trial dismissals for the three-year period. We soon realized Bexar County Criminal Court judges were using thousands and thousands of dollars in public funds to get rid of cases never even heard in court.\n\nBy joining and comparing the court records data table and the attorney payment records table we discovered more than $100,000 in taxpayer money spent by judges appointing criminal cases on the \"speedy trial docket.\" Christian was tops in this category, spending $28,076. Alonso spent $21,446, while Crouch spent $17,623 on appointed attorneys.\n\nWe found that basically the judges bypass the district attorney's office and handpick cases they want dismissed. They also handpick attorneys to help dismiss them. One lawyer, who asked not to be identified in our story, described the dismissal process.\n\nBecause of our investigation the county commissioners met to propose plans to stop judges from spending public funds dismissing cases and letting accused criminals go free. The commissioners created a task force to come up with some recommendations. The suggestions include: court coordinators and the County Warrants Division work together to make sure warrants are served; judges and the district attorney checking caseloads in criminal courts to make sure cases are heard in a timely manner; and fugitives being picked up when getting a marriage license, car registration, etc.\n\nAlso, local interest groups filed complaints with the Texas Commission on Judicial Conduct asking them to investigate and publicly reprimand the judges. The commission will not confirm or deny any investigation in progress.\n\nWithout computer-assisted reporting, the problem may not have been uncovered. We wouldn't suggest that other reporters will find the same problem in their community, but we would suggest obtaining similar data. We've produced several stories from this data and found it to be a valuable resource in the newsroom for enterprise reporting and daily use.\n\nThe key is doing one story at a time and staying focused. And when requesting and negotiating for data, never back down, matter how long the process may take.\n\nContact Joe Ellis by e-mail at jellis@kmol.com Contact Brian Collister by e-mail at bcollister@kmol.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-analysis-finds-juries-lack-color","slug":"2002-november-december-analysis-finds-juries-lack-color","url":"https://dwillis.github.io/uplink/article/2002-november-december-analysis-finds-juries-lack-color.html","year":2002,"month":"November December","monthNum":11,"headline":"Analysis finds juries lack color","kicker":"SPOTLIGHT: CRIME & JUSTICE","author_name":"Mark Houser","author_title":"","affiliation":"Tribune-Review","summary":"A Tribune-Review investigation using surveying and GIS analysis revealed that Allegheny County's jury selection process systematically underrepresents Black residents, with Black people making up only 4 percent of jurors despite comprising 11 percent of the county's adult population. The article provides guidance on how other journalists can use computer-assisted reporting techniques to examine whether jury selection in their own areas reflects demographic fairness.","keywords":["jury","representation","analysis"],"topics":["Crime & Justice","Demographics & Census","Investigative Methods"],"technologies":["ArcView","Excel","TIGER"],"full_text":"Blacks have as much right as whites to serve on a jury, according to a U.S. Supreme Court ruling that's more than a century old. You might not know it from looking in an Allegheny County jury room.\n\nThe recent Tribune-Review story \"A jury of peers?\" used surveying and GIS analysis to show how a system that's supposedly designed to randomly pick a fair cross-section of the community for jury duty consistently overlooks blacks.\n\nResidents of the county's overwhelmingly white neighborhoods are twice as likely to get called for jury duty as residents of black neighborhoods, the story showed. While blacks make up 11 percent of the county's adult population, they make up only 4 percent of those serving on jury panels.\n\nAs a result, black defendants - who outnumber white defendants in our courts - often have their fates decided by all-white juries.\n\nIt's possible that courtrooms in your area have similar problems. Just take a look inside jury duty rooms and see whether the racial makeup of the jurors matches the demographics from your county's census numbers. Here's how you can use CAR to answer some key questions.\n\nSurveying for race\n\nPick a week or two and go to the jury room every morning before the cases start. Ask the court administrators how many jurors they have that day. Then count the number of black jurors. (When I was unsure, I briefly interviewed the person and asked his race.)\n\nWhen I did this over two weeks I had counted 1,031 jurors, 4 percent of them black. That gave me a very small margin of error, plus or minus 1.2 percent at a 95 percent confidence interval.\n\nPittsburgh has almost no Hispanics, so I did not count them. If you want to look at Hispanics in the jury room, you have to ask everyone his or her ethnicity. Or you can substitute mapping analysis.\n\nMapping juror lists\n\nAnother way to test for fairness is to map the addresses of people called for jury service and see whether residents of minority neighborhoods are as likely to be called for jury duty. You'll need to get the court's jury array - the list of people called to serve each day - along with their addresses and the dates served. Try to get a least a year; I got 18 months of potential jurors in a database.\n\nEven in privacy-mad Pennsylvania (it has the fourth-worst public records laws in the United States, according to the Better Government Association, a nonpartisan watchdog group based in Chicago) court administrators must release the jury arrays.\n\nSince the arrays document everyone called for jury duty, not just the people who showed up, the list is a better measure of how well the court administrators are summoning a \"fair cross-section of the community,\" as the Supreme Court ruling requires.\n\nOnce I had the arrays, I needed to remove duplicate names. Our county court jury arrays are made public 30 days before jury duty, so there are always people on it who later are granted a postponement. Those people then show up months later on another array when they're rescheduled for duty. To find them, I sorted by last names and street names, then made a separate column with an \"=IF\" test to see if a cell repeated the one above it. (For details see sidebar on page 12).\n\nWith the duplicates removed from the data, I was ready to geocode those addresses and create a point map showing the residences of the potential jurors. ArcView 3.2, using Census TIGER street files, matched about 91 percent of the addresses off the bat. I got the match rate up to 99 percent by fixing mistakes in addresses, using the StreetMap geocoding extension and manually plotting particularly troublesome addresses.\n\nOnce I had created the shapefile of 45,000 juror points, I overlaid it onto a census block group layer with Census 2000 demographics. After that, I opened the attribute table for the potential jurors and block group layers and performed a spatial join to append the demographic data to the juror layer. Then I used the Summarize function to count the potential juror points inside each block group.\n\nUsing the Census 2000 race demographics for block groups, I created an Excel spreadsheet with two key numbers for each: the percent of the adult population that is black, and the number of jurors per 1,000 adults population, which I called the \"jury service ratio.\"\n\nThe average jury service ratio in block groups that were at least 98 percent white was 53 per 1,000. The average ratio in block groups that were at least 50 percent black was 26 per 1,000. There was my lede: Black neighborhood residents were half as likely to be called for jury duty. I selected 98 percent white to make sure I excluded suburban neighborhoods where a significant number of blacks also live. I calculated similar numbers for the county's 130 municipalities, so I could refer to them in stories and graphics, because nobody knows what block group he lives in.\n\nI calculated them for most of Pittsburgh's neighborhoods. Like many cities, Pittsburgh has several black and white enclaves. Instead of saying that in Pittsburgh 32 of 1,000 adults got called for jury duty, it was more telling to say that 20 did in (black) Larimer, and 57 did in (white) Regent Square.\n\nIf you have clearly defined Hispanic neighborhoods, you might be able to use GIS analysis to make up for difficulties in using jury room surveys to identify Latinos.\n\nExplaining patterns\n\nIf you find the similar results, the next step is to determine why. This was the most difficult part of the story, and I only was able to suggest some possibilities.\n\nMost jury pools are picked by computer from voter rolls and driver's licenses lists. Many people say, \"Aha! Blacks don't vote as often as whites.\" Maybe. But according to a Census Bureau study of the 2000 election, blacks register at almost the same rate as whites. \"Motor Voter\" laws prevent election authorities in Pennsylvania from regularly purging voter rolls, so the lists are useless in trying to see if black neighborhoods have fewer registered voters.\n\nLicense records in Pennsylvania lack a racial designation for drivers. And the state Department of Transportation is hyper-secretive about those records so it is impossible to check if there are fewer licenses per adult population in black neighborhoods.\n\nI found a very likely culprit for the jury pool discrepancies by looking elsewhere. The jury commission sends \"qualifying questionnaires\" to the people its computer picks to make sure they will be able to serve. A third of those never come back. The commission ignores the non-responses, even though at least half of them are returned because of a wrong address - in other words, the commission fails keep its address records current.\n\nWe know from the census that blacks in the county are more likely to rent. Renters move more often. Ergo, blacks are more likely to be missed by the jury commission because of a recent move.\n\nThe commission does an even worse job finding people in the neighborhood around the University of Pittsburgh. Residents there are mainly students, and they move every year. Only 10 adults per 1,000 in the Pitt neighborhood got called for jury duty.\n\nIs that another story? Not likely. The Supreme Court has ruled that people can't challenge a jury room for being disproportionately old.\n\nContact Mark Houser by e-mail at mhouser@tribweb.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-checking-for-duplicates-in-excel","slug":"2002-november-december-checking-for-duplicates-in-excel","url":"https://dwillis.github.io/uplink/article/2002-november-december-checking-for-duplicates-in-excel.html","year":2002,"month":"November December","monthNum":11,"headline":"Checking for duplicates in Excel","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A spreadsheet technique using IF formulas to identify duplicate jury duty summons by comparing last names, first names, and street addresses, allowing manual review to remove true duplicates while preserving legitimate multiple records.","keywords":["duplicates","spreadsheet","filtering"],"topics":["Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"To find duplicates in the list of people called to jury duty, I sorted the list by last name, first name, and street name. Then I ran an \"=IF\" test on all three to see if the value in one cell was the same as in the cell below it.\n\nThis statement shows you if the last name is duplicated in the cell below for the spreadsheet pictured here:\n\n(A2=B2, dup\",\n\nIf there is a duplication, Excel inserts the word \"dup\" the cell. I calculated a similar formula for first names and streets. Then by filtering I could find the triple \"dups\" and delete them, leaving only one record for the person. I checked double \"dups\" and if they appeared to be true duplicates, I deleted them.\n\nIn this example Carolyn Messenger is a triple \"dup.\" She was initially summoned in March 2002 and then summoned again two months later. That means she postponed her first jury duty summons, because you can't serve twice in a year. I deleted that row.\n\nLynda Smith has two addresses. I can't know if she's a duplicate from this list, or if there are two Lynda Smiths, so I let both records stay.\n\nRobert Sommer shows that it's important to use first names too. There are two Sommers from Galway Drive, but one's Robert and one's Roselyn. Both stayed.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-elder-care-homes-ripe-for-probes","slug":"2002-november-december-elder-care-homes-ripe-for-probes","url":"https://dwillis.github.io/uplink/article/2002-november-december-elder-care-homes-ripe-for-probes.html","year":2002,"month":"November December","monthNum":11,"headline":"Elder care homes ripe for probes","kicker":"CAR INVESTIGATIONS","author_name":"Stephanie Kang","author_title":"Contributing Editor","affiliation":"IRE and NICAR","summary":"Journalists across the country use database reporting and open records requests to investigate poorly regulated assisted living centers and nursing homes, revealing widespread understaffing, patient neglect, and inadequate state oversight of the rapidly growing elderly care industry.","keywords":["eldercare","regulation","investigation"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["FoxPro","SPSS"],"full_text":"Elder care homes ripe for probes\n\nBy Stephanie Kang, IRE and NICAR\n\nAs America's elderly population explodes, journalists use database reporting to explore the state of assisted living centers and nursing homes around the country.\n\nAt just more than 20 years old, the assisted living industry is still relatively new. Arising out of dissatisfaction with nursing homes, assisted living centers cater to the elderly with financial means and little need of intensive nursing care. Yet because they are regulated by states, not by the federal government as most nursing homes are, monitoring assisted living care can become difficult. Assisted living centers and nursing homes have come under fire for lax regulation, understaffing, and patient abuse and neglect.\n\nMonths of anecdotal stories from state inspectors prompted Milwaukee Journal Sentinel reporter Mary Zahn to investigate the conditions of Wisconsin assisted living centers. A six-month investigation resulted in \"Overwhelmed and Broken Down: Caring for the Elderly and Disabled,\" published in August 2001. Zahn used Wisconsin's open records law to obtain a database of state inspections and citations against nursing homes and assisted living centers. The series found that hundreds of elderly and disabled people in assisted living facilities had been injured and put at risk. In addition, 10,000 of the state's disabled or elderly residents had been forced to wait for months or even years to move into an assisted living facility. (Story No. 18778 in the IRE Resource Center)\n\nReporters for The Oregonian built and used existing databases while investigating the assisted living industry, focusing on Keren Wilson and the company she created, Assisted Living Concepts. For a series that ran in June 2001, they built several databases to track investigations of centers run by the company, entering information from investigative reports. They also tailored a complaints database that they got from the Oregon Senior and Disabled Services Division. An analysis of the data showed that Assisted Living Concepts had a track record that was worse than the state average. (Story No. 18972)\n\nIn \"Who Will Care For Them? Ohio's Elder Crisis\" the Dayton Daily News examined how Ohio cares for its elderly. The Daily News analyzed more than a half dozen state regulatory databases with FoxPro and SPSS. Some of the databases had never been publicly available in Ohio and the reporters had to negotiate with health department officials to obtain them.\n\nThe Daily News discovered a system \"dangerously under-scrutinized,\" where the elderly often are forced to choose between costly assisted living care centers and state-funded nursing homes. The Daily News also found that state inspection personnel reductions and a 70 percent decrease in citations for serious health and safety violations threatened to minimize the state's influence on a sub-par facility (Story No. 15917)\n\nIn the questionnaire that accompanies their stories in the IRE Resource Center, the reporters advise: \"Data is virtually limitless. Don't get overwhelmed. Use it to answer your questions, not for fishing expeditions.\"\n\nFinding data\n\nFor starters, federal law mandates that state regulators keep registries of aides working in every assisted living center and nursing home.\n\nNursing Home Compare data (www.medicare.gov/nhcompare/home.asp) allows you to check individual nursing homes in your area or download information to build your own database. The Online Survey Certification and Reporting file (OSCAR) at www.longtermcareinfo.com/crg/about_oscar.html is an administrative database of the Health Care Financing Administration. OSCAR provides survey and data. That means every institutional health-care provider in the United States that is certified to provide services under either Medicare or Medicaid (or both) is listed in OSCAR. Because different types of providers have to report different information during the survey and certification process, OSCAR results can vary.\n\nJournalists also have access to complaint investigation forms. In Missouri, the Social Services Department Division on Aging regulates nursing homes and maintains a \"Statement of Deficiencies and Plan of Correction\" record. Most state health departments will maintain a complaints and enforcement database.\n\nLook for datasets that lay out Medicaid and Medicare eligibility for nursing homes and assisted living centers, as well as a costs database, also from the Health Department.\n\nThese departments usually administer inspections and give licenses for adult care centers, adult residential care homes, intermediate care and skilled nursing centers. These datasets may include facility surveys, and provide operator information and resident demographics.\n\nFor details about the stories by the Milwaukee Journal Sentinel and The Oregonian, see the feature package about investigating assisted living centers in the November/December 2002 issue of The IRE Journal. Also included are tips and sources for covering assisted living and examples of other elder housing stories that used CAR. Order a copy through the IRE Resource Center by calling 573-882-3364 or e-mailing rescntr@nicar.org.\n\nContact Stephanie Kang by e-mail at Stephanie@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2002-november-december-stitching-points-data-when-gis-address-matches-fail","slug":"2002-november-december-stitching-points-data-when-gis-address-matches-fail","url":"https://dwillis.github.io/uplink/article/2002-november-december-stitching-points-data-when-gis-address-matches-fail.html","year":2002,"month":"November December","monthNum":11,"headline":"Stitching points, data when GIS address matches fail","kicker":"Tech tip.","author_name":"James E. Wilkerson","author_title":"GIS specialist","affiliation":"The Morning Call","summary":"A journalist used ArcView GIS software to manually map shooting incidents around housing projects in Allentown by creating a point shapefile and joining it to crime data, since inconsistent location data made automated geocoding impossible.","keywords":["mapping","ArcView","shootings"],"topics":["Investigative Methods","Technology","Crime & Justice"],"technologies":["ArcView","ArcMap","ArcCatalog","dBase","shapefile","TIGER"],"full_text":"One of our reporters recently wanted to get a visual picture of shootings around housing projects over a few months and came to me, because he know that I am using ArcView for mapping. What sounded like a simple project - plotting a list of shooting locations onto a street map of Allentown - turned out much more difficult than imagined, mostly because the location data was inconsistent.\n\nInstead of having an exact street address, the shooting location field often contained an intersection or general block on which the shooting occurred. Geocoding didn't seem to be an option, at least not for the precise placement wanted. But because we were looking at only a couple dozen incidents, I decided to map them by hand.\n\nFirst, I added a map shapefile of city streets that the city's GIS department provided. This street map - like many created by municipal agencies - is more often updated and corrected than the TIGER mapping files provided by the U.S. Census Bureau. The data did not include housing project boundaries, so reporter Keith Herbert and I drove to them and jotted down the streets around the perimeter of the projects. Later, in ArcView 8, we created a new layer for the polygon boundary.\n\nUsing computerized crime data provided by the city police department, I created a spreadsheet containing the incidents to map as points. Each record had basic information about the shooting and a unique identifier that I created, something needed to create a point map. I saved that to a dBASE file and added it as a table to the project.\n\nThen I got stuck trying to figure out how to join the crimes to points on the city map. After posting a query on the NICAR and ESRI listservs, several people suggested that I create a new point file with no data and join it to the list of shootings.\n\nHere are the steps:\n\nWith streets and project layers in a view, I created a new, separate shapefile. In ArcView 8, you do that in the ArcCatalog application. Right click on the drive location where you want to store it and choose New/Shapefile. Name the file and make sure that it is a point file.\n\nAfter adding the point shapefile to a view in ArcMap, open its attribute table and you will see that it is empty, except for three fields.\n\nSecond, I opened the dBASE file of shooting incidents and arranged the table window and map window so I could see both. With the new shapefile opened for editing, I used the Create New Feature pointer tool. For each shooting location listed in the dBASE file, I created a point on the map that was as close as possible to the location described. Each time I clicked on the map to add a new point, ArcMap created a new record in the shapefile.\n\nTo avoid confusion, I worked with one point at a time. I copied the unique ID number of the shooting into the ID field of my new point shapefile, periodically saving the file.\n\nIn a couple of hours, I had a new map of about two dozen shooting locations, which I then joined to the file of shooting data on the common ID field for further analysis. The dBASE file and new shapefile could also be merged into one file, if you prefer, but I kept mine separate.\n\nUnfortunately, the story never made it to print as we originally envisioned it, but the map gave us the ability to add new analysis to the subject, which was essential in deciding whether the story was worth pursuing.\n\nContact James E. Wilkerson by e-mail at james.wilkerson@mcall.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-get-set-for-2003-car-conference","slug":"2003-february-get-set-for-2003-car-conference","url":"https://dwillis.github.io/uplink/article/2003-february-get-set-for-2003-car-conference.html","year":2003,"month":"February","monthNum":2,"headline":"Get set for 2003 CAR conference","kicker":"CHARLOTTE","author_name":"Brant Houston","author_title":"","affiliation":"IRE and NICAR","summary":"Highlights of the upcoming Annual Computer-Assisted Reporting Conference in Charlotte, NC, including hands-on classes and workshops.","keywords":["CAR","conference","Charlotte"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":[],"full_text":"Hands-on classes extended into the evening so you don't have to miss a favorite panel. A special three-day track for those just starting computer-assisted reporting. A series of smaller workshops for those more advanced in CAR. These are just a few of the highlights of IRE's upcoming Annual Computer-Assisted Reporting Conference in Charlotte, N.C., March 14-16. Although we had to do a quick hotel switch (now we are at the Hilton & Towers because of slowed construction of the Westin) we have just as many panels and classes planned. This conference marks the 10th anniversary of IRE's first CAR conference, which also was held in North Carolina.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-february-databases-show-implant-troubles","slug":"2003-february-databases-show-implant-troubles","url":"https://dwillis.github.io/uplink/article/2003-february-databases-show-implant-troubles.html","year":2003,"month":"February","monthNum":2,"headline":"Databases show implant troubles","kicker":"SPOTLIGHT: MEDICINE & HEALTH","author_name":"Robert Gebeloff","author_title":"Reporter","affiliation":"The Newark Star-Ledger","summary":"Analysis of FDA data reveals issues with medical implants and regulatory system flaws.","keywords":["medical","implants","regulation"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"When The Star-Ledger first began examining the world of medical implants, we soon faced a major irony: We were reporting on a flawed regulatory system and, to do it, we were crunching numbers based on data coming out of this flawed system. But we found that troubled data isn't necessarily useless data. It just requires a little more thought. We already knew from documents and interviews that the Food and Drug Administration's regulation of the medical device industry was lax. The results of our analysis of FDA recall and malfunctioning device data provided numbers that illustrated the scope of the problem in our project, \"Foreign Objects: The Risky World of Medical Implants.\" It showed us what kinds of devices we needed to examine and illustrated what kinds of problems were cropping up with these devices. The idea of looking into the medical device industry, and specifically implantable devices, came out of our Washington bureau. Reporters Robert Cohen and J. Scott Orr had covered regulatory changes at the FDA for several years and had taken note of some medical device recalls that had made national headlines.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-february-digging-into-health-medicine","slug":"2003-february-digging-into-health-medicine","url":"https://dwillis.github.io/uplink/article/2003-february-digging-into-health-medicine.html","year":2003,"month":"February","monthNum":2,"headline":"Digging into health, medicine","kicker":"SPOTLIGHT:","author_name":"Megan Christensen","author_title":"","affiliation":"IRE and NICAR","summary":"Medical errors cause tens of thousands of deaths annually, prompting journalists to use databases like MAUDE and state health records to investigate faulty medical devices, nursing shortcomings, and hospital safety issues. The article provides guidance on accessing and using various data sources for investigating medical errors and public health stories.","keywords":["errors","databases","hospitals"],"topics":["Data & Databases","Health & Environment","Investigative Methods"],"technologies":["MAUDE"],"full_text":"The toll of medical errors - estimated at 44,000 to 98,000 a year - gained national attention after the National Institute of Medicine released \"To Err is Human\" a few years ago. Citing death rates greater than those from highway accidents or breast cancer, the report challenged the medical establishment to reduce errors by 50 percent in five years.\n\nJournalists were already on the story, using databases to examine faulty medical products and shortcomings in patient care. A variety of data sources can help with big-picture analysis and breaking public health news, whether it's suspicious hospital deaths or a product recall.\n\nWhen Michael Berens investigated the impact of errors by nurses for the Chicago Tribune (Story No. 17136 in the IRE Resource Center), he found that at least 1,720 patients had died and more than 9,500 had been hurt due to registered nurses \"besieged by cuts in staff and other belt-tightening in U.S. hospitals\" from 1995-2000.\n\nA major resource for the investigation was the Manufacturers User and Device Experience Database (MAUDE). This database from the U.S. Food and Drug Administration contains reports about injuries, deaths and accidents related to medical devices. It also contains information about the malfunctioning devices.\n\nThe Star-Ledger recently used MAUDE in its investigation of faulty medical devices. (See \"Databases show implant troubles\" on page 1).\n\nOne of the most useful features for journalists is a memo field detailing the incidents. It also contains references that can be used to request detailed records for each report.\n\nIt takes patience to cull through MAUDE, because incident reporting is voluntary and thus prone to gaps. Data entries also are inconsistent. Journalists using MAUDE can examine specific products or manufacturers, as the Tallahassee Democrat did for its report about automated drug pumps linked to accidental overdoses (Story No. 17311).\n\nMAUDE is available from the IRE and NICAR Database Library. For more detailed information about the data see www.ire.org/datalibrary/databases/maude/.\n\nBerens also used a database of investigative records from HCFA (now known as the Center for Medicaid and Medicare Services, or CMS) and a database of disciplinary records collected by the National Council of State Boards of Nursing (www.ncsbn.org), obtained from the state licensing agency and scanned into electronic form.\n\nAccess to health-system data varies by state. For example, Wilmington (N.C.) Morning Star reporter Cory Weiss knew that local hospitals would not share their internal investigation into rising staphylococcal infection rates. Instead, the state's death certificate database provided a window on staph-related deaths (Story No. 16583).\n\nDo some research to find out what your state regulates, what kinds of reports the statutes specify, and whether any records exist as databases. Or, if the state yields little, find local records collected nationally by the CMS.\n\nWhen planning a project, keep in mind that the medical profession has its own rigorous scientific standards for evaluating performance, and the media's methods need to be stringent. Many news organizations consult with independent experts. One major concern is making an accurate comparison among patient outcomes because of all the variables that affect risk in a diverse population with many different procedures and diagnoses. The Fort Worth Star-Telegram, for example, worked with a research scientist to analyze patient records (see article on page 8).\n\nHere are some available data sources:\n\nBirth and death records.\nInspection reports, mandated by state law or under the Medicare program.\nHealth maintenance organization financial data plus information on participating facilities and patient complaints.\nUniform billing records. These databases describe each patient's illness, any surgeries, age, insurance type, town or ZIP code and are available from state agencies. Everything will be anonymous, but you can ask for a unique patient identifier as a substitute for names.\nMedicare billing data can be obtained from CMS. Check with your local branch office for details on these requests; private companies often maintain and provide the records.\nFor more information on data sources, see tipsheet Nos. 1710, 407 and 643 in the IRE Resource Center.\nThe FDA Adverse Event Reporting System contains incidents related to pharmaceuticals and biological products like blood (available from NICAR; for details, go to www.ire.org/datalibrary/databases/AERS/).\n\nOther resources can help:\n\nNational Association of Health Data Organizations (www.nahdo.org)\nECRI (www.ecri.org) is a nonprofit specializing in medical device safety issues.\nThe Centers for Disease Control tracks public health data (www.cdc.gov). See its National Center for Health Statistics Data Warehouse (www.cdc.gov/nchs/datawh.htm)\nThe National Association of Health Data Organizations, 703-532-3282.\n\nSee the January-February 2003 IRE Journal for even more about investigating problems with hospitals. Berens of the Chicago Tribune details how he used patient billing records to help uncover infections in hospitals. The newspaper reported that three-quarters of the infections could have been prevented. Berens also shares tips for investigating your local hospital.\n\nContact Megan Christensen by e-mail at megan@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-database-library-news","slug":"2003-february-database-library-news","url":"https://dwillis.github.io/uplink/article/2003-february-database-library-news.html","year":2003,"month":"February","monthNum":2,"headline":"Database Library News","kicker":"NICAR UPDATE","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The SBA Disaster Loans and 7(a) business loan databases contain hundreds of thousands of records showing which small businesses received federal loans for disaster recovery and general operations, while updated immigration data tracks characteristics of permanent legal residents admitted to the U.S. in 1999-2000.","keywords":["loans","databases","immigrants"],"topics":["Data & Databases","Business & Economy","Demographics & Census"],"technologies":[],"full_text":"Database Library News\n\nSmall Business data (updated): The Small Business Administration won't let natural disasters and setbacks stand in the way of America's love affair with greasy pepperoni. More than $5.5 million has been awarded to businesses that serve cheesy slices and whole pizza pies from 1980 through September 2002.\n\nThe SBA's Disaster Loans database can offer details about how natural and other disasters affect businesses dealing with pizza, burritos, bagels as well as products and services that aren't edible. Journalists can discover who received loans and why.\n\nThe roughly 751,000 records include more than 9,000 recipients receiving almost $1 billion in loans to help them recover from the Sept. 11, 2001, terrorist attacks.\n\nAs winter storms hit some parts of the country, you also can find snow removal companies to which the SBA has awarded more than $1 million combined through general 7(a) business loans. Since the SBA began in 1953, it has offered loans through the 7(a) program to help small businesses in the United States.\n\nWithin the SBA 7(a) database of small business loans are nearly 830,000 records of companies that have received the guaranteed loans since 1953. Journalists can use this data to determine which lenders give small businesses the most money, as well as which businesses are receiving that money.\n\nAnother database from the SBA contains a list of companies qualifying for the 8(a) minority and other disadvantaged business programs. NICAR has offered this data in the past, but this latest update contains many new fields. Journalists can discover how many women, veterans, blacks, American Indians, Native Americans, Puerto Ricans, Hispanics and other minorities are active in the program.\n\nmoney, as well as which businesses are receiving that money.\n\nAnother database from the SBA contains a list of companies qualifying for the 8(a) minority and other disadvantaged business programs. NICAR has offered this data in the past, but this latest update contains many new fields. Journalists can discover how many women, veterans, blacks, American Indians, Native Americans, Puerto Ricans, Hispanics and other minorities are active in the program.\n\nImmigration data (updated):\n\nNICAR just updated its database of immigrants admitted to the U.S. as permanent legal residents. The data, maintained by the Immigration and Naturalization Service, contains characteristics of the aliens who were granted legal permanent residence in the U.S. during fiscal years 2000 and 1999.\n\nIn fiscal 1999, some 647,000 aliens were granted permanent residence. In fiscal year 2000, there were nearly 850,000. The database includes each immigrant's country of birth and country of last permanent residence, as well as the intended metropolitan area and state of residence. Journalists can track which countries most immigrants come from and which occupations are listed most.\n\nTo order or find out more about these databases, visit www.ire.org/datalibrary or call 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-open-source-free-but-worth-the-cost","slug":"2003-february-open-source-free-but-worth-the-cost","url":"https://dwillis.github.io/uplink/article/2003-february-open-source-free-but-worth-the-cost.html","year":2003,"month":"February","monthNum":2,"headline":"Open Source: Free, but worth the cost?","kicker":"","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"Exploration of open-source software in the newsroom, assessing its capabilities for journalism.","keywords":["open-source","software","newsroom"],"topics":["Technology","Journalism & Newsroom"],"technologies":["OpenOffice","MySQL","PostgreSQL","GRASS","Mozilla"],"full_text":"With newsrooms strapped with tight budgets, free open-source software sounds like a great deal for journalists. But is it? With this issue, Uplink starts exploring that question with a series of road tests designed to see whether these open-source packages meet the demanding needs of computer-assisted reporting. In recent months, the open-source movement has garnered heightened interest with the release of OpenOffice - targeted at users looking for an alternative to Microsoft Office. Also, Lotus Development Corp. founder Mitch Kapor gave the movement a boost when he announced his nonprofit foundation would develop an open-source personal information manager. Under open-source rules, users may download and use the software at no cost. Programmers are encouraged to view and modify the software source code. Proponents say that quickens the pace of software development and, in the end, results in better programs. Already there are scores of open source programs for all kinds of applications. Looking for a geographic information system (GIS)? There's GRASS. Need a structured query language-based database manager? You can use MySQL or PostgreSQL. How about something as simple as a Web browser? You can download Mozilla, based on the Netscape Communicator code released in 1998. With all the choices available, it's tough to know which programs are robust and which are still buggy. We'll give you answers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-february-hands-on-car-training-upcoming","slug":"2003-february-hands-on-car-training-upcoming","url":"https://dwillis.github.io/uplink/article/2003-february-hands-on-car-training-upcoming.html","year":2003,"month":"February","monthNum":2,"headline":"Hands-on CAR training upcoming","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer multiple computer-assisted reporting training opportunities including CAR Boot Camps in Columbia, Missouri and an annual conference in Washington, D.C. in June with hands-on classes and demonstrations.","keywords":["training","journalism","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet"],"full_text":"IRE and NICAR has numerous training opportunities in the coming months for journalists who want to learn computer-assisted reporting. For a complete list of training events visit www.ire.org/training/otr.html.\n\nHere are some of the highlights:\n\nThere are three CAR Boot Camps in Columbia, Mo., in the coming months for journalists who want hands-on training in how to use spreadsheets and database managers to analyze data for high-impact stories. The sessions are March 23-28, May 18-23 and Aug. 3-8.\n\nThe IRE Annual Conference, June 5-8 in Washington, D.C., will feature demonstrations of CAR techniques and hands-on classes. Members of the NICAR staff will be there to help you develop your stories and answer questions about computer-assisted reporting. Journalists can count on stellar panels and special presentations about national security, public safety, courts, the military, business, education and more.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-counting-the-faithful-with-church-roll-data","slug":"2003-february-counting-the-faithful-with-church-roll-data","url":"https://dwillis.github.io/uplink/article/2003-february-counting-the-faithful-with-church-roll-data.html","year":2003,"month":"February","monthNum":2,"headline":"Counting the faithful with church roll data","kicker":"RELIGION","author_name":"Ron Nixon","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists can use the Churches And Church Membership In The United States survey, conducted every 10 years by religious organizations, to analyze religious demographics and membership patterns in their communities, as the U.S. Census dropped its religious affiliation question in 1946. News organizations have used this data to track religious trends, though the survey has limitations including reliance on self-reported figures and coverage of only about 150 religious organizations.","keywords":["religion","census","membership"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Counting the faithful with church roll data\n\nHow many Baptists live in your community? What about Catholics or Lutherans? How many Jews or Muslims are there?\n\nFrom 1850 to 1936, you could find answers to questions like these in U.S. Census results. But the U.S. government dropped a question about religious affiliation in 1946, partly because the Christian Science Church had concerns about invasions of privacy.\n\nThat left little demographic information about religious organizations. When the Census Bureau tried to revive the question for the 1960 Census, opposition from religious groups caused the agency to scuttle the plan.\n\nFortunately, journalists can find answers by analyzing the Churches And Church Membership In The United States survey conducted every 10 years by a group of religious organizations.\n\nLed by the Association of Statisticians of American Religious Bodies, the group started to conduct its own census of religious group membership in the early 1950s. The survey provides the best available information on the membership of religious bodies in the United States.\n\nMore than three dozen news organizations recently used the 2000 data to examine religious beliefs and attendance in their communities.\n\nTracking membership\n\nThe Palm Beach Post examined church membership in South Florida and found that Palm Beach County was the second-most religious county in the state, based on the percentage of people who claim some type of religious affiliation. The paper included color maps and graphics to enhance its stories.\n\nThe Charleston Daily Mail did a story showing West Virginia ranked second among the states in the number of congregations per capita.\n\nIn contrast, The (Tacoma, Wash.) News Tribune found that Washington ranks next to last among the 50 states in church attendance and membership, dropping from 47th in 1990. One-third of Washington residents attend or belong to a church, synagogue or mosque, compared with one-half nationally. The Oregonian of Portland, Ore., found similar results in its review of the religious census.\n\nThe Herald News of West Paterson, N.J., used the data to look at religious groups that claimed the survey underestimated their membership. Mike Corn of The Hays (Kan.) Daily News found that several of the reported figures in the survey conflicted with numbers provided by local religious groups. For example, the census showed that church membership for several congregations in one county had declined. Corn traced the decline not to membership losses, but to counting problems in the study. The churches had been included in the population counts of another county.\n\nLimits in data\n\nThe current census is a survey of about 150 religious organizations. The study is the only census to provide a county-by-county breakdown of religious participation. The study provides the total population for a county, number of religious bodies and number of members and adherents.\n\nThe study is limited in many ways because the numbers are reported by the religious organizations and not all groups participate in the study. For example, in the 2000 census, 14 religious groups declined to participate, including some predominantly black congregations. Several groups participated for the first time, making comparisons between census years difficult.\n\nAlso, many Muslim groups have taken issue with the religious study, saying that it underestimates the total number of their members. An additional problem with the data is the way membership is defined by the religious groups that report. While many include only those who attend services on a regular basis, other groups include anyone with an affiliation to the religious body, including children.\n\nTo standardize the data, the ASARB uses adherents rather than members to examine growth of religious organizations. Adherents are adult members and their children. Membership totals include only those who are full regular members of a religious group.\n\nAnother problem with the data is that in a few counties, the number of adherents actually exceeds the total population. The ASARD says this could be because many people live in one county, but attend church in another.\n\nDespite these limits, the data is the best available information for journalists reporting on religious group numbers.\n\nThe data can be obtained from the Glenmary Research Institute in Nashville, Tenn. The data comes on a single CD in Microsoft Access and costs $100. Older datasets can be obtained from the American Religion Data Archive at Penn State University at www.thearda.com.\n\nContact Ron Nixon by e-mail at ron@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-other-sources-of-religion-data","slug":"2003-february-other-sources-of-religion-data","url":"https://dwillis.github.io/uplink/article/2003-february-other-sources-of-religion-data.html","year":2003,"month":"February","monthNum":2,"headline":"Other sources of religion data","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The article identifies numerous online resources and databases where journalists can access religious data beyond the standard Churches And Church Membership survey, including archives, adherent statistics, research institutes, and polling organizations.","keywords":["religion","data","resources"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Other sources of religion data\n\nWhile the Churches And Church Membership In The United States survey is considered the best collection of religious data available, it is by no means the only one. There are dozens of places to find data about religion, including the ones listed here:\n\nwww.thearda.com Located at Penn State University, the American Religion Data Archive holds dozens of datasets related to religions, some down to the local level.\n\nwww.adherents.com This is a collection of more than 41,000 adherent statistics and religious geography citations. Journalists can find adherent numbers by group and place.\n\nhttp://hirr.hartsem.edu/sociology/sociology_research_datasets.html The Hartford Institute for Religion Research at Hartford Seminary in Connecticut specializes in studying religion in public life and includes several datasets on its Web site.\n\nwww.gallup.com Opinion research firm The Gallup Organization has several polls with religion-based information.\n\nwww.nazareneresearch.org Church of the Nazarene Research Office. Good site with religion data plus interactive maps that can be queried on the Web site.\n\nFor general reporting on religion, see these sites:\n\nwww.religionwriters.com The Religion Newswriters Association is a nonprofit membership association founded in 1949 to advance the professional standards of religion reporting in the secular press and create a support network for religion reporters.\n\nJournalist's Online links for Religion www.toad.net/~andrews/jrelol.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-tv-pioneer-cox-rolls-with-changes","slug":"2003-february-tv-pioneer-cox-rolls-with-changes","url":"https://dwillis.github.io/uplink/article/2003-february-tv-pioneer-cox-rolls-with-changes.html","year":2003,"month":"February","monthNum":2,"headline":"TV pioneer Cox rolls with changes","kicker":"BROADCAST CAR","author_name":"Jaimi Dowdell","author_title":"","affiliation":"IRE and NICAR","summary":"Alan Cox, a pioneering computer-assisted reporting specialist at WCCO-Minneapolis, left the station after 25 years and is now freelancing, having helped transform how broadcast journalists use data and technology for investigative storytelling.","keywords":["pioneer","broadcast","technology"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Lotus 1-2-3"],"full_text":"TV pioneer Cox rolls with changes\n\nBy Jaimi Dowdell, IRE and NICAR\n\nOne of the first times former WCCO-Minneapolis reporter Alan Cox attempted computer-assisted reporting, he used a spare computer loaded with Lotus 1-2-3 for DOS from the TV station's accounting department.\n\nThat was in 1990, even before he was officially the station's CAR specialist. Cox's experience of endless keypunching was one of the first in a long line of projects that created respect for the reporter's work and helped change the way broadcasters use CAR.\n\nAfter a 25-year career in television, Cox left WCCO at the end of June. Since then he has been freelancing out of his home in Minnesota and has done some work for the Star-Tribune and KMSP-Minneapolis.\n\n\"He showed a whole crop of TV reporters that the same computer-assisted reporting skills that print journalists were regularly using to break big stories could also be applied to TV news,\" said Mike Wendland, technology columnist at the Detroit Free Press.\n\nBrant Houston, executive director of IRE and NICAR, said \"Alan has been a true pioneer in computer-assisted reporting in broadcast.\"\n\n\"He not only has led the way with stories, but also helped organize and teach our boot camps focused on the particular skills needed by broadcast journalists.\"\n\nStorytelling with CAR\n\nAfter several years working as a general assignment reporter and then business reporter at WCCO, Cox received an opportunity to learn and use CAR when news director John Lansing approached Cox with a proposal in 1993. Lansing told Cox that he was thinking of creating a computer-assisted reporting position and wanted him to do the job.\n\n\"Alan and I had the discussion, and he made the point that if we do it, we need the proper training and we need to do it well,\" said Lansing, now senior vice president of Scripps Howard Broadcasting.\n\nHe crunched data and assisted with investigative projects and the day-to-day reporting in the newsroom.\n\nCox was pulled off the regular schedule so he could learn how the tools of his new trade full time. Although he had little previous computer experience and did not even own a computer, he was eager to learn.\n\n\"My wife remembers that basically for the first year and a half, I stopped reading for recreation and just read computer manuals,\" Cox said. \"I remember one day, all I accomplished was to learn to change the colors in Windows.\"\n\nOnce Cox put his skills to work, the station created a set for him called \"Computer Center 4\". From there, he crunched data and assisted with investigative projects and the day-to-day reporting in the newsroom.\n\nFreelancer Julie Kramer was senior producer for investigations at WCCO and worked with Cox on many stories. \"A lot of people claim to do computer-assisted reporting, but Alan was what I call an alpha-geek,\" Kramer said. \"He knew his stuff, but he wasn't just all computer; as a journalist he understood the importance of storytelling.\"\n\nThe stories Cox developed with Kramer and the WCCO investigative team were some of his favorites while at the station. Together, they tracked voter fraud in Minnesota by matching a database of criminal history records with a list of registered voters. After this story, Cox said Kramer had the idea to use the same felons data and cross it with a database of hunting licenses.\n\n\"We found more than 500 people who weren't supposed to have a gun that were licensed to use guns,\" Cox said.\n\nFreelance route\n\nCox also contributed behind the scenes. After attending a NICAR advanced Boot Camp in 1998, he set up an intranet so WCCO journalists could access information in driver registration, voter and criminal record databases for their daily reporting.\n\n\"It helped us find people that didn't want to be found,\" Kramer said. \"It cut the paper trail time down immensely and a lot of reporters who couldn't do it on their own were able to do it quickly.\"\n\nWhen Cox's contract came up for renewal last summer, he decided it was time to try something new. While he said he's not sure what that will end up being, he remains very interested in CAR and putting in the time to do it well. Those who worked with him at WCCO recognize his contributions to the station's success as well as broadcast reporting.\n\n\"If you want to have a serious investigative unit at a TV station you need a computer-assisted reporting element and we had that in Alan Cox,\" Kramer said. \"We were able to do a lot of stories that we couldn't otherwise.\"\n\nContact Jaimi Dowdell by e-mail at jaimi@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-judges-database-helps-uncover-political-games","slug":"2003-february-judges-database-helps-uncover-political-games","url":"https://dwillis.github.io/uplink/article/2003-february-judges-database-helps-uncover-political-games.html","year":2003,"month":"February","monthNum":2,"headline":"Judges database helps uncover political games","kicker":"CAMPAIGN FINANCE","author_name":"Michael Beebe and Robert J. McCarthy","author_title":"reporter","affiliation":"The Buffalo News","summary":"A Buffalo News investigation used a database of judicial candidates to expose how political party bosses controlled the judge nomination process through improper fundraising practices and inflated convention fees, ultimately leading to the removal of the Democratic party chairman and reforms in judicial selection.","keywords":["corruption","judicial","investigation"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["spreadsheet"],"full_text":"When a respected town justice in the Buffalo, N.Y., suburb of Amherst decided to run for state Supreme Court, New York's highest trial bench, he rented some billboards to announce his entry into the race.\n\nIn no time at all, he was called on the carpet by the Democratic party chief in Buffalo's Erie County, G. Steven Pigeon.\n\n\"Listen you SOB,\" Pigeon told the judge. \"Don't you ever involve the general public in this process. It's my call. Nobody else's.\"\n\nAnd when a veteran Supreme Court justice offered to mediate a dispute involving Pigeon at a judicial nominating convention, Pigeon told him, in the presence of witnesses including Buffalo's mayor, \"Sit down and shut up you\"\n\nBoth judges went on the record to recount these incidents, a rarity, as anyone who has covered courts knows. Judges, at least in New York state, don't say much outside the courtroom.\n\nBut we were able to convince these judges and others to go public in The Buffalo News last summer because of the spadework we did in discovering how political party bosses had taken over the system used to nominate judges.\n\nA database we compiled of every state Supreme Court candidacy since 1999, and every local race since 1994, allowed us to go to these candidates and ask some tough questions. Among them:\n\nWhy did those running for judge here, even those endorsed by both major political parties, raise thousands of dollars and contribute to dozens of political campaigns when state judicial canons forbid them from doing so?\n\nWhy was it that judicial candidates here - and nowhere else in the state - were each charged $7,500 for judicial nominating conventions, when the actual costs were a couple hundred dollars to rent a hall and put out some coffee and doughnuts for delegates?\n\nWhy were sitting judges putting the arm on local lawyers for campaign contributions when these lawyers could end up appearing before the very same judges?\n\nThe three-day report helped bring down the Democratic party chairman, sparking a change in how local judges are nominated and a pledge from party leaders to stop forcing judicial candidates to raise funds for others.\n\nBefore it became a series, though, it had been one of those things that gnaws at a beat reporter, having to get a story in the paper but knowing there's a far better tale to tell if only there were time to really pull it together.\n\nContributions database\n\nRobert J. McCarthy, the political reporter at The News, kept running across the same thing each election cycle. Why were candidates for seats on New York's Supreme Court - even those guaranteed election because they were endorsed by Democrat and Republican party leaders - still raising and spending thousands of dollars?\n\nHe wrote different versions of the story several times, but they always seemed to be buried among other election pieces, lost in the crush of more important election stories.\n\nMcCarthy tried several times to set aside the time to dig into this, but the pressure of beat stories kept him from getting to it. He held onto a growing pile of campaign financial reports-the beginning of what came to be a valuable database.\n\nFinally last year, Stan Evans, an assistant managing editor at The News, allowed McCarthy to carve out some time from his beat and teamed him with projects reporter Michael Beebe to work on the judges story.\n\nThe reporters hooked up with Andrew Bailey, a researcher in The News' library and our newspaper's database and spreadsheet expert, and got to work.\n\nThe first thing we did was download all campaign finance data for every state Supreme Court race in New York from 1999 to 2002 from the state Board of Elections. The board only started computerizing these records in 1999. So for local judges we used McCarthy's paper records going back to 1994. File clerks typed in the data.\n\nWe then created a database of judicial campaign finance reports, with every contribution and expense listed for anyone who ran for state Supreme Court. Although the series would focus on candidates in the western part of New York state, known as the eighth judicial district, we wanted to know how judge candidates here ranked with their counterparts.\n\nWe also downloaded the attorney directory from the New York Unified Court System Web site. We merged this with the contributions from our database and got a list of all lawyers who contributed to those running for judge. And because judicial districts don't conform to other, more recognizable jurisdictions, we used ArcView, a GIS program that allowed us more flexibility for graphics when the series ran.\n\nInterviewing data\n\nBailey set up everything in a user-friendly format that allowed us to run any number of queries on anything that struck our fancy. After awhile, we got comfortable enough so it was like interviewing someone. In this case, we asked the questions of the data.\n\nWe found, as we suspected, that only political parties in Erie County charged a $7,500 fee for a judicial nominating convention, and that the Republicans had abandoned it in recent years.\n\nWe found that we had the most expensive state Supreme Court races in New York. We found a judge without an opponent who still managed to spend $75,000 and hire the party chairman's roommate as a campaign director.\n\nAnd we found that party chairmen got around the ban on judicial candidates donating to other campaigns through a loophole, one big enough to fit a buffet table through. The exemption allowed candidates to buy tickets to political dinners, some of them costing $5,000 each. So while the party chairman kept these candidates waiting months and months for endorsements, these potential judges kept getting billed for more and more tickets.\n\nArmed with this data, we were able to interview with considerable authority and confidence the judges and judicial candidates. Once judges and candidates knew what we had and what we intended to say, the judicial veil of secrecy was dropped. It turned out they didn't like the system either, that it was demeaning and that judges were being nominated more for who they knew and how much they could raise, rather than what their qualifications were.\n\nAfter the series ran, Pigeon, who had faced opposition before, withdrew from the race when it became clear he didn't have the votes to stay as party chairman. The new party chairman pledged there would be no more $7,500 judicial nominating fees, and one prominent Democrat ran for judge on a pledge he would accept no more than $100 from any lawyer.\n\nContact Michael Beebe by e-mail at mbeebe@buffnews.com.\n\nContact Robert J. McCarthy by e-mail at rmccarthy@buffnews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-free-stats-programs-compare-hospitals","slug":"2003-february-free-stats-programs-compare-hospitals","url":"https://dwillis.github.io/uplink/article/2003-february-free-stats-programs-compare-hospitals.html","year":2003,"month":"February","monthNum":2,"headline":"Free stats programs compare hospitals","kicker":"SPOTLIGHT: MEDICINE & HEALTH","author_name":"Andrea Adams","author_title":"computer-assisted reporting specialist","affiliation":"The Fort Worth Star-Telegram","summary":"The Fort Worth Star-Telegram analyzed 5 million hospital discharge records using a federal statistical program to uncover patterns in patient care and hospital quality across North Texas, finding discrepancies in death rates and procedural performance. The newspaper published its findings online so readers could compare hospitals based on specific procedures or diseases.","keywords":["hospitals","analysis","data"],"topics":["Data & Databases","Health & Environment","Investigative Methods"],"technologies":["SAS","SPSS","FoxPro","SQL Server","Excel","CD-ROM"],"full_text":"Analyzing millions of hospital discharge records might seem daunting, but customized statistical programs created for the federal government can churn through huge datasets in just a couple of hours - and help uncover interesting patterns in patient care.\n\nThe Fort Worth Star-Telegram, working with an outside researcher, used one of the programs to help analyze 5 million patient discharge records from the Texas Health Care Information Council for \"Vital Signs,\" a July 2002 special report about hospital quality in Tarrant and Dallas counties.\n\nThe newspaper found discrepancies in death rates for a variety of diseases and in the performance of Caesarian sections. Also, the analysis showed that hospitals using new procedures and medicines were the most effective in helping patients. Many of the dozen stories in the special report focused on specific medical topics, such as cardiac procedures and care for the elderly. The Star-Telegram posted the results of its analysis on the Web so readers could compare hospitals based on specific procedures or diseases.\n\nFirst evaluation\n\nThe Texas Health Care Information Council was created in 1995 to publish report cards about hospitals' performance rates for particular medical services. Nothing had ever been published by the agency. As a result, the Star-Telegram decided to conduct its own analysis to show readers how North Texas hospitals compared.\n\nObtaining the data was easy but costly. The newspaper paid $8,000 for two years of data, which was provided on a CD-ROM for each quarter. Jeff Claassen, the newspaper's computer-assisted reporting specialist, had decided to examine information from 1999 and 2000, the only two full years of data, to decrease the chance that a hospital's ranking might be distorted by an abnormally bad year.\n\nThe Star-Telegram decided to use the same computer program that the Texas Health Care Information Council used to analyze the patient medical procedure data: In-Patient Quality Indicators.\n\nIn-Patient Quality Indicators was created at Stanford University for the Agency for Healthcare Research and Quality, part of the U.S. Department of Health and Human Services. The program comes in two formats: SAS program files and SPSS syntax files. The agency provides the files and documentation for free download on its Web site (www.ahcpr.gov) but users need to have a SAS or SPSS license to run the programs.\n\nRather than obtain a license, the newspaper hired Wayne Lehman, a research scientist at Texas Christian University, to conduct the analysis using SAS.\n\nThe program works by assigning codes to each patient discharge record based on that patient's type and severity of illness. It then compares the patients undergoing a surgery or treatment to a national sample of patients undergoing the same, and adjusts the hospital's mortality rate using the difference between the two. The risk-adjusted rates take into account how old or sick the patients are to make hospital comparisons fairer. The program also calculates the number of times a certain procedure was performed and the rate at which a hospital performed a certain procedure.\n\nBefore analyzing the SAS program results, which were compiled in an ASCII text file, Claassen verified the numbers. He made sure the calculations were done using the entire dataset and checked them against other sources. He focused his analysis on death rates for 10 medical procedures, primarily those involving heart problems and birth. Following advice from a researcher at the agency, Claassen included only those hospitals that performed the procedure at least 60 times during the two years to reduce the risk of unreliable statistics.\n\nClaassen used FoxPro and Microsoft SQL Server to run queries, ranking the hospitals that performed best and worst at each procedure and comparing local hospitals to others in Texas. Charts of these results were compiled in Excel for medical reporter Charlotte Huff, business reporter Trebor Banstetter and Claassen to use. The three reporters called area hospitals with the best and worst rates to find out what kinds of factors can contribute to high or low mortality rates.\n\nClaassen recommends using programs like the In-Patient Quality Indicators because it provides the first widely recognized risk-adjustment methodology, decreasing the chances that a news organization's analysis would be criticized. Claassen said journalists who would like to do a story like this elsewhere should keep in mind that many hospitals are nonprofit organizations and therefore not subject to open-records laws. To get around this limitation, he suggests reading medical journals so you can ask better questions of hospitals with high or low mortality rates.\n\nContact Andrea Adams, Missouri School of Journalism","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-eatery-data-shows-poor-food-handling","slug":"2003-february-eatery-data-shows-poor-food-handling","url":"https://dwillis.github.io/uplink/article/2003-february-eatery-data-shows-poor-food-handling.html","year":2003,"month":"February","monthNum":2,"headline":"Eatery data shows poor food handling","kicker":"PUBLIC HEALTH","author_name":"Fred Vallance-Jones","author_title":"Database editor","affiliation":"The Hamilton Spectator","summary":"A Hamilton, Ontario newspaper investigation into restaurant safety practices revealed widespread food-handling violations and inadequate health department inspections, while municipal bureaucrats initially obstructed access to inspection records through excessive fees and delay tactics.","keywords":["inspection","restaurant","database"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["FoxPro"],"full_text":"The cook in the small family restaurant wanted to make sure the eggs weren't greasy.\n\nSo before the heaping plate of breakfast went to the customer, she used a reusable cloth lying on the counter to dab the eggs. She then put the grease-laden rag down, ready for the next meal. It would be used for six hours before being washed.\n\nSuch was the scene witnessed by a public health inspector in Hamilton, Ontario, an industrial city with nearly half a million residents, about an hour's drive southwest of Toronto.\n\nThe inspector wrote about the incident in his notes, one of more than 2,000 pages of inspection reports released to The Hamilton Spectator as part of our investigation into restaurant safety.\n\nThe resulting five-part series, called \"Reservations,\" revealed shocking food-handling practices in the back kitchens of well-known eateries.\n\nBut the series was more than just a recitation of horror stories about mouse infestations, rotting food and cooks who never washed their hands. Using computer-assisted reporting, it also revealed that the city health department was frequently failing to do the inspections required to protect the public, and even when serious problems were found, little or nothing was being done.\n\nSenior managers at the newspaper had wanted to do a story on restaurant safety for some time. There would be high reader interest, and similar CAR reports by other newspapers had led to reforms in several North American cities, including Toronto.\n\nThe responsibility eventually fell to me because of my special interests in CAR and public documents research. Little did I know that getting out of the starting gate was going to take the better part of a year.\n\nObstruction 101\n\nMy goal was to obtain a copy of the city's database of food premise inspections so I could see which establishments had the worst records and whether the city was doing the minimum inspections required to protect the public.\n\nBut the municipal health bureaucrats were in no hurry to hand over a disk.\n\nOntario has an open-records law that specifically applies to municipalities. Under it, the city was supposed to make a decision on my request within 30 days of my filing it in March 2000. In reality, it took five months and a meeting with the city clerk before I finally got my first response.\n\nThe answer was no. I could not have the database. But if I was willing to hand over a check for more than 1,000 Canadian dollars (about $630 U.S.), they'd be happy to make paper printouts.\n\nThe health bureaucrats said they couldn't provide an electronic copy.\n\nThe provincial health ministry had designed the database system and provided it to municipalities to make it easier for them to report inspection data to the ministry. In addition to being able to spit out what the province needed, the system allowed for some standard printed reports, including a list of inspections for any one premise. The city's staff had no idea how to make a copy of the underlying data they had themselves entered.\n\nWhen questioned they did provide a key piece of information, however. The application was written in Microsoft FoxPro.\n\nAs I had used FoxPro extensively in the past, I now knew it should be straightforward to write a query to produce the data I wanted. I called the ministry people who had designed the application, and after some initial reluctance, they agreed to write the query and provide it to the city to run.\n\nBy November 2000 I finally had a disk in hand, the cost about $30.\n\nTable troubles\n\nOnce my disk arrived, I was not surprised to find the data was dirty.\n\nThere were two tables. One contained information about establishments, each identified with a unique ID. It linked to detailed inspection information stored in the second table.\n\nIn the establishment table, a handful of the ID numbers were duplicated, with more than one establishment having the same number. Usually it happened because a premise had changed hands (and names) and someone had neglected to create a new ID. In two instances the mistake was inexplicable. I chose to eliminate those two premises from any further analysis.\n\nThe inspection table was twice as dirty. Some inspection entries were duplicated, and a later review of paper records revealed that a small number of inspections were not entered at all.\n\nI had to design queries to eliminate the duplicates, which could be identified from one of the fields. As for the missing inspections, they seemed few in number and health bureaucrats maintained the data was at least 98 percent complete. The deficiencies were later revealed to readers.\n\nTo decide which paper files I wanted to see, I used a field that recorded the number of critical violations found on an inspection visit. A critical violation is a mistake in food preparation or handling that has the potential to cause food poisoning, such as storing foods at temperatures where bacteria thrive and grow.\n\nI compiled a list of premises that either had large total numbers of critical violations or a large number of failing inspections. I then submitted an informal request for the detailed paper files.\n\nWithout the data, we could never have zeroed in so well on premises that had consistently had problems.\n\nDirty dining\n\nI received the paper records in early 2001. From there, the project sat on the backburner until May, when City Editor Jim Poling assigned me to work on it full time.\n\nBy that point, I had received an updated run of the inspection data from the city. Using Microsoft Access, I queried the database extensively and determined how many times each premise had been inspected, as well as how many inspections the department was doing.\n\nUsing Excel, I calculated how long passed between individual inspections of each establishment. All of this revealed that the city was falling far short of the inspection schedule required by provincial law. In some cases, years passed between visits, a fact that I confirmed with the owners of the establishments themselves.\n\nThere was still a key piece missing.\n\nRoger Gillespie, the night managing editor at The Spectator, observed: It was important that the reader understand the potential consequences of a system that failed. That meant documenting cases of food poisoning.\n\nI filed a new freedom of information request to another branch of the public health department, asking for records detailing outbreaks from the previous year and a half.\n\nWhile I waited for those documents, I interviewed restaurant owners, food safety experts, the inspectors and top health bureaucrats. By early September I had received records documenting several dozen outbreaks or suspected outbreaks of food poisoning and was tracking down one of the victims, who had suffered the humiliation of seeing the guests at her wedding meal fall ill.\n\nWhen \"Reservations\" launched on Nov. 24, 2001, we led with the wedding story and documented the failure of the city to live up to its obligations to protect the public.\n\nOn Monday (The Spectator, like many Canadian papers, does not publish on Sunday), we wrote about the restaurants with the worst inspection records. Three more days of stories followed, including a look at systems used by other cities to disclose the results of inspections to the public. We also ran sidebars giving necessary background on food safety issues and posted summary information derived from the city database on our Web site.\n\nThe response from the public and politicians was overwhelming in the true sense of that overused word.\n\nMy voicemail was overloaded, letters and e-mails poured in and our Web site was swamped as outraged readers found out for the first time about filthy conditions in restaurant kitchens and the city's failure to make them clean up.\n\nBy the second day of our series, the politicians were already promising a cleanup. By the time the last of the stories ran, councillors had voted to hire additional inspectors and to develop an action plan to fix the problems identified by the newspaper.\n\nIn the meantime, shortly after our series ran, the owner of that family restaurant ordered his staff to stop using the \"egg cloths.\" He still insisted the cloths posed no risk to the public, but his diners apparently thought otherwise.\n\nContact Fred Vallance-Jones by e-mail at fvjones@hamiltonspectator.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-using-sas-code-in-other-programs","slug":"2003-february-using-sas-code-in-other-programs","url":"https://dwillis.github.io/uplink/article/2003-february-using-sas-code-in-other-programs.html","year":2003,"month":"February","monthNum":2,"headline":"Using SAS code in other programs","kicker":"Tech tip","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists can leverage SAS programs from government agencies to create lookup tables and translation tools in more accessible programs like Excel, Access, and FoxPro, enabling data analysis without requiring SAS software.","keywords":["SAS","databases","journalists"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["SAS","Excel","Access","FoxPro","SQL"],"full_text":"Using SAS code in other programs\n\nBy Jeff Porter, IRE and NICAR\n\nSAS out of your budget? Think again. You can use some SAS files with the more widely used programs such as Excel, Access or FoxPro to ease your work considerably.\n\nSAS programs abound in state and federal agencies, and are often downloadable from the Web. They're valuable even for non-SAS users.\n\nUsing language in SAS programs - ending with the sas suffix and easily read with Notepad or any text editor - you can create lookup tables or UPDATE queries in Access or REPLACE commands in FoxPro.\n\nThe IRE and NICAR Database Library recently did some analysis for a project on the causes of death for the Columbia Missourian newspaper using a mortality file containing data of every death in Missouri during a 10-year period. A SAS program provided by health department authorities in Kansas helped us write a FoxPro program to categorize causes of death.\n\nThe official codes of causes of death, called the International Classification of Diseases, have changed over time. So the data has a different set of codes for deaths 1999 and after, and those before then. Those classification codes are extremely detailed. For example, there are several codes that represent a death caused by a faulty heart valve. Depending on what you need to know, that might be helpful. For the purposes of the Missourian project, though, we needed fewer, broader categories. For example, we wanted a \"heart disease\" category that would cover those heart valves and several other causes. And we preferred to have causes of death reported in plain English.\n\nFortunately, Kansas has a SAS program to do that.\n\nHere's a snippet: proc format library = death print; VALUE $TIR31GP 20044,M20051V1001 /* TUBERCULOSIS */ 2009'='1002' /* SEPTICEMIA */ 2010/='1003 /* SYPHILIS */ 2015'=*1004 /* AIDS/HIV */ '2001'-'2003,'2006' '2008','2011'-'2014 2016'.2017'='1005\" /* OTHER INFECTIONS-PARASITES */ 20181-02040'='1006 /* CANCER */ '2043' -''1007 /* DIABETES */ 2048'='1008' /* ALZHEIMERS DISEASE */ 2049 '2059'='1009 /* HEART DISEASE */ 2060'=\"1010\" /* HYPERTENSION */ '2061' -''1011' /* STROKE */ 2062'='1012 /* ATHEROSCLEROSIS */\n\nThis part of the program takes deaths from several categories and aggregates them. For example, \"2049\" through \"2059\" are grouped together into another code: \"1009.\" The text inside of the slashes are notes that are not part of the SAS program.\n\nA couple of preparations, then we're off to write some simple lines of SQL. We need two additional fields in the Missouri death data. One will include the new codes such as \"1009.\" Let's call it \"AGGCOD.\" (Stands for aggregated cause of death.) The other will include English translations of those codes, also courtesy of the SAS program borrowed from Kansas. We'll call that field \"TRANCOD.\" The field that already includes the range of codes we're going to aggregate is called \"ALLCOD.\"\n\nNow, with a lowly word processor or Excel, you can write commands to FoxPro.\n\nExample: Copy and paste a section into a Word document, then on the tool bar, activate the button that looks like a paragraph mark. It allows you to view hidden characters. In this SAS file, we noted that the line before \"OTHER INFECTIONS-PARASITES\" has an extra paragraph return. We deleted it. Also, some, but not all, of the category notes were preceded by two tabs before the first slash mark. We inserted those to make all of them the same.\n\nThen, we copied that into an Excel spreadsheet, then used Data > Text to Columns to make a new column, using the equal sign as the delimiter. Now, we have the new codes in a separate column. We made the appropriate number of columns between the old codes and the new ones - by simply counting the commas - and used Text to Columns again to delimit the columns with commas. Now, we have single codes or a range of codes in separate columns.\n\nRather than go through old ground we'll touch on just a couple of items regarding a function called BETWEEN. (MaryJo Sylwester, of USA Today, wrote an article for the September/October 2002 Uplink showing how to use Excel to write FoxPro program language.) Notice in several categories there are ranges of codes. For example, the \"OTHER INFECTIONS\" codes include \"2001\"-\"2003.\" We'll do a simple find-and-replace in Excel to change those hyphens to commas, then we're ready to write a program line to throw in a BETWEEN function.\n\nIn our example, the codes \"2001\",\"2003\" are in cell A2. The codes we eventually want to populate the just-created data field called AGGCOD are in cell C2. So in the cell B2, we'll type in this language:\n\n=IF(MID(A2,7,1)=\",\",\"REPLACE all AGGCOD with \"&TRIM(C2)&\" for BETWEEN(ALLCOD,\"&A2&\")\",\"\"\n\nIn other words, if there's a comma separating the two items - that is, if there's a range in the particular cell and not a single value - then plug in the language needed. If not, then leave the cell blank.\n\nThe result: REPLACE all AGGCOD with \"1005\" for BETWEEN(ALLCOD,\"2001\",\"2003\")\n\nThat line tells FoxPro to go to the new field AGGCOD and plug in the code \"1005\" every time the field ALLCOD values fall between \"2001\" and \"2003.\" We just repeated these steps a few more times, resulting in a long list of SQL commands.\n\nThen we were ready to move on to the next step, again from a short example of the Kansas SAS file:\n\nproc format library = death print; VALUE STIR31DS 1001r='TUBERCULOSIS' 1002r='SEPTICEMIA' 1003r='SYPHILIS' '1004'='AIDS/HIV' '1005'='OTHER INFECTIONS-PARASITES' 1006'='CANCER' '1007'='DIABETES' '1008'='ALZHEIMERS DISEASE' '1009'='HEART DISEASE' '1010'='HYPERTENSION' '1011'='STROKE' '1012'='ATHEROSCLEROSIS'\n\nThe first codes, of course, are in the AGGCOD field. The rest of the language will eventually wind up in the field called TRANCOD. Again using Excel, with the equal sign as the delimiter, the spreadsheet software becomes a tool of writing SQL commands. No BETWEEN function is needed this time; it's a straight translation for one code to a text label. Assuming that the codes are in A column and the terms are in C column, in B column, the Excel language would be:\n\n=\"REPLACE all TRANCOD with \"&TRIM(C2)&\" for AGGCOD = \"&A2\n\nThe resulting SQL line: REPLACE all TRANCOD with 'TUBERCULOSIS' for AGGCOD = 1001\n\nIf one wanted to do the same work in Access SQL language, it would be slightly different. The Excel formula looks like:\n\n=\"UPDATE deaths SET TRANCOD =\"&TRIM(C6)&\" WHERE AGGCOD = \"&A6\n\nThe results:\n\nUPDATE deaths SET TRANCOD = TUBERCULOSIS WHERE AGGCOD = '1001'\n\nTo download the FoxPro program created from the SAS file, go to www.nicar.org/techtips.html. There are even more techniques for using SAS materials without the SAS price tag. We'll share them in a future edition of Uplink.\n\nContact Jeff Porter by e-mail at jeff@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-office-package-needs-more-work","slug":"2003-february-office-package-needs-more-work","url":"https://dwillis.github.io/uplink/article/2003-february-office-package-needs-more-work.html","year":2003,"month":"February","monthNum":2,"headline":"Office package needs more work","kicker":"OPEN SOURCE","author_name":"Aron Pilhofer","author_title":"","affiliation":"Center for Public Integrity","summary":"Sun's StarOffice and the free OpenOffice.org are impressive Microsoft Office alternatives with strong word processing capabilities, but they have limitations in spreadsheet functionality and database features that make them unsuitable for serious computer-assisted reporting work.","keywords":["spreadsheet","software","alternative"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["StarOffice","OpenOffice.org","Microsoft Office","Excel","Java Runtime Environment","Windows","Linux","Adabas"],"full_text":"Office package needs more work\n\nLast spring, Sun Microsystems released a new version of its StarOffice suite for PC, promising it would deliver the same functions as Microsoft Office at about one-tenth the price.\n\nAbout the same time, an open-source version of the StarOffice suite OpenOffice.org - emerged from beta testing, with nearly the same features, but at no cost.\n\nAfter putting both to the test, I can say that they are very impressive and deliver many of the same features as MS Office. But there are enough small shortcomings - and one critical limit to the spreadsheet - that I'm not quite ready to recommend giving up on MS Office just yet.\n\nFor serious computer-assisted reporting (minimal word processing, heavy spreadsheet and database work), stick with MS Office for now.\n\nA bit of history\n\nA few years ago, Sun Microsystems bought StarOffice, then a commercial product, and later decided to release it under open-source rules. Under these rules, anyone may download, view or alter the programming source code behind the application, while the copyright owner maintains its rights.\n\nThe StarOffice code was placed in the hands of an organization of largely volunteer programmers and was renamed OpenOffice.org. Meanwhile, Sun continued to develop StarOffice internally and released it in May.\n\nThe OpenOffice project and the Sun developers worked very closely, so it's no surprise that StarOffice and OpenOffice are almost identical. Unlike OpenOffice, StarOffice includes printed documentation and technical support.\n\nInstallation\n\nI have installed OpenOffice and StarOffice on Windows ME, Windows XP Home and Professional editions, and Mandrake, Gentoo and Red Hat Linux. All went smooth as silk.\n\nBut there's one huge warning for Windows/MS Office users: During installation you must change a default setting that makes Open/StarOffice the default program for MS Office files. Otherwise, Windows will specify Open/StarOffice as the startup program for these files.\n\nWindows users may also have to download and install the free Java Runtime Environment from Sun Microsystems (www.sun.com).\n\nWhat's inside?\n\nBoth products include a word processor, spreadsheet, presentation program, a database front-end and a drawing program. Of these, I have used the first three extensively, the database front-end somewhat and the drawing program not at all. StarOffice includes a database manager, a stripped-down version of Adabas. But it is limited to databases of 300 MB and allows no more than three concurrent users. These make the program largely useless for CAR. Otherwise the two office suites are nearly identical.\n\nWord processor\n\nThe people at Sun probably realized that a good word processing application is a must for an office suite. So they obviously spent most of the development on this part of the Star/Open office suite, and it shows.\n\nThis is the one application that rivals its MS Office equivalent. I edited large and small files in OpenOffice, including a 100-page book filled with odd formatting, embedding comments and change tracking codes - and OpenOffice handled that flawlessly.\n\nEverything imported from MS Word perfectly, with some tiny exceptions. Images were slightly larger that before, and sometimes just a hair off their correct position. MS Word users will find it easy to switch to the OpenOffice word processor because almost every feature in Word can be found in OpenOffice. There are some nice extras in the OpenOffice package that makes working in that environment easier.\n\nSpreadsheet\n\nThis application, called Calc, is slightly less polished, and has one huge shortcoming: It only accepts 32,000 rows of data, compared to 65,500 or so in Excel. This is bad for journalists doing CAR, but not necessarily fatal. If you import a spreadsheet that is too long, it will simply lop off all rows past number 32,000 and tell you so.\n\nBeyond that, there is a lot good about Calc. Excel users will feel right at home. You'll find similar features used in CAR: string functions, pivot tables (called \"datapilots\") and charts.\n\nIt also has some nice extras. For example, you can do a three-level filter in Calc, compared to two levels in Excel. You also can insert graphics, form elements and other automation tools right into the sheet.\n\nConnecting the spreadsheet to an existing data source (such as an external Excel spreadsheet or .dbf table) is also very easy. Like the word processor, Calc imports data seamlessly from any version of MS Office, right through version XP.\n\nOne annoyance: Unlike Excel, Calc does not allow you to select non-adjoining cells using the control key.\n\nDatabase manager\n\nGood news and bad news here. The good news: Connecting OpenOffice to an external data source is an easy, wizard-driven process. Once you connect you can manipulate the data through a number of means. You can use the database to create a merge file with your word processor. You can manipulate the numbers in Calc. Or you can send SQL commands directly to the data source.\n\nOpenOffice also offers a nice front-end that looks and works very much like the Access query by example grid, though this feature is buried. You can link tables and send queries, just like you do in Access.\n\nNow the bad news: You can't connect to an Access database directly. You can connect via ODBC, but that is slower and somewhat limits what you can do.\n\nIn addition, it seems you are restricted to select queries via the OpenOffice grid, which limits how much crunching you can do without writing SQL. And if you're going to write SQL, you might as well use a database application with some power, such as MySQL or Postgres.\n\nRecommendation:\n\nThis is the one part of the OpenOffice suite that has changed very little since Sun's purchase of StarOffice. Unfortunately, it is the one program in the suite that journalists doing CAR would care about most.\n\nThose who aren't able to afford any office suite software would do well to download OpenOffice at www.openoffice.org. It does just about everything as well as MS Office - with a few exceptions.\n\nFor users stuck with an older version of MS Office (Office 95 or earlier), the OpenOffice programs would provide significant improvements, except for the database manager.\n\nEven older versions of Access are superior to the database manager in StarOffice. But, for CAR power users, there are plenty of other excellent open source database applications that run on Windows, Mac or Linux. MySQL rivals Oracle in terms of power and speed. The typical journalist doing CAR will find that the simplicity of Access cannot be beat, and users of Office 97, 2000 and XP should probably stick with what they have, at least for now.\n\nContact Aron Pilhofer by e-mail at apilhofer@publicintegrity.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-february-data-library-tackles-specialty-data-jobs","slug":"2003-february-data-library-tackles-specialty-data-jobs","url":"https://dwillis.github.io/uplink/article/2003-february-data-library-tackles-specialty-data-jobs.html","year":2003,"month":"February","monthNum":2,"headline":"Data Library tackles specialty data jobs","kicker":"NICAR SERVICES","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library offers specialized data processing and analysis services for news organizations, recently completing projects including Census data transformation for the Associated Press and speeding ticket analysis for a Cleveland news station.","keywords":["data","analysis","journalism"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["dBase","CD-ROM","9-track tape"],"full_text":"Data Library tackles specialty data jobs\n\nThe IRE and NICAR Database Library has looked at every block group in the United States and every speeding ticket in Cleveland. Those are just two of the processing or data analysis jobs the Database Library has completed during the past few months.\n\nUnder contract with The Associated Press, NICAR downloaded Census 2000 Summary File 3 data files, transformed the raw text files into dBASE IV files and added some geographic elements to every table. NICAR's work for the AP began in March 2001, when the redistricting Census data became available and continued through the Summary File 1 and the SF3 releases.\n\nNICAR analyzed a year's worth of speeding ticket data from the state of Ohio and the city of Cleveland. For WEWS-Cleveland, we discovered the most likely places to get a ticket, found the officers who wrote the most, and discovered how much faster than the speed limit a driver can go before getting a ticket.\n\nTranscripts of the stories can be found at www.newsnet5.com/theinvestigator1781686/detail.html and www.newsnet5.com/theinvestigator/1784946/detail.html\n\nFor a nominal cost, the Database Library can conduct data analysis and transfer data from tape to CD for news organizations. For more information, call 573-884-7711.\n\nContact Jeff Porter by e-mail at jeff@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-data-access-obtaining-restricted-files","slug":"2003-marchapril-data-access-obtaining-restricted-files","url":"https://dwillis.github.io/uplink/article/2003-marchapril-data-access-obtaining-restricted-files.html","year":2003,"month":"March/April","monthNum":3,"headline":"DATA ACCESS Obtaining restricted files","kicker":"","author_name":"Megan Christensen","author_title":"","affiliation":"IRE & NICAR","summary":"The article discusses how journalists can gain access to restricted data for computer-assisted reporting by signing research agreements.","keywords":["data access","journalism","restricted files"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["ASCII"],"full_text":"You think you have found the perfect data: thorough, current, topical, ASCH text. Then, the fine print. \"This data is provided for statistical research purposes only\" or \"Authorized for release to qualified research personnel or institutions.\"\n\nThe good news is that there is no special data club only for people with Ph.Ds. Journalists, too, can sign research agreements and gain access to databases. These arrangements can open the door to a wide variety of data for computer-assisted reporting.\n\nBut before accepting data with legal strings attached, journalists who've considered these agreements recommend exploring: Is there any other way to get the story? Can I report the story the way I want under the terms of the agreement?\n\nAgencies restrict access to many of these data sets because they contain sensitive information. Although you can't always avoid such obstacles, you should be certain that this is the only way to obtain the information you need for your story.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-marchapril-key-data-resources-for-covering-conflict","slug":"2003-marchapril-key-data-resources-for-covering-conflict","url":"https://dwillis.github.io/uplink/article/2003-marchapril-key-data-resources-for-covering-conflict.html","year":2003,"month":"March/April","monthNum":3,"headline":"Key data, resources for covering conflict","kicker":"SPOTLIGHT: CAR FOR WARTIME","author_name":"Brant Houston","author_title":"","affiliation":"IRE","summary":"A guide to organizing databases and resources for covering war-related stories, with focus on military and security data sources including Census Bureau information and Defense Department deployment records.","keywords":["databases","military","resources"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Internet","World Wide Web","Adobe Acrobat"],"full_text":"Databases and resources abound for covering war-related stories, but organizing them in a useful way can be a challenge. At IRE's recent Annual Computer-Assisted Reporting Conference in Charlotte, several speakers slotted the resources into these categories.\n\nMilitary and security\n\nThe U.S. Census Bureau has an enormous amount of data on the military, veterans, educational background of military and so on. Summary File 3 of the Census 2000 release is jam-packed with information. You can use American Factfinder (http://factfinder.census.gov) to zero in on what you need.\nNational Guard and reserves deployment data can be obtained from the Defense Department DefenseLink Web site in Adobe Acrobat format at www.defenselink.mil/news/Apr2003/d20030409ngr.pdf. Check the DefenseLink news releases section for updated lists.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-help-from-ire-and-nicar","slug":"2003-marchapril-help-from-ire-and-nicar","url":"https://dwillis.github.io/uplink/article/2003-marchapril-help-from-ire-and-nicar.html","year":2003,"month":"March/April","monthNum":3,"headline":"Help from IRE & NICAR","kicker":"","author_name":"David Herzog","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer resources and databases to help journalists cover post-Iraq War issues, including military presence, reconstruction, and terrorism, with training opportunities at upcoming conferences in Charlotte, Washington D.C., and Copenhagen.","keywords":["Iraq","journalism","databases"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Excel"],"full_text":"The U.S.-led war with Iraq is over but key questions remain unanswered. How long will coalition forces remain in the country? Will troops continue to battle pockets of resistance? Who will profit from rebuilding Iraq? No one has the answers, but you can count on IRE and NICAR to provide the vital information resources and assistance that you'll need for your own wartime news coverage.\n\nFor starters, you'll find timely articles and resources for covering the military, terrorism and cross-border issues in this issue of Uplink. Brant Houston, executive director of IRE and NICAR, writes here about the must-have databases. Mark Bixler of The Atlanta Journal-Constitution shares how he used Excel and INS data to show how the federal government stepped up deportations of visitors from primarily Muslim countries after the terrorist attacks of Sept. 11, 2001. We'll have even more in the May-June edition, including an in-depth look at reporting about military contractors in your region.\n\nThe Annual Computer-Assisted Reporting Conference in Charlotte, N.C., in early March featured a number of panels about using CAR for wartime coverage, everything from obtaining data about the military to the battle over access to information. We'll post any tipsheets from those sessions on the IRE War Resources Web page www.ire.org/resourcecenter/war_resources.html. Check back regularly for updates.\n\nWe'll offer even more opportunities to get up to speed on covering those topics at the IRE Annual Conference June 5-8 in Washington, D.C. The conference kicks off with the optional CAR day and includes hands-on classes where you can learn how to analyze data with spreadsheets and database managers.\n\nLooking for a multi-national perspective? You can join journalists from around the world at the second Global Investigative Journalism Conference this May 1-4 in Copenhagen and discuss strategies for covering war, arms smuggling and terrorism. There will also be hands-on CAR classes. You can find more information at www.ire.org/training/globalconference Program organizers include IRE and the Danish Institute for Computer Assisted Reporting.\n\nThe IRE and NICAR Database Library has been assisting journalists covering the military and terrorism by providing U.S. government databases regarding federal contracting and immigration. See the IRE War Resources Web page for more.\n\nContact David Herzog by e-mail at dherzog@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-hands-on-car-training-upcoming","slug":"2003-marchapril-hands-on-car-training-upcoming","url":"https://dwillis.github.io/uplink/article/2003-marchapril-hands-on-car-training-upcoming.html","year":2003,"month":"March/April","monthNum":3,"headline":"Hands-on CAR training upcoming","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer computer-assisted reporting training opportunities, including CAR Boot Camps in Columbia, Missouri scheduled for May and August that teach journalists to use spreadsheets and databases for data analysis.","keywords":["training","journalism","data"],"topics":["Journalism & Newsroom","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"IRE and NICAR has numerous training opportunities in the coming months for journalists who want to learn computer-assisted reporting. For a complete list of training events visit www.ire.org/training/otr.html\n\nHere are some of the highlights:\n\nThere are two CAR Boot Camps in Columbia, Mo., in the coming months for journalists who want hands-on training in how to use spreadsheets and database managers to analyze data for high-impact stories. The sessions are May 18-23 and Aug. 3-8.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-new-audit-data","slug":"2003-marchapril-new-audit-data","url":"https://dwillis.github.io/uplink/article/2003-marchapril-new-audit-data.html","year":2003,"month":"March/April","monthNum":3,"headline":"New audit data","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library have released a new Federal Audit Clearinghouse database containing 170,900 audit records from 1997-2003 that track how government agencies and nonprofits spent federal funds of $300,000 or more annually. The database includes audit findings and contact information for audited organizations and auditors, providing journalists with tools to investigate federal spending and identify problems.","keywords":["audit","federal","spending"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["Database Library","Federal Audit Clearinghouse","CFFR","FAADS"],"full_text":"New audit data\n\nHow are government agencies and nonprofit groups spending money they received from Uncle Sam? A new database from the IRE and NICAR Database Library can help journalists find out. The Federal Audit Clearinghouse database lists all audits of state and local governments, and nonprofits that spent $300,000 or more a year in federal money. The database contains audits from 1997, when the federal government started requiring them, through February 2003. There are 170,900 records in all.\n\nUnlike other databases available from the Database Library namely the Consolidated Federal Funds Report (CFFR) and the Federal Assistance Award Data System (FAADS) - the Federal Audit database looks at the spending of federal assistance. It also includes audits of sub-recipients of federal aid. So if a state government gives some of its federal award money to local agencies, audits of those local agencies will also appear in the database.\n\nThe data contains information about whether the audits have revealed any problems, plus valuable contact information, including names, addresses and phone numbers for the audited organizations and the auditors. To order the database, call the Database Library at 573-884-7711 or go online to www.ire.org/datalibrary/databases/fedaudit.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-tracking-deportations-by-country-of-origin","slug":"2003-marchapril-tracking-deportations-by-country-of-origin","url":"https://dwillis.github.io/uplink/article/2003-marchapril-tracking-deportations-by-country-of-origin.html","year":2003,"month":"March/April","monthNum":3,"headline":"Tracking deportations by country of origin","kicker":"SPOTLIGHT: CAR FOR WARTIME","author_name":"Mark Bixler","author_title":"reporter","affiliation":"Atlanta Journal-Constitution","summary":"The article outlines how to use computer-assisted reporting to track and analyze deportations by nationality post 9/11.","keywords":["deportation","immigration","data analysis"],"topics":["Investigative Methods","Government & Politics","Demographics & Census"],"technologies":["Excel"],"full_text":"Not long ago, computer-assisted reporting was a language that other reporters spoke, as foreign to me as Serbo-Croatian. I had a hard enough time opening e-mail attachments, so slicing and dicing numbers by computer seemed beyond my grasp. Then my newspaper sent me to a four-day workshop that it sponsored with the National Institute for Computer-Assisted Reporting. An instructor demystified spreadsheets and databases. I saw the possibilities.\n\nI cover immigrants and immigration for the Atlanta Journal-Constitution and have written extensively about changes affecting non-citizens since Sept. 11, 2001. The Immigration and Naturalization Service detained hundreds of people from Arab and Muslim nations for violations of immigration law that were once considered minor, such as overstaying a visa. It has held many of them in secret and displayed great reluctance to divulge information about them.\n\nWouldn't it be great, I thought, to know the nationality of people deported after Sept. 11?\n\nA simple phone call to the INS public affairs office in Washington got a Microsoft Excel spreadsheet in my e-mail inbox that showed the nationality of all people removed from the United States for the past six budget years. It listed countries alphabetically; the nations were not ranked by the number deported in any particular year.\n\nThe spreadsheet had more than 200 rows for the countries, from Afghanistan to Zimbabwe. The 18 columns showed for each of the past six years the number of people expelled, the number of criminal expulsions and the number of noncriminal expulsions.\n\nThe findings from the data included that the number of foreign nationals expelled to North Africa, the Middle East, and South Asia multiplied faster than for citizens of nearly all other nations in the year after Sept. 11, 2001. Eight of the 10 countries with the most dramatic increases in deportation were Muslim nations. The countries with the most dramatic increases in order were Tunisia, Morocco, Egypt, Yemen, Guyana, Jordan, Lebanon, Pakistan, Samoa, and Algeria. By contrast, overall deportations went down 18 percent. The number of immigrants deported to Mexico, which sends the vast majority of immigrants to the United States, went down 24 percent.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-marchapril-extra-extra","slug":"2003-marchapril-extra-extra","url":"https://dwillis.github.io/uplink/article/2003-marchapril-extra-extra.html","year":2003,"month":"March/April","monthNum":3,"headline":"Extra! Extra!","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE","summary":"Extra! Extra! is an online resource sponsored by IRE that provides summaries of investigative and computer-assisted reporting stories, searchable by category including health, homeland security, and politics. The Resource Center offers access to older stories through its website or by phone.","keywords":["investigative","journalism","resources"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"For some inspiration, check out Extra! Extra!, an online guide to investigative and computer-assisted reporting stories. The Web page, sponsored by IRE, contains summaries of recent investigative pieces and links to the stories. Journalists can also search for stories by category, from health, homeland security and politics to nonprofits. The Web page is at www.ire.org/extraextra. To find older stories through the Resource Center, go to www.ire.org/resourcecenter or call 573-882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-kbs-boot-camp","slug":"2003-marchapril-kbs-boot-camp","url":"https://dwillis.github.io/uplink/article/2003-marchapril-kbs-boot-camp.html","year":2003,"month":"March/April","monthNum":3,"headline":"KBS boot camp","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"Journalists from Korea's national broadcasting service attended a computer-assisted reporting boot camp at the University of Missouri, learning data analysis and spreadsheet skills to apply in their newsrooms. IRE and NICAR offer specialized CAR training programs for news organizations worldwide.","keywords":["journalists","training","data"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet","Internet"],"full_text":"Journalists come from all over the world to learn computer-assisted reporting in boot camps sponsored by IRE and NICAR. In November, visiting journalists from Korea came to the University of Missouri School of Journalism to learn CAR. For a week the reporters from the Korean Broadcasting System, South Korea's national broadcasting service, learned the basics of spreadsheets, database managers and importing data from the Internet, and returned home eager to apply their new skills.\n\nIRE and NICAR can arrange similar specialized training for news organizations, inside or outside the United States. For more information, contact Ron Nixon, IRE and NICAR training director, ron@ire.org or 410-995-0343.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-federal-spending-getting-a-handle-on-shuttle-contractors","slug":"2003-marchapril-federal-spending-getting-a-handle-on-shuttle-contractors","url":"https://dwillis.github.io/uplink/article/2003-marchapril-federal-spending-getting-a-handle-on-shuttle-contractors.html","year":2003,"month":"March/April","monthNum":3,"headline":"FEDERAL SPENDING Getting a handle on shuttle contractors","kicker":"","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"Following the Space Shuttle Columbia disaster, journalists used the Federal Procurement Data System to analyze NASA contractor spending, with the IRE and NICAR Database Library quickly providing federal contract data from 1992 onward to help news organizations identify local connections to the space program. The article explains how to effectively use this database, including its structure, limitations, and how contract modifications are recorded.","keywords":["contracts","NASA","database"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Federal Procurement Data System","FoxPro","Access","SQL Server","FTP"],"full_text":"Space shuttle Columbia crashed in pieces Feb. 1, and all the NASA shuttle program money going to contractors quickly became part of the story.\n\nJournalists covering that angle of the story found that a database of federal contracts of $25,000 or more -called the Federal Procurement Data System - is a great source of information for looking at program spending.\n\nThe IRE and NICAR Database Library maintains copies of the data from the General Services Administration back to 1992, covering about a half-million federal contracts each fiscal year. The day of the shuttle disaster, the Database Library culled all of the NASA contracts for each of those years, then posted the data and documentation on its FTP server for a quick download.\n\nTwo dozen news organizations obtained the data. In some instances, it allowed journalists to show a local connection to the space program, because the data could show exactly how much money NASA gave to specific contractors in a given state, city or county.\n\nThe database is fairly straightforward with a few wrinkles.\n\nThere's just one data table, although IRE and NICAR provide a few lookup tables with geographic information and details about the contractors. The primary data table includes the agency, name and address of the company, the amount of the contract, the type of work and where the work was performed.\n\nThe type of work listed is not specific enough for certain news reporting. For example, it doesn't say the company worked on the left wing of the space shuttle Columbia. Instead, it might say the company's product or service involved \"space vehicles\" or \"space vehicle components.\"\n\nNext, contracts can be modified over time, and it's important to understand how the changes are reflected in the data. The GSA data system has a separate record for each of these modifications with each record bearing the same contract number. Take, for instance, contract NAS850000 (with the Boeing Co. for space vehicles), modified 104 times from fiscal year 1992 through 2001.\n\nThe records show modification No 135 in 2001 and an additional obligation of $16.7 million. The next modification resulted in a reduction of the obligation by $1.2 million. So if you add all the numbers together (sometimes over several fiscal years), you can find the total.\n\nComplicating matters further, the GSA has added some fields over time. For fiscal year 2001, the IRE and NICAR Database Library created a program that processes the data, which will smooth some consistency with field names and how dollar amounts are handled over the long run. All those are reflected in the documentation the Database Library provides. Those documents should be examined carefully to account for any of those changes.\n\nIt's possible to turn the 10 yearly data files into one. The Database Library kept them separate so it could deliver the data more quickly to journalists the day of the shuttle disaster.\n\nNow, let's turn to the data. We'll go over a basic series of Visual FoxPro commands to choose the fields we're interested in and string the years together, then make some basic queries of the data. Any number of other database managers - Access, SQL Server, example-could do the trick, too.\n\nFirst, we studied the documentation and, for our purposes, decided to go with 18 fields, chosen because of their consistency year to year and their importance. They include the contracting agency, the number identifying the contract and any modification, the amount of money, the four fields describing the type of work, the three different fields that could list the company name, plus the locating information on where the company is based and where the work was performed. (See the record layout at www.ire.org/datalibrary/databases/fedcontacts/layout.txt).\n\nThe complete Visual FoxPro program can be found at www.nicar.org/techtips.html. Here are some key parts of the program.\n\nuse nasa01\n\nselect contractor, ;\nconnum, ;\nmodnum, ;\nentrytype, ;\ncontaction, ;\ndollaramt, ;\nproded, prodnomen, naicscd, naics, contnmicar, ;\ncontnmcif1 ;\ncontnmcif2 contcity, contsted, contzip, stateperf, ;\nperfcity from nasa01;\ninto table m:\\dataprob\\nasa\n\nWe continued to make separate tables of each year. If a field was named differently in a previous year, we told the program to change the name, simply like this:\n\n... siccd naicscd, sicnomen naics,\n\nThe GSA had changed its reference from the SIC (Standard Industrial Classification) to NAICS (North American Industry Classification System) from one year to the next. The language simply tells Fox to select the fields refer to SIC and call them the same field names as the 2001 data calls them.\n\nBefore 2001, the data implies three zeroes after the dollar amount. So to make it consistent, we multiplied all the dollar amount fields by 1,000 like this:\n\n.dollars*1000 dollaramt,\n\nThat line simply does the math and names the field the same as the 2001 data. The 1999 data had a slight variation for the dollar amounts. Instead of indicating a negative number, the data included a field, called NEGATIVE, showing whether the number is negative or positive. So for the 1999 data file, before we ran our main program (although we could have incorporated it into that program), we created a new dollar amount field, then ran a couple of replacements:\n\nReplace all dollar2 with dollar\nReplace all dollar2 with 0-dollar for negative = '-'\n\nBefore we stitched all those years together, we needed a way to tell which year is which, so we added a field to each table and told Fox to fill that in with the appropriate year:\n\nuse nasa98\nalter table nasa98;\nadd column year C (4)\nreplace all year with '1998'\nclose databases\n\nFinally, we reopened the 2001 table we simply called NASA, then appended all the years together.\n\nThere are other ways around this. For example, one could have added the years, then run a UNION ALL query to string the tables together.\n\nWith the data file covering 1992-2001, containing 114,126 records, a few queries could trigger some potential thoughts. Here's a very simple one:\n\nselect year, sum (dollaramt) total;\nfrom nasa;\nwhere prodcd like '18%';\ngroup by 1;\norder by 1\n\nIt shows a 10-year timeline of the money related to space vehicles. Interested in a particular state?\n\nselect year, sum (dollaramt) total;\nfrom nasa;\nwhere prodcd like '18%' and (contsted = 'TX' or stateperf = 'TX')\ngroup by 1;\norder by 1\n\nThat includes the same timeline, but limiting it to contracts where the company is located in Texas or the work was performed in Texas. Locate the biggest contractors for Texas, on space vehicles work?\n\nselect contested, contnmicar, contnmcif1, (dollaramt) total;\nfrom nasa;\nwhere prodcd like '18%' and (contsted = 'TX' or stateperf = 'TX') ;\ngroup by 1;\norder by 4 desc\n\nThis query groups by the Dun and Bradstreet number for the company, the field called CONTESTCD. That's often useful because the data is somewhat inconsistent in how the company is named. It's important to note, though, that some companies have multiple Dun and Bradstreet numbers, so you might need to do a little data checking and cleaning for accuracy and consistency.\n\nNancy Amons of WSMV-Nashville, was able to localize the shuttle disaster with federal contract data from IRE and NICAR, reporting \"NASA pumps millions into our local economy,\" identifying companies and finding people to interview, guided by the data.\n\n\"In all,\" Amons reported, \"NASA awarded nearly $21 million in contracts to Tennessee companies in 2001. Saturday's disaster could mean even more.\"\n\nContact Jeff Porter by e-mail at jeff@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-the-business-of-war","slug":"2003-marchapril-the-business-of-war","url":"https://dwillis.github.io/uplink/article/2003-marchapril-the-business-of-war.html","year":2003,"month":"March/April","monthNum":3,"headline":"The Business of War","kicker":"MAPPING IT OUT The latest uses of mapping in news reporting.","author_name":"Ron Nixon","author_title":"","affiliation":"Center for Public Integrity","summary":"The International Consortium of Investigative Journalists used ArcView mapping software to create maps illustrating global weapons trade connections and natural resource locations for their investigation into the profitable business of war. The project combined data from multiple sources including government factbooks, news archives, and corporate websites to map weapons dealers, military operations, and valuable resources in conflict zones.","keywords":["mapping","weapons","resources"],"topics":["Investigative Methods","Technology","Business & Economy"],"technologies":["ArcView","ArcAtlas","Internet","Google","World Wide Web"],"full_text":"Wars and ethnic conflicts have wreaked havoc on the economies and internal stability of developing countries around the world. Many of these are countries in name only, as the strife has fractured them into sections. For a small number of people around the world, however, wars and conflicts have been profitable.\n\nMany operate out of places and countries that few Americans have ever heard of, so when the Center for Public Integrity's International Consortium of Investigative Journalists began its special report called \"Making a Killing: The Business of War,\" it used ArcView 3.2 to create maps that illustrated the global connections of the weapons trade.\n\nThe center used ArcView to show places like Ostend, Belgium, an operating base of key weapons dealer Victor Bout. It also used the mapping software to show the location of the oil-rich Gulf of Guinea, where several states employ private military companies (once called mercenaries) and the location of the rich diamond deposits in the Congo.\n\nThe maps were largely locator maps and none required much advanced GIS work, but they did require research to locate information. The mapping work consisted of overlaying country shapefiles, national capitals and resource features from ArcAtlas: Our Earth, an ESRI data set. ArcAtlas lacked some of the resources that the Center wanted to show. For example, the location of oil fields in most cases was included, but not for those offshore. Additionally, resources such as uranium tailings in Angola were not included in the shapefiles.\n\nTo get the locations of other resources, the Center turned to other sources. One was the CIA World Factbook (www.cia.gov/cia/publications/factbook/) with maps and information about every country.\n\nGoogle searches found the Internet sites of global oil companies and information about their offshore sites.\n\nSearches of the French newspaper Le Monde were used to locate some of the natural resources, such as diamond mines and oil fields. The Center used the National Geographic Web site, located at www.nationalgeographic.com, to find many of the cities not included in the ArcView shapefile of major world cities.\n\nThe Center used the ArcView legend editor and symbol palette to create unique symbols for the oil fields and diamond mines. The Center also changed the background color of the view from the default of none to blue, so the oceans appeared as blue.\n\nThough the maps were a small part of the project, they helped by showing the locations of the places and key natural resources discussed in the project.\n\n\"Making a Killing: The Business of War\" can be found online at www.icij.org/dtaweb/icij_bow.asp.\n\nContact Ron Nixon by e-mail at ron@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-first-venture-simple-queries-find-prince-of-the-city","slug":"2003-marchapril-first-venture-simple-queries-find-prince-of-the-city","url":"https://dwillis.github.io/uplink/article/2003-marchapril-first-venture-simple-queries-find-prince-of-the-city.html","year":2003,"month":"March/April","monthNum":3,"headline":"FIRST VENTURE Simple queries find prince of the city","kicker":"","author_name":"Michael Lafleur","author_title":"","affiliation":"The (Lowell, Mass.) Sun","summary":"A journalist's first computer-assisted reporting project reveals that obtaining data is easier than determining how to use it effectively, ultimately leading to an unexpected story about a major property owner rather than the original investigation into building inspections.","keywords":["data","journalism","investigation"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Access"],"full_text":"I had always thought that the most difficult part of any computer-assisted reporting project would be negotiating for the data.\n\nBut in my first real attempt at a CAR story, I found the opposite to be true. I got the data - a catalog of complaints about alleged building and health code violations at properties in the city of Lowell - after a fight, but not an intolerable one. The most difficult part of my project was figuring out what to do with the data.\n\nMy initial story idea turned out to be undoable with my time and budget constraints, forcing me to explore other avenues with the data. What began as an attempt to examine the effectiveness of Lowell's property inspection system ended up as a profile of an outspoken and flashy Nigerian prince who was one of the largest owners of run-down properties in the city.\n\nThe project started in late summer, after several gang-related shootings in one of the city's poorest neighborhoods. Politicians in Lowell, a former textile manufacturing boomtown, called for a police crackdown on the gangs. City administrators began focusing on the absentee landlords whose properties police said served as hangouts for the gangs.\n\nWith the new emphasis on enforcing state building and health codes, we wanted to quantify what had been done prior to the high-profile crackdown. We especially were interested in determining the frequency of property inspections and the city's overall success rate for abating problem properties. I had attended a NICAR Boot Camp a few months earlier and thought this project presented a good opportunity for me to incorporate what I had learned.\n\nAs I reported, I discovered that city building inspectors kept a Microsoft Access database of their activities dating back to 1998. I asked for it, making what seemed to be the first request for electronic records that the city of Lowell ever had. My interest in the data initially triggered consternation from city officials. The building commissioner accused me of targeting him for unfair scrutiny and said I would have to pay 50 cents a record for each of the 12,000 records in the database. City legal staff asked me to place stricter limits on my request. At the time, I wasn't exactly sure what the data would show, but I hoped that it would at least shed light on the effectiveness of building department inspections. Unfortunately, that did not turn out to be the case.\n\nThe database contained complaints about problem properties called into the building department and a description of the response to them, but I was told it was not comprehensive. I was beginning to get a bit discouraged. I soon learned that building inspectors enforce only state building codes, which encompass construction guidelines and minimum structural soundness rules. They become involved with properties when there are problems such as a lack of exits from an apartment or a balcony railing is unsafe. Cockroach and rodent infestations, broken plumbing, leaking sewage, heat and electricity shutoffs - almost any problem in the interior of an apartment - are health code violations. These account for the bulk of citations generated by the city and are handled by Lowell's Health Department.\n\nHealth and building inspectors kept records separately on paper, under a property's address, and neither did it very well. The state housing court for my region lists all the Lowell property citations that are heard by a judge, but court records only are available on paper at 50 cents a page and offer no insight on how many complaints city inspectors actually resolve. Reporting a story on the total number of inspections conducted in Lowell and their outcome would have required me to sift through large numbers of files and far exceeded the time I had available for the project and possibly my newspaper's budget.\n\nI discovered many of these shortcomings about the same time I negotiated my public records request with the city legal staff. From the outset, the legal staff repeatedly asked me to explain specifically what I was looking for and to limit my request to just that. My insistence upon receiving all of the data really flustered the city attorney I dealt with. She was unfamiliar with the concept of a database and seemed very nervous about giving me something that could reflect badly upon her later. At one point, she said she would need months to review every bit of information in the database before she could release it to me. But we finally scheduled a formal meeting.\n\nShe brought along the city's chief information officer, who reinforced her demand that I explain exactly what I wanted by unleashing a stream of techno-jargon at me about how difficult it could be to fulfill my request. This was all very frustrating. It was now nearly two months after I had asked for the data and no one whom I was dealing with had even looked at it. Had I been unaware that the database was relatively simple and small, the protests would have sounded completely valid.\n\nLuckily, the paper's executive editor, Kevin G. Keene, had agreed to accompany me to the meeting. His willingness to stand behind my request, I think, was the deciding factor in my success. He asked the two city officials to look at the database and report back to us on how difficult it would be to provide a copy. Otherwise, he told them, we were willing to pursue legal action. They agreed and two weeks later I had a copy on compact disc for $20. The chief information officer, on instructions from the attorney, redacted only the names of people who filed complaints. She argued that releasing the complainants' names would violate their privacy rights under state law. I didn't challenge that because I was more interested in the number and focus of the complaints than who complained. (The public records attorney in the Massachusetts Secretary of State's office later told me he would have sided with the attorney's interpretation if I had appealed.)\n\nThe NICAR-L listserv proved to be invaluable during the negotiations. I was able to solicit the advice of veteran reporters who had already gone through what I was experiencing. Every time I dealt with the city attorney I asked journalists on the list about how to respond. I also relied upon the free legal defense hotline maintained by The Reporters Committee for Freedom of the Press (www.rcfp.org), which was another very handy resource.\n\nUnfortunately, my suspicions about the shortcomings in the data proved true. The database held two main tables, a listing of building permits granted and the catalog of property complaints - each containing between 10,000 and 15,000 records - dating to 1998. And then there were three smaller tables that referred to fields in one of the two larger tables. But there would be no way for me to quantify the efficiency of all city property inspections. While I could have done a story on how well building inspectors were addressing the complaints they received, I would not be able to provide a similar comprehensive look at health inspectors, who field many more property complaints.\n\nNow the pressure was on. My boss had invested money for me to attend the Boot Camp and his time in helping me get the data. Figuring out what to do once I realized my initial idea would not work was probably the most intimidating part of my first experience with CAR.\n\nSifting through the data I came across several complaints logged against a man named Prince Charles Eweka, who piqued my interest. It turned out the prince had a fascinating story. The scion of a Nigerian royal family, Eweka was one of the largest property owners in Lowell's low-income neighborhoods. He insisted that he was trying to reinvigorate the area, but the prince's properties had myriad structural problems and a long history of building and health code violations, which he typically did not correct until receiving a court order. But I found all this through interviews and manually reviewing city and court records.\n\nMy data analysis was very simple, and Boot Camp had prepared me well for what I needed to know. Using Microsoft Access 97, I focused my attention on the complaint table. I performed a group by name and count query to see which names appeared most often. I didn't have time to thoroughly clean the data, but because the table had relatively few records I printed it and eyeballed it, looking for alternate spellings that had escaped my initial query. I corrected only the names of people who appeared several times. In that way, I came across Eweka and decided to investigate him further.\n\nThe story, \"The Prince vs. the City,\" ran in December 2002 and detailed the city's complaints against Eweka.\n\nLooking back, I think my nervousness about developing a story from the data was just first-time jitters. Now that I've incorporated CAR into a story once, I'm much more comfortable with doing it again. Next time, I won't be so surprised if my initial preconceptions aren't borne out by the data. Though I have yet to figure out a way to quantify the efficiency of Lowell's inspection departments, I still haven't exhausted all the story possibilities for the complaint database, and I'm currently using it in another project that involves more data analysis.\n\nContact Michael Lafleur by e-mail at maolafleur@attbi.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-education-analysis-helps-uncover-college-funding-scheme","slug":"2003-marchapril-education-analysis-helps-uncover-college-funding-scheme","url":"https://dwillis.github.io/uplink/article/2003-marchapril-education-analysis-helps-uncover-college-funding-scheme.html","year":2003,"month":"March/April","monthNum":3,"headline":"EDUCATION Analysis helps uncover college funding scheme","kicker":"","author_name":"Natalya Shulyakovskaya","author_title":"Reporter","affiliation":"The Orange County Register","summary":"A newspaper investigation uncovered a scheme where California community colleges enrolled high school athletes in fake college physical education courses taught by their high school coaches, allowing them to earn college credits without completing the work.","keywords":["credits","fraud","colleges"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["ASCII"],"full_text":"Our \"Phantom Classes\" series started with a tip from a parent: His two sons had received community college credits for playing baseball in high school. Their college transcripts from Cypress College showed they had gotten A's even though they had to transfer to another high school and did not even complete the course.\n\nBy the time I joined the project in late August 2002, my colleagues Scott Reid and Marla Jo Fisher had already found that Orange County community colleges enrolled high school athletes in droves in what they called college physical education courses. Their high school coaches rounded them up for the classes and got paid to run the phantom college courses. These classes, called Bridge P.E., are a variation of the legitimate \"Bridge Program\" that allows qualified high school students to \"bridge\" gaps in their educational programs by taking tougher community college classes that meet their educational needs.\n\nAfter I joined the project, we turned to data analysis to get a clear picture of the extent of the program. In early September, we requested enrollment, course, student and faculty data from the California Community Colleges Chancellor's Office, using the California Public Records Law.\n\nIt took two months of negotiations - e-mails, letters and even appeals to state legislators - until we received some of the data we needed.\n\nThe Chancellor's Office argued that it could not release student-level data because the federal Family Educational Rights and Privacy Act prohibits the release of easily identifiable student records. Citing the same law, it argued that it could not strip all the identifiable information and replace the unique identification numbers that link the student and course tables. We accepted course summary data under protest.\n\nThe data came from the Chancellor's Office Management Information System (MIS), which gathers information from California's 108 community colleges. We were not charged for the data, which the office provided as ASCII fixed format files and allowed us to download from its Web site.\n\nTracking enrollment\n\nThe data covered 10 academic years starting from 1992-93 and contained records for 1.5 million courses, listing the number of students, the number of high school students and the number of full-time equivalent students, or FTEs, each course generated. The colleges use FTEs to track their workload and apply for funding.\n\nThe Register also obtained three additional tables from the MIS office. The tables listed more details for each course, including whether the courses were offered for credit. The state allocates a much larger chunk of funding based on the number of for-credit FTEs. In fact, more than 80 percent of taxpayer funding is allocated to community colleges on the basis of for-credit FTEs, and we focused our analysis on those.\n\nOddly, some of the totals the colleges reported for budget apportionment purposes were different from the numbers they had submitted for the MIS database. It took some tests, conversations with colleges and college finance experts to figure out how to handle the discrepancies - especially since the numbers fluctuated both ways.\n\nOur analysis of the data showed that while the entire community college enrollment grew by 20 percent in the last 10 years, growth in the physical education classes outpaced most other courses, increasing seven-fold. Since the 1992-93 academic year, it ballooned from a mere 0.3 percent of the entire California community college system's enrollment to almost 2 percent.\n\nCalculating costs\n\nIn order to figure out how much the phantom P.E. courses were worth, we waded though the state apportionment reports and built a database detailing funding allocations for each community college district and for the state, based on enrollment. The apportionment calculations are used to distribute money to community college districts. They are based on numbers the districts report to the state.\n\nFive years worth of apportionment reports, in Adobe Acrobat format, was available on the Chancellor's Office Web site. Each contained at least 72 pages densely covered with numbers and text.\n\nWhen I asked for the 10 years of the data in tables, I was told it would take about 12 weeks to compile. I saved the data in the Acrobat files as text. It then took me a day to write, test and modify a FoxPro program that parsed each report into a database.\n\nMost of the muscle work, including rebuilding and condensing the tables, was done in Microsoft SQL Server. SPSS came in handy for the final analysis.\n\nTo understand the community college financing, I retraced steps used for calculating apportionment funding. I used three main sources for understanding how the complicated funding formulas worked: materials the Chancellor's Office provided to the college administrators during its annual statewide budget information workshops; Title V regulations that outlined funding algorithms, and spreadsheets used by the colleges to estimate their funding.\n\nWhen my results came within 0.009 percent of the state's own numbers, I felt confident enough to start estimating the worth of different courses to community colleges.\n\nOur most conservative analysis showed that in the 2001-02 academic year, high school athletes counted as community college students generated at least $56 million for the colleges. Our reporting showed that in their quest for higher enrollment and, ultimately, more taxpayer dollars, the colleges broke the rules defining what could be called a college course. The Bridge P.E. classes amounted to nothing more than regular high school sports practices. Some classes were not held at all. Colleges paid millions to high-school coaches to sign up and train the enrolled high school kids.\n\nDoing the math\n\nHere is how I calculated the key numbers:\n\n$56 million received for Bridge P.E. classes\n\nUsing the MIS database, I calculated a percentage of total credit FTEs that the colleges reported based on their Bridge P.E. enrollment. For example, in 2001-02, Bridge P.E. accounted for 1.8 percent of all for-credit FTEs. That year the state distributed $3.3 billion based on for-credit FTEs.\n\nBecause of budget growth limits established by the state, the state funded only 96 percent of all for-credit enrollments at community colleges in 2001-02. So we adjusted the final worth of Bridge P.E. to reflect that.\n\nSome administrators argued that because Bridge P.E. represents roughly 2 percent of the entire enrollment these sports classes essentially are run at no cost to taxpayers. They base their argument on the fact that the funding shortfall is 4 percent. But the state doesn't calculate funding this way, records and interviews show. In reality, colleges apply for funding based on their total enrollment - including Bridge P.E.\n\nAt least 77 of the 108 California community colleges conducted Bridge P.E. classes at high schools in spring 2002.\n\nI counted only the colleges that had more than 100 high school athletes enrolled.\n\nTo come up with the most conservative estimate, The Register adjusted the numbers of students counted by the Chancellor's Office downward to reflect differences between those numbers and the numbers the colleges themselves report to the Chancellor's Office fiscal unit.\n\nFor some colleges, counts in the MIS data were lower than those reported to the fiscal unit. For others, they were higher. The Register adjusted down the numbers for over-reporting colleges. The adjustment was done only to analyze district-level data.\n\nStatewide data was not adjusted because the state does not adjust its data. The Register also ran tests to make sure the differences between the two different state databases did not affect the overall trends.\n\nTypical P.E. class\n\nThe Register calculated the median for the 2001-02 Bridge P.E. courses: 166 high school students.\n\nPercent of state-funded growth for community colleges\n\nThe Register used the state budget apportionment data to calculate funded growth. Then the growth in Bridge P.E. was calculated using the MIS data. Because enrollment has not been funded fully, growth in Bridge P.E. was adjusted accordingly. Then I calculated percentages.\n\nAs a result of our questions and stories, the state announced it was withholding $80 million from the community colleges this year. The Chancellor's Office and the state Department of Finance announced they were going to conduct a system-wide audit. California legislators have called for an independent audit. Some colleges have canceled their planned P.E. classes for high school students. Our phones and mailboxes were flooded by messages from outraged readers and parents. The stories - especially in the midst of the California budget crisis - have really touched a nerve.\n\nFor broader look at the reporting behind the series, read the March/April 2003 IRE Journal.\n\nYou can read the stories online at www.ocregister.com/features/phantom_classes.\n\nContact Natalya Shulyakovskaya by e-mail at natalyas@ocregister.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-marchapril-tech-tip-using-census-files-to-link-1990-and-2000-geography","slug":"2003-marchapril-tech-tip-using-census-files-to-link-1990-and-2000-geography","url":"https://dwillis.github.io/uplink/article/2003-marchapril-tech-tip-using-census-files-to-link-1990-and-2000-geography.html","year":2003,"month":"March/April","monthNum":3,"headline":"Tech tip... Using Census files to link 1990 and 2000 geography","kicker":"","author_name":"Dan Keating","author_title":"","affiliation":"The Washington Post","summary":"Census reporters face challenges when Census tract boundaries shift over time, making historical comparisons difficult. The article explains how to use the Census Bureau's tract-to-tract relationship files to reconcile these geographic changes and track data across decades.","keywords":["Census","geography","data"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":["SAS","TIGER","GIS","ASCII","spreadsheet"],"full_text":"No one inconvenience can be dubbed the bane of a Census reporter's existence since there is a whole list of banes, from shifting release schedules to questionnaire wording changes to margins of error. One nominee for the worst, however, is the shifts in Census geography that can make calculating changes over time a challenge. I once naively thought that tracts either split or combined. Oh, if it were only true. They shift, re-combine in parts and otherwise heave to and fro like a sailboat tossed at sea.\n\nCensus reporters have tried various approaches for reconciling the moving tracts. Richard O'Reilly at the Los Angeles Times devised a brilliant approach in SAS that read the raw text of the TIGER\\Line mapping files, looked at the vertices of each polygon and divvyed them up. Others have chosen to make the comparisons right in their GIS programs by stacking the geographic layers and computing how much the polygons overlap.\n\nI have chosen, however, to go with the Census Bureau's tract-to-tract relationship files. While the files can be tricky to explain and are befuddling to look at, the good news is that they're not difficult to use. For each 1990 tract, the files show how much of its population went into any 2000 tract. So if a tract was divided with 20 percent of its population going into one tract, 30 percent into a second tract and 50 percent into a third, the file would indicate that. You can then use those percentages to assign any 1990 count data into 2000 tracts and easily compare changes over time.\n\nInside the files\n\nThe 1990 to 2000 tract relationship files are available at the Census Web site at www.census.gov/geo/www/relate/rel_tract.html. The files I used are the Population-based Census Tract Relationship files. You can find files for each state and a national file, all in fixed-width ASCII, along with the record layouts you'll need to import them into a spreadsheet or database program.\n\nThere are 18 fields. Four fields make up the 1990 FIPS code (state, county, tract base and tract suffix), and another four make up the codes for 2000. There are additional fields for 2000 with the state abbreviation and county name.\n\nI suggest importing the relationship tables into the same database program where you store your Census data. That way you can easily link to the tables. Make sure you format the tract FIPS codes the same way you have them in your data. So if you put periods between the tract base and the tract suffix, make sure you do the same in the relationship tables. You'll be linking these tables to your 1990 and 2000 Census data.\n\nEach record describes the relationship between a 1990 tract and a 2000 tract. Tracts that do not overlap have no record. The key parts of each record are two fields that describe the relationship between the 1990 and 2000 tracts. One is labeled \"Percentage of 1990 census tract population.\" The other is \"Percentage of 2000 census tract population.\" If a tract didn't change, both of those numbers will be 100 percent, since none of the population has been split up.\n\nThere are also flags for 1990 and 2000 indicating whether the record represents part of a tract. If a 1990 tract was split over two 2000 tracts, there will be two records in the table for that 1990 tract - one record to describe the relationship to each of the 2000 tracts. In that case, each of the records will have a \"P\" on the 1990 tract FIPS code to show that the record only describes a portion of the tract.\n\nThere's one very important limit to using these files: You can use them only on counts, not on median values, averages or any other calculated fields. You can, however, use them to assign data and generate your own calculated fields.\n\nAn example with the files\n\nI wrestled with the partial-tract flags and the two numbers describing \"Percentage of 1990 census tract population\" and \"Percentage of 2000 census tract population\" for a while until I realized that I could throw most of them away. If I want to assemble 1990 data in 2000 Census tracts the only number I need is the \"Percentage of 1990 Census Tract population.\" I named that field \"tract90mult,\" which means that is the multiplier that I use for the 1990 tract to get the proper share for the 2000 tract.\n\nLet's look at a small example.\n\nThe relationship table tells us that 1990 Tract 0001 has 40 percent of its population in 2000 Tract 0002 and 60 percent of its population in 2000 Tract 0003. The pertinent parts of the table would have two records:\n\n1990 Tract FIP (simplified) 2000 Tract FIP (simplified) Tract90mult\n0001 0002 40\n0001 0003 60\n\nSo if I'm doing a story about African-American homeowners and I want to have a tract-to-tract comparison, I would divide the 1990 tract into the 2000 tracts using the tract90mult value. Note that the value is expressed as a percent. But if you multiply by those values, you'll get results 100 times too large. You have to divide the tract90mult by 100 first. I did that when I imported it. So if the tract90mult was 100, I turned it into 1. In this case, the values become 0.4 and 0.6.\n\nHere are the three steps needed to calculate:\n\n(1) Link the tract relationship file to the 1990 data by tract FIPS and multiply the 1990 value by the tract90mult.\n(2) Sum the new values grouping by the 2000 tract FIPS.\n(3) Link the result to the 2000 data using the 2000 tract FIPS to do comparisons.\n\nHere is how to calculate the rate of African-American homeowners using SQL. The 1990 data is in STF1 table H9. The 2000 data is in SF1 table H15B.\n\nselect .tractfip90, b.tractfip00,\ntract90mult*a.H0090002 + b.tract90mult*a.H0090007 as black_hhlds_share,\nb.tract90mult*a.H0090002 as black_own_share,\nfrom tblSTF190 a join lkpTractConvert90_00 b on tractfip90,tractfip00 a.tractfip90 = b.tractfip90\nwhere a.sumlev = '140'\n\nThe first step creates the values black_hhlds_share and black_own_share that have adjusted values. Now I have to aggregate those 1990 values into the 2000 tracts.\n\nselect tractfip00, sum(black_hhlds_share) as black_hhlds90, sum(black_own_share) as black_own90,\nfrom [the previous SQL query]\ngroup by tractfip00\n\nNow I have a table with one record for each 2000 tract with a properly apportioned share of the 1990 data. All I have to do is link it to my 2000 data compare by tract.\n\nselect h15bi1 as black_hhlds00, h15bi2 as black_own00, black_hhlds90, black_own90, h15bi2 - black_own90 as black_owngain\nfrom tblSF12000 a join [the previous SQL query] b on a.tractfip00 = b.tractfip00\n\nYou can loop these three queries into one, which is how I did it. It looked something like this:\n\nselect h15bi1 as black_hhlds00, h15bi2 as black_own00, black_hhlds90, black_own90, black_rent90, h15bi2 - black_own90 as black_owngain\nfrom tblSF12000 h00 join (select tractfip00, round(sum(black_hhlds_share),0) as black_hhlds90, round(sum(black_own_share),0) as black_own90,\nfrom (select tract90mult*h0090002 + tract90mult*H0090007 as black_hhlds_share, tract90mult*h0090002 as black_own_share\nfrom tblSTF190 t0 join lkpTractConvert90_00 cv on t0.tractfip00 = cv.tractfip90\nwhere t0.sumlev = '140' and localyn = 'Y') a group by tractfip00) b on h00.tractfip00 = b.tractfip00\n\nGetting the stories\n\nThe first waves of Census stories have come and gone. Tools like the tract-to-tract relationship files can enable news organizations to delve deeper into patterns. The Washington Post used a version of the query above to help map home ownership for a story that looked at how African-Americans had fared over the 1990s in the Washington, D.C., region and elsewhere.\n\nBy using dot density representation of the counts at the tract level, we were able to display the 1990 values and additions in the decade on the same map. It showed that the old pattern of African-American homeowners concentrated in the District of Columbia had been replaced by a wide belt of suburban homeownership.\n\nBy the way, if tracts are too big for you, the Census has block relationship files that work the same as the tract relationship files. They're on the Census Web site at www.census.gov/geo/www/relate/rel_blk.html. There are not any block group relationship files.\n\nContact Dan Keating by e-mail at keatingd@washpost.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-may-june-homemade-files-track-priests","slug":"2003-may-june-homemade-files-track-priests","url":"https://dwillis.github.io/uplink/article/2003-may-june-homemade-files-track-priests.html","year":2003,"month":"May • June","monthNum":5,"headline":"Homemade files track priests","kicker":"RELIGION","author_name":"Jaimi Dowdell","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists from The Boston Globe and other organizations overcame data shortages by creating internal databases to investigate sexual misconduct within the Catholic Church.","keywords":["Catholic Church","sexual misconduct","The Boston Globe"],"topics":["Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Many journalists who recently investigated sexual misconduct within the Roman Catholic Church faced a major hurdle: a lack of data about allegations of abuse by priests. Some news organizations found the answer to this problem by creating their own databases. The stories that resulted show how useful these homegrown databases can be.\n\nIn January 2002, The Boston Globe began publishing \"Crisis in the Catholic Church,\" a series that offered a picture of a Catholic archdiocese riddled with scandal. More than 800 stories later, The Globe's series has garnered many awards including a Pulitzer Prize for public service and the IRE Medal. Conducting an intense investigation into the church's private realm, the newspaper's Spotlight Team found success through the use of shoe-leather reporting, documents and a database that greatly supported the re-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-may-june-finding-backyard-military-contractors","slug":"2003-may-june-finding-backyard-military-contractors","url":"https://dwillis.github.io/uplink/article/2003-may-june-finding-backyard-military-contractors.html","year":2003,"month":"May • June","monthNum":5,"headline":"Finding backyard military contractors","kicker":"CAR FOR WARTIME","author_name":"Adam Bell","author_title":"","affiliation":"The Charlotte Observer","summary":"The Charlotte Observer used federal procurement data to identify Department of Defense contractors in the Carolinas during the lead-up to the Iraq war.","keywords":["military contractors","Federal Procurement Data System","Department of Defense"],"topics":["Business & Economy","Investigative Methods","Government & Politics"],"technologies":["Access","Federal Procurement Data System"],"full_text":"With war looming in Iraq and the demand for related local news stories rising a few months ago, we decided to look at which companies in the Carolinas benefited from military contracts.\n\nWe obtained the Federal Procurement Data System from the IRE and NICAR Database Library and imported it to Access 2000 for our initial queries. Because this database compiled by the U.S. General Services Administration contains all federal contracts with a value of more than $25,000, we needed to filter our queries to limit our results to Department of Defense contracts. Then we decided to omit contracts granted by the civilian Army Corps of Engineers since the story focused on military activities.\n\nNow we were ready to explore the data two ways: for work performed in the Carolinas and work performed by contractors based in the Carolinas. We decided that it was more interesting to","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-may-june-segregation-in-black-white-shades-of-gray","slug":"2003-may-june-segregation-in-black-white-shades-of-gray","url":"https://dwillis.github.io/uplink/article/2003-may-june-segregation-in-black-white-shades-of-gray.html","year":2003,"month":"May • June","monthNum":5,"headline":"Segregation in black white, shades of gray","kicker":"SPOTLIGHT: RACE","author_name":"Holly Hacker","author_title":"","affiliation":"IRE and NICAR","summary":"The article examines the complexities and various methodologies researchers use to measure residential segregation in American metropolitan areas.","keywords":["segregation","residential data","metropolitan areas"],"topics":["Investigative Methods","Demographics & Census"],"technologies":[],"full_text":"What is the most segregated metropolitan area in the United States: Detroit, Milwaukee, Minneapolis or New York? They all are. It just depends on how you measure segregation. And there are lots of yardsticks to choose from. Researchers have devised at least 20 ways to measure residential segregation. That's great for sociologists who have months to write detailed articles for scholarly journals, but not for journalists who must pound out stories on deadline for a general audience.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-may-june-bits-and-bytes","slug":"2003-may-june-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2003-may-june-bits-and-bytes.html","year":2003,"month":"May • June","monthNum":5,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Updates on the Mortality, Multiple Cause-of-Death database, upcoming ArcView GIS mapping training, and available tipsheets from the 2003 CAR conference.","keywords":["mortality database","GIS mapping","CAR conference"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["ArcView","Access","CD-ROM"],"full_text":"The public is becoming increasingly aware of the risks associated with obesity. A study published in April in The New England Journal of Medicine reports that being overweight increases the risk for death associated with cancer. In 2000, more than 1,000 people died in the United States because of causes directly related to obesity, according to the Mortality, Multiple Cause-of-Death database recently added to the IRE and NICAR Database Library collection of federal government databases. Journalists can use this database to track many diseases and other health issues, such as infectious disease and cancer, within their state or community. The data consist of detailed information about each death compiled from death certificates. Geographic information is complete for metropolitan areas with populations of 100,000 or more. The 1988 through 2000 data include death certificate records for the entire United States and U.S. territories. Detailed demographic information about the deceased, including age, race and marital status can be determined. Causes of death can also be analyzed by using the International Classification of Disease (ICD) field. Specific details about each death are also included, such as whether it was an accident, occurred in the workplace or was a suicide. For more information about this database go to www.ire.org/datalibrary/databases/health. Learn how ArcView 8 GIS can help you explore data for great stories. The Mapping Data for News Stories mini-boot camp July 25-27 in Columbia, Mo., will provide journalists with instruction on how to combine data with geocoding, thematic mapping, overlay and spatial analysis to enhance their reporting. For more information about this mini-boot camp, including how to register, go to www.ire.org/training/mapJuly2003.html; for information on other IRE and NICAR training and events go to www.ire.org/training/otr.html. If you missed the 2003 Annual Computer-Assisted Reporting Conference in Charlotte, N.C., there are still opportunities to learn what other CAR professionals presented at the event. More than 65 tipsheets are available from the conference, providing CAR techniques and story ideas for just about every beat. They include information on Access forms, Census data, spatial analysis, social network analysis and much more. To download the tipsheets, go to www.ire.org/training/charlotte and click on \"Tipsheets from this Conference\".","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-may-june-car-practitioners-share-tricks","slug":"2003-may-june-car-practitioners-share-tricks","url":"https://dwillis.github.io/uplink/article/2003-may-june-car-practitioners-share-tricks.html","year":2003,"month":"May • June","monthNum":5,"headline":"CAR practitioners share tricks","kicker":"","author_name":"David Herzog","author_title":"","affiliation":"University of Missouri and NICAR","summary":"David Herzog reflects on the collaborative environment and diverse training sessions at the 2003 Annual CAR conference in Charlotte.","keywords":["CAR conference","Charlotte","journalism training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["CD-ROM"],"full_text":"More than 250 journalists from across the globe gathered in Charlotte in March for IRE and NICAR's Annual Computer-Assisted Reporting conference. Attendance was smaller than many of the other recent CAR conferences, but those who attended were part of a committed bunch. How committed? Let me put it this way: Some of the conference-goers helped load CDs full of software onto the PCs in training rooms. Others performed manual labor, hauling PCs and monitors out of packing crates and lugging them into the training rooms. It was a great reminder that IRE and NICAR work because the journalists at the grass roots work by sharing their time, experience and knowledge. For those of you who missed it, the conference was packed with panels and demo sessions on all kinds of topics: weaving CAR into the beat, data for covering CAR during wartime and spatial analysis for investigations. And then there were special advanced workshops geared to CAR power users and hands-on classes for everyone. With the prospect of war looming, many of the panels had a special focus on defense and homeland security. Likewise, the spotlight panel focused on a timely subject: the battle over access to information. If you want to catch up on the happenings at the conference, you can order tipsheets provided by panelists. For more information about ordering contact the IRE Resource Center at (www.ire.org/resourcecenter or 573-882-3364). The tipsheets are a great way to learn lessons from other journalists. Thanks to everyone for making this a great conference. Contact David Herzog by e-mail at dherzog@nicar.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-may-june-car-training-upcoming","slug":"2003-may-june-car-training-upcoming","url":"https://dwillis.github.io/uplink/article/2003-may-june-car-training-upcoming.html","year":2003,"month":"May • June","monthNum":5,"headline":"CAR training upcoming","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"A list of upcoming training events for journalists, including GIS mapping workshops in Columbia, Missouri, and Better Watchdog Workshops across the U.S.","keywords":["GIS mapping","Better Watchdog Workshops","IRE training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["GIS"],"full_text":"IRE and NICAR has numerous training opportunities in the coming months for journalists who want to learn computer-assisted reporting. For a complete list of training events visit www.ire.org/training/otr.html. Here are some of the highlights: Mapping Data for News Stories, July 25-27 in Columbia, Mo., is for journalists who want to extend their CAR skills by using geographic information systems (GIS) to enhance their reporting. Trainers will be Jeff Porter and David Herzog of the NICAR staff, both longtime users of GIS in news reporting. Herzog is the author of \"Mapping the News,\" a case study book published by ESRI Press that shows how journalists use GIS. Journalists can get an overview of computer-assisted reporting at Better Watchdog Workshops offered by IRE and the Society for Professional Journalists. Upcoming workshops include Sept. 11 in Tampa, Fla., Oct. 4 in Eugene, Ore., and Oct. 25 in State College, Pa.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-may-june-car-fuels-tv-other-division-winners","slug":"2003-may-june-car-fuels-tv-other-division-winners","url":"https://dwillis.github.io/uplink/article/2003-may-june-car-fuels-tv-other-division-winners.html","year":2003,"month":"May • June","monthNum":5,"headline":"CAR fuels TV, other division winners","kicker":"IRE AWARDS","author_name":"Rachel Schaff","author_title":"","affiliation":"University of Missouri","summary":"An overview of 2002 IRE Award winners and finalists across television and magazine categories that used computer-assisted reporting to investigate corruption, medical malpractice, and fraud.","keywords":["IRE Awards","investigative reporting","data analysis"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["Excel","Access"],"full_text":"CAR fuels TV, other division winners\n\nBy Rachel Schaff, University of Missouri\n\nIRE received a flood of entries for its 2002 investigative reporting contest. Many of the entries used computer-assisted reporting, along with traditional reporting techniques. The following is an overview of some of the winners and finalists that used CAR in the television, other media and special categories.\n\nTelevision\n\nTop 20 Market – Finalist\nWFAA-Dallas: \"Fake Drugs, Real Lives\" by Brett Shipp, Mark Smith, Kraig Kirchem and David Duitch. This investigation examined the legitimacy of drug busts made by Dallas police. The journalists found that nearly half of the police department's cocaine seizures contained little or no illegal drugs. (Story No. 19889 in the IRE Resource Center).\n\nInstead, the substance seized was gypsum, a main component of billiard chalk. Further, the arrests often involved recent Mexican immigrants. The reporters developed a Microsoft Excel spreadsheet of Dallas County felony drug cases for 2000 and 2001. Reporters used the spreadsheet to separate defendants with Hispanic surnames and determine whether the seizures involved illegal drugs.\n\nThe story has won many other awards, including a George Foster Peabody Award; an Alfred I. duPont-Columbia University Silver Baton; a Harvard University Goldsmith Award Finalist for Investigative Reporting; Texas Gavel Award; the Katie Legacy Award; and Katie Award for Television Investigative Reporting.\n\nBelow Top 20 Market – Certificate\nWTVF-Nashville: \"Friends in High Places\" by Phil Williams and Brian Staples. This seven-month investigation exposed the questionable relationship between Tennessee Gov. Don Sundquist and some state contractors. Journalists used Microsoft Excel to analyze spreadsheets of all noncompetitive contracts awarded by the state of Tennessee. (Story No. 19649)\n\nThis story has also won a George Polk award, a National Headliner Award and a Sigma Delta Chi award.\n\nBelow Top 20 Market – Finalists\nWITI-Milwaukee: \"The Pain Doctor\" by Bob Segall and Diane Carbonara. This investigation focused on a local doctor who used questionable medical and billing practices and acted abusively toward his patients. Journalists used CAR to analyze information acquired through interviews with dissatisfied patients and to analyze pharmacy prescription data to determine the amounts and types of medications prescribed by the doctor. \"The Pain Doctor\" also has won a regional Edward E. Murrow award, Best Series Reporting from the Wisconsin Associated Press, and Best Investigative Report from the Milwaukee Press Club. (Story No. 19674)\n\nWOAI-San Antonio: \"Councilman & Judge – A Cozy Connection\" by Brian Collister, Joe Ellis and Stephanie Kline. This piece investigated a San Antonio criminal defense attorney (who is also a city councilman) and his relationship with a female judge. The journalists used Microsoft Access and Excel to analyze databases of criminal court records and county payment records. They determined which attorneys received the most court appointments and made the most money, both in each court and across all courts. Eventually the results helped show that the councilman and the judge had a romantic relationship, shared a bank account, shared an address at one time, and possibly committed fraud together. This story received an Honorable Mention in the Investigative category from Texas Associated Press Broadcasters and also won first place for Best Continuing Coverage by the Texas Associated Press Broadcasters. (Story No. 19671)\n\nOther Media\n\nMagazine/Specialty Publication – Finalists\nTime Magazine: \"Look Who's Cashing in at Indian Casinos\" by Donald L. Barlett and James B. Steele. This investigation revealed that Indian reservation casinos do not benefit the majority of Native Americans; most of the proceeds go to non-Indian investors and a handful of small tribes with fabulously profitable casinos. CAR played a large role in the investigation; Barlett and Steele made extensive use of databases, the Internet and spreadsheets. One of the most important databases, the Indian Labor Force Report, contains information about individual tribal enrollment as well as the employment rate and poverty levels of each tribe. (Story No. 19517)\n\nU.S. News and World Report: \"The New Math of Old Age: Why the Nursing Home Industry's Cries of Poverty Don't Add Up\" by Christopher Schmitt. The nursing home industry has long contended that the federal aid it receives is inadequate; in 2002 nursing home operators lobbied the federal government for additional billions of dollars of aid. However, this investigation showed that the nursing home industry is, in fact, profitable and growing, with many nursing homes earning 20 percent to 30 percent profit margins. The investigation required many different databases of nursing home cost reports. Schmitt used Paradox for database management, FoxPro for data import and Excel for spreadsheet analysis. (Story No. 19495)\n\nOnline – Finalist\nCenter for Public Integrity: \"Enron Top Brass Accused of Dumping Stock Were Big Political Donors\" by John Dunbar, Robert Moore and MaryJo Sylwester. This investigation looked at the col-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-may-june-bits-and-bytes-2","slug":"2003-may-june-bits-and-bytes-2","url":"https://dwillis.github.io/uplink/article/2003-may-june-bits-and-bytes-2.html","year":2003,"month":"May • June","monthNum":5,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Updates on the Mortality, Multiple Cause-of-Death database, upcoming ArcView GIS mapping training, and available tipsheets from the 2003 CAR conference.","keywords":["mortality database","GIS mapping","CAR conference"],"topics":["Health & Environment","Investigative Methods","Technology"],"technologies":["ArcView","Access"],"full_text":"The public is becoming increasingly aware of the risks associated with obesity. A study published in April in The New England Journal of Medicine reports that being overweight increases the risk for death associated with cancer. In 2000, more than 1,000 people died in the United States because of causes directly related to obesity, according to the Mortality, Multiple Cause-of-Death database recently added to the IRE and NICAR Database Library collection of federal government databases. Journalists can use this database to track many diseases and other health issues, such as infectious disease and cancer, within their state or community. The data consist of detailed information about each death compiled from death certificates. Geographic information is complete for metropolitan areas with populations of 100,000 or more. The 1988 through 2000 data include death certificate records for the entire United States and U.S. territories. Detailed demographic information about the deceased, including age, race and marital status can be determined. Causes of death can also be analyzed by using the International Classification of Disease (ICD) field. Specific details about each death are also included, such as whether it was an accident, occurred in the workplace or was a suicide. For more information about this database go to www.ire.org/datalibrary/databases/health. Learn how ArcView 8 GIS can help you explore data for great stories. The Mapping Data for News Stories mini-boot camp July 25-27 in Columbia, Mo., will provide journalists with instruction on how to combine data with geocoding, thematic mapping, overlay and spatial analysis to enhance their reporting. For more information about this mini-boot camp, including how to register, go to www.ire.org/training/mapJuly2003.html; for information on other IRE and NICAR training and events go to www.ire.org/training/otr.html. If you missed the 2003 Annual Computer-Assisted Reporting Conference in Charlotte, N.C., there are still opportunities to learn what other CAR professionals presented at the event. More than 65 tipsheets are available from the conference, providing CAR techniques and story ideas for just about every beat. They include information on Access forms, Census data, spatial analysis, social network analysis and much more. To download the tipsheets, go to www.ire.org/training/charlotte and click on \"Tipsheets from this Conference\".","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-may-june-database-sleuthing-uncovers-the-missing","slug":"2003-may-june-database-sleuthing-uncovers-the-missing","url":"https://dwillis.github.io/uplink/article/2003-may-june-database-sleuthing-uncovers-the-missing.html","year":2003,"month":"May • June","monthNum":5,"headline":"Database sleuthing uncovers the missing","kicker":"CRIME","author_name":"Lise Olsen","author_title":"Reporter","affiliation":"Seattle Post-Intelligencer","summary":"The Seattle Post-Intelligencer investigated two decades of missing-persons cases in Washington, finding that police negligence and database errors led to unsolved murders and unidentified remains.","keywords":["missing persons","Seattle Post-Intelligencer","police negligence"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["NCIC"],"full_text":"Deep in the heart of Green River Killer country our initial goal was to try to identify probable serial killer victims. We did that and found a lot more, but not without a struggle. We learned that when someone went missing his or her records often disappeared too. That was one of the challenges reporters at the Post-Intelligencer faced when we attempted to analyze 20 years of missing-persons reports from more than 300 police agencies in the state of Washington. The police had misplaced or purged dozens of reports for people still listed as missing in the official state and national databases. The official databases also were inaccurate. Many people who should have been included weren't, and other cases that should have been purged remained in the system. The consequences were dramatic: Murder cases went unsolved, bodies remained unidentified and killers stayed free. The databases we built and those we obtained from police, coupled with traditional investigative reporting, allowed us to make such sweeping conclusions with confidence. We also found that agencies had lost or destroyed more than 100 reports for active missing-persons cases. Coroners and medical examiners had lost files, physical evidence and even remains. Most police agencies routinely violated a law that requires them to seek dental records in long-term missing person cases, even though the records can provide key clues in identifying corpses. The database steered us to dramatic human stories: a foster child whose disappearance and mysterious death in Mexico was never adequately investigated. A toddler, the unidentified victim of murder in 1987, whose image had never been publicized until our series ran. A missing woman whose frantic family could not convince police to take a report after she slit her wrists and drove away. The anecdotes alone were appalling but our database allowed us to combine the tales with hard numbers. Our investigation took nearly a year, mainly because it was so difficult to obtain records, most of them police reports from open investigations. But it also took time to track down families of the victims. Reporter Lewis Kamb filed the first public disclosure requests in January 2002, asking all the departments across the state for unsolved missing persons reports. From the beginning, the newspaper specified that it wanted reports that were at least a year old. We didn't want to track the thousands of teen-agers and adults each year who return home after a few days. Federal and state auditors regularly are supposed to review whether police accurately track long-term missing persons cases in the National Crime Information Center (NCIC) database. But it immediately became clear that most departments didn't have their records in order. While some departments responded to the public records requests within a week or two, most dragged their feet. Some flatly refused to provide records. Others provided information only in summary form or redacted names. It took the Seattle Police Department, the state's largest agency, nine months to respond. Kamb was relentless in his pursuit of the records. He pointed out to the reluctant police departments that some major police agencies had immediately provided copies. After several months, I got involved in building the database. Kamb had boxes full of reports. But, using a list of official missing persons provided by the state, I quickly realized that many departments had not given us everything. We wrote back again and called some departments dozens of times. By late fall, we had assembled more than 600 reports in all sorts of formats, some handwritten, some barely readable, but most usable. I built an index to our reports that included fundamental information, such as the department name, date, victim name, age, sex, race, where disappeared, usual residence, a description of the victim and a summary of circumstances behind the disappearance. I marked others that seemed to bear more investigation as \"interesting\" in the database. I also attempted to categorize the reports by the reasons police had given in the reports to possibly explain the disappearance. Foul play was suspected in many. A key category for us was serial killer victim. This involved a lot of investigation into alleged serial killers who had operated in Washington since the 1980s. In additional to the missing-persons database, I built a database of unidentified dead persons. Next we compared the information more closely to official records. We received parts of the missing persons data and much of the unidentified persons data from the state. Journalists can obtain similar data from the authorities in their own states. We found that more than 60 percent of all missing persons had no dental reports on file. Despite the limits of the data, I was able to analyze it and discover that some probable murder victims had never been entered as official missing-persons. The NCIC tries to match the tables of the missing and the dead but, our reporting found, the automated search engine in the database was flawed. So I matched the tables we obtained to see whether I could find matches overlooked by the police. So far, the identities of two long-unidentified bodies have been confirmed based on information generated by the series. I hope that more will be added to that list over time. To see the series and a portion of the databases we used, see http://seattlepi.nwsource.com/missing. Contact Lise Olsen by e-mail at LiseOlsen@seattlepi.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-may-june-adding-the-3rd-dimension","slug":"2003-may-june-adding-the-3rd-dimension","url":"https://dwillis.github.io/uplink/article/2003-may-june-adding-the-3rd-dimension.html","year":2003,"month":"May • June","monthNum":5,"headline":"Adding the 3rd dimension","kicker":"MAPPING IT OUT","author_name":"Stephen K. Doig","author_title":"journalist","affiliation":"Arizona State University","summary":"Stephen Doig explains how to use ArcView's 3D Analyst extension to create three-dimensional maps for news reporting, using a St. Paul income study as an example.","keywords":["3D Analyst","GIS mapping","ArcView"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["ArcView","3D Analyst","ArcScene","GIS","shapefile"],"full_text":"A few months ago, I was wowed by a demonstration of a mapping tool that was new to me. It was 3D Analyst, an extension to ESRI's popular ArcView geographical information system (GIS) software. 3D Analyst can take a two-dimensional map, such as a thematic map of census tracts, and transform it into a three-dimensional image in which the height of each area on the map can represent another variable. The resulting depiction of data, which looks like a cityscape of skyscrapers, can then be \"explored\" by turning it every which way, zooming in and out and even flying through it up close. I decided I had to learn how to use this cool toy. Like most journalists interested in computer-assisted reporting, I learn new tools best when I have some real journalistic task to accomplish. The trick was to find something interesting to do. Janet Roberts of the St. Paul Pioneer-Press quickly took up my offer. Having seen a nice 3D map of median income change data created by the Chicago Tribune, she wanted to do something similar for the city of St. Paul, looking at percentage change in median household income since 1989. Turns out I needn't have worried. If you already know how to use ArcView, doing the basic work with 3D Analyst is a snap. So here's a handy step-by-step for ArcView 8.x in case you get your hands on the extension: The first step is to activate the 3D Analyst extension. Hit the Add Data button and find your shapefile. To make a 3D image, launch ArcScene, which is 3D Analyst's workspace. Now right-click on the layer name and open the Properties window. Click on the \"Extrude features in a layer\" box in the Extrusion tab. In the Expression Builder window, click on a field you want to have provide the height variable. If the variable's value by itself won't do what you want, go back to the expression builder window and multiply or divide by some arbitrary number. Now explore the image using the Navigate and other pointers on the ArcScene toolbar. Finally, go to File\\Export Scene\\2D on the menu to save the image in whatever graphics format you want. 3D Analyst will do much more than simply making a 3D version of a flat thematic map. Contact Steve Doig by e-mail at steve.doig@asu.edu.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-may-june-ticket-data-shows-search-rate-gaps","slug":"2003-may-june-ticket-data-shows-search-rate-gaps","url":"https://dwillis.github.io/uplink/article/2003-may-june-ticket-data-shows-search-rate-gaps.html","year":2003,"month":"May • June","monthNum":5,"headline":"Ticket data shows search rate gaps","kicker":"SPOTLIGHT: RACE","author_name":"Karisa King","author_title":"Reporter","affiliation":"San Antonio Express-News","summary":"The San Antonio Express-News analyzed traffic stop data, finding that black drivers were three times as likely to be searched as white drivers despite similar contraband find rates.","keywords":["racial profiling","San Antonio","traffic stops"],"topics":["Crime & Justice","Demographics & Census","Investigative Methods"],"technologies":["Access"],"full_text":"The story began with a Texas law that went into effect in January 2002 and required every law enforcement agency in the state to track the race of drivers stopped and searched. Seeking to go beyond this debate, the San Antonio Express-News filed a state open records request and obtained the first full year's worth of data for the city. Using Microsoft Access I analyzed nearly 290,000 records detailing all the police traffic and pedestrian stops for 2002. This was my first computer-assisted reporting story, done shortly after I attended a NICAR Boot Camp in March. Before we could put the findings into perspective we had to leap one of our biggest hurdles: finding a point of comparison. Profiling experts agree that no convenient standard exists for comparing data. Census figures reflect a community's population, not the demographics of drivers. In the end, we settled on using Census data for San Antonio residents but told readers up front about the debate over comparing the numbers and emphasized that our benchmark was imperfect. We determined the stop rates by summing the total number of drivers stopped for traffic reasons for each race and calculated that as a percent of the total number of drivers stopped. Then we compared those percentages to the Census population percentages. To determine the search rates we compared the total number of drivers of each race who were searched to the total number of drivers of each race stopped for traffic reasons. Among the findings: Police stop minorities and whites for traffic reasons at rates that are roughly proportional to their share of the population. But during probable cause and consensual searches – instances in which officers use discretion – blacks are more than three times as likely as whites to face a search during traffic and pedestrian stops. Although blacks are more likely to face consensual searches, police found contraband — some type of evidence of a crime, stolen property or drugs — at about the same rates for all three groups. With those findings in hand it was time for the traditional reporting. I interviewed police officers and administrators, civil rights advocates, profiling experts and drivers. I pulled the ticket numbers from the data and submitted an open records request to the San Antonio Municipal Court for copies of about 40 traffic tickets. In response to the article, San Antonio police acknowledged that the difference between the search rates of whites and blacks caught their attention. Contact Karisa King by e-mail at kking@express-news.net.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-may-june-toronto-profiling-stories-spark-a-firestorm","slug":"2003-may-june-toronto-profiling-stories-spark-a-firestorm","url":"https://dwillis.github.io/uplink/article/2003-may-june-toronto-profiling-stories-spark-a-firestorm.html","year":2003,"month":"May • June","monthNum":5,"headline":"Toronto profiling stories spark a firestorm","kicker":"SPOTLIGHT: RACE","author_name":"Stephanie Kang","author_title":"","affiliation":"IRE and NICAR","summary":"The Toronto Star received the Michener Award for its 'Race and Crime' series, which used a landmark database request to investigate police racial profiling.","keywords":["Toronto Star","Michener Award","racial profiling"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":[],"full_text":"When Toronto Star reporter-photographer Jim Rankin submitted a Freedom of Information request for a database detailing arrests in Toronto, he knew the police wouldn't just hand over the data. There was no precedent for such an expansive request to a Canadian law enforcement agency and no database editor at the Star to help him.\n\nAlthough the Star had already lost plenty of FOI battles to the Toronto police, Rankin felt that the paper's commitment to his request was strong enough to prevail this time. He had hope.\n\nThat hope was rewarded April 10 when Rankin and four other Star reporters received the Michener Award for meritorious public service journalism for their series, \"Race and Crime.\" This year marked the first time that the award, one of the most prestigious awarded to Canadian media, recognized a computer-assisted reporting project done","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-taxes-finding-the-biggest-deadbeats","slug":"2003-julyaugust-taxes-finding-the-biggest-deadbeats","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-taxes-finding-the-biggest-deadbeats.html","year":2003,"month":"July/August","monthNum":7,"headline":"TAXES Finding the biggest deadbeats","kicker":"","author_name":"Mark Greenblatt","author_title":"Reporter","affiliation":"WBBH-Fort Myers/Naples","summary":"The article highlights the investigation on tax delinquents in Charlotte County to identify top property tax defaulters, revealing significant uncollected taxes due to inaction by the tax collector's office.","keywords":["taxes","delinquents","Charlotte County"],"topics":["Business & Economy","Investigative Methods","Government & Politics"],"technologies":["Access","CD-ROM","mainframe","ASCII"],"full_text":"Our project began with a search for the top 10 tax delinquents in Charlotte County. With government budgets facing some of their tightest years in state history, my television station wanted to know who wasn't paying their fair share of property taxes in southwest Florida. By the end of the reporting, we discovered a much more important story that could potentially help collect millions of dollars in lost revenue for Charlotte County. I began by contacting the Charlotte County tax collector and requesting a complete database of all tax certificates issued to people who hadn't paid their real property taxes in the last seven years. We wanted to know who the delinquents were and how much they owed in taxes and penalties. Within a week of the initial request, I received a comprehensive set of 'deadbeat data' from the tax collector on CD-ROM. Since Charlotte County stores delinquent taxpayer information on a mainframe system, I asked for the data in ASCII. They provided the entire database for free. I imported the text into Microsoft Access 97. Not long after beginning a search through all the records, I discovered my first roadblock. The tax collector kept detailed information on each delinquent, but surprisingly, no one had tallied up how much people owe in back taxes over the years. In fact, I was told no one had ever done a countywide calculation showing how much each delinquent owed in penalties over multiple years. Also, because interest kept adding up, the dollar figure changed every day. But the tax collector's database showed only how much delinquents originally failed to pay. To get an accurate picture of the top tax delinquents, I needed to calculate the back tax information for more than 40,000 records. After talking with the information technology office for the tax collector, I began writing a program in Access to calculate back taxes. To accomplish our task, we had to first calculate how many months had passed since each delinquency began. The more months without a payment, the more in back taxes owed. I created two new fields in Access and ran the calculations to show the number of delinquent months and the monthly interest rate for each delinquent. We used this information to calculate how much each delinquent owed in back taxes, but I still had to add a $4.25 penalty to each record to cover the cost of advertising the delinquency in a local newspaper. After completing the work, we finally thought we had all the information we needed. It didn't take long for us to discover I needed to merge the file with another county database to get an accurate picture of local delinquents. The database of tax certificates showed only the name of the person who owned the property the year its taxes became delinquent. We needed a second database to see the name of the current owner. Florida law states the current owner of property is responsible for all back taxes. I called the property appraiser's office and arranged to receive the electronic list of properties in Charlotte County. I merged the tax collector's information with the property appraiser's list by joining the files using the 'strap number' field that serves as a property identifier. Finally, with information about current property owners in hand, we were able to generate the true list of top tax delinquents. Our queries found that one company had piled up more than $950,000 in back taxes on more than 500 properties. With a quick visit to the county clerk's office, we discovered the county's top tax delinquent was going to have a very difficult time paying its bill. Court records showed the company had gone bankrupt and hadn't paid its taxes for seven years in a row. Despite this fact, the county continued to spend tax dollars sending out delinquent notices and advertising its delinquency in the newspaper year after year. This struck us as odd, since Florida law allows county governments to auction property if the owner remains delinquent for more than two years. For some reason the county was bypassing the option to auction the properties off, a move that could recoup money and return the real estate to the tax rolls. By law, the tax collector must spend tax dollars advertising delinquencies three times a year for each piece of property that qualifies. We began more detailed queries, searching for all the delinquents that hadn't paid for years but were still draining the county's coffers. We calculated the tax collector had spent $251,712 since the mid-1990s trying to collect from deadbeats it could have cracked down on long ago. We eventually uncovered a total of $8.3 million in uncollected taxes across the county since the mid-1990s. It's money that wasn't getting to essential services such as schools, fire departments and road construction crews. More than $1 million of that would have gone to the Charlotte County Fire Department. Fire Chief Dennis DiDio told us it could have added new fire trucks, firefighters and equipment to fight bio-terrorism. Further research revealed Florida law requires county governments to auction off all land worth more than $5,000, if the land has been delinquent for more than two years. If it's worth less than $5,000, counties can decide for themselves if they want to go after the property. The $8.3 million in uncollected revenue we discovered was largely made up of properties worth less than $5,000. We found the county failed to put together all the pieces on its own. It looked only at how much each delinquent owed on individual pieces of land. No one ever thought to keep track of just how much delinquent land existed under one person's name. We eventually discovered systematic proof that the county wouldn't crack down on a majority of delinquents who owned land assessed at $5,000 or less. We took our findings to the tax collector and the real property manager for Charlotte County. The tax collector told us she wanted to immediately respond by auctioning off all the delinquent land we discovered. However, that wasn't her decision to make. The real property manager decides which land to sell at tax auction. When we showed him our findings, he pledged his support for the tax collector's idea. The real property manager told us he would begin the process of auctioning off all the delinquent properties that qualified for the program. The county clerk has agreed to postpone some of its normal fees to speed the sale of the large number of properties that will be auctioned off on the courthouse steps. The Charlotte County tax collector and the real property manager credit WBBH for discovering a potential source of millions of dollars in uncollected revenue. We're told Charlotte County will complete its auctions over the course of the next year.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-julyaugust-simple-sorts-reveal-cops-overtime-jackpot","slug":"2003-julyaugust-simple-sorts-reveal-cops-overtime-jackpot","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-simple-sorts-reveal-cops-overtime-jackpot.html","year":2003,"month":"July/August","monthNum":7,"headline":"Simple sorts reveal cops' overtime jackpot","kicker":"SPOTLIGHT: JOBS, PAY & PERKS","author_name":"Mark Pazniokas","author_title":"Reporter","affiliation":"The Hartford Courant","summary":"The Hartford Courant's analysis of municipal payroll data revealed that Hartford police officers earned a median income of $75,000 last year through overtime opportunities, with some officers making over $100,000 despite modest base salaries, surprising city officials who didn't track individual officer overtime.","keywords":["overtime","payroll","Hartford"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Excel","CD-ROM"],"full_text":"Hartford police complained for years about being underpaid. Starting pay is just $34,554 - lower than in many nearby suburbs. Patrol officers' base pay tops out after five years at $56,095.\n\nSo it was startling news when The Hartford Courant reported that the median income for the city's police officers actually was $75,000 last year, with 20 percent of the force's 377 officers making at least $90,000. Abundant overtime pay and a contract that allows police to work double shifts every day helped three dozen rank-and-file cops earn more than $100,000. The top-earning cop was a patrolman: Joseph Smith grossed $143,405, with overtime making up $86,215 of his total pay.\n\nThe findings, published on May 25 in an article by Matt Burgard, our city police reporter, and me surprised not only Hartford residents, but also police administrators and union officials. The Hartford Police Department did not monitor overtime earnings by individual officers, only by units.\n\n\"Is this right?\" the police chief asked, when shown a printout of the top overtime earners. Then he turned to an aide and said, \"Who the hell is Joseph Smith?\"\n\nWe uncovered the high salaries by analyzing data that all municipal beat reporters should have on their computer desktops: a city worker payroll list, broken down by department, base pay and overtime.\n\nI requested the data earlier this year with no particular story in mind. At a minimum, I figured the payroll would be helpful in checking names and salaries on deadline.\n\nObtaining the city payroll data was easy, and analyzing it was simple even for a reporter using an Excel spreadsheet for the first time.\n\nLaying groundwork\n\nBefore requesting the data under the Connecticut Freedom of Information Act, I consulted with the city's information technology director and asked questions. How did the city maintain the records? What would it take to copy the entire payroll file to a CD-ROM? I had made a point of getting to know the director months earlier, asking him to give me a tour of the recently merged city-school board information technology system. By the time I filed my FOI request by e-mail, I knew that producing a copy of the payroll on a CD-ROM was a matter of a few keystrokes for one of his staffers.\n\nSimilar requests by The Courant have not always gone so smoothly.\n\nA few years ago, the Connecticut State Police tried to charge Jack Dolan, our resident computer-assisted reporting guru, a heart-stopping sum for a copy of its criminal convictions database. The state police figured copies of individual records cost $25 and there were 815,000 records in the database. So they multiplied $25 by 815,000.\n\nThe bill came out to more than $20.3 million. Try burying that in an expense report.\n\nThe newspaper instead opted for a lawsuit, resolved last year by the Connecticut Supreme Court. As a result, it is well-established law in Connecticut that public agencies can charge only reasonable fees for copying electronic records.\n\nThe city of Hartford gave me a copy of its payroll data at no charge.\n\nWith brief tutoring by Dolan, I started playing with the city data.\n\nThe city provided the data on an Excel spreadsheet at my request. I quickly realized, however, the data was incomplete. The city had provided me every employee's salary, not the actual pay. Missing were overtime and longevity payments, small bonuses paid for years of service.\n\nI amended my FOI request to include all payments to employees, including longevity payments and overtime. This time the I.T. director gave me the bad news: Marrying overtime records with the payroll would require staff time for which he would have to charge me. We agreed on a price not to exceed $100 - a bargain for 6,700 payroll records.\n\nThe payroll records listed each employee in alphabetical order with six other columns of data: department, regular pay, overtime, longevity pay, other pay and gross pay.\n\nSorting by salary, I created a list of the city and school employees in descending order of gross pay. The list showed rank-and-file cops making more money than the city manager, police chief and other department heads. To find all the top overtime earners, I sorted again by overtime.\n\nMore reporting\n\nWe put the analysis into a broader context by reporting more. Opportunities for overtime had been increasing every year as the department's force had been allowed to shrink. The police contract encouraged officers to take overtime when offered, especially as they neared retirement. Police pensions in Hartford are based on an officer's \"final average income,\" taken from the best three of the last five years before retirement.\n\nPatrolman Smith declined to talk to us, though we were able to answer the chief's question: \"Who the hell is Joseph Smith?\" It turned out that Smith had achieved a measure of notoriety under a previous administration. He was accused of loafing on the job and stealing city time. He was reinstated after a jury acquitted him in 1999.\n\nOfficer Plummer Carroll, who finished just behind Smith with $80,606 in overtime, was happy to talk about his industrious ways. He took all the overtime offered to run up his pension calculation. He estimated that he averaged about 80 hours a week, with a high of 110 hours, over three years.\n\n\"I missed out on a lot of beautiful things: holidays, weekends,\" Carroll said.\n\nHis reward: an annual pension of $74,093-nearly $20,000 more than his base pay of $56,095.\n\nficers making at least $90,000. Abundant overtime pay and a contract that allows police to work double shifts every day helped three dozen rank-and-file cops earn more than $100,000. The top-earning cop was a patrolman: Joseph Smith grossed $143,405, with overtime making up $86,215 of his total pay.\n\nThe findings, published on May 25 in an article by Matt Burgard, our city police reporter, and me surprised not only Hartford residents, but also police administrators and union officials. The Hartford Police Department did not monitor overtime earnings by individual officers, only by units.\n\n\"Is this right?\" the police chief asked, when shown a printout of the top overtime earners. Then he turned to an aide and said, \"Who the hell is Joseph Smith?\"\n\nWe uncovered the high salaries by analyzing data that all municipal beat reporters should have on their computer desktops: a city worker payroll list, broken down by department, base pay and overtime.\n\nI requested the data earlier this year with no particular story in mind. At a minimum, I figured the payroll would be helpful in checking names and salaries on deadline.\n\nObtaining the city payroll data was easy, and analyzing it was simple even for a reporter using an Excel spreadsheet for the first time.\n\nLaying groundwork\n\nBefore requesting the data under the Connecticut Freedom of Information Act, I consulted with the city's information technology director and asked questions. How did the city maintain the records? What would it take to copy the entire payroll file to a CD-ROM? I had made a point of getting to know the director months earlier, asking him to give me a tour of the recently merged city-school board information technology system. By the time I filed my FOI request by e-mail, I knew that producing a copy of the payroll on a CD-ROM was a matter of a few keystrokes for one of his staffers.\n\nSimilar requests by The Courant have not always gone so smoothly.\n\nA few years ago, the Connecticut State Police tried to charge Jack Dolan, our resident computer-assisted reporting guru, a heart-stopping sum for a copy of its criminal convictions database. The state police figured copies of individual records cost $25 and there were 815,000 records in the database. So they multiplied $25 by 815,000.\n\nThe bill came out to more than $20.3 million. Try burying that in an expense report.\n\nThe newspaper instead opted for a lawsuit, resolved last year by the Connecticut Supreme Court. As a result, it is well-established law in Connecticut that public agencies can charge only reasonable fees for copying electronic records.\n\nThe city of Hartford gave me a copy of its payroll data at no charge.\n\nWith brief tutoring by Dolan, I started playing with the city data.\n\nThe city provided the data on an Excel spreadsheet at my request. I quickly realized, however, the data was incomplete. The city had provided me every employee's salary, not the actual pay. Missing were overtime and longevity payments, small bonuses paid for years of service.\n\nI amended my FOI request to include all payments to employees, including longevity payments and overtime. This time the I.T. director gave me the bad news: Marrying overtime records with the payroll would require staff time for which he would have to charge me. We agreed on a price not to exceed $100 - a bargain for 6,700 payroll records.\n\nThe payroll records listed each employee in alphabetical order with six other columns of data: department, regular pay, overtime, longevity pay, other pay and gross pay.\n\nSorting by salary, I created a list of the city and school employees in descending order of gross pay. The list showed rank-and-file cops making more money than the city manager, police chief and other department heads. To find all the top overtime earners, I sorted again by overtime.\n\nMore reporting\n\nWe put the analysis into a broader context by reporting more. Opportunities for overtime had been increasing every year as the department's force had been allowed to shrink. The police contract encouraged officers to take overtime when offered, especially as they neared retirement. Police pensions in Hartford are based on an officer's \"final average income,\" taken from the best three of the last five years before retirement.\n\nPatrolman Smith declined to talk to us, though we were able to answer the chief's question: \"Who the hell is Joseph Smith?\" It turned out that Smith had achieved a measure of notoriety under a previous administration. He was accused of loafing on the job and stealing city time. He was reinstated after a jury acquitted him in 1999.\n\nOfficer Plummer Carroll, who finished just behind Smith with $80,606 in overtime, was happy to talk about his industrious ways. He took all the overtime offered to run up his pension calculation. He estimated that he averaged about 80 hours a week, with a high of 110 hours, over three years.\n\n\"I missed out on a lot of beautiful things: holidays, weekends,\" Carroll said.\n\nHis reward: an annual pension of $74,093-nearly $20,000 more than his base pay of $56,095.\n\nContact Mark Pazniokas by e-mail at impazniokas@courant.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-learning-lessons-while-tracing-pols-spending","slug":"2003-julyaugust-learning-lessons-while-tracing-pols-spending","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-learning-lessons-while-tracing-pols-spending.html","year":2003,"month":"July/August","monthNum":7,"headline":"Learning lessons while tracing pol's spending","kicker":"FIRST VENTURE","author_name":"Todd McAdam","author_title":"Reporter","affiliation":"Press & Sun-Bulletin (Binghamton, NY)","summary":"An exploration into a New York state senator's campaign finance, raising questions about the implications of campaign spending and gifts related to political influence and ethics.","keywords":["campaign finance","spending","New York"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Excel"],"full_text":"As I walked out of a prominent state senator's office following an interview, he and his spokeswoman handed me a plastic bag. In it were two bars of high-quality chocolate imprinted with his campaign logo, a coffee mug with the same design, a bottle of a spice rub suitable for beef, chicken or fish, and a bottle of steak sauce regularly sold at the politician's premier fundraiser. 'You know,' I told the spokeswoman, a longtime source, 'I can take this only if I do a story on it someday.' She laughed. Six months later, I sat at my desk and stared at the mug. It was gathering dust on a corner of my desk. I had just returned home from an IRE and NICAR Boot Camp and was eager to prove that I don't believe in junkets - and wouldn't go to Columbia, Mo., for one even if I did. Why is steak sauce important to a political campaign? That was the key element of my first computer-assisted reporting project: A question. Finding New York's campaign finance database online was easy. After that, I had to fall back on my Boot Camp training. Two pieces of advice became immediately useful - even critical - in doing the story: First, find a buddy to review your work. In my case, I tapped Brad Heath. He's a reporter in Detroit now, but he was my predecessor as Binghamton's CAR person and understands my experience as a reporter and my virtually nonexistent database skills. In fact, he knew the data I was working with, too, because he had briefly played with it. Second, give the database to the source. The campaign treasurer had done all the paperwork in filing the information, and it was in his interest to make sure I got my facts straight. He found a couple of errors in my queries. And with more than 3,000 potential sources, I still kept total control of what the story said. I learned a few other lessons. Do everything two or three times. It wasn't until I exported my query results to Excel and checked the math that I realized the numbers didn't add up, and it was because of a problem in my query. At Boot Camp, they called these data integrity checks, and they are important. Review the data. The quick and easy task is to find the extremes in your data, and that frequently can be useful. But go through the rest of it, too. You'll come up with more questions. Why, for example, couldn't I find the local Republican leaders contributing to the area's most prominent Republican, but I could find their spouses? You're not going to notice that unless you know both the data and your subject. I'm primarily a beat reporter so that kind of analysis was easy, even fun, once my co-workers and I made a game of finding how many names we recognized. I don't know how a reporter could do that coming into the data cold. I also had to learn what the data wasn't telling me, which meant I had to learn what it was supposed to say. Candidates for New York's state legislature generally contribute to a party 'housekeeping' fund to pay for legal advice, polls and whatnot. Other politicians had contributed to it, according to my analysis, but the state senator hadn't. Either he was the state's biggest political cheapskate (unlikely) or there was another fund out there the data didn't have a record of. I had to learn to keep focus on my story - how the senator's campaign spent its money. There were plenty of other tempting angles to pursue. Why was the ambassador to Grenada my boy's biggest individual contributor? And was the fact that he was indicted about 15 years ago (and later exonerated) on charges alleging he blew up trailers of a union protesting one of his facilities relevant? Fascinating stuff, but it wasn't part of the story I had in mind. Maybe next time. That's the lure of a good database. It gives you a garbage load of ideas. You can't do them all. Even if you could... you probably shouldn't. Overkill can be a nasty habit, and with all the time I sank into this project, it was a temptation. CAR stories are inexpensive compared with other investigative stories of similar depth, but they're inefficient compared with traditional methods. That can be an issue at a community newspaper, where cutting somebody free from the daily need to feed the beast can be a burden on everybody else. Fortunately, Heath taught my editors that so I didn't have to. Through all of this, Heath double-checked my work and cheered me up when I felt dumb. With his help, I learned to rewrite queries to make them as simple as possible. The easy.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-julyaugust-historical-spending-data","slug":"2003-julyaugust-historical-spending-data","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-historical-spending-data.html","year":2003,"month":"July/August","monthNum":7,"headline":"Historical spending data","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library provides journalists with comprehensive archived federal spending data from 1979-2002, including contracts and assistance awards, enabling them to track government spending patterns across decades and communities. The library also offers data analysis services, as demonstrated by their recent environmental mapping project for a Chicago news station.","keywords":["spending","databases","analysis"],"topics":["Data & Databases","Business & Economy","Health & Environment"],"technologies":["ArcView","Spatial Analyst"],"full_text":"The IRE and NICAR Database Library has compiled archived data to provide journalists with the most thorough tools for covering federal spending. This historical data allows journalists the unique opportunity to go back in time and track federal spending in their communities across decades. Journalists can use the data to show how different presidential administrations, economic conditions and pivotal events affected spending.\n\nThe IRE and NICAR Database Library now has Federal Contracts data covering fiscal years 1979-2002. The database lists almost all federal contracts worth more than $25,000 and includes the amount, name and location of the company, the agency, the type of work and where the work was performed.\n\nThe Database Library has also added Federal Assistance Award Data System (FAADS) covering fiscal years 1983-2002. Journalists can use the FAADS data to determine what federal money has been obligated to individuals, state and local governments, institutions of higher education, profit organizations, nonprofit agencies and others.\n\nFor more details on these datasets, including prices and ordering information, go to www.ire.org/datalibrary/databases or contact the IRE and NICAR Database Library at 573-884-7711.\n\nThe IRE and NICAR Database Library recently conducted an environmental data analysis for a story that aired on WMAQ-Chicago in May. The analysis identified areas with a large number of polluters under state remediation programs. Using ArcView, the Database Library mapped polluters by ZIP code, by latitude and longitude and, using the Spatial Analyst extension, found the neighborhoods with the highest concentrations of polluters and pollutants. To read the story, go to www.nbc5.com/news/2203768/detail.html. For more information on how IRE and NICAR can help your news organization conduct data analysis, contact the Database Library at 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-ires-chief-trainer-moves-on","slug":"2003-julyaugust-ires-chief-trainer-moves-on","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-ires-chief-trainer-moves-on.html","year":2003,"month":"July/August","monthNum":7,"headline":"IRE's chief trainer moves on","kicker":"","author_name":"David Herzog","author_title":"","affiliation":"IRE and NICAR","summary":"Ron Nixon, a veteran training director for IRE and NICAR who has trained thousands of journalists in computer-assisted reporting, is returning to newspaper work as the computer-assisted reporting editor for the Minneapolis Star Tribune.","keywords":["journalism","training","reporter"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["GIS"],"full_text":"Ron Nixon, who's trained thousands of journalists across the globe in computer-assisted reporting for IRE and NICAR, is headed back to the newsroom.\n\nAfter two and a half years on the road, Ron is set to go home to Minneapolis, where he will become the computer-assisted reporting editor on the Star Tribune projects team in early September.\n\nAs the training director for IRE and NICAR, Ron logged countless miles to help print and broadcast journalists do their jobs better and earned their praise. He's trained in newsrooms, national journalism association events and at events conducted by IRE and NICAR.\n\nRon joined IRE and NICAR as the director of the Campaign Finance Information Center (www.campaignfinance.org) and later moved into the top training post. Before joining IRE and NICAR, Ron was an investigative reporter for The Roanoke (Va.) Times with years of experience working with local, state and federal campaign finance data.\n\nIn his spare time at home, Ron kept his skills sharp by performing database and geographic information system analysis for clients of the IRE and NICAR database library.\n\nRon's last official day with IRE and NICAR is Aug. 29. Drop him a note at ron@ire.org and join us in wishing him well as he returns to newspaper journalism.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-hands-on-car-training-upcoming","slug":"2003-julyaugust-hands-on-car-training-upcoming","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-hands-on-car-training-upcoming.html","year":2003,"month":"July/August","monthNum":7,"headline":"Hands-on CAR training upcoming","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer multiple computer-assisted reporting training opportunities including Better Watchdog Workshops, CAR Boot Camps, and an annual conference throughout the coming months.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet"],"full_text":"IRE and NICAR has numerous training opportunities in the coming months for journalists who want to learn computer-assisted reporting. For a complete list of training events visit www.ire.org/training/otr.html.\n\nHere are some of the highlights:\n\nJournalists can get an overview of computer-assisted reporting at Better Watchdog Workshops offered by IRE and the Society for Professional Journalists. Upcoming workshops include Sept. 11 in Tampa, Fla., Oct. 4-5 in Eugene, Ore., and Oct. 25 in State College, Pa.\n\nThere are four CAR Boot Camps in Columbia, Mo., next year for journalists who want hands-on training in how to use spreadsheets and database managers to analyze data for high-impact stories. The sessions are Jan. 4-9, March 21-26, May 16-21 and Aug. 1-6.\n\nAlso, mark your calendar for the annual CAR conference, March 12-14 in Cincinnati. The conference promises to be great for tips, story ideas and hands-on training.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-car-gives-contest-winners-boost","slug":"2003-julyaugust-car-gives-contest-winners-boost","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-car-gives-contest-winners-boost.html","year":2003,"month":"July/August","monthNum":7,"headline":"CAR gives contest winners boost","kicker":"IRE AWARDS","author_name":"Rachel Schaff","author_title":"","affiliation":"IRE","summary":"IRE's investigative reporting contest winners and finalists demonstrated how computer-assisted reporting, including spreadsheets and databases, uncovered corruption, government failures, and public health issues across various newspaper categories.","keywords":["investigation","reporting","data"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["Excel","Access","FoxPro"],"full_text":"The May/June issue of Uplink highlighted the television, special categories and other media winners and finalists of IRE's annual investigative reporting contest that used computer-assisted reporting. Here we feature the newspaper, FOI and Tom Renner award winners and finalists that used CAR.\n\nLocal circulation weeklies\nFinalist:\n\nAtlanta Business Chronicle: \"Northern Arc: Road to Riches\" by Sarah Rubenstein and Walter Woods. Rubenstein and Woods exposed how the planned construction of a new highway north of Atlanta, called the Northern Arc, would benefit government officials who owned property along the proposed road. The reporters based their investigation on land parcel maps and corresponding Microsoft Excel spreadsheets for the counties through which the Northern Arc would pass. Rubenstein and Woods were named journalists of the year by the Atlanta Press Club and the story won Best Scoop from Atlanta Magazine. (Story No. 19576 in the IRE Resource Center)\n\nSmall newspapers\n(Circulation less than 100,000)\n\nFinalists:\n\nCharleston (W.Va.) Gazette: \"The Long Haul/Broken Promises\" by Eric Eyre and Scott Finn. This investigation examined the effects of school consolidation in West Virginia, including the increased amount of time students had to spend traveling to and from school. Eyre and Finn requested bus logs under FOIA for 35 counties. With the information, they built a spreadsheet of 1,569 bus runs, forming the foundation for the story. The series won the Education Writers Association's Fred M. Hechinger Grand Prize for Distinguished Education Reporting. See \"Measuring long bus rides for rural children\" in the September/October 2002 Uplink for a full account of the data analysis. (Story No. 19601)\n\nTri-City Herald (Pasco, Wash.): \"Bitter Harvest\" by Mike Lee. This story investigated a series of deaths, environmental damage and accidents that were all traced back to one Columbia Basin farm, the largest organic farm in Washington state. Lee showed how the farm's owners ignored environmental regulations, and he exposed the government's failure to take any decisive action. Lee used spreadsheets to track story threads and piece together information from a dozen agencies. The story - which relied on more than 20 public document requests from state and federal agencies - won in the investigative reporting category in Sigma Delta Chi's competition for newspapers with a circulation under 100,000 and first place for agriculture/business reporting in the Inland Northwest SPJ competition. (Story No. 19608)\n\nMedium newspapers\n(100,000-250,000 circulation)\nCertificate:\n\nThe Fresno Bee: \"Last Gasp\" by Russell Clemings, Barbara Anderson and Mark Grossi. This special report traced how California's San Joaquin Valley air basin became one of America's most polluted. The reporters used FOI requests to acquire information from many sources, including the U.S. Environmental Protection Agency. They then used Access, Excel and FoxPro to compare current levels of pollution with historical estimates and eventually to understand the growing toxicity of the basin. \"Last Gasp\" also won first place in the Associated Press News Executives Council's contest for public service reporting; first place in the Best of the West explanatory reporting category; the Meade Clean Air Prize; the Coalition for Clean Air's Corporate Clean Air Leadership Award; and the EPA's Environmental Achievement Award. See the September/October 2003 issue of The IRE Journal for details of their investigation. (Story No. 19677)\n\nFinalists:\n\nThe Birmingham News: \"The Black Belt: Alabama's Third World\" by John Archibald, Jeff Hansen, Carla Crowder, Thomas Spencer and Marie Jones. This piece focused on a small region of Alabama called the \"Black Belt.\" People in this region, mostly poverty-stricken African-Americans, live in conditions much worse than those in the rest of the state. The reporters relied on databases of all land records for 10 Alabama counties as well as census records, mortality and birth records, data from the Alabama Department of Public Health, property tax records, and other state databases. The authors built many of their own databases using Excel and Access. This series also won awards from the Alabama Associated Press Managing Editors, Alabama Press Association and the Medical Association of Alabama. (Story No. 19724)\n\nDayton Daily News: \"Deadly Consequences, Ohio's Broken Mental Retardation System\" by Tom Beyerlein and Steve Bennish. The Dayton Daily News identified 30 cases of neglect within Ohio's mental retardation system by matching death cases against addresses of group homes for the mentally retarded. The reporters used Excel and Access to analyze 400,000 death records and develop their own database of deaths in group homes. (Story No. 19107)\n\nThe Virginian-Pilot: \"Operating Behind Closed Doors\" by Liz Szabo. This story illustrates the failures of Virginia's system for protecting patients from dangerous doctors by highlighting the last decade in the","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-training","slug":"2003-julyaugust-training","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-training.html","year":2003,"month":"July/August","monthNum":7,"headline":"Training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering Computer-Assisted Reporting Boot Camps in Columbia, Missouri throughout 2004 with four scheduled sessions, providing specialized training opportunities for journalists.","keywords":["training","journalism","CAR"],"topics":["Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"Although no IRE and NICAR Boot Camps remain this year, journalists will have many opportunities in 2004 to take advantage of the specialty training that these high-impact CAR seminars offer. Dates for the 2004 Boot Camps in Columbia, Mo., are Jan. 4-9, March 21-26, May 16-21 and Aug. 1-6.\n\nFor more details about the Boot Camps, including fellowship and registration information and a typical daily training schedule, go to www.ire.org/training/bootcamps.html. For information on other training opportunities available from IRE and NICAR, go to www.ire.org/training.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-tallying-the-bill-for-unused-sick-time","slug":"2003-julyaugust-tallying-the-bill-for-unused-sick-time","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-tallying-the-bill-for-unused-sick-time.html","year":2003,"month":"July/August","monthNum":7,"headline":"Tallying the bill for unused sick time","kicker":"SPOTLIGHT: JOBS, PAY & PERKS","author_name":"Robert Gebeloff","author_title":"Reporter","affiliation":"The Star-Ledger","summary":"The Star-Ledger used computer-assisted reporting to uncover that New Jersey local government agencies face nearly $1.5 billion in liabilities for unused sick and vacation time payouts to retiring employees, a benefit that evolved from Depression-era protections into a costly modern practice.","keywords":["payouts","government","benefits"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":["spreadsheet"],"full_text":"When we use computer-assisted reporting for our government coverage, salary data is often at the top of our list.\n\nWe obtain government payrolls and rank them. We compare median salaries of teachers among our school districts. When a government worker gets suspended for official misconduct, we don't report just that. We write that the worker was suspended from a $90,000-a-year job.\n\nBut salary data is just the tip of the iceberg in reporting on government workers and their benefits. There's police overtime, government-issue cell phones and credit cards.\n\nAnd in New Jersey there's the matter of sick and vacation time. Seemingly innocuous, except, as we found, that local government agencies are now on the hook for nearly $1.5 billion to reimburse employees who fail to use their allotted time.\n\nEver since the Great Depression, civil servants in New Jersey have essentially been guaranteed 15 paid sick days a year. Over time, government agencies began allowing workers to carry unused days over into sick day banks, a practice later codified in union contracts.\n\nIt's a progressive benefit intended to protect workers who suffer prolonged illness. But in the modern era, it has become a bonanza for retirees. As public employee unions gained strength in the 1970s, they managed to negotiate mandatory payouts for unused sick and vacation time.\n\nIn almost all cases the payouts are made to retirees at their final pay rate, not the salary at the time the days were banked.\n\nThe Star-Ledger was certainly not the first paper to notice how common it had become for a police chief or schools superintendent to collect a six-figure payout - on top of a generous pension - just for retiring.\n\nIn fact, when I was with a regional paper in New Jersey a few years ago, we did a pretty good CAR story that looked at the potential cost of this system.\n\nBut nobody had ever been able to report precisely what the statewide impact was, and thus we had a mission.\n\nWhen it came to gathering data, we had one thing going for us. The state had recently enacted new regulations that required municipalities to annually calculate the value of \"compensated absences.\"\n\nWe found that school districts had been making the calculations for years and listed the amount as a long-term liability on their annual audits.\n\nThat was the good news. The bad news was that none of this information was available electronically. We wanted a number for 1,177 different agencies throughout the state and it was all on paper. We discovered that all of the paper ends up in one place: Trenton.\n\nSo with a research assistant and laptop in tow, I made a series of visits to the state capital. It took us about six days to pull the 1,177 documents out of filing cabinets one by one, find the sick time number on each and enter them into a spreadsheet.\n\nOnce we had the numbers it was easy to add it all up and come up with the $1.5 billion total.\n\nMore challenging was coming up with a method for compiling town-by-town statistics. In addition to the state's 566 municipalities, we had to account for school districts and county agencies that collected taxes across town borders.\n\nSince we wanted to measure the impact on the basis of property taxes, we used property tax data to help us divvy up the burden.\n\nFor example, employees of the West Essex Regional district had $400,000 in the sick-time bank. Roseland is a town served by the district, and about one quarter of the district's tax base is located in Roseland. Therefore, we assigned one quarter of the obligation to pay West Essex retirees to Roseland residents.\n\nWe did the same with the obligation to county employees so at the end of the process, we had one figure for each town that incorporated what was owed to municipal workers, school employees and county employees.\n\nAfter we had a single number for each town, we wanted to add context.\n\nObviously, the biggest cities would have the most employees - and likely the highest total obligation.\n\nYet these towns would also be able to spread their obligation among a larger base of taxpayers. So we needed to come up with a way of quantifying the number. We came up with two.\n\nFirst, we did a simple per-capita calculation. Then we plugged the number into the town's property tax formula and expressed it in terms of the average homeowner.\n\nWe used both in the story. Per capita, we decided, was an easy-to-understand ratio that generally put the total obligation into context. The average homeowner cost, meanwhile, was a more precise measure that told readers \"This is exactly what it's going to cost you in coming years.\"\n\nWe found that in a tiny urban enclave called Harrison, the average homeowner was on the hook for nearly $3,100. Overall, the debt was $251 per person in the state's 30 cities, but only $152 per person in the suburbs.\n\nWe also found that individual golden parachutes were being proffered all over the state. In one of the audits we pulled, for a tiny rural community called Stillwater, there was a footnote from the auditor explaining that the town's obligation was unclear, pending resolution of a conflict with a soon-to-retire police chief.\n\nUpon closer examination, we found a community where the police force could be counted on one hand, the number of annual crimes on two, and the police chief was walking away with a severance package of $117,587 for sick and vacation time he didn't use.\n\nAfter the story ran, a reader tipped us off to another case that lead to a follow-up: The police chief in Jersey City had recently been granted a compensation package that provided him with an instant bank of 150 sick days-days he could cash in upon retirement.\n\nThat wasn't necessarily unusual, except that the chief had already retired once from the force, as a lieutenant, and cashed in a six-figure payout. Now he was in line to get a second helping of taxpayer generosity.\n\nThere were, of course, arguments included in these stories for why such payouts are justified. Unions told us that because these benefits are often negotiated for their rank and file, it means the unions had to give up something else to get such a deal.\n\nStill, the point of our stories was not to pass judgment on the system. Rather, our goal was to tell readers the real cost of such a system - a figure that had been unknown until we completed our project.\n\nRead the story, \"Cashing Out,\" online at www.nj.com/specialprojects.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-gis-shows-close-call-for-dallas","slug":"2003-julyaugust-gis-shows-close-call-for-dallas","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-gis-shows-close-call-for-dallas.html","year":2003,"month":"July/August","monthNum":7,"headline":"GIS shows close call for Dallas","kicker":"MAPPING The latest uses of mapping in news reporting.","author_name":"John Kelly","author_title":"Reporter","affiliation":"Florida Today","summary":"Journalists used GIS mapping software to analyze the debris field from the Space Shuttle Columbia disaster and determine that approximately 216,000 people were in the path of falling wreckage, while showing that the shuttle breaking up just one minute earlier would have sent debris into Dallas suburbs, potentially affecting three times as many people.","keywords":["shuttle","mapping","debris"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["ArcView","GIS"],"full_text":"GIS shows close call for Dallas\n\nBy John Kelly, Florida Today\n\n\"How many are dead on the ground?\"\n\nI must have asked that question a dozen or so times the morning of Feb. 1, as my colleagues and I scrambled to find out what happened to the shuttle Columbia in the far reaches of the Texas sky. Watching the televised images of giant steel balls, mangled fragments of wing and other heavy metal objects strewn across Texas, it seemed obvious the death toll was going to be higher than the seven astronauts. The question was, \"How many?\"\n\nThe still-dumbfounding answer is zero. NASA got lucky.\n\nNASA seemed to be ignoring how many people on the ground had been in harm's way. It was a fundamental public safety issue. We weren't about to ignore it. We could have just thrown together color from Texas with dueling expert quotes for a \"what-a-miracle\" story. Instead, we wanted to show in a definitive way the very real risk of flying the space shuttle over populated areas.\n\nWe set out to apply the same methods that NASA, the U.S. Air Force and big aerospace contractors use when they try to assess danger to people on the ground during space launches, or when one of their unmanned satellites or rockets eventually plummets back to Earth.\n\nThat turned out to be the right approach for two reasons.\n\nFirst, my colleague Todd Halvorson and I piled up a mountain of public records showing how carefully the government and aerospace companies make sure that debris from re-entering spacecraft and rockets does not strike land, let alone come down on populated areas. By contrast, Halvorson found NASA had never studied the danger to people on the ground during the very risky re-entry of its space shuttles.\n\nSecond, we learned the industry's accepted method for minimizing death and destruction on the ground was to predict the footprint, or area, where the debris would hit, and to do everything possible to control the re-entering space junk so the danger zone would contain few, if any, residents.\n\nI knew we could use ESRI ArcView mapping software to determine the number of people who were in the path of shuttle parts. Using ArcView we layered the debris footprint over U.S. Census population data and determined that about 216,000 people lived in the largely rural area where most of the debris hit. We also used the shuttle's flight path to show the more grim consequences had the shuttle broken up about a minute earlier.\n\nIn that case, the storm of metal would have hit Dallas' suburbs. About three times as many people and houses would have been in the path of falling wreckage.\n\nComputer-assisted reporting nerd that I am, I overcomplicated things from the start.\n\nGIS and computer mapping tools were being used in the recovery effort. It was clear search teams were recording the precise coordinates of each piece of shuttle found. NASA was making computer maps.\n\nI went to Stephen F. Austin University, which was helping NASA with the mapping. They made available online ArcView shapefiles of the region of Texas where debris was found. I wanted the shapefiles that were the basis of NASA's nifty handouts, showing every piece of debris as a dot on the Texas map. I had the big idea that I was going to calculate how many people lived a set distance from any one of those dots.\n\nThe university asked NASA if they could give me the data. NASA said no. We could FOIA the database, but we did not have time to wait.\n\nI went back to what the university released online. I found a shapefile of the 20-kilometer buffer zone calculated from the centerline of the debris field. The oval was as close as we could get to an \"official\" debris footprint.\n\nWe used a free ArcView 3.x extension called Two-Theme Analyst. I laid the approximated debris footprint over Texas Census data and then used Two-Theme Analyst to approximate the number of people living within the debris oval.\n\nWe ran the analysis using Census data at the block, block group and tract level. We got three sets of very similar numbers. In the end, we went with block groups because it worked best for the maps we ultimately published in the newspaper.\n\nWe also shifted the footprint northwest along the flight path to get \"what-if\" numbers for a breakup over Dallas-Ft. Worth. I then re-created the maps in ArcView 8 and worked closely with graphic artists to convert from ArcView to Adobe Illustrator a set of maps that showed readers the dangerous consequences of the two scenarios.\n\nThat could not have been done in words alone - and certainly not without the computer analysis.\n\nWe ran a pair of population density maps side by side in the paper that we prepared with ArcView. One showed the real debris field. The other showed the scenario. The simple contrast of population density made a very clear point about NASA's good fortune. Detailed counts of people and homes were given too.\n\nOur online staff turned the maps into a telling interactive presentation on the Web. See the story at www.floridatoday.com/columbia/columbiastory2A52178A.htm\n\nMeanwhile, Halvorson and I also were doing the research and reporting to frame the issue in the appropriate context.\n\nWe wanted to show how NASA fretted in the past about where debris from satellites the size of school buses might hit. Yet for weeks leading up to the publication of our report, the agency said it never deemed necessary a study of what might happen with a craft the size of a jetliner. Officials explained the space shuttles were designed to survive re-entry and the chance of a breakup was so remote that studying where debris might fall was neither necessary nor practical.\n\nOur story acknowledged NASA's options are limited. There are only so many paths the shuttles can fly to get back from space. But there are options to reach landing strips in Florida and California while substantially reducing the number of population centers beneath the flight path.\n\nWithin a week of the story running, the Columbia Accident Investigation Board said it was looking for contractors to study the issue we raised. NASA Associate Administrator Bill Readdy told The Washington Post the agency commissioned a similar study. Neither will say if the studies were initiated before our report or because of it.\n\nThe official government investigation report will ask if public safety should have dictated that NASA try to land Columbia in California that morning instead of flying over so many major population centers from San Francisco to Dallas. NASA is not finished studying its re-entry options for the future, but investigators agree changes to shuttle flight rules are inevitable.\n\nContact John Kelly by e-mail at jkelly@flatoday.net","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-combing-through-data-finds-top-paid-workers","slug":"2003-julyaugust-combing-through-data-finds-top-paid-workers","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-combing-through-data-finds-top-paid-workers.html","year":2003,"month":"July/August","monthNum":7,"headline":"Combing through data finds top-paid workers","kicker":"SPOTLIGHT: JOBS, PAY & PERKS","author_name":"Hanah Cho","author_title":"Reporter","affiliation":"Capital News Service","summary":"A journalist describes her experience updating a 1990 analysis of Maryland's highest-paid public employees, detailing her challenges obtaining salary data through public information requests and navigating disputes with state agencies over data format and accessibility.","keywords":["transparency","database","bureaucracy"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["Excel","Access"],"full_text":"The story was supposed to be simple. My task was to update a 1990 analysis by Capital News Service, a wire service operated by the University of Maryland, examining the highest-paid public employees in the state. The analysis then found that the highest-paid state employees were mostly white and male and came primarily from the academic world, particularly from the public medical schools.\n\nAfter preliminary research and interviews, I determined that state employee salary data in Maryland would come from two sources - the state Office of Personnel Services and Benefits, and the University System of Maryland. (Looking back, after attending a NICAR Boot Camp this year, I also should have contacted the state comptroller's office for salary information.)\n\nI didn't have much time, maybe about two months, to finish the project. So, after pinpointing the right contact person at each agency, I immediately filed public information act requests with both agencies. I was very specific in my request and asked for a number of items, including the top 100 highest-paid employees, names, salary, job title, department, gender and race, even though I was unsure I would get the latter because race may be considered private information. I also asked for the information in electronic form, such as Excel or Access. If I had had more time, would have requested the agencies' complete payroll database so I could perform a more extensive analysis.\n\nThe university system complied with the request within weeks, providing the information in an Excel spreadsheet at no charge. The state Office of Personnel Services and Benefits wasn't as efficient or quick. The agency's lawyer argued that my request would essentially create new records, since it would require the agency to reprogram its database something that's not required under the public information act. The lawyer also argued that my request imposed a burden on the agency because it would require extra work.\n\nThe lawyer was strict in her interpretation of the law, while I believed the law left room for negotiation. For example, Maryland's open records act ultimately leaves the choice of format up to the agency. However, the law also urges the agencies to voluntarily agree to the requester's choice of format if it does not require much work or significant cost. I knew my request would take a simple query or two, especially since the university system appeared to have no problems with it.\n\nI found that the database did not tell the complete story.\n\nSo the lawyer and I wrangled back and forth on the law's intent. As the last straw, I put our position in writing to the lawyer as well as to the Maryland state attorney.\n\nMeanwhile, I began analyzing the university employees' data, a table that included all the requested fields except race. The seven columns had the last name, first name, salary, title, department, university and gender. The first thing I did was make a copy of the database, a point that was drilled into my head by my computer-assisted reporting professor, who warned of the dangers of working off the original file. And, although the database was fairly clear, I ran an integrity check to see that there were in fact 100 rows representing the 100 highest-paid employees and whether there were any null fields.\n\nI found discrepancies in how the names of departments were entered into the database, a quirk that I had to straighten out. I manually typed in the full name of departments by individually going through the department fields, because at that point my query skills were pretty basic.\n\nAfter that, I imported the file into Access to run some queries, such as group by department and gender and count. I noticed something immediately. The University of Maryland men's football and basketball team coaches - Ralph Friedgen and Gary Williams, respectively - appeared quite low on the list, even though the media had reported that they make salaries in the millions. So I inquired about the coaches' contracts. But the university's athletic department refused to disclose contract details, arguing that the university was required only to release state-funded salaries.\n\nYet the state law makes it clear that bonuses and performance awards, which Friedgen and Williams most likely received, are considered public. So I filed a public information request even though I wasn't sure I would get the information before my story deadline. My editors also talked to the lawyer.\n\nAbout a month before my deadline, the state finally relented and gave me its top 100 highest-paid state employees in the form I requested. After the databases were combined (which I did manually by copying and pasting, since most of the highest-paid employees came from the university), it was easy to see that the highest-paid employees came mostly from the university.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-analysis-spots-hiring-during-state-crunch","slug":"2003-julyaugust-analysis-spots-hiring-during-state-crunch","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-analysis-spots-hiring-during-state-crunch.html","year":2003,"month":"July/August","monthNum":7,"headline":"Analysis spots hiring during state crunch","kicker":"SPOTLIGHT: JOBS, PAY, & PERKS","author_name":"Spencer Hunt","author_title":"","affiliation":"Cincinnati Enquirer","summary":"An Ohio newspaper analysis revealed that state agencies hired thousands of workers, including highly paid administrators, despite Gov. Bob Taft's hiring freeze during a budget crisis, with nearly 1,000 non-essential hires costing taxpayers up to $39 million annually.","keywords":["hiring","budget","analysis"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Excel"],"full_text":"In the midst of a growing budget crisis, Ohio Gov. Bob Taft outlined his short-term plan to help keep the state's finances in order.\n\nIn addition to cutting back spending at state agencies and raiding the state's \"rainy day fund\" for $350 million, Taft told reporters he would halt hiring. \"With the exception of people critical to health, safety and those involved in revenue-generating activity, we will put all hiring on hold effective immediately,\" the governor said.\n\nThat statement, an incorrect tip from a good source, and several e-mails from the state's payroll system helped lay the groundwork for an Enquirer analysis that showed state departments and agencies hired thousands of workers, including several highly paid administrators, while the budget continued to tank.\n\nThe tip was a complaint that Taft's own cabinet leaders ignored his freeze and hired new people \"left and right.\" It turned out that the tip was wrong, but that rumor had a kernel of truth.\n\nA few months after he announced the freeze, Taft lifted it. That was on July 1, 2002, the beginning of the state's new fiscal year.\n\nThere was still a story to write when a cursory glance at state payroll data in an Excel spreadsheet revealed more than 5,000 people had been hired or promoted since July\n\nThese hires were made while dwindling income and sales tax revenue led Taft to cut spending again. This time he eliminated $140 million that would have gone to schools and universities. He'd also recently announced his plans to raise more than $3.1 billion in new taxes.\n\nThe story, published March 30, showed 3,859 people had been hired or promoted to new jobs since July 1. Nearly 1,000 of those people were paid for work not directly linked to health care, public safety or highway maintenance at a taxpayer cost of up to $39 million a year.\n\nThe analysis was done in Excel, using data obtained under the state public records act from the Ohio Department of Administrative Services. This agency handles the payroll for all non-university state employees.\n\nGetting data was not a problem. Getting the best, most useful data was. The data arrived by e-mail in delimited text at no cost, usually within a day of a request. The spreadsheet had about 5,000 rows, one line per person.\n\nThe first spreadsheet showed the names, agencies, hiring dates and hourly pay rates. I eventually got a spreadsheet that included a more detailed job description, instead of a code, and columns showing if the person hired was still employed.\n\nI sorted the data by job description, name and hiring date to identify people and take out those who weren't included in the analysis. For instance, there were hundreds of college interns and temporary state fair workers who had to be cut out because they worked for only a matter of weeks.\n\nThere were people who had been hired under a federal program that supplied all the money. These hires would have occurred regardless of the freeze. Since the payroll records didn't identify which jobs were paid by federal funds, I had to do some additional reporting by calling agencies that had hired a significant number of new employees.\n\nIn several cases in which more than 20 people had been hired during the same month with the same job description, it turned out that the hiring was done with funds from a new grant or federal program. Additional reporting with officials in other agencies turned up even more federally paid employees.\n\nAt the end of this process, 3,859 employees remained. Because state agency officials argued without exception that every hire would have been made under the freeze, my editors and I had to come up with our own definition of an essential employee. We considered anyone directly involved in making sure others were healthy and safe to be essential. We included prison guards, highway patrol cadets, cooks, hospital workers, mental health workers, workers at state institutions for the mentally retarded, nurses, doctors and psychiatrists. Psychiatrists were a tough call because almost every one was paid more than $100,000 a year. We also included highway maintenance workers, because it's essential to keep the roads in good repair.\n\nThat eliminated another 2,870 employees. Finding out exactly how much money the non-essential state employees cost taxpayers was another problem because the data did not show whether they worked full time or part time. Other states' payroll records may show this, and anyone else thinking about doing a similar story should certainly ask for it.\n\nIn this case, the $39 million figure we arrived at had to be qualified. We wrote in the story that the figure is probably high, but we also decided not to include the cost of health insurance and other benefits, which payroll officials say tack on another 30 percent onto the base pay.\n\nOnce the data was in shape and I had created a separate spreadsheet showing only the non-essential hires, the remaining analysis consisted of simple sorts in Excel. Sorting by pay revealed the most expensive hires. Sorting by date showed who had been hired on or shortly after July 1.\n\nIn a few cases, expensive employees were hired right after the freeze was lifted. The prime example was a Department of Education employee who was hired July 8 for $120,000 to help make sure state education reforms dovetailed with the federal \"No Child Left Behind\" initiative. Sorting by job description showed how many of each type of employee was hired. Sorting by state agency showed which sector of government did the most hiring. Ohio's lead welfare agency, the Department of Job and Family Services, hired 140 people.\n\nI also used Excel to calculate the average cost of a subset of new hires. For example, I determined that the Department of Education hired 28 education consultants, people who act as liaisons with local school districts, at an average $53,500 a year.\n\nThis kind of analysis takes some time especially if you're learning just what payroll records can and cannot show you along the way. The result, I believe, is a story that offers a perspective on state finances and spending that only an independent observer can provide.\n\nOf course, you will also have to deal with state officials who probably will argue that every single person they've hired was out of an absolute necessity.\n\nThe best response came from the Ohio Department of Taxation, which hired 55 auditors in December and January to beef up collections. While each auditor was paid an average $34,500 salary, they claimed each collected about $457,000 in unpaid taxes.\n\nThat prompted a spokesperson to say, \"They are more than carrying their weight.\"\n\nContact Spencer Hunt by e-mail at shunt@cincinna.gannett.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-gps-units-add-depth-to-reporting","slug":"2003-julyaugust-gps-units-add-depth-to-reporting","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-gps-units-add-depth-to-reporting.html","year":2003,"month":"July/August","monthNum":7,"headline":"GPS units add depth to reporting","kicker":"CAR TOOL","author_name":"Matthew Waite","author_title":"Reporter","affiliation":"St. Petersburg Times","summary":"A journalist explains how GPS technology and geographic mapping tools have become invaluable for reporting, allowing him to quickly create location-based analyses and maps for both deadline and investigative stories.","keywords":["GPS","mapping","journalism"],"topics":["Technology","Investigative Methods"],"technologies":["GPS","ArcView","Maptitude","GIS"],"full_text":"You won't hear me complaining about military spending anytime soon. The U.S. government - bless it - dropped $90 million to add another Global Positioning System satellite at the end of March, and I gleefully paid my taxes two weeks later.\n\nGiddy about paying taxes? Sure. I'm happy to contribute my small sum to the $2.5 billion Global Positioning System constellation of 28 satellites that surrounds the planet and blankets it with low-power radio waves. Before you report me to the mental health authorities, let me explain why I'm so happy: It helps me do my job as a reporter.\n\nI believe GPS is the best little-used tool in journalism. A GPS receiver, mapping software and a little know-how are all you need to do your own geographic analysis. Think of it as building your own databases, only with maps.\n\nYou can do this all very quickly and accurately. In the past, a reporter going after a plane that crashed in a remote area had to try to explain the accident location over the phone. Now, a reporter with a GPS unit can, depending on how close he gets to the crash, plot the crash location within a few feet.\n\nGPS is a handy tool on deadline and off. Off deadline, I used GPS to create what the state of Florida lacked a few years ago: a map of traffic signal locations. The state had only engineering drawings and computer-aided drafting files showing the locations. So I decided to create a map. I got in the car with my GPS and my wife at the wheel. Within an hour I had a point map of every traffic signal on a stretch of U.S. 19, a deadly road I was writing about. That map became an important part of my analysis in ArcView, because I could layer deaths and traffic control lights. On one section of roadway with the most traffic control lights, there were no deaths in more than a decade.\n\nOf all deaths, more than 80 percent were people crossing away from crosswalks. (See the story at www.sptimes.com/2002/01/20/Pasco/At_any_time__crossing.shtml)\n\nOn deadline, GPS can help locate events or create maps of new roads.\n\nHere's a step-by-step example of how I used GPS for a daily news story:\n\nOne night, after deadline, I heard an emergency search call on the police radio scanner for a missing elderly man. This happens all the time in Florida, so I wasn't very interested. Then I heard someone call for a U.S. Coast Guard helicopter to pull him out of a swamp in the middle of the night. That sounded like news.\n\nSo I went to the scene - a golf course covered in the darkness - and watched as a huge helicopter hovered over the man, lowered a basket down to a group of sopping wet sheriff's deputies and hoisted him away to a local hospital.\n\nThat night, I grabbed as many interviews with deputies that I could, and I grabbed a scorecard for the golf course.\n\nEarly the next morning I returned with my GPS unit and laptop. I plotted the points where the man went into the swamp looking for golf balls and where the sheriff's deputies went into the swamp to find him. I also plotted the points of the tee box and cup from the nearest hole to where the man was found. Why do this? You can get details like distances and reference points. And, using a satellite map of the area, you can also make great graphics. (See the story, without the graphic, at www.sptimes.com/2002/04/09/TampaBay/Off_the_course_and_st.shtml)\n\nThere are several ways to get GPS data from your unit into your geographic information system software. The one I used until recently is Maptitude 4.05 or later. Within the past few months, ArcView 8.3, through a free download, has added GPS support that rivals Maptitude's. ArcView 3.x users can find shareware and free scripts for almost any GPS device, though they are not nearly as easy or intuitive as either Maptitude or ArcView 8.3.\n\nMaptitude (www.caliper.com/maptovu.htm) and now ArcView 8.3 (http://support.esri.com index.cfm?fa=downloa.samplesUtilities.viewSample&PID=43&MetalD=537) allow you to plug in your GPS unit via a serial cable (or USB, depending on the unit) that's often included with the unit. It plots your location live on a map. In one menu, you can configure and control your GPS unit, and control what data is displayed.\n\nUsing either software package you can record events (points) and save them in various GIS formats. You can use your GIS program's distance measuring tool to calculate the distances between points. That's how I measured how far the elderly man on the golf course was from the tee box.\n\nBelieve it or not, GPS units aren't just for recording a position. One day I covered a new toll highway that was opening to traffic. Many, including my editors, wondered what the road would be like, how fast you could go from point A to point B, and so on. I went out with my GPS unit and drove my county's leg from one county line to the other several times. The first time I did it mostly to make a GIS map of the road that I could include in other maps of the county. But my GPS - a Garmin GPS III+ - also records speed: how fast was I going when the plot was taken. A few more times up and down the road, with traffic, stopping for tolls, and I had a pretty good average speed for the roadway.\n\nThe analysis gave readers one sentence that no one else had: Average speed on the new road was 58 in a 65 mph zone. Not bad when you consider a driver has to stop twice unless his vehicle has an E-ZPass electronic tag that allows automated toll collection.\n\nSome journalists like to call those statements computer-assisted paragraphs, and GPS is great for them if you can get used to operating your unit quickly. For instance, I covered an experimental-plane crash deep in a nature preserve. We had to get an escort from the sheriff's office to take us nearly two miles into the preserve where the plane went down. The experimental plane apparently lost power and tried to land on a nature trail. The cut in the trees wasn't wide enough for the wings, and you can figure out the rest.\n\nWith my GPS, I was able to plot my location near the downed plane and the takeoff location (the distance was less than a half mile). I told our graphic artist about the location and they created a graphic that ran in the paper. I guess you call that computer-assisted graphics.\n\nThe applications for GPS are as limitless as the uses of GIS. With GPS units starting under $100, this is an inexpensive tool for the newsroom, one that we as a profession are missing out on.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-buying-a-gps","slug":"2003-julyaugust-buying-a-gps","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-buying-a-gps.html","year":2003,"month":"July/August","monthNum":7,"headline":"Buying a GPS","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A guide to purchasing GPS units for newsroom use, covering price ranges, key considerations for different use cases, and essential features like ruggedness and map displays versus basic coordinate-only models.","keywords":["GPS","equipment","newsroom"],"topics":["Technology"],"technologies":["GPS","ArcPad","GIS","Windows Pocket PC","Excel","Adobe Acrobat","UltraEdit"],"full_text":"Buying a GPS\n\nIn a few short years, the GPS market has taken off. Buyers will find plenty of choices and a range of prices to fit almost any newsroom budget.\n\nIf you have a few thousand dollars to burn, Trimble and ESRI offer a promotional deal on a Trimble GPS, a Windows Pocket PC and ArcPad, ESRI's GIS for Pocket PC. At the other end of the spectrum, you can find a Magellan GPS 310 for $99 on Amazon.com.\n\nWhat you should buy depends on how you plan to use the unit. Are you, the GIS user, going to be the only one using it? Or are you going to circulate it in the newsroom? Do you need a map on screen, or can you live with just seeing coordinate points?\n\nThe basics: If you don't already have one, you'll need a GIS. You also will need a GPS unit with a PC connection of some kind. Most, but not all, have them. You should probably get one that's a little more rugged, since journalists seem to have a habit of dropping electronics.\n\nThe extras: If you want to save a considerable amount of money and are comfortable dealing with latitude and longitude coordinates, buy a GPS unit without a built-in map and screen. Low-end GPS units, which take position points just as well as more expensive models, don't include a scrolling map on their screen, just a compass. However, if you plan to hand the unit to novices, having a screen with a map on it is a good idea: reporters tend to be more comfortable with the idea of seeing a map, pushing a button, and having a dot appear where they are.\n\nthe end of the rankings and move it to Column B where it belongs. To do that, you could do a LEFT function for Column A (example: =LEFT(A82,2) for two-digit rankings, =LEFT(A104,3) for three digits) and a CONCATENATE for Column B. (Insert a new column and type -CONCATENATE(RIGHT(A82,1)&B82).)\n\nSo, is there any way to import data into Excel and keep the exact formatting of columns? The good news - if you're willing to break the process into steps - is that yes, it's possible with Acrobat Reader 6.0. It has a new feature that lets you select a column of text. Just click the \"Select Text\" tool and move the cursor toward the column until it looks like a vertical bar with a box around it.\n\nThen highlight the column (See figure 8), copy it, and paste into Excel. With steady mouse skills, you can even strip the column of rankings and keep them separate. Be creative - sometimes it's easier to copy a column starting somewhere besides the upper-left hand corner.\n\nIt works almost perfectly. The one big goof was that the column for newspaper ownership was blank in a couple of cases, but the data pasted into Excel without the spaces - so it stopped two rows short. That means you have to go back and reinsert the blank cells where they belong. You can also import the column headers with the data, but that requires tinkering, too. For instance, \"Rank in top 200\" filled up four vertical cells in Column A, just as it looks in the original.\n\nThe other big drawback is that the Column Select tool highlights only the data that appears on your screen. So for this table, which runs six pages, you'd have to select each column and paste for each page. This is okay for a small table - but if you've got hundreds of pages, you might want to stick with one of the free downloads.\n\nThe Challenge included a third document that several lists of the top-scoring schools in Missouri, motop 10.pdf. The same methods work with that document that worked with the ASNE table. Willis, for one, copied and pasted the data into UltraEdit and replaced the ending parentheses in the school column with tabs, coded in the program as \")\"t\". He then replaced every number with spaces with tabs, coded in UltraEdit as ^1^t Willis imported that file into Excel, added the header row and saved it. (Note: UltraEdit costs $35.)\n\nIn summary, here are some tips to keep in mind:\n\nMake sure you've got the most recent version of Acrobat Reader. Check back for updates. Here's a link for the latest version: www.adobe.com/products/acrobat/readermain.html\nYou can convert documents for free or close to nothing (less than $40), especially if you're willing to do a little cleaning.\nYes, cleaning. No matter how you import something, chances are it won't be perfect. Spot-check your imported version with the original. Also be prepared to do a little work to get the column headers in the right place.\nExperiment. You can import the same file using more than one method.\n\nWith some practice and patience, you might find that importing PDFs into Excel is almost as simple as A-B-C.\n\nContact Holly Hacker by e-mail at holly@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-julyaugust-data-for-covering-workers-compensation","slug":"2003-julyaugust-data-for-covering-workers-compensation","url":"https://dwillis.github.io/uplink/article/2003-julyaugust-data-for-covering-workers-compensation.html","year":2003,"month":"July/August","monthNum":7,"headline":"Data for covering workers, compensation","kicker":"SPOTLIGHT: JOBS, PAY & PERKS","author_name":"David Herzog","author_title":"","affiliation":"NICAR","summary":"Journalists can access three key databases—payrolls, pensions, and check registers—to investigate public employee compensation, benefits, and spending patterns.","keywords":["payrolls","pensions","databases"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"Journalists can use several key databases to cover public employees, their salaries and benefits. Here are a few of the most valuable and what you can expect to find inside them:\n\nPayrolls\n\nPayrolls list the names, departments, jobs and pay of government workers. The workers' hiring dates, pay rates, work status (part- or full-time) also should be included. Check the pay field carefully because agencies sometimes mix the hourly pay rates of wage earners in the same field with the monthly or weekly salaries of professional workers. Also, make sure you request the most detailed pay records you can get so you can look at overtime pay.\n\nDepending on your state's open records law, you might be able to get even more detailed information about workers, including home addresses and dates of birth. The more detailed information you can get, the better.\n\nRequest this data from agency human resources or personnel offices. Sometimes finance offices maintain the data. Update this data every year so you can build an archive that shows salaries over the years.\n\nPensions\n\nThese databases list the names of pensioners, last date of employment and amounts of annual pensions. Many of these databases include the addresses of the pensioners and the types of pensions (disability or retirement).\n\nDepending on the complexity of the retirement system, there may be several types of pensions with distinct rules and cost-of-living adjustments.\n\nIf there is more than one pension system, make sure your data includes a field that indicates the type of pension.\n\nJournalists have used pension data to show how some pensioners get more money off the job than when they worked. They've also used it to project how much pensioners will collect in the future.\n\nYou can request this data from agency pension boards. If they do not keep the data, try the human resources, personnel or finance offices.\n\nCheck register\n\nThe check register can be a great place to find information about public agency worker travel. A check register is a database that contains a record of every check issued by a government agency.\n\nRegisters often include the check number, name of person or entity that the check was issued to, date, amount, department and purpose of payment (often a code). Sometimes the data includes an employee identification number.\n\nJournalists can use this data to scour for employee travel, which may or may not be legitimate, and reimbursed business expenses. In addition, journalists can use the data to look for payments to consultants and contract workers who are not paid through the usual payroll channels.\n\nTry requesting this data from the treasurer, comptroller or another government fiscal officer.\n\nContact David Herzog by e-mail at dherzog@nicar.org.\n\nuniversity's medical schools, a finding that was true back in 1990.\n\nThe breakdown was almost too good and too easy to be true. So I did more reporting, calling officials at the medical schools and experts on medical school salaries. I found that the database did not tell the complete story. At face value the highest-paid state employee, a renowned surgeon, made about $600,000 a year from the state. But, as my reporting later revealed, most of the medical school deans and professors also practice medicine, thereby earning most of their so-called state salaries from private sources.\n\nAfter culling through names of individuals who fit this category, I found that the highest-paid employee, whose $359,570 salary was completely state-funded, was the chancellor of the University System of Maryland. That revelation changed the focus of the story, especially since the salary and deferred compensation package of the incoming chancellor had recently raised questions at a time when the state was trying to close a $1 billion deficit.\n\nInstead of analyzing the database in the aggregate, I decided to focus my story on the chancellor's salary and look at why medical school personnel received such high six-figure salaries, outranking the governor, judges and heads of state agencies.\n\nI also decided to write a separate story on the two coaches whose state salaries did not accurately reflect their overall compensation package. The database showed that the two coaches, Friedgen and Williams, made $183,820 and $207,584, respectively, when in fact, they were making nearly triple the pay of the chancellor. Because I was still waiting for my request to be answered at the time of deadline, I relied on published reports of the coaches' salaries to write my story. The story also touched upon the controversial issue of paying college coaches million-dollar salaries.\n\nThe stories I found went further than the original intent of my reporting: to update the 1990 analysis of the 100 highest-paid state employees. If I had taken the database at face value, I would have incorrectly written that medical school personnel received the top salaries, a trend unchanged from 1990. Instead, I found that the university chancellor was the highest-paid employee whose salary was paid completely from state funds.\n\nThe salary database did not require complicated queries or have many quirks. But the project was worthwhile because I learned a valuable lesson: No database is a substitute for good, thorough reporting. I also learned it's crucial to know the state's public information law so you can be armed when going into battle with lawyers or agency administrators who don't want to give up information easily.\n\nAs a final note, a few weeks after the story ran, the University of Maryland athletic department responded to my request and provided more details of the coaches' salaries, such as the amount of bonuses tied to performance. It pays to be persistent.\n\nContact Hanah Cho by e-mail at hanah.cho@latimes.com.\n\nSince Charlotte County stores delinquent taxpayer information on a mainframe system, I asked for the data in ASCII. They provided the entire database for free. I imported the text into Microsoft Access 97.\n\nNot long after beginning a search through all the records, I discovered my first roadblock. The tax collector kept detailed information on each delinquent, but surprisingly, no one had tallied up how much people owe in back taxes over the years. In fact, I was told no one had ever done a countywide calculation showing how much each delinquent owed in penalties over multiple years. Also, because interest kept adding up, the dollar figure changed every day. But the tax collector's database showed only how much delinquents originally failed to pay. To get an accurate picture of the top tax delinquents, I needed to calculate the back tax information for more than 40,000 records.\n\nPrep work\n\nAfter talking with the information technology office for the tax collector, began writing a program in Access to calculate back taxes. To accomplish our task, we had to first calculate how many months had passed since each delinquency began. The more months without a payment, the more in back taxes owed.\n\nI created two new fields in Access and ran the calculations to show the number of delinquent months and the monthly interest rate for each delinquent. We used this information to calculate how much each delinquent owed in back taxes, but I still had to add a $4.25 penalty to each record to cover the cost of advertising the delinquency in a local newspaper. After completing the work, I finally thought we had all the information we needed.\n\nIt didn't take long for us to discover I needed to merge the file with another county database to get an accurate picture of local delinquents. The database of tax certificates showed only the name of the person who owned the property the year its taxes became delinquent. We needed a second database to see the name of the current owner. Florida law states the current owner of property is responsible for all back taxes.\n\nI called the property appraiser's office and arranged to receive the electronic list of properties in Charlotte County. I merged the tax collector's information with the property appraiser's list by joining the files using the \"strap number\" field that serves as a property identifier. Finally, with information about current property owners in hand, we were able to generate the true list of top tax delinquents.\n\nBeyond the data\n\nOur queries found that one company had piled up more than $950,000 in back taxes on more than 500 properties. With a quick visit to the county clerk's office, we discovered the county's top tax delinquent was going to have a very difficult time paying its bill. Court records showed the company had gone bankrupt and hadn't paid its taxes for seven years in a row. Despite this fact, the county continued to spend tax dollars sending out delinquent notices and advertising its delinquency in the newspaper year after year.\n\nThis struck us as odd, since Florida law allows county governments to auction property if the owner remains delinquent for more than two years. For some reason the county was bypassing the option to auction the properties off, a move that could recoup money and return the real estate to the tax rolls.\n\nBy law, the tax collector must spend tax dollars advertising delinquencies three times a year for each piece of property that qualifies. We began more detailed queries, searching for all the delinquents that hadn't paid for years but were still draining the county's coffers. We calculated the tax collector had spent $251,712 since the mid-1990s trying to collect from deadbeats it could have cracked down on long ago.\n\nWe eventually uncovered a total of $8.3 million in uncollected taxes across the county since the mid-1990s. It's money that wasn't getting to essential services such as schools, fire departments and road construction crews. More than $1 million of that would have gone to the Charlotte County Fire Department. Fire Chief Dennis DiDio told us it could have added new fire trucks, firefighters and equipment to fight bio-terrorism.\n\nPutting it together\n\nFurther research revealed Florida law requires county governments to auction off all land worth more than $5,000, if the land has been delinquent for more than two years. If it's worth less than $5,000, counties can decide for themselves if they want to go after the property. The $8.3 million in uncollected revenue we discovered was largely made up of properties worth less than $5,000.\n\nWe found the county failed to put together all the pieces on its own. It looked only at how much each delinquent owed on individual pieces of land. No one ever thought to keep track of just how much delinquent land existed under one person's name. We eventually discovered systematic proof that the county wouldn't crack down on a majority of delinquents who owned land assessed at $5,000 or less.\n\nWe took our findings to the tax collector and the real property manager for Charlotte County. The tax collector told us she wanted to immediately respond by auctioning off all the delinquent land we discovered. However, that wasn't her decision to make. The real property manager decides which land to sell at tax auction. When we showed him our findings, he pledged his support for the tax collector's idea. The real property manager told us he would begin the process of auctioning off all the delinquent properties that qualified for the program.\n\nThe county clerk has agreed to postpone some of its normal fees to speed the sale of the large number of properties that will be auctioned off on the courthouse steps. The Charlotte County tax collector and the real property manager credit WBBH for discovering a potential source of millions of dollars in uncollected revenue. We're told Charlotte County will complete its auctions over the course of the next year.\n\nContact Mark Greenblatt by e-mail at mark.greenblatt@nbc-2.com\n\nWhy is steak sauce important to a political campaign? That was the key element of my first computer-assisted reporting project: A question.\n\nFinding New York's campaign finance database online was easy. After that, I had to fall back on my Boot Camp training. Two pieces of advice became immediately useful - even critical - in doing the story:\n\nFirst, find a buddy to review your work. In my case, I tapped Brad Heath. He's a reporter in Detroit now, but he was my predecessor as Binghamton's CAR person and understands my experience as a reporter and my virtually nonexistent database skills. In fact, he knew the data I was working with, too, because he had briefly played with it.\n\nSecond, give the database to the source. The campaign treasurer had done all the paperwork in filing the information, and it was in his interest to make sure I got my facts straight. He found a couple of errors in my queries. And with more than 3,000 potential sources, I still kept total control of what the story said.\n\nI learned a few other lessons.\n\nDo everything two or three times. It wasn't until I exported my query results to Excel and checked the math that I realized the numbers didn't add up, and it was because of a problem in my query. At Boot Camp, they called these data integrity checks, and they are important.\n\nReview the data. The quick and easy task is to find the extremes in your data, and that frequently can be useful. But go through the rest of it, too. You'll come up with more questions. Why, for example, couldn't I find the local Republican leaders contributing to the area's most prominent Republican, but I could find their spouses? You're not going to notice that unless you know both the data and your subject.\n\nI'm primarily a beat reporter so that kind of analysis was easy, even fun, once my co-workers and I made a game of finding how many names we recognized. I don't know how a reporter could do that coming into the data cold.\n\nI also had to learn what the data wasn't telling me, which meant I had to learn what it was supposed to say. Candidates for New York's state legislature generally contribute to a party \"housekeeping\" fund to pay for legal advice, polls and whatnot. Other politicians had contributed to it, according to my analysis, but the state senator hadn't. Either he was the state's biggest political cheapskate (unlikely) or there was another fund out there the data didn't have a record of.\n\nI had to learn to keep focus on my story - how the senator's campaign spent its money. There were plenty of other tempting angles to pursue. Why was the ambassador to Grenada my boy's biggest individual contributor? And was the fact that he was indicted about 15 years ago (and later exonerated) on charges alleging he blew up trailers of a union protesting one of his facilities relevant? Fascinating stuff, but it wasn't part of the story I had in mind. Maybe next time.\n\nThat's the lure of a good database. It gives you a garbage load of ideas. You can't do them all. Even if you could, you probably shouldn't. Overkill can be a nasty habit, and with all the time I sank into this project, it was a temptation.\n\nThe story described how campaigning works and what it means to a community.\n\nAnd that was the toughest lesson of all. Making the database work took a solid 20 hours - as much as I normally devote to a basic, no-frills enterprise package. And because a database generates about three numbers and a nut graf - and maybe some charts or lists or facts for a box - that meant that once I found the good stuff, I still had 15 or 20 hours of work remaining.\n\nCAR stories are inexpensive compared with other investigative stories of similar depth, but they're inefficient compared with traditional methods. That can be an issue at a community newspaper, where cutting somebody free from the daily need to feed the beast can be a burden on everybody else. Fortunately, Heath taught my editors that so I didn't have to.\n\nThrough all of this, Heath double-checked my work and cheered me up when I felt dumb. With his help, I learned to rewrite queries to make them as simple as possible. The easy","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-tracking-quality-of-life","slug":"2003-septemberoctober-tracking-quality-of-life","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-tracking-quality-of-life.html","year":2003,"month":"September/October","monthNum":9,"headline":"Tracking quality of life","kicker":"CHILDREN","author_name":"Dave Davis","author_title":"Reporter","affiliation":"The Cleveland Plain Dealer","summary":"An in-depth examination of the state of children in Northeast Ohio using data-driven storytelling to create a report-card approach.","keywords":["children","Ohio","quality of life"],"topics":["Data & Databases","Investigative Methods","Health & Environment"],"technologies":["ArcView"],"full_text":"When Editor Doug Clifton called us into the conference room a year ago and said he wanted us to undertake an in-depth examination of the state of children in Northeast Ohio, we politely rolled our eyes. It was clearly one of those dreaded editor-assigned projects, an important subject but one that had been more than a bit worn out in the press. Clifton came to the table with more than simply the idea of 'children' but not too much more. He wanted the stories to be data-driven, for us to take a report-card approach, compiling indicators that would somehow shine light on children in Cleveland and allow us to compare the condition of their lives to that of kids elsewhere. We spent the next few months searching for data. We wanted to touch on all the important areas: health, education, crime, housing. We wanted figures that could be compiled on a city-by-city basis. And for big cities, we wanted to be able to examine what was going on in individual neighborhoods. Along the way, we found a couple of new wrinkles to this much-written story. (Much of our effort went into reporting on environmental health issues, such as identifying the 1.5 million youngsters statewide who either live next door to a toxic waste site or in poor housing). We also came up with some creative uses of Census 2000 data. The result was a five-day series, 'Children Left Behind,' published in late July that showed how kids in Cleveland are so much worse off than those in other places in Ohio or in comparable big cities across the country. We found that Cleveland students are twice as likely to drop out of high school than those in Philadelphia or Milwaukee. A larger share of our kids lives in poverty, are lead poisoned and become teenage parents. We also found that nearly half of Cleveland's children are living in risky housing, meaning that they are at risk for injuries, lead poisoning, and potential asthma triggers like cockroaches and rats. 'We've got Third World numbers for kids,' one community leader told us. And our data analysis showed he was right. By geocoding the addresses in five years of Ohio death records obtained from the Ohio Health Department with ArcView 3.3, we identified some city neighborhoods where infants are dying in rates that rival Guatemala, Guyana, and Romania. This wasn't tricky, really. We used the age field in the death records to find babies less than 1 year old. Then we used ArcView's StreetMap geocoding extension to create a point map by matching the home addresses in the death records against a streets map. Later we used ArcView to attach a Census tract number to each death record, so that we could then compile infant deaths by neighborhood. Using Ohio's birth records, which show the mother's age and baby's weight and gestational age, we were able to determine that premature and low-birth weight babies born to teenage mothers cost state taxpayers and hospitals an estimated $161 million from 1997 through 2001, the latest year for which we had records. All told, we analyzed more than a million records gathered from Ohio agencies, other states and the 2000 Census to produce our indicators and rank our cities. To make comparisons between cities fair, we used Census data to group places into eight categories based on population, median household income and the percent of residents living in urban, rural or suburban setting. Then we used a simple rank order approach to rank cities within each category. Here's how we did it: We ranked all of the cities in each category eight different times, once for each indicator. We summed the ranks for each city, divided by eight, and re-ranked the cities on the final number. We gave each indicator equal weight.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-septemberoctober-taking-the-pulse-of-a-body-of-water","slug":"2003-septemberoctober-taking-the-pulse-of-a-body-of-water","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-taking-the-pulse-of-a-body-of-water.html","year":2003,"month":"September/October","monthNum":9,"headline":"Taking the pulse of a body of water","kicker":"SPOTLIGHT: POLLUTION","author_name":"Lisa Stiffler","author_title":"reporter","affiliation":"Seattle Post-Intelligencer","summary":"A look at the health of Puget Sound using various indicators of pollution, exploring sources and impacts on marine life.","keywords":["pollution","Puget Sound","marine life"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Excel","Internet"],"full_text":"Located on the shores of the Puget Sound, our newsroom has a front-row seat to our region's primary water body. We watch the ferries, kids playing on the beaches, fishermen tromping out to the piers. But when colleague Robert McClure and I were charged with evaluating the health of the sound, we had to figure out how to check its vital signs. We looked at different indicators: pollution - its sources past, present, and future; populations of fish and other marine life; and contamination in seafood. Lise Olsen, then the computer-assisted reporting specialist at the newspaper, and I sifted through state and federal databases to identify who was still harming the sound and checked whether regulators adequately protected the public and environment. The results were surprising. We used databases of cleanup sites to identify old polluters and places where contaminants could be leaching from tainted sediment. Some of those locations had been placed under the federal Superfund program, others under state regulation. The U.S. Environmental Protection Agency has a searchable site for some federal cleanups (www.epa.gov/superfund/sites/npl/locate.htm), but that didn't provide the data we needed for our analysis. For that, we went again to the Ecology Department. We downloaded from its Web site an Excel file called the Confirmed and Suspected Contaminated Sites (CSCS) Report. It had loads of information including what the pollutant was; if it was in the land, air or water; the progress of the cleanup and, roughly, how long the contamination had been known about. We found that the pollution was getting better overall and that the main backsliding came from pollution associated with cars, not industry. These results led us in search of the pollution sources - where did this stuff come from, and was it still pouring in? We examined current contamination sources and permits data, trying to learn how many square miles of contamination existed and what was in the water. After figuring out what we could about pollution in the environment and where it was coming from, we wanted to know what that meant for the critters soaking in it, plus the people eating the local seafood. As for support, both Microsoft and Red Hat charge for it.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-septemberoctober-database-reporting-shows-no-bid-deals","slug":"2003-septemberoctober-database-reporting-shows-no-bid-deals","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-database-reporting-shows-no-bid-deals.html","year":2003,"month":"September/October","monthNum":9,"headline":"Database reporting shows no-bid deals","kicker":"GOVERNMENT CONTRACTS","author_name":"Phil Williams","author_title":"Reporter","affiliation":"WTVF-Nashville","summary":"An investigation into no-bid government contracts in Tennessee, revealing potential corruption and cronyism.","keywords":["government contracts","Tennessee","investigation"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":["Access","dBase"],"full_text":"John Stamps was one heck of a talented guy with all sorts of unique experiences that were needed by Tennessee taxpayers - that is if you believed the bureaucrats who worked for Stamps' friend, former Tennessee Gov. Don Sundquist. The longer I sat at my computer, examining state contracts data, the more incredible the insurance agent's talents - especially for tapping into his political connections to land state contracts - appeared. Ultimately, we discovered Stamps' involvement in more than $200 million in state contracts handed out by the Sundquist administration. Our 'Friends in High Places' investigation led to a federal-state criminal probe, involving the FBI and at least three other agencies. Investigators raided at least two companies with ties to Stamps, and a federal grand jury subpoenaed hundreds of thousands of email messages based on information unearthed by our reporting. No charges have resulted so far, but the grand jury is continuing its probe. Unlike a lot of investigations, this one didn't begin with a tip. Instead, it began from mere curiosity during a brainstorming session. With Tennessee having just enacted the largest tax increase in state history, my photojournalist partner Bryan Staples and I wondered who was getting taxpayers' money. It was a question that couldn't be answered easily without a computer. Using Tennessee's Public Records Law, I requested a database of state contracts. The state's Department of Finance and Administration quickly e-mailed the data in a .dbf file, which was then imported to Access. It contained hundreds of records, which required a targeted approach. Among the fields of information was a description of the bidding process, including one that I immediately found of great interest: 'noncompetitive' contracts. Those are contracts that are awarded without any sort of bidding process because the vendors are deemed to offer a unique service unavailable anywhere else. Slicing out the no-bid contracts from the data, I then began to background the companies involved in some of the larger contracts. To accomplish that, I began with a basic Web search, then moved on to AutotrackXP's online public record service. One contract, with a company called Workforce Strategists, set off immediate alarms. The company had been awarded a $1.9 million no-bid contract to provide intensive counseling and coaching to help the unemployed get back to work. 'Obviously, with six days, their experience is rather limited,' Rep. Buck said. The state had awarded millions to companies linked to political donations, but there were few records of actual work completed. Our investigation prompted legislators to pass a new law that requires more scrutiny of no-bid contracts. Tennessee's new governor, Phil Bredesen, invited us to privately witness his signing of the legislation. And he has imposed his own tighter controls. 'It's primarily an issue,' the new governor said, 'of having the public have some confidence that what we are doing is in their interests.'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-septemberoctober-fresh-migration-aircraft-data","slug":"2003-septemberoctober-fresh-migration-aircraft-data","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-fresh-migration-aircraft-data.html","year":2003,"month":"September/October","monthNum":9,"headline":"Fresh migration, aircraft data","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"The San Diego Union-Tribune used IRS migration data to report on residents leaving the region for cheaper housing in the West, illustrating how journalists can track population movement and income patterns across counties. The article highlights IRS migration databases and other public records available through IRE and NICAR that enable reporting on demographic and economic trends.","keywords":["migration","housing","data"],"topics":["Data & Databases","Demographics & Census","Government & Politics"],"technologies":["FAA Aircraft Registry"],"full_text":"\"San Diego County may have a stunning coastline, incomparable weather and a world-famous zoo, but all that has not been enough to stem the flow of residents leaving the region for fast-growing areas in the West where housing is far cheaper.\"\n\nSo began a recent story by the San Diego Union-Tribune that used Internal Revenue Service migration data, which is offered by the IRE and NICAR Database Library. The U.S. Census Bureau compiles the data, based on personal income tax return information, for the IRS.\n\nThe database allows journalists to track the movement of people and their incomes, in and out of counties, from 1992 through 2002. USA Today used the migration data in an October 2002 report about how migration of affluent Californians has led to the dominance of Republican politicians in the mountain west states.\n\nOther recent updates include the U.S. Federal Aviation Administration Aircraft Registry, which lists all aircraft and aircraft owners registered in the United States (current through February 2003). Find out more at www.ire.org/datalibrary/databases or call 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-breaking-news-help","slug":"2003-septemberoctober-breaking-news-help","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-breaking-news-help.html","year":2003,"month":"September/October","monthNum":9,"headline":"Breaking news help","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"IRE's 'In the News' resource helps journalists quickly find databases and information sources when major disasters and news events occur, as demonstrated during the recent power blackouts across North America.","keywords":["databases","blackouts","journalism"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"Power blackouts. Nightclub fires. Wars. Plane crashes. When disasters and other big news occur, journalists can find help in a hurry at IRE's \"In the News,\" online at www.ire.org/inthenews archive.\n\nWhen power blackouts hit New York and other major cities in the U.S. and Canada, a special \"In the News\" section listed useful databases, publications, Internet sites and other resources. For instance, the site linked to several database and spreadsheet files from the Energy Department's Energy Information Administration. Also, the IRE and NICAR Database Library has data on federal contracts that includes the company, contact information, agency, type of work performed and place where the work was performed. Certainly there were companies with federal contracts involved in the blackout.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-new-beat-book","slug":"2003-septemberoctober-new-beat-book","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-new-beat-book.html","year":2003,"month":"September/October","monthNum":9,"headline":"New Beat Book","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE (Investigative Reporters and Editors)","summary":"IRE publishes \"Unstacking the Deck: A Reporter's Guide To Campaign Finance,\" a resource written by veteran journalists that provides guidance on investigating campaign finance through documents, data, and databases from sources like the FEC.","keywords":["campaign","finance","reporting"],"topics":["Government & Politics","Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"From lobbying to loopholes, soft money to hard money, reporters will find useful information and inspiration in \"Unstacking the Deck: A Reporter's Guide To Campaign Finance.\" Written by veteran journalists Michael A. Weber, Aron Pilhofer and Derek Willis, this is the latest publication in IRE's Beat Book series. It includes chapters on gathering documents and data and using campaign finance databases, such as those from the Federal Election Commission. There's also information on finding local and state data, and putting together your own database. The book costs $15 for IRE members and $25 for nonmembers. For details, see www.ire.org/store/books/campaign.html or call 573-882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-contribute-your-files-to-database-library","slug":"2003-septemberoctober-contribute-your-files-to-database-library","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-contribute-your-files-to-database-library.html","year":2003,"month":"September/October","monthNum":9,"headline":"Contribute your files to database library","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"The IRE and NICAR Database Library is seeking journalists to donate raw government databases they've used for stories, reviving the original co-op model to expand their collection of data resources for news organizations.","keywords":["databases","journalists","donation"],"topics":["Data & Databases","Journalism & Newsroom"],"technologies":[],"full_text":"Contribute your files to database library\n\nBy David Herzog, NICAR and Missouri School of Journalism\n\nJournalists hungry for high-quality data have turned to the IRE and NICAR Database Library for more than a decade.\n\nFor those of you who are unfamiliar with it, the Database Library acquires data from government agencies and provides it in useful formats for journalists and journalism organizations. The library staff, directed by Jeff Porter, processes the data and often fixes problems in the data so customers can focus on analyzing it for news stories. The staff also creates documentation that helps journalists understand the data better.\n\nThe number of Database Library offerings has multiplied many times since the early days. Journalists can obtain data on a wide range of topics - transportation, public safety and business, to name a few - for low fees on a sliding scale. Smaller news organizations pay less than the bigger outfits.\n\nWhen the Database Library started more than a decade ago, its initial offerings were databases that journalists had used for news stories and then donated. It was like a journalists' co-op for data.\n\nWe want to revive that practice and ask you to donate your databases - the raw files you obtained from government agencies - to IRE and NICAR after your stories run. We are mainly interested in national databases. Send us your data, along with any documentation and notes about quirks that you discovered. We'll take a look at the data to see if it's something we would like to add to our collection.\n\nJeff is leading this push to add more data. You can contact him at jeff@ire.org if you have data that you'd like to share.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-hands-on-car-training-upcoming","slug":"2003-septemberoctober-hands-on-car-training-upcoming","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-hands-on-car-training-upcoming.html","year":2003,"month":"September/October","monthNum":9,"headline":"Hands-on CAR training upcoming","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering computer-assisted reporting training programs including CAR Boot Camps, mapping workshops, and statistics training at various dates throughout the year.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet","GIS"],"full_text":"IRE and NICAR have a number of training opportunities coming up for journalists who want to learn more about computer-assisted reporting.\n\nJournalists can get intensive hands-on training in using spreadsheets and database managers to produce high-impact news stories at the CAR Boot Camps, which will be held four times next year in Columbia, Mo. The sessions are Jan. 4-9, March 21-26, May 16-21 and Aug. 1-6.\n\nAn Advanced Boot Camp on Mapping, scheduled for Jan. 9-11, will offer intensive hands-on training using mapping software. An Advanced Statistics Workshop, taught by Steve Doig of Arizona State University and Sarah Cohen of The Washington Post, will be Feb. 13-15 for reporters who want to move beyond basic CAR and use statistical analysis.\n\nFor more information, see the IRE training calendar at www.ire.org/training/otr.html.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-state-cars-sit-waste-tax-money","slug":"2003-septemberoctober-state-cars-sit-waste-tax-money","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-state-cars-sit-waste-tax-money.html","year":2003,"month":"September/October","monthNum":9,"headline":"State cars sit, waste tax money","kicker":"FIRST VENTURE","author_name":"Morgan Loew","author_title":"Journalist","affiliation":"KPHO-Phoenix","summary":"An examination of underused state vehicles in Arizona, highlighting wasteful spending amidst a budget shortfall.","keywords":["Arizona","state vehicles","budget"],"topics":["Investigative Methods","Government & Politics","Business & Economy"],"technologies":[],"full_text":"Heading into the 2003 legislative session, the state of Arizona was facing a $1 billion budget shortfall. As in nearly every other state in the country, revenues had fallen, programs were getting slashed and lawmakers were desperately looking for ways to cut expenses. Journalists were also under the gun to find wasteful spending and report it in print or on the air as fast as possible. We found it in an unusual place: state vehicles that were used so little, the state's own guidelines said they weren't worth the cost of owning them. Compared to hardcore computer-assisted reporting projects that can take months of spreadsheet and database analysis, this story was pretty simple to do using the skills I learned in a NICAR Boot Camp in January. We found a recent audit of one state agency that referred to an 'efficient use guideline' for state vehicles. It concluded state vehicles needed to be driven at least 10,000 miles per year to justify the cost of acquiring and maintaining the vehicles. With that minimum mileage standard in mind, we sent a public records request to the Arizona Department of Administration asking for the entire state vehicle database. The department buys most state vehicles and leases them to individual agencies. We specifically asked for the following: license plate numbers; agency the vehicles are leased to; year, make and model of the vehicle; mileage at the beginning of the last fiscal year; and mileage at the end of the last fiscal year. Querying the data, the result was that nearly half the vehicles were underused, according to the state's own guidelines. We listed the agencies with the most underused vehicles by using queries. So we had a story, but we needed a good way to tell it. My photographer and I performed surveillance next to one of the lots, and watched as several state vehicles just sat there day after day. Next, I contacted the state Department of Education and got a list of relatively inexpensive state programs that were facing the budget axe. We ended up profiling a program for underprivileged preschoolers and another that helped adults get their high school diplomas. The story boiled down to this: 'State leaders say there is no money lying around to fund programs like preschool and adult education, but we found lots of extra tax dollars - literally 'parked' all over the state Capitol.'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-septemberoctober-pinpointing-polluters-in-chicago-area","slug":"2003-septemberoctober-pinpointing-polluters-in-chicago-area","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-pinpointing-polluters-in-chicago-area.html","year":2003,"month":"September/October","monthNum":9,"headline":"Pinpointing polluters in Chicago area","kicker":"MAPPING","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists can use environmental data from EPA and state agencies combined with GIS mapping tools to create visualizations of pollution hotspots and contaminated sites for investigative reporting.","keywords":["mapping","environmental","data"],"topics":["Health & Environment","Investigative Methods","Technology"],"technologies":["ArcView","Spatial Analyst","dBase","FoxPro","Internet","World Wide Web"],"full_text":"Environmental reporting is a natural - pun intended - for journalists looking to use their analytic mapping skills and find information for news stories.\n\nEnvironmental data that can be mapped from state and federal agencies is often a simple download away on the Internet. For example, go to the U.S. Environmental Protection Agency Web site at www.epa.gov and search for the phrase \"underground storage tank.\" You'll get a list of state agencies that manage data about leaking tanks and, often, links to that data.\n\nThe environmental agencies often include the locations, reported in latitude and longitude, of regulated entities. For example, the Toxics Release Inventory data (www.epa.gov/tri) from the EPA includes the latitude and longitude. A journalist using a geographic information system can create a pin map based on those coordinates and, going further, use density mapping tools to show high concentrations.\n\nThe Toxics Release Inventory and other environmental datasets include even more geographic information that can be mapped: counties, cities and ZIP codes. While ZIP code areas may or may not have precise boundaries, most people know their own Zip codes.\n\nThe IRE and NICAR Database Library recently used those geographic elements when it assisted WMAQ-Chicago in pinpointing contaminated sites targeted for cleanup by the Illinois Environmental Protection Agency. The data included hazardous waste sites and leaking underground storage tanks. The Database Library used ArcView 3.3 and the Spatial Analyst extension to conduct the analysis.\n\nThe data, obtained by station producer Michele Youngerman, came from the Illinois EPA in dBASE files, easily opened in ArcView after we cleaned up the data using Visual Fox Pro to convert nine-digit ZIP codes to five-digit codes and convert the coordinates from text to number.\n\n(A note of caution about latitudes and longitudes: Make sure you have them reported properly. ArcView 3.x requires decimal degrees. Occasionally the coordinates are reported in degrees, minutes and seconds and will have to be converted.)\n\nAfter we cleaned up the data, our analysis took several steps.\n\n1. We created a point map using the numeric latitude and longitude coordinates via the View\\Add Event Theme menu command.\n2. The first map was too laden with dots; we wanted to restrict our results. So we used the definition query tool to include only larger polluters.\n3. With some advice from GIS expert Matthew Waite of the St. Petersburg Times, we used the Spatial Analyst extension to identify \"hot spots\" of regulated sites. The resulting density maps weren't pretty, but they helped us focus on specific areas and confirmed some suspicions for the station.\n4. We limited the maps to the specific ZIP codes provided by WMAQ that define its viewing area by using definition queries. Then we used ArcView's Geoprocessing wizard extension to clip base map layers - roads, cities and rivers - for just the viewing area.\n5. Using the clipped data, we summarized the number of regulated sites by ZIP code and mapped the results.\n\nThe resulting maps highlighted areas where large polluters - generating, hauling or managing hazardous materials - are also close to residential areas. The television station reported about thousands of contaminated sites near homes and playgrounds, although homeowners often don't know about the contamination. The analysis also helped give perspective for the story of three families facing serious illness. All three families believe the illnesses were caused by companies releasing toxic waste.\n\nThe stories can be found at: www.nbc5.com/nbc5/22037681detail.html\n\nContact Jeff Porter by e-mail at jeff@ire.org.\n\nWould you be willing to share a mapping example with fellow journalists? Send an electronic copy of the map along with details to David Herzog at dherzog@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-mapping-soil-tests-shows-legacy-of-lead","slug":"2003-septemberoctober-mapping-soil-tests-shows-legacy-of-lead","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-mapping-soil-tests-shows-legacy-of-lead.html","year":2003,"month":"September/October","monthNum":9,"headline":"Mapping soil tests shows legacy of lead","kicker":"SPOTLIGHT: POLLUTION","author_name":"Tina Lam and Wendy Wendland-Bowyer","author_title":"","affiliation":"Detroit Free Press","summary":"Detroit-area journalists conducted a seven-month investigation using soil sampling, GIS mapping, and interviews to reveal that lead contamination in soil from old smelters and leaded gasoline remains a significant public health threat, particularly in inner-city neighborhoods. The team employed over 400 soil samples and advanced mapping technology to document lead exposure risks near schools and residential areas.","keywords":["lead","soil","Detroit"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["ArcView","GIS","EPA's TRI Explorer"],"full_text":"Lead poisoning is still a big problem in Detroit and other Michigan cities. Because many of our readers think it is old news, we wanted to bring a fresh perspective.\n\n\"We didn't want to write an anecdotal story,\" said Alison Young, then project editor and now an investigative reporter for Knight Ridder's Washington bureau. \"We wanted to break news and give readers new information that would get their attention.\" Our early research showed little had been done to alert the public to the dangers of lead-contaminated soil and the legacy of long-forgotten lead smelters.\n\nEight reporters, a data analyst and three photographers spent seven months working on a series published in January. We interviewed 300 people, examined more than 10,000 pages of documents, filed a dozen public records requests and performed computer-assisted reporting. Since January, we've written follow-ups and three other major stories, including an analysis of a state database of children's lead test results.\n\nWe hired a soil expert who has studied lead left in soil from leaded gasoline Howard Mielke, professor of environmental toxicology at Xavier University. His team took more than 400 soil samples, spread randomly along lines drawn through the Detroit metro area, from downtown radiating out into the suburbs. They used global positional satellite gear to determine where to dig each sample and then tested them in their lab in New Orleans for lead and other heavy metals.\n\nWhen we got the soil test data, it arrived with coordinates of where each sample was taken. Victoria Turk, our data analyst, loaded the data into the ArcView 3.2 geographic information system (GIS) program to plot the locations of samples using the latitude and longitude coordinates. She then placed the sample point file over a streets layer and on printed maps. Turk also added a map layer containing the locations of schools hosting children from kindergarten through fifth grade to see how close they were to the contamination locations. Turk obtained the school data from the National Center for Education Statistics Web site (http://nces.ed.gov) and geocoded it in ArcView to create a point map.\n\nTurk had to convert the schools data to match the projection of the soil sample layer.\n\nUsing the sample maps, reporters and photographers fanned out, interviewing people who lived near the spots we'd sampled. The research showed that lead was highest in the soil in the inner city, with levels dropping near and in the suburbs. Our story talked about similar findings in other cities and the fact that some scientists think contaminated soil is as great a contributor to lead poisoning among children as lead paint and dust.\n\nOne of the five days of the series was devoted to a defunct lead smelter in Detroit and the Environmental Protection Agency's failure to notify people nearby that lead might have blown into their homes and yards. We asked the Xavier University team to return to Detroit and sample areas around the smelter. They found elevated levels. The EPA had taken just seven samples before determining there was no neighborhood contamination. We did 100 and found that there was contamination, some of the levels astronomically high. Turk again used ArcView to map the results. She also used EPA's TRI Explorer on the Web (www.epa.gov/triexplorer) to find out how much lead and lead compounds had been emitted in the three counties near Detroit.\n\nOur series included graphics showing what we found. It also had plenty of human tales, portraits of families all over the state who had been affected by lead poisoning. One story focused on how the city and state fumbled efforts to abate houses for lead paint and left millions of dollars unspent or used on homes where no children lived.\n\nDuring the project, we filed a FOIA request asking the Michigan Department of Community Health for its database of the results of all blood-lead tests for young children during the previous five years. The state requires that all clinics or physicians who do blood-lead tests report them to the state. After denying the database existed, the state eventually agreed to give it to us, but only on paper and only for $2,445. It was already too late for the series. But we sued the Department of Community Health in January in Oakland County Circuit Court and won our case and $10,000 in attorney fees in June.\n\nThe state provided nearly half a million records with names redacted. The main fields were a specimen ID (unique for each test, not each child), test date, result and the child's birth date, address to the block level, race and sex. We wanted to find the most-poisoned blocks in the state and see whether health officials were targeting them.\n\nMegan Christensen, a former NICAR data analyst interning at the Free Press, joined us on the story. We used VEDIT to remove formatting glitches in the original file and then ran the addresses through ParseRat to separate a single field into components, such as block, street, direction and P.O. Box. We wanted a clean block and street number for each record. (For more about ParseRat, see p. 16 in this issue of Uplink).\n\nThe state had rounded the house numbers to the 100s. For example, 1500 Elm represented all houses from 1450 Elm to 1549 Elm, so that's how we defined our blocks. But we also had to separate Southwest Elm from Northeast Elm. We had to visit some of the streets and cross-reference others in street directories to make sure we had it right.\n\nOnce we separated the records by block, we had to throw out the duplicates. Some records were redundant because the same child was tested in multiple years. Because we had no names, we settled for a grouping of birth date, race, sex and address to identify each child. Unfortunately, we did not come up with a reliable programming solution to throw out duplicates. Instead, we used Microsoft Access to query the data and group results by block. Then we printed out block results and picked out the duplicates by hand based on matches of birth date, race and sex. Then we ran new block sums to determine how many separate children were poisoned on each block.\n\nThis forced us to focus on only the top 10-15 blocks because it would have been too time consuming for every block statewide. Had we been able to eliminate all the duplicates, the story would have produced more concrete numbers. But we were still able to focus on the blocks with the most poisonings and report on those families.\n\nThe analyst for the Michigan Department of Community Health admitted they'd never done this. He thought the data was too dirty to eliminate double counting, so had never attempted it. Our story showed that most health officials didn't know which were the worst blocks and were not targeting those areas. The state had even denied funding to one county with some of the worst blocks.\n\nOur lead stories have triggered results. Embarrassed health officials have already started knocking on doors in the poisoned blocks that we found. Days after the January series ended, the EPA announced it would finish the cleanup of the abandoned lead smelter and sample the yards near it. The agency set up a satellite office in the neighborhood, got permission to test widely and discovered that 75 percent of the homes they sampled within one-quarter mile of the smelter had lead-contaminated yards. They're working on cleaning them up.\n\nAfter another story we did on 16 other old smelters in Detroit that hadn't been investigated, the state sent environmental officials to examine every site. They've discovered that all but three were indeed lead smelters and could still be causing contamination. Soil sampling is in the works.\n\nOur lead stories also led directly to legislation requiring inspectors to check for lead in day care centers. In August the governor introduced a 33-page plan to mount a new attack on lead poisoning, citing our work in the second paragraph. It should generate new legislation on several fronts, including requiring more lead testing and penalizing landlords who don't repair lead-contaminated apartments. We'll be watching.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-showing-sources-effects-of-emmissions","slug":"2003-septemberoctober-showing-sources-effects-of-emmissions","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-showing-sources-effects-of-emmissions.html","year":2003,"month":"September/October","monthNum":9,"headline":"Showing sources, effects of emmissions","kicker":"SPOTLIGHT: POLLUTION","author_name":"James Bruggers and Mark Schaver","author_title":"Environmental Reporter and Data Analyst","affiliation":"The (Louisville, Ky.) Courier-Journal","summary":"Data analysis revealed that Louisville's air quality poses significant health risks, with toxic chemical concentrations hundreds of times higher than EPA safety standards, identified through the use of databases and geographic mapping tools.","keywords":["pollution","analysis","health"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["Access","Excel","ArcView","Perl"],"full_text":"Data analysis played a major role in two recent projects that showed the severity of dirty air in Louisville and identified the area's biggest polluters.\n\nStories published in May showed how the city's dirty air could cause a significant number of additional cancers and other health problems, and that the concentrations of certain chemicals were hundreds of times higher than levels considered safe by pollution regulators.\n\nIn June the newspaper followed up with a six-page special section that contained graphics pinpointing the sources of pollution and looked at emissions trends.\n\nWe used Microsoft Access, Excel, ArcView 8.3 geographic information system and Perl to wrestle with large quantities of data from local and federal air pollution regulators.\n\nFor the first project, environmental reporter James Bruggers, under the Kentucky open records law, obtained air-sampling results in an Excel spreadsheet from the Louisville Metro Air Pollution Control District. The district enforces the federal Clean Air Act in Louisville.\n\nThe spreadsheet contained, among other things, the average, median and 95-percent upper confidence limit values for 13 monitoring stations spread over the Louisville area, as well as the number of times an individual chemical had been detected over a 12-month period in 2000 and 2001. Roughly 100 of about 160 chemicals that were sampled had been detected at least once during that period.\n\nTo see which of those chemicals might be of concern, we wanted to compare the monitoring results with health-risk benchmarks developed by two regional offices (San Francisco and Philadelphia) of the U.S. Environmental Protection Agency. We downloaded them from the Web. The EPA lacks national benchmarks. These thresholds are sometimes called risk-based concentrations or preliminary remediation goals and were developed for screening potential toxic dumps.\n\nWe decided that we would focus on chemicals whose concentrations exceeded both benchmarks. We used Access to join the air sampling results table with the threshold tables. We joined the tables using the unique Chemical Abstracts Service number that identified each chemical.\n\nThen we calculated a ratio showing which chemicals exceeded the thresholds. We exported the results into Excel and used conditional formatting to highlight in red the sites that exceeded the thresholds.\n\nA week before we published the stories, the Philadelphia regional office posted a revised set of screening levels, which required us to perform our analysis again. Fortunately, there were only a few changes.\n\nOur results were striking: Concentrations of at least 18 toxic chemicals and compounds measured in the air exceeded EPA benchmarks. Some were at levels hundreds of times higher than what the government considers acceptable. Concentrations of one chemical, the human carcinogen 1,3-butadiene, were 2,400 times higher than the EPA health threshold at one sampling site. With ArcView, we produced a map showing where the air samples were taken and used proportional symbols to show the relative quantity of butadiene emissions measured at each.\n\nThe number crunching was fairly straightforward; the real difficulty was making sure, through interviews with a variety of experts, that the newspaper's analysis was legitimate. So we talked to university-based toxicologists, environmental statisticians, former environmental regulators and physicians. We also relied heavily on federal government Web sites for information about the chemicals and compounds that we found in high concentrations.\n\nAfter the stories ran, the newspaper obtained a just-completed environmental consulting firm's draft analysis of the air sampling results that confirmed our findings of a significant air quality problem.\n\nFinding polluters\n\nNext we wanted to analyze data to find pollution sources and examine trends. This is where it got a little complicated: There's more than one database with emissions information, and some of the data overlap. Some databases track emissions for which there are no national air standards, while others track emissions for which there are standards. And, because reporting rules have changed over the years, it can be difficult to identify yearly trends.\n\nWe started by analyzing the granddaddy of all pollution inventories: the EPA's Toxics Release Inventory, which tracks emissions of some 650 different chemicals. This data is available on the Web at www.epa.gov/tri, and can be easily sorted and manipulated using the EPA's Web-based interface; it can also be downloaded and put into Excel for additional analysis. We did both.\n\nWhen we examined countywide trends, we limited our analysis to what the EPA calls its \"1995 core chemicals,\" and \"original industries.\" This gave us most of the chemicals and most of the industries, excluding power plants, which we analyzed separately. By slicing the data this way, we found that overall TRI emissions in Louisville-Jefferson County had declined 39 percent between 1988 and 2001.\n\nWe had a spreadsheet for each year and combined the data into a single worksheet to make it easier to look at year-to-year trends. We used Excel pivot tables to quickly summarize data for each company.\n\nWhen we examined the emissions trends from the top 25 TRI sources, we used the EPA's Envirofacts Warehouse, a very useful online resource, at www.epa.gov/enviro. Its emissions data is more current, and includes yearly releases for each facility on one screen. After a few points and clicks, a color-coded trend chart popped up on our computer screen.\n\nTRI, however, fails to account for all emissions. Other categories of pollutants are reported by industry and compiled into different databases that can be obtained from the EPA, and sometimes, state and local environmental agencies. We obtained ours from the local air pollution control district.\n\nThe agency at first told us that it didn't keep the records in a form that our computers could understand. But they agreed to extract the data we were requested and gave it to us electronically. Some came in an Excel spreadsheet; others in text tables.\n\nTo get the data in a useful form, we used a Perl script to loop through the text files, pulling out the name of each plant and the pollutants associated with it.\n\nWith this data, we used Excel to rank each source of industrial pollution for each category of pollutant, and determined yearly trends. Some of what we found we expected, such as the major emissions from power plants. But we also found that a bourbon distillery was the largest source of volatile organic compounds, a contributor to summer smog. The alcohol vapors that escape from aging casks of Old Forester and Early Times are called \"the angel's share\" by industry insiders and are unregulated.\n\nWe ran our results by the local air pollution officials as well as some of the companies that came out on top. We then realized that one of the databases we had was faulty. The computer program that local air officials had used to extract the data on hazardous air pollutants had left some industries out. The local officials fixed the problem, and we ran our analysis again.\n\nWith stories, maps and charts, we presented a complete picture of industrial air pollution sources in Louisville. While much attention over the years had been focused on an area with 11 chemical plants, our analysis identified major sources far from there. We found that a single power plant in Indiana just across the Ohio River from Louisville produced more TRI emissions than two Louisville based power plants combined. We found that particulates, which can damage lungs and cause cancer, had increased by 76 percent since 1988, and that the hazardous air pollutants were on the rise since 1998.\n\nAt the same time, we showed that industrial emissions of volatile organic compounds had declined by 60 percent since 1988.\n\nWe used graphics and maps to help tell the story and keep most of the numbers there. Readers could identify the plants nearest them, and then see whether air emissions were going up or down at those facilities.\n\nIn ArcView, we made a map showing the location of the 50 counties in the U.S. with the highest air emissions based on the TRI data. We used ArcView's graduated symbols feature to show the relative size of each of those counties' emissions. We wanted to compare our county to others in the nation, and found Louisville-Jefferson County ranked 23rd.\n\nWe told the story through the people who work in these plants and those who live nearby. We also followed up the special section with a package on how University of Louisville medical experts had identified certain \"asthma clusters\" in Louisville, and that many people with asthma in Louisville blame poor air quality on making their medical condition worse.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-squeezing-trends-from-air-emissions-data","slug":"2003-septemberoctober-squeezing-trends-from-air-emissions-data","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-squeezing-trends-from-air-emissions-data.html","year":2003,"month":"September/October","monthNum":9,"headline":"Squeezing trends from air emissions data","kicker":"SPOTLIGHT: POLLUTION","author_name":"Russell Clemings","author_title":"","affiliation":"The Fresno Bee","summary":"Journalists investigating air pollution trends in California's Central Valley discovered that standard statistical methods like linear regression fail with seasonal data, so they adopted a 365-day moving average to reveal long-term patterns that would be understandable to readers.","keywords":["pollution","statistics","trends"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["CD-ROM","dBase","Access","Excel","Internet","World Wide Web"],"full_text":"Even the experts can't agree on the best way to find long-term trends in air pollution data. But we had to figure something out for our \"Last Gasp\" project, which made the point that air pollution was worsening in parts of California's Central Valley even as it improved in Los Angeles.\n\nHere's what we did. Our data came from California's Air Resources Board, which offers a free CD with data, maps and a nifty interactive program for displaying both. The board has a Web site (www.arb.ca.gov/aqd/aqdcd/aqdcd.htm) with downloadable hourly monitoring data back to 1980.\n\nMost of the data is in dBASE files that can be imported into Microsoft Access with a couple of keystrokes. From there, you can construct queries to extract data for a single monitoring station (station ID numbers and locations are in a separate file) and export the resulting data to Excel.\n\nThe files have readings for a dozen pollutants, presented more than 20 different ways. Unfortunately, the raw data is just a jumble of numbers. Even if you scanned the entire file from top to bottom, it would be impossible to figure out the trend.\n\nYou can use the Excel Chart Wizard to create, for example, a line graph for the date and one-hour ozone levels, but even that doesn't help much.\n\nBecause ozone is a summertime pollutant, the levels are high in the summer and approach zero in the winter. If you eyeball the graph, it looks as if the summertime peaks are edging upward, but it's hard to say for sure.\n\nWhat we needed was a way to boil down these peaks and valleys to a long-term trend.\n\nOne of the ways that statisticians find patterns in data is to do a linear regression, and Excel fortunately has a built-in way to add a regression line to the chart we built. After you've finished the chart, just right-click on the data line and click Add Trendline, and you can lay down a nice, clean regression line over the data.\n\nDoing that may show what appears to be a long-term rising trend in the data. There's just one thing wrong: As any competent statistician will tell you (and lucky for us, one did), a simple linear regression doesn't work with data that involves a time series like this one. The seasonal pattern in this data is so strong that it overwhelms any other pattern that a regression analysis might show you.\n\nThis problem is not unique to air pollution data. Economists run into it when they're studying things like retail sales, which rise and fall with the calendar. And there are tools available for dealing with it, such as the Census Bureau's X-ARIMA model. But even if you have the intellectual capacity to use something like that, you're still left with a significant problem - explaining it to your readers. Just saying, \"Trust me, my computer says there's a trend here\" probably isn't the best approach.\n\nHere's another alternative: A 365-day (or annual) moving average. It's not too hard to explain: for each day, it's just the average of the past year's daily values. That's why it's called a \"moving\" average. If today is June 5, 2003, then the 365-day moving average for today is the average of all daily values from June 6, 2002 through today. Tomorrow's 365-day moving average, in turn, would be the average of daily values from June 7, 2002 through June 6, 2003.\n\nHere's how you write a formula to get that from Excel. Suppose you have date labels in column A and data for several years in column B. Calculate the first annual average in cell C366 with this formula: =average(b2:b366\n\nCopy that formula down and graph the result, and you can say with some confidence that the 1990s saw one-hour daily ozone peaks decline edge upward or downward, or remain about the same, depending on the data. The above chart shows contrasting trends (using the 365-day moving average) at two stations - one in southern California that is declining and one in the San Joaquin Valley near Fresno that is rising somewhat.\n\nAt this point in our reporting, we are ready for quality control. When we did this in real life, we showed our graphs to various experts at the state and local air pollution control agencies. We also explained how we did the analysis and asked for their comments on it.\n\nAll agreed that the 365-day moving average was a valid way to show the long-term trend. But one expert went a step further. He encouraged us to throw out the winter values and look only at the summer data. His reasoning: Although no one knows for sure, it's considered likely that very low levels of ozone are not a health threat. The state's one-hour ozone standard of 0.09 parts per million is exceeded regularly in the summer. But in winter, typical levels are far less than that. And an increase from 0.02 to 0.03 is clearly not as important as an increase from 0.12 to 0.13. Yet our 365-day moving average would regard them both as important.\n\nThe expert suggested a number of alternative approaches. We finally chose one that was doubly elegant, being both easy to calculate and easy to explain. For each year, we calculated and plotted just one value - the average daily peak for the summer smog season, May 1 through Oct. 31.\n\nYou can read the \"Last Gasp\" project on the Web at www.valleyairquality.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-making-sense-of-navy-aviation-crash-data","slug":"2003-septemberoctober-making-sense-of-navy-aviation-crash-data","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-making-sense-of-navy-aviation-crash-data.html","year":2003,"month":"September/October","monthNum":9,"headline":"Making sense of Navy aviation crash data","kicker":"Tech tip...","author_name":"Richard O'Reilly","author_title":"","affiliation":"Los Angeles Times","summary":"The Los Angeles Times used the Naval Aviation Safety Database to investigate U.S. Marine Corps Harrier jet crashes, but encountered significant FOIA restrictions that excluded narrative fields and conclusionary information while providing only basic coded data.","keywords":["database","military","FOIA"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["SAS","UltraEdit"],"full_text":"Making sense of Navy aviation crash data\n\nThe Naval Aviation Safety Database was one of the research sources for the Los Angeles Times' Pulitzer Prize-winning series on the U.S. Marine Corps Harrier attack fighter jet. The four-part series, published Dec. 15-2002, also won an IRE Certificate.\n\nReporters Alan C. Miller and Kevin Sack found the bulk of their material through numerous interviews and nearly 90 Judge Advocate General accident reports obtained under Freedom of Information Act requests. But an analysis of the Harrier through the Naval Aviation Safety data provided scope, context and nuance to the project.\n\nBrief summaries from the database are available on the Navy Safety Center Web site, www.safetycenter.navy.mil The full database must be requested via FOIA from Josie Bowden, 757-444-3520, ext. 7045 (JBowden@safety.center.navy.mil).\n\nIt took us months to get the database after filing the request. And when we received it, we found that data from 218 fields, about 27 percent of the 807 fields then in the database, were omitted on privacy and national security grounds. An appeal failed. What we got instead was the brief public summary of recent Harrier accidents supplied by e-mail. It was exactly the same data we could have copied from their Web site, which was useless for analytical purposes.\n\nVirtually all of the extensive narrative fields contained in the database were excluded. That had the effect of removing all conclusionary information. Personnel data on injuries, causes and even the severity of injuries was withheld. Gender, date of birth, rank and unit were provided. But the field containing first and last initials was omitted.\n\nAlso missing were fields that identified recurring problems, identified instances in which planes were not in compliance with Marine standards and in which problem parts had been cannibalized from other aircraft. A field that provided a detailed identification of the component damage or failure was omitted. However, a field that identified materials or components involved in the incident was provided.\n\nData on Marine aircraft are contained in the Navy database because Marine funding is routed through the Navy, and the Navy is responsible for maintaining safety data for the Marines.\n\nWhat the Navy eventually provided was a complex, well-documented, clean set of related data tables in tab-delimited format. There is a mix of data tables and code tables. Many of the fields in the data tables are coded. Code translation tables were provided for each of the coded fields we received. However, the translation tables for coded fields that were omitted also were omitted.\n\nA large-format table relationship sheet was provided, as was a table list text file and a data dictionary text file. The table list (with the file name av_tablist.rtf) gave a simple list of table names, field names, types and sizes and whether the field was omitted or not. The dictionary file (av_dicty.txt) gave an English translation of field names plus an explanation. For example, the field \"bank_degs_left\" in the table \"av_actt_at_esc\" was described as follows: \"LEFT BANK DEGREES-The left bank of the event aircraft in degrees at the time of escape.\"\n\nWhere fields were omitted in tables, the field place was held by a null value (hex 00) between the tab characters. That caused problems for me when reading the data into SAS. Therefore, I copied the original tables and then, using UltraEdit, did a search and replace across all the file copies in one operation to translate a tab-null-tab string into a tab-space-tab string.\n\nSome of the code fields proved to contain hierarchical codes, which were not explicitly documented. An example is \"incdt_phase_ops_c\" in the table \"av_incdt_occurn.\" It is described in the dictionary as \"Significant Incident Phase Operation~The phase of operation in which the event aircraft was engaged, at the time the coded subject significant incident occurred, in the evolution of the reportable event.\"\n\nAn example entry in the three-character field is \"33Z.\" The code translation table renders that as \"No further breakdown,\" which is meaningless. In the same code translation table, however, you find that \"3\" translates as \"Takeoff forward motion on run/catapult until attaining climb schedule.\" And \"33\" translates into \"Aircraft breaks ground climb schedule not yet attained.\" In other words, that single three-character field can be decoded into three separate, related meanings.\n\nThere are other examples. The five-character field \"imc_c\" in the table \"av_eng_cmpnt\" is described in the dictionary as \"Involved Material Component~Identifies the Naval Safety Center code relating to the category of the event aircraft's component or engine (power plant).\"\n\nOne such entry is \"T1007\", which is translated in the field code table as \"Compressor pressure limiter section.\" The \"T\" alone translates into \"Turbo-jet engine,\" and the first three characters, \"T10,\" mean \"Regulator-not detailed.\" Thus the code must be parsed to gain its full meaning.\n\nMy solution for all of the hierarchical codes (which were identified by inspecting the code table translations for partial string definitions) was to create new fields in the table to contain just the partial strings that had separate meanings. Thus I created an \"imc_a\" field that was only the first character of \"imc_c,\" and an \"imc_b\" field containing the first three characters of \"imc_c.\" This was easily done because all of the hierarchical fields contained the full complement of characters specified in the table list and dictionary files.\n\nThe Naval Aviation Safety Database contains records (more than 48,000 in the mid-2002 version we received) of \"events\" from Jan. 1, 1980, to within the last several months. An event can range from a no-damage, no-injury maintenance or operational problem that could have a safety impact and thus meets the criteria for reporting, to loss of an aircraft and its crew. The most dramatic events are those known as \"Class A,\" with at least $1 million in damage or fatal injury.\n\nThere are three primary trajectories for analyzing the data - events, persons or aircraft - each of which utilizes its own key field for table joins. Within those trajectories there are subsidiary keys that join more detailed tables. As a result, you can end up with a lot of tables joined together, which can lead to a lot of rows being added as details embellish what started out as a single row for a single event. Much of the data will be constant and repeated on those rows, such as the total cost of the event, or the event category, or even the tail number of the aircraft. Other fields on the rows will contain unique data, as when several different components are involved, causing several rows from the component table to be joined to the aircraft table.\n\nUltimately, it is necessary to recode and collapse data to analyze it accurately. There is a lot of missing data, beyond what was withheld. The data collected has undergone a number of revisions since 1980. Later events have more complete data than earlier events. Sometimes data is missing because it is not relevant to a particular event.\n\nTo compute accurate percents for occurrences, the denominator has to be just the events for which the data exists. For instance, some reports for Class A events involving the Harrier did not have any data on the components involved. I created a new logical field called Parts, with \"Yes\" meaning there was parts data and \"No\" when it was missing. Then when calculating the percentage of jet engine problems, for instance, I divided by the count of Parts = Yes.\n\nIn some cases I forced empty records to sort at the bottom of a list instead of the top by recoding a blank field to \"zzz.\"\n\nIf the summary being created involves a number of fields, it probably will be necessary to make several intermediate tables by doing record-by-record inspections and editing. That's because it probably will not be possible to sort across several fields and end up with the data desired from each field rising to the top of the sort.\n\nThe most important advice I can give is to be skeptical of every result you derive and try to figure out an alternate way to get it. If you come up with the same result two different ways, it is probably reliable. But if you get different results, you'll need to figure out why and fix it.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-using-parserat-to-carve-up-names","slug":"2003-septemberoctober-using-parserat-to-carve-up-names","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-using-parserat-to-carve-up-names.html","year":2003,"month":"September/October","monthNum":9,"headline":"Using ParseRat to carve up names","kicker":"CAR TOOL","author_name":"James E. Wilkerson","author_title":"","affiliation":"The (Allentown, Pa.) Morning Call","summary":"ParseRat is a $50 software tool that efficiently parses and cleans large datasets, particularly names, by breaking them into manageable fields and converting various data formats for use in databases and spreadsheets. The program handles multiple data formats and includes features like gender detection and phonetic coding, though it requires some preparation and has a complex interface.","keywords":["ParseRat","parsing","data"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["ParseRat","UltraEdit","XML","EBCDIC","mainframe","dBase","COBOL","Excel","Access"],"full_text":"If you've ever sat in on a session about data cleaning at one of the annual IRE and NICAR computer-assisted reporting conferences, you've heard one question that comes up again and again: \"What about names?\"\n\nMany of us have gazed longingly at a huge table containing jumbled names, wondering what sort of havoc we could wreak if we could just break them up into more manageable fields.\n\nA $50 program called ParseRat can do that by flashing through thousands of names in a few seconds to produce a nice, clean list parsed however you like.\n\nWhile I've primarily used ParseRat (www.guysoftware.com/ParseRat.htm) to carve names, the program is capable of doing much more. The compact program file takes only a few minutes to download but can handle very large files in a variety of formats, from the antiquated EBCDIC data exported from mainframes to data-heavy Web pages using modern XML tagging.\n\nParseRat steps\n\nFor example, you can easily grab large tables from Web pages and, with the help of scripts included with the program, modify them for importing into a spreadsheet or database manager. Or you can use the program to parse data from a simple .dbf or complicated COBOL file. In fact, you can grab text in just about any format: delimited, fixed-width, \"tagged\" with HTML, XML or other coding, among others.\n\nThere are routines for converting unusual date and time formats. The program can even convert from one measurement system to another, such as metric to U.S. standard, and allows you to write new conversion rules for measurements not included.\n\nParseRat will even take a stab at determining gender from names, and will generate Soundex codes useful for searching for names phonetically. But it's obvious from the start that ParseRat was created with mailing lists in mind and there are a bunch of built-in options to parse names, addresses and phone numbers.\n\nThe program's major drawback: The interface is somewhat complicated, with tons of options available through small buttons in an often-jumbled package. The help files have a homemade feel that can be confusing at times. But after spending an hour or so with the documentation, I was able to navigate around the program with relative ease.\n\nFirst, preparation\n\nYou should also be ready to do some prep work on your data before running it through ParseRat. I use a powerful but inexpensive text editor, UltraEdit, available for $35 at www.ultraedit.com There are a number of text editors out there and having one is a must, because they can crank through very large text files that would choke a conventional desktop word processor such as Microsoft Word or the trusty Windows Notepad.\n\nBefore using ParseRat, make sure all header information and extraneous material has been stripped from the top of the text because the program will use the first line of data to make judgment calls about parsing the entire file. The cleaner the file, the better the output, so take some time to ensure everything's as uniform and as clean as possible. I recommend isolating only those fields on which you want to work to keep things simple and speed up the process.\n\nHere's a walk-through on parsing names, which I do regularly.\n\nFirst, use your text editor to choose the most complicated name you can find and move it to the very top of the list. By first showing ParseRat a name with the most elements - \"Mr. James E. Wilkerson III,\" for example - you prime the program to look for all of those elements in each record. And, because you only get to see the first record when making parsing decisions, it's easier to do with a name that contains all of the possible elements.\n\nSometimes it's easier to make up that first name than dig through your data to find the perfect model. If you make up a name just remember to delete it later.\n\nSoftware interface\n\nAfter that, fire up ParseRat and choose Open\\Input file and browse for the text file. The next screen will ask you to confirm the type of data - delimited, fixed width, etc.\n\nThe interface gets pretty clunky here, but it's easy enough to navigate once you get the basics down. We will focus on the Input Type, Name and Output tabs.\n\nAfter confirming the text type, click on the Name tab. This is where you specify the field (or fields) containing the name you want to clean, and tell ParseRat what you want the output to look like. To the left of the tabs, you'll see an input selection to the left and an output indicator to the right. (You should have at least one field visible in the input section - if not, go back to the Input Type and try another type until you do.)\n\nNear the top of the dialog box, click on the first box available in the source fields input. Then, click on the appropriate source field to the left. The name of that field will appear in the source fields box, and the first name in your text will appear in the elements boxes underneath.\n\nParseRat takes a stab at determining the order in which the elements of the name appear, and ranks them from one to five using the radio buttons to the left of the sample name, starting with \"Title\" and ending with the full name. You can change that order by clicking on the buttons. You can change the output order by using the similar feature to the right of the sample name.\n\nAfter setting up the input and output order on the name tab, you need to specify the fields to output. Click on the Output tab to get a lot of options. The most important is the parser-generated fields, which show you the output that ParseRat will generate. Scroll down and take a look at the results to make sure they match your expectations.\n\nTo the right of parser-generated fields is a small button labeled \"Add Field.\" Clicking on that button will add a blank output field to the right of the dialog box. Count the number of fields you want to output and create that many blank fields with that button.\n\nChoices, choices\n\nClick on the first output field at the far right and choose the parser-generated fields name that you want to place there by clicking on it. Click on the next output field name and do the same, repeat until you have all of the output fields you need in the order you want.\n\nIf you want to keep the original record, place it in one of the output fields by clicking on it at the far right of the dialog box.\n\nMake sure you have selected the appropriate output format - usually use comma-delimited text - and click the output button in the top tool bar. Then sit back and wait while ParseRat generates your cleaned names.\n\nIt should be obvious when you start poking around on it that the program is capable of much more than I touched on here. Take some time and read the documentation, and you'll soon be flying through parsing jobs you might have thought impossible.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-septemberoctober-new-linux-offering-great-for-serving-data","slug":"2003-septemberoctober-new-linux-offering-great-for-serving-data","url":"https://dwillis.github.io/uplink/article/2003-septemberoctober-new-linux-offering-great-for-serving-data.html","year":2003,"month":"September/October","monthNum":9,"headline":"New Linux offering great for serving data","kicker":"OPEN SOURCE","author_name":"Aron Pilhofer","author_title":"","affiliation":"Center for Public Integrity","summary":"Computer-assisted reporting in newsrooms is evolving beyond basic spreadsheets to require server-based data management solutions, with Red Hat Linux 9.0 offering a cost-effective alternative to expensive Microsoft Windows-based systems for centralizing newsroom data operations.","keywords":["Linux","database","newsroom"],"topics":["Technology","Data & Databases","Journalism & Newsroom"],"technologies":["Linux","SQL","Access","FoxPro","Windows","IIS","SQL Server","Internet","World Wide Web"],"full_text":"It used to be that all the tools a journalist doing computer-assisted reporting needed were a spreadsheet, database manager and a decent SQL book. But in these days of networked newsrooms with a PC on every desk, the CAR specialist's job has morphed to include such diverse skills as Web design, server-side scripting, database administration and data warehousing.\n\nAn increasing number of reporters and editors want access to data directly. They want the ability to do quick queries on multiple databases through an easy-to-use interface, such as an intranet Web form. The more tech-savvy journalists want to do their own data analysis using more advanced tools, such as Microsoft Access database manager.\n\nStand-alone database managers (Access, FoxPro and the like) aren't well suited for the task of sharing data. For that, you need a central server capable of making data available over a network.\n\nWhich brings us to Red Hat Linux 9.0, the latest version of the most popular Linux distribution on the market. After more than two months working with Red Hat 9.0, I can say that it is by far the best distribution of Linux ever released and an ideal solution for any newsroom looking to centralize its data operations at a fraction of the cost of other alternatives.\n\nAssuming you already have the hardware on hand, a very basic Microsoft Windows/Internet Information Server/SQL Server installation will range from $2,500 to more than $10,000 (not including support) depending on features desired and number of users.\n\nRed Hat 9.0 can be downloaded for free. A boxed version costs $39.95 and comes with printed manuals and 30 days of Web-based support. Red Hat also sells higher end enterprise editions (the ES series) that range from $350 to $800 and come packaged with up to a year of live telephone support.\n\nLike all open-source software, Red Hat's license allows administrators freedom to install it on an unlimited number of servers accessible to any number of networked users. Microsoft generally requires a separate license for each user or system on which its software will be installed.\n\nAs for support, both Microsoft and Red Hat charge for it. But while some kind of formal support is a nice safety net, it isn't absolutely necessary for Linux any longer because the installation, configuration and administration tools have improved so much in the past few years.\n\nAnd among the new breed of simpler, easier Linux releases, Red Hat 9.0 is without a doubt the best.\n\nInstallation\n\nThe boxed version can be purchased at any major computer store, but the three installation disks can be downloaded from the Red Hat Web site, www.redhat.com/download/products.html. Be aware that downloading will take several hours even with a very fast Internet connection. The boxed version might be worthwhile for those new to Linux because of the printed documentation and support.\n\nOnce you have the disks, installation should be a simple matter of booting from the CD and following on-screen instructions. I installed Red Hat 9.0 on two desktop configurations and a laptop without a hint of trouble. Users will most likely be able to simply select all the default settings and continue.\n\nAlthough there may be a few speed bumps along the way—especially for new users—the Red Hat Web site and printed documentation should cover these topics adequately. If not, there are several excellent Web sites that cover Linux basics, including the Linux Documentation Project (www.tldp.org) and Linux Planet (www.linuxplanet.com). It might help to have an information technology person nearby just in case.\n\nRed Hat has made selecting the right mix of software a snap. Users can initially select from several broad installation categories such as server, workstation, desktop, and the appropriate software for that function will be installed. Users can also customize their installation by selecting among the hundreds of individual applications that come packaged with Red Hat.\n\nOnce the software has been selected, the actual installation should go automatically. Reboot and you should have a working Linux machine.\n\nSoftware\n\nRed Hat 9.0 includes the two critical applications any newsroom data center would need: a Web server and a high-end database application.\n\nApache 2.x is the standard Web server packaged with Red Hat 9.0, and it's outstanding. It is by far the most popular Web server in the world. According to the most recent statistics, roughly two of every three Web sites run Apache.\n\nIt's popular because it is secure, stable and easy to set up. The default configuration probably will work just fine for internal newsroom use but, if users need to tweak any settings, that can be done by editing a master configuration file.\n\nRed Hat 9.0 offers several database management solutions. The best is MySQL, which is optimized for speed, stability, reliability and power, and ideal for serving data. (For details see \"MySQL primed for heavy-duty action\" in the March-April 2003 Uplink).\n\nGoodbye stand-alone Access databases and random .dbf files floating around the network. No more problems with corrupted data on deadline. So long to worries about users accidentally deleting or altering data. MySQL, like any high-end database server, allows administrators to give users just the amount of access they need and no more, or the ability to query a database but not to update or delete records.\n\nRed Hat 9.0 comes packaged with a number of scripting languages - PHP, Perl, Python, Ruby - and development tools that allow you to combine the power of Apache and MySQL to create simple Web-based database search pages.\n\nAlthough there are many ways of installing software under Red Hat, including compiling from source code, the simplest way is to use prepackaged applications called RPMs. Just as Windows has its .exe, the software packaged for Red Hat machines carries the .rpm extension. Several years ago, the company created a special program to handle RPMs called the Red Hat Package Manager, which works a little bit like the add/remove programs control panel in Windows.\n\nRPM did make installing software easier, but it was far from perfect. As long as the user stuck to programs bundled with the distribution, things were fine. But users sometimes want to install more bleeding edge versions of programs, or applications that were not packaged with the system - and that could cause problems.\n\nRed Hat 9.0 solves this problem by simply removing it. The new package manager will not install anything but the applications that come with the Red Hat distribution. Users can install other RPMs, but that must be done from the command line, not through a graphical interface - an excellent compromise for newer users and power users alike.\n\nMaintaining Red Hat\n\nIn addition to being easy to install, Red Hat also is simple to maintain because of the Red Hat Network. All users (even those who downloaded the software) are given access to it. RHN is similar to the Windows \"update\" feature but much more powerful because it updates not only the Linux operating system but any standard installed software.\n\nRed Hat is quite good about posting bug fixes and updates to the Red Hat Network almost as quickly as they are released, thus ensuring that your system is as up to date as possible.\n\nManaging your server is easy and secure with Webmin (www.webmin.com), a Web-based administration tool that is bar none the best available. Administrators can administer a Linux server over a network or even over the Internet, although that is not recommended for security reasons.\n\nWebmin is simple to install, and very intuitive. It is also open source, meaning it is totally free. There are modules available to help manage everything from a firewall to user accounts to disk partitions. Webmin, unfortunately, is not among the standard packages that come with Red Hat 9.0, but its excellent documentation can be downloaded from the Webmin Web site.\n\nThe major problem with Red Hat 9.0 is its slightly out-of-date version of MySQL. The 4.x release, which adds Union queries and full support for full-text indexes, is now considered the production version. But Red Hat still inexplicably packages the older 3.2x version of MySQL.\n\nUpgrading to the 4.x series is not as easy as installing Webmin and, if you have an existing installation of MySQL on your machine, it can cause problems. There is excellent documentation on the MySQL Web site (www.mysql.com) that covers everything you'll need to know to install the latest version of MySQL.\n\nConclusion\n\nRed Hat 9.0 ranks as the single most painless installation I have ever seen in my 20 years of computing. It won't cost you a cent to download, but offers the power and features of systems costing tens of thousands of dollars.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-remote-sensing-images-boost-gis-work","slug":"2003-novemberdecember-remote-sensing-images-boost-gis-work","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-remote-sensing-images-boost-gis-work.html","year":2003,"month":"November/December","monthNum":11,"headline":"REMOTE SENSING Images boost GIS work","kicker":"","author_name":"Holly Hacker","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists are using remote sensing combined with GIS to uncover insightful stories. The St. Louis Post-Dispatch utilized satellite images to analyze development on the Upper Mississippi River Basin post-1993 flood, revealing significant flood plain development.","keywords":["remote sensing","GIS","satellite"],"topics":["Technology","Investigative Methods","Health & Environment"],"technologies":["GIS","ArcEditor","ArcInfo","shapefile","geotiff","satellite imagery"],"full_text":"Most GIS-savvy journalists are familiar with shapefiles, points and polygons. They can place dots on a map to show crimes in their neighborhood, or code Census tracts to identify neighborhoods that gained or lost the most residents. But for journalists who use a geographic information system, the next spatial frontier to conquer is remote sensing, according to experts in the business. The St. Louis Post-Dispatch recently discovered the dynamic stories that can be told by combining satellite images with other information in a GIS. The newspaper hired researchers in Minnesota to study development on the Upper Mississippi River Basin since the flood of 1993. The analysis, which blended satellite images, Census figures and flood area data, confirmed the lead reporter's suspicion: Missouri has developed more flood plain than any other state affected by the disaster. Those who study or work in the field of remote sensing expect to see similar projects from news organizations down the road. In a nutshell, remote sensing is the science of gathering information about the Earth from a distance. The process typically involves sensing energy (such as heat, visible light or infrared radiation) that is emitted or reflected by the Earth's surface or atmosphere. This can be done in several ways, from attaching a camera to a hot-air balloon to sending a satellite hundreds of miles into space. Remote sensing really isn't new to the news business. Think of TV weather maps showing satellite images of clouds or rainfall, or aerial photographs in newspapers. News organizations have embraced remote sensing more in the past year, largely because of the war in Iraq. They have shown satellite images of Baghdad, debris from the Columbia space shuttle crash over Texas, and areas in Washington, D.C., where the snipers attacked. The Post-Dispatch series took remote sensing another step by blending other data with satellite images to find and tell a story. Environment reporter Sara Shipley grew up near St. Louis and remembers big floods in the 1980s. When a job at the Post-Dispatch brought her back to the area last year, she was struck by all of the businesses and roads that had sprouted on the flood plain. Shipley wanted to measure how much development had occurred since the 1993 flood, and to compare Missouri to other Midwestern states. Existing documents and data, such as permits issued for flood plain development, could not answer her questions. (The permits are required only for federally backed flood insurance, which not everyone obtains. Plus, the permits don't cover all areas of a flood plain.) So the newspaper hired eoSpatial Services (www.gis.smumn.edu), a research group at Saint Mary's University in Minnesota, to analyze development in areas of seven states affected by the flood. To mark the flood plain and areas that flooded in 1993, the researchers used data from the Scientific Assessment and Strategy Team, a government group formed to study the disaster. The researchers also obtained current and pre-flood satellite images from the U.S. Geological Survey. Their third major source of data, showing changes in population and housing, came from the U.S. Census for 1990 and 2000. After some cleaning and processing, Saint Mary's researcher Jim Hipple imported the satellite images (as geotiff files), flooding and flood plain boundaries (as shapefiles) and Census block data (as database files) into ArcEditor, ESRI's mid-level GIS. The Census figures helped Hipple zoom in on areas with big jumps in population. He also scrutinized the pre- and post-flood satellite images to scout out new highways, parking lots, businesses and homes. The satellite images had 30-meter resolution - meaning each pixel of the image represents a 30-by-30 meter surface. To get clearer pictures with higher resolution, Hipple used aerial photographs to study areas with intense development. Hipple traced the outlines of post-flood development using his mouse to create polygons and place them in one of four categories: industrial/commercial, residential, highway/interchange and recreational. Then he tallied the acres of land developed in flood plain and flooded areas since 1993. The analysis revealed that Missouri had developed more than 4,200 acres of flood plain, more than any other state affected by the flood. And nearly all of Missouri's development had occurred in the St. Louis area, in areas that sat under water 10 years ago. The series (www.stitoday.com/stitoday/news/special/flood93. front?openview&count=2000) then explained how lax state regulation and the political climate in the St. Louis region allowed so much development in areas that could someday end up under water again. 'We were able to quantify it. That was the biggest thing,' Shipley said. And the satellite images taken before and after the flood drove home just how much development had taken place. 'When you're sitting here on the ground, you don't really get that perspective.' Remote sensing can be costly. The Post-Dispatch paid $10,000 for the analysis. Nor is buying remote pictures from a commercial company cheap, at least by most newsroom standards. Images from Space Imaging (www.spaceimaging.com) start at $350, while those from DigitalGlobe (www.digitalglobe.com) start at $562. However, in an effort to tap into the media market, both companies have given images to some news organizations for free. But journalists can do other projects and use other data that cost less. 'There are many very good shoestring-type projects that can be done using imagery, sometimes as a thing unto itself, other times as a layer in a GIS,' said Chris Simpson, who teaches journalism at American University and uses remote sensing to track development in urban areas. For instance, the U.S. Geological Survey's Earth Explorer (earthexplorer.usgs.gov) and Earthshots page (earthshots.usgs.gov) have free satellite images and tips for analyzing them. NASA has free images too, at visibleearth.nasa.gov and rapidfire.sci.gsfc.nasa.gov, and its World Wind software (nasaworldwind.com) lets users scroll through the planet and zoom in on shared satellite photos. Images from both agencies should meet the needs of many newsrooms. In higher-resolution shots, rooftops are visible but individual people are not. There's a wealth of potential stories in remote sensing for journalists who find the dollars or ingenuity in-house to do it themselves. Consider satellite images of urban sprawl, how irrigated fields in the Midwest partly explain the Ogallala aquifer's decline, mountains near Atlanta flattened by quarries providing stone for new homes. Earthviews could offer an amazing blend of facts and visuals on pollution, urban decay, environmental deterioration or regeneration. ' Imagery is imagery is imagery,' Simpson said. 'One way or another, you're looking at it from a distance...'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2003-novemberdecember-data-based-approach-reaps-great-stories","slug":"2003-novemberdecember-data-based-approach-reaps-great-stories","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-data-based-approach-reaps-great-stories.html","year":2003,"month":"November/December","monthNum":11,"headline":"Data-based approach reaps great stories","kicker":"SPOTLIGHT: CITY GOVERNMENT","author_name":"Michael Mansur","author_title":"Reporter","affiliation":"The Kansas City Star","summary":"A local government reporter describes how learning Microsoft Excel enabled them to analyze public data sets like budgets and payroll records, leading to stories that provided original information about government spending and official compensation.","keywords":["Excel","government","data"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Excel","FoxPro","dBase","Access"],"full_text":"Two years ago, I thought Excel was a meatpacking plant somewhere in central Kansas. I was spreadsheet ignorant.\n\nThis was particularly startling because I had worked on stories with other reporters who had analyzed large amounts of data using Microsoft FoxPro. And years ago, I had even sorted and analyzed data in dBASE. But for a variety of reasons, spreadsheets were foreign to me.\n\nLast year I began a new assignment covering local government. And I almost immediately saw the need to quickly add, subtract, order from highest to lowest, find the average or median and calculate percentage increases. I had to learn Microsoft Excel.\n\nNow, I realize I risk ridicule here. Many readers of this publication are interested in how to write the shortest program to sort through millions of government records in Microsoft Access or FoxPro.\n\nBut I also know many local government reporters who find themselves too busy to regularly build spreadsheets for their stories. I've come to find those spreadsheets are essential in almost every story. In fact, they can help a reporter take more control of the beat and its stories by generating information - and stories - that no one else has.\n\nWhat I will attempt to detail here is some of the key data sets on the local government beat, how I used them to build spreadsheets and the stories that resulted.\n\nThe questions that prompted my requests for the data or documents I used to build the spreadsheets are basic to most any beat.\n\nHow much does that government official make?\n\nHow much does that perk or program cost taxpayers?\n\nStart with your local government's annual budget. It should break down expenses in a general way. Look for any interesting notes and be sure to compare that budget to previous years, to search for major changes.\n\nAlso be sure to obtain a copy of the annual management letter, normally done by outside auditors. It should detail problems in financial controls, reconciliation of accounts and other problems related to revenue collections.\n\nThe city budget also should total personnel costs. But you'll need to dig a bit deeper to find out which government employees have the best deals in the way of pay.\n\nPersonnel and payroll\n\nYou need basic salary information about everyone. That should include name, department, salary, overtime and any other pay. A quick and easy story could focus on how many officials earn more than $100,000. You might be surprised by what you find. In Kansas City, we found 60 city officials in the \"$100,000 club.\"\n\nThen focus on the overtime. Examine by departments. What are the trends over the years? Examine also by individuals. Did any employees double or greatly boost their annual pay with overtime?\n\nStart with your local government's annual budget.\n\nI obtained some of this information electronically. It was in a rather clunky form, however. It was organized by payroll date - so each employee had 26 entries per year. I imported the data to Excel, then created pivot tables on the names to group all payroll information. Then, I sorted in descending order.\n\nAnother angle to pursue on overtime is to look at how often city officials, including police and fire, are paid for overtime they don't actually work. In Kansas City, police are paid a minimum of four hours overtime for a callback, even if they only work one hour. The Star obtained a few weeks of the police department's callback sheets, which detailed the actual time worked versus the amount paid.\n\nUsing a spreadsheet, I logged each callback, the amount paid in overtime and the amount actually worked.\n\nPerks\n\nA number of basic, but interesting, city or county records will be a good starting point in determining the perks that your local government employees enjoy.\n\nEarly retirement programs are popular today as local governments attempt to trim budgets. Obtain a list of every employee, position, salary, years of service and monthly retirement payment. I used such a list, building most of it by entering the data myself, when about 350 employees opted for an early retirement program in Kansas City.\n\nWith the spreadsheet, I was able to detail who was receiving the largest monthly retirement payments, the average payment, the years of service lost to the city and losses by department.\n\nAnd then there are cell phones. When I set up shop in city hall, I quickly found it difficult to find any official by leaving a message on their office phone. The best way to reach them, I found, was on their city cell phone. That, of course, prompted the question: How much does that cost taxpayers?\n\nUsing an open-records request, I obtained electronic data about each cell phone, its city user, department and total monthly bill for three months. We totaled the three months of bills by user, then ranked them.\n\nWe filed another open-records request, asking for detailed billing information about the top 25 users. That revealed that many, including two city council members, were abusing their phones or racking up exorbitant bills. Others were doing work on their cell phones, but they were on the wrong plans, regularly exceeding their allotted \"free\" minutes.\n\nTake-home vehicles traditionally are another city or county government perk. We requested the number and assignments of vehicles by city department. The city auditor had studied this a couple years earlier, so we compared our update figures to those in the city audit, creating a spreadsheet by department to track the increases or decreases.\n\nTo flesh out the story a bit more, we also obtained logs of city employees assigned to vehicles, which showed how often they use the vehicles for something other than commuting to and from work.\n\nProperty records\n\nBe sure to obtain copies of county assessment records. These could be a foundation for a number of stories. Who has had the highest or smallest increases in assessment?\n\nAlso ask for delinquent property records, which may tip you off about local developers and businesses in trouble. Who owes the most in back taxes? Do many property owners allow their property tax bills to go delinquent for two years or more before paying the bill?\n\nIn addition, cities must create inventories of their city property. Check the city's real estate lists, matching city-owned addresses against lists of code violators. I recently found that the city owned land on which illegal junkyards were being operated.\n\nOther fun records\n\nInspection records of restaurants, swimming pools, day-care centers and even salvage yards can produce great stories. Plug into a spreadsheet the inspection dates, findings, investigator's names and other document information. Patterns will emerge that you might not have caught otherwise. Also, spreadsheets are a great tool for keeping the information close at hand.\n\nI recently had fun by obtaining city water department listings of water use at city-owned sites, including the city zoo and numerous fountains. With that information in a spreadsheet, we could calculate the amount of \"free\" water going to city sites.\n\nThe city kept spreadsheets on its free water users, including their addresses and the amount of water per month. With that information, we calculated usage per hour for some fountains and compared it to the average monthly usage of a family of four. We also toured several of these sites with water officials to refine our calculations.\n\nThen there was water lost through leaks. In some areas the department lost as much as half of its water and ended up collecting fees for only two-thirds of the water it had treated for distribution. Big leaks happen not only in Kansas City but also in almost every city east of the Mississippi River because of the age of the city water distribution systems.\n\nFinally, The Star also recently obtained the parking records for municipal judges, who had long been accused of working short hours in the city courthouse.\n\nFor many courthouses, this idea of using parking records to mark their workweeks would probably not work. But for judges whose cases are all heard from the bench in a docket-style fashion and where little legal research is required, it seemed like a good way to determine how much time the judges were spending on city business in the courthouse.\n\nFor this story, though, I had to advance beyond spreadsheets, enlisting the support of Greg Reeves, the newspaper's FoxPro guru, to sort through the electronic swipes of judges, who used cards to come in and out of the courthouse parking garage.\n\nWith Reeves' help, I had moved beyond Excel. But it would never have been possible without those initial steps 18 months ago into the world of spreadsheets.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-newsroom-data-depots-no-longer-a-luxury","slug":"2003-novemberdecember-newsroom-data-depots-no-longer-a-luxury","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-newsroom-data-depots-no-longer-a-luxury.html","year":2003,"month":"November/December","monthNum":11,"headline":"Newsroom data depots no longer a luxury","kicker":"INTRANETS","author_name":"Matt Carroll","author_title":"","affiliation":"The Boston Globe","summary":"Searchable databases have become essential tools in modern newsrooms, allowing journalists to quickly access public records and connect information across multiple sources to enhance investigative reporting.","keywords":["databases","journalists","records"],"topics":["Data & Databases","Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"In a few short years, searchable databases on newsroom intranets have gone from novelty to necessity.\n\nMost newsrooms either already have databases - ranging from state payrolls to police bookings, from licenses for dogs to licenses for burglar alarms - or have someone thumbing through old \"Dummies\" computer books, learning how to use them.\n\nJournalists who use the databases understand their value as a tremendous tool for finding people on deadline. With a few keystrokes they can find the address of the juror who voted to acquit in the murder trial, the unlisted phone of an indicted CEO, and all the campaign contributions from the real estate industry to the new mayor.\n\nOther papers, such as The New York Times, have gone the next step and linked databases together so that journalists can intuitively follow the flow of","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-bits-and-bytes","slug":"2003-novemberdecember-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-bits-and-bytes.html","year":2003,"month":"November/December","monthNum":11,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library has updated its FDA and NASA safety reporting databases with recent data, providing journalists with resources to investigate pharmaceutical and aviation issues. The article also announces the upcoming 2004 Computer-Assisted Reporting Conference in Cincinnati and describes how IRE and NICAR provide timely resources during breaking news events.","keywords":["databases","journalism","reporting"],"topics":["Journalism & Newsroom","Data & Databases","Technology"],"technologies":["SQL","spreadsheet","Internet","World Wide Web"],"full_text":"Bits & Bytes\n\nUpdated files\n\nThe IRE and NICAR Database Library now has FDA Adverse Events Reporting System data spanning 1969 through December 2002 and NASA Air Safety Reporting data covering 1988 through September 2003.\n\nThe FDA database contains more than 2 million medical problems associated with pharmaceutical products reported by manufacturers, health-care professionals and consumers. The database holds information about each drug's probable role in a patient's death, disability or other such event.\n\nIn 2000, for example, the Los Angeles Times found information suggesting that the FDA ignored warnings from its own scientists and sided with drug companies chasing profits.\n\nJournalists have used the NASA data to identify aviation incidents and gather information about them for news stories. The database is composed of voluntary and anonymous reports about aviation safety filed by passengers, pilots, and controllers - anyone is eligible to file. The data is rich with detailed narratives about events such as the 1998 near collision between two foreign jumbo jets at JFK International Airport in New York.\n\nTo learn more about these and other databases, visit the IRE and NICAR Database Library on the Web at www.ire.org/datalibrary/databases or call 573-884-7711.\n\nCAR Conference\n\nPlanning is under way for the 2004 Computer-Assisted Reporting Conference in Cincinnati, scheduled for March 12-14.\n\nPanels and workshops will address campaign finance, census, military, federal contracts, international data, environment, local government, business, sports, health, education and much more.\n\nFor CAR beginners, special panels will be coordinated with hands-on classes. A special \"Start to Finish\" track will focus on broadcast needs and issues. Special pre-conference workshops will be offered March 11 for advanced CAR users.\n\nHands-on classes will cover basic to advanced skills in spreadsheets, database managers, the Internet, mapping, statistics, social network analysis, SQL and more.\n\nFor more information, registration and the latest schedule, visit www.ire.org/cincy04.\n\nBreaking news\n\nAs major news breaks, IRE and NICAR update their Web pages to help journalists get a head start in reporting. When Hurricane Isabel struck North Carolina and Virginia, IRE and NICAR added hurricane story ideas, tipsheets and databases to the growing list of breaking news resources.\n\nAlthough this year's hurricane season is over, many of the resources can be tapped during other disasters. See www.ire.org/inthenews archive.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-new-faces-changes-at-database-library","slug":"2003-novemberdecember-new-faces-changes-at-database-library","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-new-faces-changes-at-database-library.html","year":2003,"month":"November/December","monthNum":11,"headline":"New faces, changes at database library","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"Database Library staff","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library introduces new graduate assistants and volunteers for the fall semester, including Christina Caron, Megan Clarke, Hsuju Ho, and Tad Vezner, while congratulating departing staff members who have moved on to reporting positions at major news organizations.","keywords":["staff","database","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Fall is a season of change - something that holds true at the IRE and NICAR offices. Each new school year at the University of Missouri brings new staff members to our Database Library.\n\nSo I thought I'd introduce the new graduate assistants working in the database library. Chances are, if you call to order data or ask us a question, you'll speak to one of them.\n\nChristina Caron is a first-year master's student who has spent the past three years managing and collecting outcomes data at the Dana-Farber Cancer Institute in Boston.\n\nMegan Clarke is also a first-year master's student who has interned with KEVN-Rapid City, S.D., and Wisconsin Public Radio.\n\nHsuju Ho is a second-year master's student who started working in the Database Library last year as a volunteer. She has interned at major English and Chinese newspapers in Taipei, Taiwan.\n\nTad Vezner is a second-year master's student who switched to journalism after a career in social work and public relations. He was director of communications for the Epilepsy Foundation of Greater Chicago. Since making the switch to journalism, he has reported for The (Toledo) Blade.\n\nWe also have two new Database Library volunteers: incoming master's students Florence Kizza and Andrea Lorenz.\n\nSecond-year master's students Jaimi Dowdell and Holly Hacker have returned to the Database Library.\n\nTwo former Database Library staffers have departed. Megan Christensen got her master's degree and is reporting for the Detroit Free Press. Stephanie Kang is reporting for The Wall Street Journal in Los Angeles.\n\nContact David Herzog by e-mail at dherzog@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-hands-on-car-training","slug":"2003-novemberdecember-hands-on-car-training","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-hands-on-car-training.html","year":2003,"month":"November/December","monthNum":11,"headline":"Hands-on CAR training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering multiple computer-assisted reporting training workshops and boot camps for journalists in 2024, including sessions on mapping data, advanced statistics, and spreadsheet/database skills.","keywords":["training","journalism","data"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["ArcView","spreadsheet"],"full_text":"IRE and NICAR have a number of training opportunities coming up for journalists who want to learn more about computer-assisted reporting.\n\nJournalists can learn how to use geographic information system software to map data and uncover compelling information for news stories at the Mapping Data for News Stories Workshop Jan. 9-11 in Columbia, Mo.\n\nJournalists who want to expand their knowledge of data analysis can register for the Advanced Statistics Workshop Feb. 13-15 in Tempe, Ariz. Pulitzer Prize winners Steve Doig of Arizona State University and Sarah Cohen of The Washington Post will lead the session.\n\nIRE and NICAR also will conduct intensive CAR Boot Camps for journalists who want to learn how to use spreadsheets and database managers to produce high-impact news stories in Columbia, Mo. The dates are March 21-26, May 16-21 and Aug. 1-6.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-party-town-arrest-patterns-uncovered","slug":"2003-novemberdecember-party-town-arrest-patterns-uncovered","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-party-town-arrest-patterns-uncovered.html","year":2003,"month":"November/December","monthNum":11,"headline":"Party town arrest patterns uncovered","kicker":"FIRST VENTURE","author_name":"Shelley Nelson","author_title":"Reporter","affiliation":"Duluth (Minn.) News-Tribune","summary":"A Duluth journalist uses computer-assisted reporting techniques to analyze arrest data and determine where Superior, Wisconsin's crime problems originate, examining whether extending bar hours in Duluth would reduce Superior's law enforcement burden.","keywords":["reporting","arrest","data"],"topics":["Investigative Methods","Crime & Justice","Data & Databases"],"technologies":["Excel","Access"],"full_text":"Duluth's city council has a theory - keep the bars open an hour later and you'll keep the money from flowing over the bridges that connect Duluth, Minn., and Superior, Wis.\n\nSuperior police officers scoff at the idea.\n\nSuperior's reputation as a \"party town\" is nearly as old and well-documented as the city's history. The availability of cheap alcohol and a wide variety of bars and nightclubs in a 10-block area of a 45-square-mile city create a nightlife culture that is far different from Duluth's.\n\nAs I talked to Superior police officers trying to gauge the expected impact of a later bar closing time in Duluth on law enforcement in Superior, I discovered most police officers didn't think it would make a difference. The size of Superior's larger-than-average police department has long been attributed to late-night partying and violations committed when alcohol is consumed.\n\nOne night shift officer told me that if the police department could close off the bridges at night, about half their problems would go away. Several others on the night shift agreed with him.\n\nI was skeptical.\n\nI read police arrest logs routinely. Simple observation suggests a lot more Superior residents are arrested than Duluth residents.\n\nThat's where the computer-assisted reporting techniques that I learned at an IRE and NICAR Boot Camp earlier this year came in.\n\nAddresses are key\n\nGetting the data to answer those questions was the easy part. Department command staffers were willing to accommodate my request.\n\nThe database itself was a problem.\n\nThe lack of a system expert proved to be the biggest difficulty. The department doesn't have the know-how on staff to retrieve the full body of information. The system expert retired several months before and was never replaced because of budget constraints.\n\nThat meant I had to manually enter home addresses of arrestees from reports to find out where Superior's crime problems were coming from. While the data included arrest dates and times, arresting officer, arrest district, incident numbers, the suspect's name and birth date, and charges leveled at the time of arrest, staff had no way of retrieving the data to include the arrestees' home addresses.\n\nThe data was provided in text format. I did an initial import into Microsoft Excel 2003 to conduct a visual review of the data to make sure it imported cleanly. From there, I saved it as an Excel file to import into Microsoft Access.\n\nDeadlines limited the number of records that could be updated with the addresses of arrestees, but throughout the input process - every 100 records - I tested the database to determine how much difference there was in the number of arrestees from various communities and states. I saved the query to avoid having to rewrite it each time.\n\nReady to write\n\nThroughout the gathering process, the difference - the percentage of people arrested coming from various local communities - was fairly small, varying from 0.3 to 1.2 percent at any point in the collection process.\n\nAfter acquiring home addresses for about 1,800 of more than 2,300 arrest reports, covering a year ending in August, it was time to write. The consistent testing made me comfortable that the data fairly represented the arrestees.\n\nWith Access, I looked at the times of day and locations of arrest to determine where and when Superior police had their greatest problems. I looked at the types of charges leveled at the time of arrest. I looked at the home address of arrestees citywide, and then focused on the trouble areas and times, determining the home addresses of people arrested in the entertainment district and where those arrestees were coming from.\n\nI copied and pasted Access query results into an Excel workbook for analysis. I could easily tab through the information as I wrote.\n\nThat allowed me to test my query logic without relying on recollection, and in one case, catch an erroneous result. It also allowed me to look at the results to see what was happening and determine what type of graphics would best tell the story.\n\nIt also proved handy when it came time to pass along information to the Duluth News-Tribune's one-man graphics department.\n\nData findings, interviews\n\nThe findings substantiated what officers were saying about the draw of Superior's party-town culture and what they observed on patrol.\n\nSuperior residents accounted for about 60 percent of all arrests made by the department, but Duluth residents accounted for more arrests made in the entertainment district between 11 p.m. and 3 a.m. In that time frame and area, only about 33 percent of people arrested in the entertainment district and 27 percent arrested during those hours actually lived in Superior. Overall, Duluth residents only accounted for about 18 percent of all arrests, but 37 percent of the arrests made in the entertainment district and 40 percent of the arrests made between 11 p.m. and 3 a.m.\n\nI was surprised by the results, but when I went back to talk to officers who work the night shift about the problems they deal with, they weren't surprised at all. That was what they expected based on their patrol experience and what they were talking about during earlier discussions.\n\nSince the story ran on Sept. 8, the Superior city council has gotten behind the chief's efforts to create a satellite police office in the city's entertainment district. It's also prompted the chief to look into ways to make people who live outside the community responsible for some of the law enforcement costs they generate and city residents bear.\n\nIt has also prompted the chief to seek better training for his records staff so they can look at the information in different ways. It's training that will be useful a year after the new Duluth bar closing time goes into effect and I look back to see if the extended drinking time makes a difference in Superior's law enforcement efforts.\n\nThe one finding that really took me by surprise - and the records staff was able to verify - is the number of underage drinking arrests police make. The vast majority of those types of arrests in the entertainment district occur around one nightclub during its \"dry\" nights, a weekly alcohol-free event.\n\nThat's a story for another day.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-car-conference","slug":"2003-novemberdecember-car-conference","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-car-conference.html","year":2003,"month":"November/December","monthNum":11,"headline":"CAR Conference","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The 2004 Computer-Assisted Reporting Conference will take place in Cincinnati from March 12-14, offering panels, workshops, and hands-on classes covering various reporting topics and technical skills for journalists of all levels.","keywords":["conference","reporting","workshops"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet","Internet","SQL"],"full_text":"Planning is under way for the 2004 Computer-Assisted Reporting Conference in Cincinnati, scheduled for March 12-14.\n\nPanels and workshops will address campaign finance, census, military, federal contracts, international data, environment, local government, business, sports, health, education and much more.\n\nFor CAR beginners, special panels will be coordinated with hands-on classes. A special \"Start to Finish\" track will focus on broadcast needs and issues. Special pre-conference workshops will be offered March 11 for advanced CAR users.\n\nHands-on classes will cover basic to advanced skills in spreadsheets, database managers, the Internet, mapping, statistics, social network analysis, SQL and more.\n\nFor more information, registration and the latest schedule, visit www.ire.org/cincy04.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-breaking-news","slug":"2003-novemberdecember-breaking-news","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-breaking-news.html","year":2003,"month":"November/December","monthNum":11,"headline":"Breaking news","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR provide journalists with resources like tipsheets and databases during breaking news events, including hurricane coverage, with many resources applicable to other disasters.","keywords":["resources","disasters","journalists"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":["World Wide Web"],"full_text":"As major news breaks, IRE and NICAR update their Web pages to help journalists get a head start in reporting. When Hurricane Isabel struck North Carolina and Virginia, IRE and NICAR added hurricane story ideas, tipsheets and databases to the growing list of breaking news resources.\n\nAlthough this year's hurricane season is over, many of the resources can be tapped during other disasters. See www.ire.org/inthenews archive.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-new-service-opens-door-to-most-recent-data","slug":"2003-novemberdecember-new-service-opens-door-to-most-recent-data","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-new-service-opens-door-to-most-recent-data.html","year":2003,"month":"November/December","monthNum":11,"headline":"New service opens door to most recent data","kicker":"NICAR UPDATE","author_name":"Hsuju Ho","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library launched an online subscription service in spring 2003 that provides journalists with cleaned, updated government databases in real-time, eliminating the need to process raw data and enabling faster reporting on breaking news.","keywords":["databases","journalists","subscription"],"topics":["Data & Databases","Journalism & Newsroom","Technology"],"technologies":["Access","FoxPro","CD-ROM","FTP","Internet","World Wide Web"],"full_text":"Deadline pressure can force journalists to set aside their data because of the time it takes to process and digest it. The IRE and NICAR Database Library now has a solution to that problem: a user-friendly online subscription service that provides cleaned and updated databases for instant use.\n\nIn the past, the Database Library staff took government data orders from journalists and made the data available on CD or via FTP download. Journalists can still purchase data this way and use the online subscription service to report on breaking news.\n\nThe online subscription service, introduced in spring 2003, allows journalists to obtain data as fast as a government agency updates it. IRE and NICAR set up a special Web page for each subscriber and links that page to the latest available data files.\n\nTwenty news organizations have signed on so far.\n\nThe Database Library processes the data and puts it in a format that is easy to use with Microsoft Access, FoxPro or other database managers. The service also includes the record layouts and other documentation.\n\nThe service not only provides a shortcut for journalists who lack the time to deal with raw data, it also helps save time for journalists who could process the data themselves but would rather spend their time on other pursuits.\n\nAnother advantage of the online service is that it makes data available to subscribers any time. Subscribers can download the data when they like, even on weekends.\n\nThe databases that are offered are released weekly or monthly by government agencies and then immediately processed by the Database Library. News organizations can update their own database as frequently as they wish.\n\nWeekly data updates offered on the service include the Federal Aviation Administration Accidents and Incident database, the FAA Service Difficulty Reports and Federal Election Commission campaign contributions.\n\nMonthly data updates include the FAA Aircraft Registry, FAA Airmen Directory and the Internal Revenue Service tax-exempt organization database.\n\nThe Database Library is adding databases from its extensive collection slowly to ensure proper processing and the accuracy of the data. Keep an eye on the IRE and NICAR listservs for additions to the online service.\n\nSubscribers also have the option of getting the data updates on CD every month.\n\nFor the latest database announcements and more information about the online subscription service, contact Jeff Porter, director of the Database Library by e-mail at jeff@ire.org, or see www.ire.org/datalibrary/databases.\n\nContact Hsuju Ho by e-mail at hsuju@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-finding-a-snow-job-after-storm","slug":"2003-novemberdecember-finding-a-snow-job-after-storm","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-finding-a-snow-job-after-storm.html","year":2003,"month":"November/December","monthNum":11,"headline":"Finding a snow job after storm","kicker":"MAPPING The latest uses of mapping in news reporting.","author_name":"Janet Roberts","author_title":"","affiliation":"St. Paul Pioneer Press","summary":"A St. Paul reporter used geographic information system software to analyze snow-emergency towing data and discovered whether certain neighborhoods were towed more heavily than others after significant snowfalls.","keywords":["towing","analysis","neighborhoods"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["ArcView","Spatial Analyst","SQL Server","VB Script","Excel"],"full_text":"Finding a snow job after storm\n\nBy Janet Roberts\nSt. Paul Pioneer Press\n\nOur story began when a reporter's friend woke up one winter morning to find her car had been towed. She'd come face to face with St. Paul's confusing snow-emergency rules, which govern on-street parking after a significant snowfall.\n\nOver lunch that day, the reporter and his colleagues wondered whether the rules are applied fairly. Are some neighborhoods towed more heavily than others?\n\nWith a few computer-assisted investigations under his belt, reporter Chuck Laszewski knew he could answer that question. ESRI ArcView 3.2 geographic information system (GIS) software - and a powerful extension called Spatial Analyst - would make it possible. All Laszewski needed was the right data.\n\nConfusion aside, the snow-emergency rules make sense: Plows can't get the snow off the streets if parked cars are in the way. If streets aren't plowed clean, neighbors cope with a mess of frozen slop that can last all winter long.\n\nBut the rules cause a lot of consternation for newcomers - and even long-time residents who pay them too little mind.\n\nLaszewski set out to get ticketing and towing data.\n\nIt would be the following winter before we sat down at our computer and began mapping. To make a long story short, it took more than six months - and the intervention of our lawyer - to wrest towing records from the St. Paul Police Department. The ticket records proved to be an even bigger hurdle, so Laszewski spent about three weeks' time, spread over five months, typing in ticket addresses for one snow emergency - about 1,200 in all.\n\nNow we were ready to fire up ArcView, right? Not so fast.\n\nImagine it's just snowed more than 3 inches. A bunch of tow-truck drivers go to work. It's cold. It's wet. It might even be dark. The drivers get out of their trucks, hitch the car to their rigs and - oh, yeah - scribble down the address. Then it becomes the job of a clerk at the Public Works Department to decipher that scrawl and type the address into the database.\n\nYou can imagine how clean the address field was. We had gems such as \"CAR X HUH?\" Other records contained street names but no house numbers. The \"good\" addresses were listed this way: \"Sherburne/1033.\" I used Microsoft SQL Server and VB Script to parse the address field and format it in a way ArcView would recognize.\n\nMy first pass at geocoding the records asking ArcView to plot the tow locations on a street map - yielded a disappointing success rate, so it was back to address cleaning. Fewer than two-thirds of the addresses plotted, so I spent the better part of a week examining the records that failed to geocode and correcting misspellings. Ultimately, we had an 88 percent success rate - lower than I like, but the best we could do for this project.\n\nConcentration of tows\n\nFinally, we could begin our analysis. Immediately, we noticed the tows appeared to concentrate north of Grand Avenue, a commercial thoroughfare that roughly divides the city in half from east to west. Indeed, a quick count in ArcView and a few calculations in Microsoft Excel showed 78 percent of the tows were north of Grand. We had a story.\n\nThe Grand Avenue observation would become part of our nut graf: \"It's almost as if the towing companies and city officials stood on Grand Avenue and drew a line in the snow. To the south, considerably fewer cars get hauled away.\"\n\nNext, we overlaid a map of the neighborhood boundaries and did a spatial join to assign each towing record to its neighborhood. Exporting the data back to SQL Server, I summed the records by neighborhood. Clearly, towing was heavier in some neighborhoods. But we wondered if that was because more scofflaws parked in those neighborhoods. Enter our ticketing data.\n\nIn ArcView, I geocoded the ticketing data Laszewski had gathered. Here, there was no \"line in the snow.\" Ticketing was dispersed fairly evenly across the city. Again, I summed by neighborhood. With those totals in hand, I was able to compute each neighborhood's tow rate: the number of tows as a percentage of tickets issued.\n\nWe found wild variation: The neighborhoods with the most tickets had some of the lowest tow rates. Neighborhoods with far fewer tickets had high tow rates.\n\nFine-tuning with extension\n\nOur story was getting better. But we wanted to make more sense of our data. In particular, we wondered where the densest concentrations of tows were. Common sense told us they probably were in areas closest to the impound lot, where towed cars are taken. We knew - because we used ArcView to draw one-mile buffers around the impound lot and counted the tows in each circle - that 68 percent happened within four miles of the impound lot.\n\nIt was difficult, however, to see concentrations in the geocoded points. Because we mapped so many tows, they all blended together into a blob of largely undecipherable color.\n\nEnter Spatial Analyst, an ArcView extension that can do density analysis. We didn't have the software, and at $2,500, it's not cheap. So we sought help from a college mapping instructor. We brought our data to her lab, and she showed us how to use Spatial Analyst to draw our hot spots. The result was eye opening.\n\nWe drew separate hot-spot maps for the ticketing and the towing data. The density patterns did not correspond. In particular, a definite concentration of ticketing on the city's West Side lacked any corresponding concentration in the towing map.\n\nWhere did the towing hot spots concentrate? Around the impound lot.\n\nNow, not only did we have a story, we had a great publishable map and good reason to buy the Spatial Analyst software.\n\nWith time to play with our own software we tested several more theories: that towing patterns would correspond with density of population, rental housing, retail development and immigrant populations (language barriers make it hard for immigrants to understand the complicated parking rules).\n\nAll of these theories had some truth.\n\nBut they didn't explain why the city wasn't towing on the densely populated West Side, an area with many Hispanic immigrants, dense clusters of rental housing and a high ticketing rate.\n\nThis is where Laszewski's shoe-leather reporting was invaluable. Tow-truck operators told him they are loath to cross the Mississippi River to tow on the West Side because it is a long round-trip to the impound lot. The city's towing contracts offer incentives to operators the more cars they tow. Towing operators said they focus their efforts north of Grand Avenue because that strategy yields the most cars.\n\nThe city's public works director, who coincidentally lives in the seldom-towed West Side neighborhood, said he would review city policies in light of our story. Across the river in Minneapolis, city officials were so enchanted with our analysis, they did a similar one of their own, finding the same disparate patterns.\n\nRead the story online at www.twincities.com/mld pioneerpress/4658246.htm.\n\nContact Janet Roberts by e-mail at jroberts@pioneerpress.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-showing-contributions-from-strip-club-workers","slug":"2003-novemberdecember-showing-contributions-from-strip-club-workers","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-showing-contributions-from-strip-club-workers.html","year":2003,"month":"November/December","monthNum":11,"headline":"Showing contributions from strip-club workers","kicker":"SPOTLIGHT: CITY GOVERNMENT","author_name":"David Washburn","author_title":"database editor","affiliation":"Union-Tribune","summary":"A Union-Tribune reporter used a campaign finance database to uncover connections between San Diego city councilmen and a Las Vegas strip club owner suspected of bribery, revealing suspicious donation patterns from the adult entertainment industry.","keywords":["bribery","campaign","database"],"topics":["Business & Economy","Investigative Methods","Government & Politics"],"technologies":["Access"],"full_text":"At first blush it sounds like the premise for an episode of a TV drama chronicling the adventures of a hard-nosed newspaper reporter:\n\nThe reporter hears rumors that a strip club owner from Las Vegas bribed a few city councilmen, hoping that they will push through legislation that would allow more touching between strippers and their customers. The FBI gets wind of it and plans an afternoon raid at city hall.\n\nThe reporter gets a tip about the raid from her sources at city hall. The raid goes off as planned and the reporter has a blockbuster story.\n\nThat's what happened in San Diego. FBI agents in May visited the offices of city council members Ralph Inzunza, Charles Lewis and Michael Zucchet and seized their files, telephone records and computer hard drives. Federal agents suspected the three had solicited bribes from Michael Galardi, the owner of two strip clubs in San Diego and several more in Las Vegas.\n\nThat was where the similarities between a TV script and reality ended.\n\nIn TV-land, the strip club kingpin hires a couple of goons to whack the city councilmen. The reporter walks in on the goons getting ready to do the dirty deed. Using her martial arts skills, she knocks out the goons and the grateful councilmen confess.\n\nBut in real life everybody hired lawyers, the FBI stayed quiet and all the journalists in town hounded their frazzled city hall sources. Following the money and establishing a link between the councilmen and the strip club owner through traditional reporting was far easier said than done.\n\nThis is the spot Union-Tribune reporter Caitlin Rother found herself in during the days after the raid. She heard lots of rumors about Galardi and the councilmen but little in the way of hard facts.\n\nShe later got some of the hard facts from a database of campaign contributions that we built on the fly using Microsoft Access.\n\nAn analysis of the database showed that people involved in the adult entertainment industry had given more than $20,000 to the three councilmen and two other council members whose offices were not searched by the FBI.\n\nWe were able to show that strippers and bouncers were giving to the campaigns of the three councilmen. Also, we found thousands of dollars in contributions coming from Nevada, especially from Las Vegas \"homemakers.\"\n\nHere is what else we found:\n\nCampaign finance reports varied in the way they listed people associated with the adult entertainment industry and their occupations, employers and home addresses. For example, someone listed as an entertainer or waitress on one report would be listed on another as a student, homemaker or some other less obvious affiliation. Some donors had unlisted occupations.\nSeveral people who were listed as giving the maximum $250 contribution to more than one candidate listed apartment addresses in less-than-affluent neighborhoods that usually are not home to political donors.\nMany contributions from people in the industry were given on the same day. This fact helped Rother confirm low-profile fund-raisers held by lobbyists for the industry.\n\nI was thrilled with the final results, but wish that the database had been built prior to the raid. The newsroom had not realized before how important it is to have a database of local campaign contributions on hand.\n\nBuilding the database\n\nRother needed to do a thorough analysis of years worth of campaign contributions, and she needed to do it fast. The problem was that San Diego, like the vast majority of municipalities, does not require candidates to file their contribution reports electronically. So she ended up with a box filled with dozens of reports containing thousands of contributions.\n\nI uttered a swear word and began pounding my head on my desk. I had been pushing for a few years to have the newsroom create a database of local contributions. These databases are great for seeing who candidates may be indebted to politically.\n\nThe editors had agreed and made creating the database a newsroom priority. But, as is so often happens in newsrooms, the news of the day trumped preparing for the future. The database never got built, so we were caught unprepared when the city's biggest political scandals in a decade hit.\n\nNonetheless, we decided that a database of contributions to the three councilmen and two other council members who we knew had received money from Galardi was crucial for our reporting.\n\nI created a table for the contributions and then built a data-entry form using Access' form wizard. Whenever possible, I used tools on the form to speed the data entry. For instance, someone could pick the name of a city council member from a list to enter that name into the recipient field of the table.\n\nAfter that, we rounded up some news assistants, brought in a temp with data-entry skills and went to work entering details such as the name, address, occupation and employer of the contributor. We also entered the amount given and to which council member.\n\nAfter more than a week of grueling data entry we had three years of contributions for the five council members.\n\nNot long after that I caught a mistake. Some contributions had been entered twice because of addendum reports filed by the candidates. When candidates need to make a correction or addition to a report, they file the entire report over again, which means that, in some cases, the same contribution is reported twice.\n\nIt is easy to spot an addendum if you are looking for it. But a few slipped through. So I spent another day typing the data from the addendums into a table and then joining that table to the main table to identify the duplicates and remove them using a delete records query. We ended up with 7,500 records.\n\nWhile the news assistants and I built the database, Rother mined her sources for names of people associated with Galardi. She compiled more than two dozen names with suspected ties. We queried the database to see whether they had contributed and found that almost everyone on the list had given to the candidates. And we found that a vast majority of those people worked for Galardi. In addition, we noticed that their occupations had been listed differently.\n\nA federal grand jury indicted the three councilmen, Galardi and two lobbyists in August. All of them were charged with wire fraud and five of the six with extortion.\n\nThe time and the effort that went into the database were worth it and bolstered the story. Our earlier stories about the federal probe would have been much stronger if we had built the database sooner and used it.\n\nContact David Washburn by e-mail at david.washburrn@uniontrib.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-spreadsheet-good-eye-help-find-missing-money","slug":"2003-novemberdecember-spreadsheet-good-eye-help-find-missing-money","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-spreadsheet-good-eye-help-find-missing-money.html","year":2003,"month":"November/December","monthNum":11,"headline":"Spreadsheet, good eye help find missing money","kicker":"SPOTLIGHT: CITY GOVERNMENT","author_name":"Paul Egan","author_title":"","affiliation":"The Detroit News","summary":"The Detroit News used Excel spreadsheets to compare PAC-reported contributions with candidate-reported receipts, uncovering that Detroit Mayor Kwame Kilpatrick failed to report $16,650 in campaign contributions from 30 PACs during 1999-2000.","keywords":["contributions","database","reporting"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Excel","Access"],"full_text":"When political candidates fail to report receiving campaign contributions, the money has essentially disappeared.\n\nUnless the contributor reports the candidate's failure to publicly report the money, which is unlikely, there is almost no way of knowing the candidate ever received it.\n\nThe exception is with contributions from political action committees. Unlike individuals, whose contributions are only reported by the candidates who receive them, PACs must file public reports that detail their political contributions.\n\nUsing a spreadsheet or database manager to compare the contributions that PACs report making to a candidate with those the candidate reported receiving from the same PACs is an excellent way of testing the accuracy and integrity of a candidate's campaign-finance reporting.\n\nIn July, The Detroit News used Microsoft Excel to conduct such an examination of Detroit Mayor Kwame Kilpatrick, a former Democratic minority leader in the Michigan House of Representatives. Over the two-year period studied, 1999-2000, The News found that PACs reported writing 30 checks totaling $16,650 to Kilpatrick's state House campaign fund that Kilpatrick never reported receiving. Journalists could use a database manager, such as Microsoft Access, to match the data, especially where there are many contributions.\n\nRaising questions\n\nThree factors prompted the News to take a look.\n\nFirst, the PAC money Kilpatrick's state House fund reported receiving - a little over $120,000 over the two-year period - appeared low relative to other state legislators in leadership positions.\n\nSecond, Kilpatrick had a history of noncompliance with state campaign finance laws, accumulating more than $13,000 in fines over five years for late filing of reports to the state and county.\n\nThird, the state had recently refused Kilpatrick's request to dissolve his state House committee, because of reporting discrepancies of between $1,400 and $2,900 - much smaller discrepancies than those uncovered by the News.\n\nWithout a computer the analysis would have been next to impossible. Even with one, it required considerable drudgework.\n\nIn Michigan, where some committees file electronically and others still file paper reports, data on candidate receipts and PAC contributions is supposed to be easily accessible in electronic format. Data filed electronically is added to the Michigan Secretary of State campaign finance database that can be queried on the Web (see http://miboecfr.nicusa.com/cgi-bin/cfr/contrib_anis.cgi.) State workers manually enter information filed on the paper reports into the same database.\n\nGetting a file of Kilpatrick's reported contributions was simple: I punched in the candidate's name, set the date parameters for 1999 and 2000, and downloaded an Excel file with all the reported receipts. Then I deleted the records of contributions from individual contributors, leaving only 109 PAC contributions.\n\nGetting a file of contributions that PACs reported making to Kilpatrick in 1999 and 2000 was much more difficult. Most PACs file on paper. The state uses the candidate reports as its source of PAC contribution data. So Kirkpatrick's unreported PAC contributions would be missing.\n\nThe only way to get the data was to examine scanned reports from all 1,000 or so Michigan PACs during 1999 and 2000 and manually create a spreadsheet of contributions they reported making to Kilpatrick. Using a similar format to the first spreadsheet, my new table included a column for the name of the contributor, one for the amount of the contribution, and one for the reported contribution date.\n\nContributions reported to Kilpatrick by the PACs had 145 records. Since the databases were relatively small, I sorted each spreadsheet alphabetically by contributor and used my eyes to do the matching, striking out pairs of contributions and receipts until I was left with a list of what appeared to be unreported contributions. If I were to try a similar analysis again, I'd clean up the files, load them into Access and try joining them by contributor name and amount.\n\nChecking on the PACs\n\nLooking at the unmatched contributions, I saw that I still had plenty of work ahead.\n\nPACs are sometimes known by more than one name, and it's important to check for inconsistencies between the name used by the contributor and the one used by the recipient when recording the contribution.\n\nWe found instances with matching contributors but differing contribution amounts. Because there was no easy way to determine which table contained the correct amount, the News did not include these contributions in its results and focused solely on contributions that were not reported at all by Kilpatrick's committee.\n\nIt's also possible that a contributor reported making the contribution within the time period covered by the analysis, but the candidate did not report receiving it until a later time period. For that reason, we searched all reports filed by the candidate after the time period covered by the analysis.\n\nContributions can go astray in other ways. For example, the News found instances in which a PAC reported contributing to Kilpatrick's campaign committee, but Kilpatrick reported depositing the money into his own political action committee, called Generations. Since he reported the contributions as received somewhere, these were not included as missing.\n\nInterestingly, among the missing contributions the News found were two totaling $3,150 that Kilpatrick's PAC reported making to his campaign committee but his campaign committee never reported receiving.\n\nLost in the mail?\n\nAnother problem to be aware of is contribution checks that get lost in the mail or become outdated because they are not cashed quickly enough. In those cases, Michigan law requires the PAC to adjust the balance by reporting the amount of the lost or uncashed checks as \"other income\" in a subsequent report.\n\nIt makes sense, then, that the time period chosen for the analysis should be far enough in the past so such anomalies can be discovered and accounted for in the reports. That was one of the reasons the News examined the final two years when Kilpatrick was in the state House, rather than the period during his more recent job as mayor of Detroit.\n\nAs a further check, the News contacted a sample of the PACs that reported making contributions to Kilpatrick that Kilpatrick never reported receiving. Contributors confirmed that at least eight of the unreported checks, totaling $5,150, had cleared the bank.\n\nAfter the News published its report, a spokesman for Kilpatrick acknowledged that there appeared to be a reporting problem and said Kilpatrick's committee had hired an attorney who specialized in financial matters to try to resolve the issues.\n\nPaul Egan is a former staff writer for The Detroit News and can be contacted by e-mail at paulegan@sympatico.ca.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-readmetxt-ire-tipsheets-on-campaign-finance","slug":"2003-novemberdecember-readmetxt-ire-tipsheets-on-campaign-finance","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-readmetxt-ire-tipsheets-on-campaign-finance.html","year":2003,"month":"November/December","monthNum":11,"headline":"readme.txt (IRE tipsheets on campaign finance)","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE offers multiple tipsheets to help journalists use databases for investigating local, state, and federal campaign finance, including guidance on accessing data from reluctant officials, analyzing donor information, and cleaning databases for analysis.","keywords":["campaign","finance","databases"],"topics":["Government & Politics","Data & Databases","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"IRE offers tipsheets for journalists interested in using databases to cover local campaign finance.\n\nMichael A. Weber, co-author of the IRE Beat Book \"Unstacking the Deck: A Reporter's Guide to Campaign Finance,\" offers a list of Web addresses where you can find local, federal and state campaign finance data in Tipsheet No. 1919.\n\nTipsheet No. 1222 provides guidance to journalists for learning about campaign contributors and their employers. This tipsheet also explains how to use and sort a donations database to look for story ideas.\n\nIn Tipsheet No. 1221 Bob Warner of the Philadelphia Daily News offers advice on obtaining campaign finance databases from local and state officials who may be reluctant to hand them over. Warner also provides tips for cleaning the data before you analyze it.\n\nIRE members can download the tipsheets from www.ire.org/resourcecenter. You can also order tipsheets from the IRE Resource Center at 573-882-3364 or rescntr@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-free-tool-reads-converts-sas-data","slug":"2003-novemberdecember-free-tool-reads-converts-sas-data","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-free-tool-reads-converts-sas-data.html","year":2003,"month":"November/December","monthNum":11,"headline":"Free tool reads, converts SAS data","kicker":"CAR TOOL","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"A free SAS System Viewer tool available from SAS's website allows journalists to read and convert SAS data files into formats compatible with spreadsheets and database managers. The article provides step-by-step instructions for downloading, installing, and using the viewer with sample USDA nutrition data.","keywords":["SAS","software","data"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["SAS","Excel","CSV"],"full_text":"Free tool reads, converts SAS data\n\nBy Jeff Porter, IRE and NICAR\n\nSAS, the expensive statistical software used widely in government and academic circles, might be out of reach of many newsroom budgets. But there's still a way - for free, no less - to read SAS data files and translate them into a format your spreadsheet or database manager can use.\n\nFrom the company's Web site, you can download the SAS System Viewer at no cost. To find it, go to http://support.sas.com and click on the link to the left called \"Software Downloads.\" That takes you to a page of downloads, categorized by product type. Click on the link called \"Base SAS Software.\" You should see a short list of items available for download, including the SAS System Viewer. Click away, and follow the screen instructions. You'll have to create a login, including your e-mail address.\n\nInstallation should be a snap. Download the executable file and just double-click on it to start the installation.\n\nAfter the program installs, it's time for some practice. Go to the Web page v.usda.gov/cnpp healthyeating.html There you can download data on the USDA Healthy Eating Index, a summary measure of overall diet quality. It provides a picture of the type and quantity of food people eat and the degree to which diets comply with specific recommendations in the Dietary Guidelines and the Food Guide Pyramid.\n\nChoose to download the 1999-2000 Healthy Eating Index data in compressed, self-extracting SAS 6.12 format. Also download the 1999-2000 Food Guide Pyramid Servings database. The files are relational, creating a summary of food intake by respondent.\n\nThis will not be the most exciting database you've ever worked with. It does, however, give you some practice in using SAS System Viewer.\n\nYou'll need to extract two files: hei9900.sd2 and pyraserv9900.sd2. Start the SAS System Viewer, then go to File > Open. Keep the default setting under Files of Type as All SAS Datasets, then navigate to the appropriate folder and open the hei9900.sd2 file. It should look something like Figure 1.\n\nNow, let's check out a few features of SAS System Viewer. First, notice that the column headers are somewhat cryptic. So go to the View menu item and choose Headers and Names. Now, you'll see something a little nicer-looking in Figure 2. Prefer just the shorter version? Toggle back to headers.\n\nYou can also view the variables, too, under the same View menu, and you'll see a list of all the variables (also known as columns) in the SAS dataset. To go back to viewing the data, just go back to the View menu and choose Formatted.\n\nNow, some final few steps to make the data useful in different formats. Choose how you prefer to view the data - with headers or the lengthier names. Typically, if I'm going to use a file in Microsoft Excel, I like longer names. If I'm going to throw something into a database manager, shorter field names are my preference. So consider how you're going to use your SAS file, then simply go to File > Save As and choose your File of Type.\n\nNow, you've saved the file as a text file of your choice, ready to be opened in a spreadsheet or database manager.\n\nOf course, the viewer doesn't offer many of the features of the full-blown version SAS. You can download, review, even edit SAS program files with the extension .sas, but you can't run those programs with the viewer alone. There's a long list of SAS-type files you can just open in read-only mode, including the data files such as .sd2.\n\nOf course, you can then save those as text, reopen them with different software then edit data tables. Text files opened with the SAS viewer - such as .CSV, txt, prn, .dat - can be edited, although there are better applications for that.\n\nSAS isn't the most common data type that government agencies post on their Web sites, so you probably won't use the SAS System Viewer every day. Even government agencies that process data with SAS have the ability to provide the data in almost any way you request - text, Excel, Access, dBASE so theoretically, all you have to do is call or e-mail with a request and they'll send it right out to you.\n\nBut if you're on deadline, sometimes you don't have that luxury of time, so you can at least view and convert SAS data files with a free addition to your CAR toolbox.\n\nContact Jeff Porter by e-mail at jeff@nicar.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-geometry-scripting-find-nearby-points","slug":"2003-novemberdecember-geometry-scripting-find-nearby-points","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-geometry-scripting-find-nearby-points.html","year":2003,"month":"November/December","monthNum":11,"headline":"Geometry, scripting find nearby points","kicker":"Tech tip","author_name":"Brad Heath","author_title":"","affiliation":"The Detroit News","summary":"A guide to using GIS software (ArcView 3.2) and its Avenue scripting language to analyze the spatial relationship between convicted sex offenders and day-care centers, specifically to determine how many offenders live within a mile of each facility and measure distances using coordinate geometry.","keywords":["GIS","geocoding","coordinates"],"topics":["Technology","Investigative Methods"],"technologies":["ArcView","Avenue","Access"],"full_text":"Your state is pondering a new law: No convicted sex offenders should live within a mile of a day-care center. But what's the context? To find out, you need to answer two questions: How many sex offenders actually live near each day-care center in your neighborhood and, for each one, exactly how far away is the nearest offender?\n\nYou know how to start answering these questions using your geographic information system (GIS) program - I used ESRI ArcView 3.2. You've already used geocoding to turn a data table of day-care center street addresses in your community into a point map, and did the same for a table of convicted sex offenders. So now all you have to do is count up which ones are near each other.\n\nHow?\n\nThat's where it gets interesting.\n\nArcView will need to go through and check each day-care center for the closest sex offender. And it will have to count how many offender points are within a mile of each center. That's simple to do by hand if you're only looking at one or two facilities. But what if you're looking at dozens, or thousands?\n\nArcView doesn't come with that ability. You have to build it yourself using ArcView's scripting language, called Avenue. Once you learn Avenue you can make ArcView do pretty much anything.\n\nIn this case, it's using coordinate geometry. To answer the questions, you'll need to fill out both themes with accurate X and y coordinates. Then you can use scripts or queries to search through the data to figure out which sex offenders are closest to the day-care centers.\n\nStep one is creating the coordinates. If you have a point theme in ArcView, odds are you're dealing in latitude and longitude - decimal degrees. While these are flexible, they aren't the best for measuring distance, so you need to put some coordinates in your data.\n\nStart by changing the projection of your view. Switch to the NAD 1983 projection for your state plane. (These projections are built in to ArcView 3.x.) Also, change the distance and map units to feet. Then open the attributes table for the theme you want to work on first. Start editing it and add two fields: \"ProjX\" and \"ProjY\". Set them up as numeric and be sure to allow for several decimal places. Close the table but don't hit the \"Stop Editing\" button yet.\n\nNow you're ready to add some coordinates. To do it, you'll need an Avenue script that cycles through each record and retrieves the X and y coordinates that correspond to the map projection you've chosen. The code looks like this:\n\ncurView = av.GetProject. FindDoc(\"mymap\"\nthmToUpdate = curView.FindTheme(\"shp\")\ntabToUpdate = thmToUpdate.GetFTab\nprjView = curView.GetProjection\n\nfldShape = tabToUpdate.FindField(\"Shape\")\nfldProjX = tabToUpdate.FindField(\"projx\")\nfldProjY = FindField\n\nfor each rec in tabToUpdate\n\ncurRect = tabToUpdate.ReturnValue(fldShape, rec)\n\nProjX = curRec.ReturnProjected(prjView).GetX\nProjY = curRec.ReturnProjected(prjView).GetY\n\ntabToUpdate.SetValue(fldProjX, rec, ProjX)\ntabToUpdate.SetValue(fldProjY, rec, ProjY)\n\nend\n\nAfter you enter the code, make sure you update it with the name of your view and theme. Then compile it with the checkmark icon and run it.\n\nNow back to your table. Look at the ProjX and ProjY fields and they should be filled in with numbers - the eastings and northings of your point. Instead of using latitude and longitude, these coordinates identify a point based on how many feet north and east it is from a predefined point on the map. Click \"Stop Editing\" and save the results. Then export the table you just finished and repeat the process for your second table. Import both into Access.\n\nThis leaves you with a database, two tables that lack a common field for joining. But Microsoft Access and other database managers will let you use more sophisticated relationships.\n\nInstead of using a join, you can link the points in the two tables using the Pythagorean distance formula, which can calculate the distance between any two points on a plane using their coordinates.\n\nHere's how you'd use Access SQL to link the tables based upon points that are at most a mile (5,280 feet) apart:\n\nWHERE sqr((((daycare.ProjX - offenders.ProjX)^2)+(daycare.ProjY- offenders.ProjY)^2)) <= 5280\n\nNow you can insert that into a SQL statement to answer the two questions.\n\nFirst: How many sex offenders live within a mile of each day-care center? You'll want a query that will check the data and count every point where the distance is less than 5,280 feet. Here's the SQL:\n\nSELECT daycare.Name, Count(offenders.Name) AS offenders_within_mile FROM daycare, offenders WHERE\n(((Sqr((((offenders.projx-daycare.projx)^2)+((offenders.projy-daycare.projy)^2))))<5280) AND ((daycare.projX) Is Not Null) AND ((offenders.projX) Is Not Null))\nGROUP BY daycare.Name;\n\nIt's important that your query filter out null values, which will appear in any records that you were unable to geocode. Even if only one record wasn't coded to a spot on the map, the null value will cause Access to choke.\n\nSecond: How far from each day-care center is the nearest sex offender? You'll want a query that, for each day-care center, chooses the minimum result from the distance calculation. Here's the SQL:\n\nSELECT daycare.Name, Min(Sqr((((offenders.projx-daycare.projx)^2)+(offenders.projy-daycare.projy)^2))) AS distance FROM daycare, offenders\nWHERE (((daycare.projx) Is Not Null) AND ((offenders.projx) Is Not Null))\nGROUP BY daycare.Name;\n\nIt's possible to handle these calculations with Avenue (and you don't even have to get Pythagorean to do it, because the language has a built-in distance function). But bringing it back into a database is easier, not to mention much faster. And once you've figured out how to do spatial math in Access, it's easy to change your analysis, ask different questions of your data or bring in other data, maybe the location of sex crimes or neighborhood police stations.\n\nWhat else can you do?\n\nWhy not find out how many sewer pipes dump out near your community's beaches? Or look at whether fires are happening close to the fire departments. You could add the distance calculations to an analysis of ambulance response times. And you could incorporate it into your Web site by creating searches that let users find the day-care centers closest to their homes.\n\nContact Brad Heath by e-mail at bheath@detnews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-readmetxt-car-tool-tip","slug":"2003-novemberdecember-readmetxt-car-tool-tip","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-readmetxt-car-tool-tip.html","year":2003,"month":"November/December","monthNum":11,"headline":"readme.txt (CAR tool tip)","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR is soliciting recommendations from Uplink readers for their favorite CAR (Computer-Assisted Reporting) tools to feature.","keywords":["tools","journalism","reporting"],"topics":["Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Have a favorite CAR tool that Uplink readers should know about? Drop us a note at uplink@nicar.org and tell us about it.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-upcoming-car-workshops","slug":"2003-novemberdecember-upcoming-car-workshops","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-upcoming-car-workshops.html","year":2003,"month":"November/December","monthNum":11,"headline":"Upcoming CAR Workshops","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering computer-assisted reporting workshops in 2004, including CAR boot camps, mapping data training, and advanced statistics workshops for journalists. Fellowships are available to help cover costs for qualified participants.","keywords":["journalism","training","workshops"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["Access","FoxPro","GIS"],"full_text":"Upcoming CAR Workshops\n\nComputer-Assisted Reporting Boot Camps\n\nThese unique seminars, taught by IRE and NICAR's experts, train journalists to acquire electronic information, use spreadsheets and databases to analyze the information and to translate it into high-impact stories. In addition, IRE and NICAR provide follow-up help when participants return to their news organizations. Four CAR Boot Camps are scheduled for 2004 in Columbia, Mo.\n\nJan. 4-9 March 21-26\nMay 16-26 Aug. 1-6\n\nMapping Data for News\n\nJan. 9-11 Columbia, Mo. Intensive hands-on training using mapping software for news stories will be offered by David Herzog, of NICAR and the Missouri School of Journalism, and Jennifer LaFleur, computer-assisted reporting editor at The Dallas Morning News. Participants are asked to have a basic knowledge in using relational database programs such as Access or FoxPro.\n\nAdvanced Statistics Workshop\n\nFeb. 13-15 Arizona State University Pulitzer Prize-winning journalists Steve Doig, interim director of the Walter Cronkite School of Journalism at Arizona State University, and Sarah Cohen, database editor of The Washington Post, team for this workshop. This session is aimed at strengthening the skills of reporters who want to move beyond basic CAR and use statistical analysis in their work. Reporters should know spreadsheet and database manager applications and have experience in CAR.\n\nFellowships\n\nA limited number of fellowships are available. Visit www.ire.org/training/fellowships.html for more information and an application.\n\nMore information is available at www.ire.org/training/bootcamps.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2003-novemberdecember-readmetxt-newsroom-intranets-tipsheets","slug":"2003-novemberdecember-readmetxt-newsroom-intranets-tipsheets","url":"https://dwillis.github.io/uplink/article/2003-novemberdecember-readmetxt-newsroom-intranets-tipsheets.html","year":2003,"month":"November/December","monthNum":11,"headline":"readme.txt (newsroom intranets tipsheets)","kicker":"","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"The IRE Resource Center offers multiple tipsheets with practical guidance on creating and managing effective newsroom intranets, covering topics from content strategy to technical setup and useful resources for journalists.","keywords":["intranet","journalism","resources"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"The IRE Resource Center has tipsheets offering ideas on how to create, develop and improve newsroom intranets. Here are just a few:\n\nRobert Gebeloff of The Star-Ledger in Newark, N.J., offers a primer on what newsroom intranets should include so journalists will use them. (Tipsheet No. 1770)\n\nLiz Donovan of The Miami Herald created a list of suggestions for directories, public records, news sources, people finders, searches and general reference data for newsroom intranets. (Tipsheet No. 1327)\n\nGeorge Landau of NewsEngin, Inc., provides tips and advice on setting up and running a newsroom intranet. (Tipsheet No. 895)\n\nIRE members can download copies of the tipsheets at www.ire.org/resourcecenter. You can order copies from the Resource Center at rescntr@ire.org or 573-882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-january-february-finding-files-for-mapping","slug":"2004-january-february-finding-files-for-mapping","url":"https://dwillis.github.io/uplink/article/2004-january-february-finding-files-for-mapping.html","year":2004,"month":"January • February","monthNum":1,"headline":"Finding files for mapping","kicker":"GIS DATA","author_name":"Jaimi Dowdell","author_title":"","affiliation":"IRE and NICAR","summary":"This article provides a guide for journalists seeking free GIS data, highlighting government portals like geodata.gov.","keywords":["GIS","Mapping","Databases"],"topics":["Technology","Data & Databases"],"technologies":["GIS","World Wide Web"],"full_text":"In the sea of resources on the Web, a great deal of geographic information system (GIS) data is available free for download. The catch is you just have to know how and where to find it.\n\nWhile general government agency Web sites like the U.S. Geological Survey (www.usgs.gov) and the U.S. Census Bureau (www.census.gov/geo/www/maps) contain plenty of GIS data ready for download, much of the data available on the Web is so widely scattered and hidden so deep within government agency holdings that it is difficult to know where to begin.\n\nTo help remedy this problem, the federal government has developed www.geodata.gov. As part of the Geospatial One-Stop initiative, this site is intended to serve as a portal for access to federal, state and local GIS data.\n\nThe intent of the portal is to","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":false}},{"id":"2004-january-february-data-shows-woes-for-middle-class-blacks","slug":"2004-january-february-data-shows-woes-for-middle-class-blacks","url":"https://dwillis.github.io/uplink/article/2004-january-february-data-shows-woes-for-middle-class-blacks.html","year":2004,"month":"January • February","monthNum":1,"headline":"Data shows woes for middle-class blacks","kicker":"SPOTLIGHT: CENSUS REPORTING","author_name":"David Mendell and Darnell Little","author_title":"demographics reporter","affiliation":"Chicago Tribune","summary":"Reporters from the Chicago Tribune discuss their strategy of using U.S. Census Bureau data to investigate human-interest stories regarding the middle class.","keywords":["Census","Demographics","Reporting"],"topics":["Demographics & Census","Data & Databases"],"technologies":["spreadsheet"],"full_text":"Almost any demographics reporter can attest to this: Poring over spreadsheet after spreadsheet of U.S. Census Bureau data can be a mind-numbing experience. And even if done with the keenest demographic eye, it often leads to a numbers-heavy story with little potential for readership.\n\nSo as the two-man, tag team of demographics/Census reporters at the Tribune, we decided to take the opposite tack: Look at a fascinating human phenomenon that seems crystal clear to us in the real world, and then turn to the numbers to see if they","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-january-february-white-collar-convicts-get-lighter-sentences","slug":"2004-january-february-white-collar-convicts-get-lighter-sentences","url":"https://dwillis.github.io/uplink/article/2004-january-february-white-collar-convicts-get-lighter-sentences.html","year":2004,"month":"January • February","monthNum":1,"headline":"White-collar convicts get lighter sentences","kicker":"CRIMINAL JUSTICE","author_name":"John Archibald","author_title":"","affiliation":"The Birmingham News","summary":"The Birmingham News investigates disparities in federal sentencing, specifically focusing on a case where a white-collar thief received a minimal sentence.","keywords":["Sentencing","Crime","Courts"],"topics":["Crime & Justice","Investigative Methods"],"technologies":[],"full_text":"The decision to take a long look at white-collar sentencing in North Alabama's federal courts was easy.\n\nThe late U.S. District Judge Edwin Nelson pretty much made the decision for us when he set Mari Sanders free.\n\nThough Sanders in 2001 was convicted of using an 11-month scheme to steal more than $187,000 from her bank employer and, though evidence revealed she had stolen more than $33,000 from a previous employer, Nelson sentenced her to eight hours in custody.\n\nThe 11th U.S. Circuit Court of Appeals ordered the judge to reconsider the sentence. But Nelson stuck to his guns, again sentencing her to eight hours.\n\nFederal courts reporter Val Walton had seen drug defendants go to prison for years. She also had repeatedly seen white-collar thieves like Sanders go","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-january-february-bits-and-bytes","slug":"2004-january-february-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2004-january-february-bits-and-bytes.html","year":2004,"month":"January • February","monthNum":1,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"A collection of brief reports covering editor training boot camps, patterns of teacher sick leave abuse, and updates to the Home Mortgage Disclosure Act database.","keywords":["Training","Education","Mortgages"],"topics":["Journalism & Newsroom","Investigative Methods","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"Journalists from around the globe come to Columbia, Mo., to learn computer-assisted reporting at boot camps sponsored by IRE and NICAR several times each year. Now editors and other newsroom managers can get into the CAR game on April 16-18 with an abbreviated version of the regular boot camp that is specifically tailored to their needs. Managers will learn how to make CAR succeed in their newsrooms. They'll get just enough hands-on experience to understand what their reporters and producers are doing, and a taste for what more is possible. They'll hear from others who have experience editing and guiding computer-assisted reporting. They'll also learn how to negotiate for electronic records. In just three days, editors will learn the skills that will allow them to enhance news coverage back home. For additional information and a registration form visit www.ire.org/training/bootcamps.html. Looking for training that's more specific to your newsroom? IRE and NICAR can arrange specialized training for news organizations inside or outside the United States. For more information e-mail training @ire.org or call (573) 882-2042. Does something about Valentine's Day make public school teachers ill? Sounds a bit unromantic, but producer Ron Regan of WEWS-Cleveland – with the help of IRE and NICAR Database Library analysts – found that Feb. 14 was a day many public school teachers called in sick. Based on the analysis of how public school teachers used their sick days, WEWS reported on patterns of abuse in its local school system and estimated the costs to taxpayers in the Cleveland area. Check out the full story at www.newsnet5.com/news/2656080/detail.html. For those who want to know more about home mortgage lending practices – including demographics of loan applicants – in their area, IRE and NICAR offer the 2002 update of the Home Mortgage Disclosure Act database. The IRE Beat Book \"Home Mortgage Lending\" by Jo Craven McGinty is a great guide for journalists who want to learn how to analyze the data and detect compelling news stories. Order the book online at www.ire.org/store/books or call (573) 882-3364. For more information about the HMDA data or to order contact the Database Library at (573) 884-7711 or (573) 884-7332.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-january-february-getting-ready-for-the-big-show","slug":"2004-january-february-getting-ready-for-the-big-show","url":"https://dwillis.github.io/uplink/article/2004-january-february-getting-ready-for-the-big-show.html","year":2004,"month":"January • February","monthNum":1,"headline":"Getting ready for the big show","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"NICAR and Missouri School of Journalism","affiliation":"IRE and NICAR","summary":"NICAR director David Herzog previews the upcoming Annual Computer-Assisted Reporting Conference in Cincinnati, detailing tracks for beginners, broadcasters, and advanced users.","keywords":["Conference","NICAR","Training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet"],"full_text":"This is a busy time of year for us here at IRE and NICAR, as we get ready for the Annual Computer-Assisted Reporting Conference. This year we're meeting in Cincinnati, where former Reds star Pete Rose, who is back in the news, barreled his way to countless head-first stolen bases. That means, by the time you read this, the staff around the IRE and NICAR offices will be charging just as hard to put together the best possible program for the journalists and journalism educators from around the globe who attend this event. We're also putting together a roster of hands-on classes — everything from spreadsheets to industrial-strength database managers – taught by some of the best journalists in the business. There's a beginner's track for newcomers who want to learn CAR that will include special panels and a mini-boot camp of hands-on classes. Another track, for broadcast journalists, will address the unique challenges of getting CAR stories on the air. On Thursday afternoon there will be pre-conference sessions for journalists with advanced CAR skills. There's a lot on tap for everyone, beginner to expert. If you haven't registered for the conference yet, now's the time. You can register by using the form on page 3.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-january-february-hands-on-car-training","slug":"2004-january-february-hands-on-car-training","url":"https://dwillis.github.io/uplink/article/2004-january-february-hands-on-car-training.html","year":2004,"month":"January • February","monthNum":1,"headline":"Hands-on CAR training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"An announcement of upcoming computer-assisted reporting workshops, boot camps, and sessions at the IRE Annual Conference.","keywords":["Bootcamp","Workshops","CAR"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet"],"full_text":"IRE and NICAR have a number of training opportunities coming up for journalists who want to learn more about computer-assisted reporting. IRE and NICAR will conduct intensive CAR Boot Camps for journalists who want to learn how to use spreadsheets and database manager to produce high-impact news stories. The dates are March 21-26, May 16-21 and Aug. 1-6 in Columbia, Mo. Journalists can also get hands-on CAR training at some of the Better Watchdog Workshops offered by IRE and the Society of Professional Journalists. Upcoming workshops with optional CAR sessions include Feb. 7-8 in Fullerton, Calif., and Feb. 21-22 in Austin, Texas. The IRE Annual Conference, June 17-20 in Atlanta, also will have plenty of CAR offerings. The optional CAR day (June 17) will include panels for beginners and experts alike, and IRE and NICAR will offer hands-on classes in spreadsheets and database managers throughout the conference.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-january-february-lovesick-teachers","slug":"2004-january-february-lovesick-teachers","url":"https://dwillis.github.io/uplink/article/2004-january-february-lovesick-teachers.html","year":2004,"month":"January • February","monthNum":1,"headline":"Lovesick teachers?","kicker":"","author_name":"","author_title":"producer","affiliation":"WEWS-Cleveland","summary":"A Cleveland television station analyzed public school teacher sick day records and discovered an unusual pattern of absences on Valentine's Day, revealing potential abuse of sick leave policies and associated costs to taxpayers.","keywords":["teachers","absences","investigation"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Does something about Valentine's Day make public school teachers ill? Sounds a bit unromantic, but producer Ron Regan of WEWS-Cleveland – with the help of IRE and NICAR Database Library analysts – found that Feb. 14 was a day many public school teachers called in sick.\n\nBased on the analysis of how public school teachers used their sick days, WEWS reported on patterns of abuse in its local school system and estimated the costs to taxpayers in the Cleveland area.\n\nCheck out the full story at www.newsnet5.com/news/2656080/detail.html.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-january-february-loans-updated","slug":"2004-january-february-loans-updated","url":"https://dwillis.github.io/uplink/article/2004-january-february-loans-updated.html","year":2004,"month":"January • February","monthNum":1,"headline":"Loans updated","kicker":"","author_name":" Home Mortgage Lending\" by Jo Craven McGinty is a great guide for journalists who want to learn how to analyze the data and detect compelling news stories. Order the book online at www.ire.org/store/books or call (573) 882-3364.\n\nFor more information about the HMDA data or to order contact the Database Library at (573) 884-7711 or (573) 884-7332.","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR have released an updated Home Mortgage Disclosure Act database for 2002, along with a guide by Jo Craven McGinty to help journalists analyze lending data and identify news stories.","keywords":["mortgages","database","lending"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":[],"full_text":"For those who want to know more about home mortgage lending practices – including demographics of loan applicants – in their area, IRE and NICAR offer the 2002 update of the Home Mortgage Disclosure Act database.\n\nThe IRE Beat Book \"Home Mortgage Lending\" by Jo Craven McGinty is a great guide for journalists who want to learn how to analyze the data and detect compelling news stories. Order the book online at www.ire.org/store/books or call (573) 882-3364.\n\nFor more information about the HMDA data or to order contact the Database Library at (573) 884-7711 or (573) 884-7332.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-january-february-delving-into-ancestry-with-sf4-data","slug":"2004-january-february-delving-into-ancestry-with-sf4-data","url":"https://dwillis.github.io/uplink/article/2004-january-february-delving-into-ancestry-with-sf4-data.html","year":2004,"month":"January • February","monthNum":1,"headline":"Delving into ancestry with SF4 data","kicker":"SPOTLIGHT: CENSUS REPORTING","author_name":"Paul Overberg","author_title":"","affiliation":"USA Today","summary":"Paul Overberg explains the power of Census Summary File 4 for profiling 336 distinct racial and ethnic groups, highlighting reporting examples and methodological constraints like sampling error.","keywords":["Census","SF4","Demographics"],"topics":["Demographics & Census","Data & Databases","Investigative Methods"],"technologies":["American FactFinder"],"full_text":"Think of the Census 2000 summary files (SF1 through SF4) as a choir that sings the story of your community.\n\nThe full choir is SF1, all the voices in simple melodies. SF3 is smaller, chosen from the full choir to simplify the director's task of blending representative voices in more complex melodies.\n\nBy design, some voices emerge at times from each choir to sing solo, or in counterpoint. These are the separate tabulations by race and Hispanic ethnicity. Their stories ring clearly.\n\nBut we know the choir has more distinct voices embedded in its harmony. If we could hear them alone, we could hear their stories clearly. We could even hear the subtle differences between the melodies of Welsh and Scots, Kiowa and Comanche, Bulgarians and Romanians, Lebanese and Syrians.\n\nHence SF2 and SF4. In these files, each voice can emerge to sing solo. Are you a good enough director to lead them?","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-january-february-recent-uplink-census-stories","slug":"2004-january-february-recent-uplink-census-stories","url":"https://dwillis.github.io/uplink/article/2004-january-february-recent-uplink-census-stories.html","year":2004,"month":"January • February","monthNum":1,"headline":"Recent Uplink Census stories","kicker":"","author_name":"Megan Clarke","author_title":"Database Library staff","affiliation":"IRE and NICAR","summary":"A bibliography of past Uplink articles detailing how journalists used Census data to report on race, poverty, and geography.","keywords":["Census","Index","Reporting"],"topics":["Demographics & Census","Investigative Methods","Data & Databases"],"technologies":["GIS"],"full_text":"Uplink regularly features articles that provide tips from journalists about using Census data. It also highlights stories that have used Census data analysis as a key element in the reporting. Here are some recent articles: \"Ticket data shows search rate gaps.\" Karisa King of the San Antonio Express-News recounts how she used city police traffic-stop and Census data to uncover disparities in search rates by race. (May-June 2003). \"Mapping the Census\" shows how journalists used geographic information system software to map Census data and discover patterns in poverty levels, non-English speaking areas and commutes. (November-December 2002). \"Children: tracking quality of life.\" Dave Davis of The (Cleveland) Plain Dealer explains how he used Census data along with Ohio health and environmental data to reveal how Cleveland children are living in worse conditions than other areas in Ohio and other major cities nationwide. (September-October 2003). \"Using Census files to link 1990 and 2000 geography.\" Dan Keating of The Washington Post explains one way to deal with Census geography shifts using tract-to-tract relationship files. Keating provides step-by-step guidance. (March-April 2003). \"Segregation in black, white, shades of gray.\" Holly Hacker of IRE and NICAR offers advice on using Census data to measure segregation and explore the racial make-up of cities, counties and other geographic areas. (May-June 2003). \"Plotting race by population.\" In the \"Mapping It Out\" feature, Uplink illustrates three journalists' stories that use Census data and GIS to map patterns of blacks and whites, looking for segregation and migration changes. (May-June 2002). \"Covering income, commuting with SF3.\" Jeff Porter, director of the IRE and NICAR Database Library, provides a guide to the Summary File 3 release from the 2000 Census and highlights some stories that used the data. (November-December 2002). Compiled by Megan Clarke, IRE and NICAR Database Library.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-january-february-showing-deadly-crashes","slug":"2004-january-february-showing-deadly-crashes","url":"https://dwillis.github.io/uplink/article/2004-january-february-showing-deadly-crashes.html","year":2004,"month":"January • February","monthNum":1,"headline":"Showing deadly crashes","kicker":"MAPPING IT OUT","author_name":"Garry Lenton","author_title":"Reporter","affiliation":"Patriot-News","summary":"Garry Lenton describes how The Patriot-News used the FARS database and GIS mapping to show that Cumberland County had the highest truck-accident death rate along Interstate 81.","keywords":["GIS","FARS","Trucking"],"topics":["Investigative Methods","Public Safety","Technology"],"technologies":["FARS","Access","ArcView"],"full_text":"Pennsylvania isn't called the Keystone State for nothing. It's hard to reach any location in the northeastern United States without crossing the state. And nowhere is this phenomenon more apparent than in the state's capital city, Harrisburg. This town is like the hub of a wheel, criss-crossed by three major highways, and railroads heading off in all directions. This fact is not lost on the trucking and warehousing industry, which has become one of the largest employers in the region. Truckers can reach every major market in the eastern United States within 10 hours. But, along with the high-paying jobs that the industry brought, is an unwelcome consequence: fatal crashes with 80,000-pound trucks at almost three times the national average. My editors wanted to get a handle on how our region compared to others when it came to fatal crashes with large trucks. So we turned to the U.S. Department of Transportation's Fatality Analysis Reporting System (FARS) database – purchased from the IRE and NICAR Database Library – for help. It took a while to get familiar with the tables, but Database Library staffers and the DOT helped us sift out the data we needed – the number of accidents involving tractor-trailer trucks that occurred in our region, and the death count. But the FARS data could only take us so far. Our editors were impressed, but not really sure what they were looking at. Frankly, I wasn't so sure myself. We needed to focus the story on something readers could understand. That's when George Ginter, the assistant city editor overseeing the project, chipped in: What's the most dangerous highway we have? The answer was easy: Interstate 81. The question then became, just how dangerous is it? And how do we communicate that to readers? The answer to the second question turned out to be a map. But before we could turn to ESRI ArcView 3.2, our geographic information system software (GIS), we had to go back to the FARS data. We aggregated the data in Microsoft Access to get a table that I could map. That took grouping the deaths along I-81 by the county of the accident. The result was the hook for the story. From 1997-2001 no county along the 855-mile highway had more deaths from accidents with tractor-trailers than Cumberland County, which is the heart of our circulation area. In our area, 40 percent of all people killed in crashes with a truck died on I-81. In the process of investigating the story we also found that the proportion of trucks on the highway was approaching 43 percent in some regions. That meant that the slightest error in judgment by a driver could result in a fatal accident with a truck. Motorists who used the highway told us that they hated it. Too congested, too many trucks. But they also told us something else: It was the passenger-car drivers they most often saw driving erratically, not the truckers. The data verified that and we interviewed the authors of studies showing that car drivers accounted for the 80 percent of all crashes between trucks and cars. The map helped us to bring that message home to readers most directly. The first step in the GIS analysis was to chart I-81's 855-mile course from the New York-Canada line into Tennessee. I used only three layers to build the map: roads, counties, and state boundaries – all files that came with ArcView. First, I mapped all the U.S. counties. This was overkill, but I later pared this down to just the counties that I-81 passes through. Next I added the roads shapefile. This turned the map nearly black with a web of lines representing everything from the simplest two-lane highway up to the interstates. Using the theme properties options, I filtered out everything but I-81. I learned that it helps to spend some time scrutinizing the road layer's data table first to see how the highway is identified in each state. Sometimes local authorities rename portions of highways, rendering the Name field useless as a filter. I tried filtering by Name and got just a dotted line for my trouble. But I-81 was named consistently in the Sign1 and Sign2 fields of the table. So I used this syntax to get the entire highway: ([sign1]=\"I81\") or ([Sign2j=\"I8i\") Now I had a single red line stretching across a mosaic of multi-colored counties. But what we wanted was a map of the highway and just the counties it passed through. I identified those counties by using the Theme\\Select by Theme menu item. I activated the roads layer and then used Select by Theme to have ArcView highlight each county feature that I-81 intersected. Then I opened the table for the county layer and hit the promote button on the tool bar. This moved all of the selected features to the top of the table. Now it was time to create a layer of the selected counties. I closed the data table and selected Theme\\Convert to Shapefile from the menu and added the new file to my view. Then I added the deaths data table to my project, joined it to the counties map and, using the legend editor, shaded each county along I-81 by its death count. For reference, I added the state shapefile and, in the theme properties, selected the six states that I-81 passes through. I exported the map to a PostScript file and e-mailed it to the newspaper's graphics editor. The end product was a centerpiece map graphic for the story, which attracted a lot of positive attention from readers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-january-february-measuring-migration-with-pums-data","slug":"2004-january-february-measuring-migration-with-pums-data","url":"https://dwillis.github.io/uplink/article/2004-january-february-measuring-migration-with-pums-data.html","year":2004,"month":"January • February","monthNum":1,"headline":"Measuring migration with PUMS data","kicker":"SPOTLIGHT: CENSUS REPORTING","author_name":"Ron Shawgo","author_title":"Census coordinator","affiliation":"The (Fort Wayne) Journal Gazette","summary":"Ron Shawgo explains how to use Public Use Microdata Sample (PUMS) data and the IPUMS website to analyze population migration and education trends.","keywords":["PUMS","Migration","Census"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["SPSS","SAS","Stata","Internet","World Wide Web"],"full_text":"If someone would have asked me a year ago what PUMS stood for I might have said a \"Pretty Ugly Mess of Statistics,\" with the meaning of the \"s-word\" negotiable. I've known for some time that the U.S. Census Bureau's Public Use Microdata Sample allowed users to dive deeper into the seemingly bottomless pit of Census data. But after three years as my newspaper's Census coordinator and writing numerous Census stories, I was ready to relegate PUMS to the \"Isn't That Nice\" pile. What more could be written? That changed when journalists like Robert Gebeloff of The (Newark N.J.) Star Ledger, Mark Houser of Pittsburgh's Tribune-Review and The Charleston Daily Mail's Brian Bowling mentioned stories using PUMS on the IRE and NICAR listservs. All three wrote about population migration, in particular about the loss of people to other states. Borrowing from their ideas, I later told our readers that Indiana had the largest estimated net loss of college-educated 25- to 34-year-olds of any state between 1995 and 2000. It would have been difficult to do the story without the Integrated Public Use Microdata Series (IPUMS) maintained on a user-friendly Web site by the University of Minnesota. More on that later. In barest terms, PUMS data contains responses from individual Census returns with identifying information such as names and addresses removed. The results are weighted so the data from a single return might represent 100 people. Journalists and other researchers unsatisfied with the tabulations the Census Bureau routinely releases can pick and choose the information they want for their own tabulations. Simply sum the number of people in the weighted column and you're on your way. PUMS comes in two flavors: 1 percent and 5 percent samples of the population. The 5 percent files are much larger and were just being released as I began my research. I chose the 1 percent sample for that reason and because it was available on the IPUMS Web site. Mindful of sampling error in the data I went no smaller than the state level for my analysis. The Web site, www.ipums.org, allows users to download extracts containing just the information they need, including figures for all states. This ability to compare states is important. Otherwise, journalists looking at state trends would have to download large files for all 50 states (51 including Washington D.C.) from the Census Bureau Web site. I don't have that kind of computer storage space. One caveat: IPUMS data file options are SPSS, SAS and Stata, all statistical programs. Anyone using the site should read the online documentation. IPUMS is a free service, but registration is required. After registering, you start the IPUMS download at the \"Create New Extract\" link. The page asks for your e-mail address and lets you chose the type of file you want (regular density for the 1 percent sample). Selecting \"Only the most commonly requested samples,\" will serve up the 2000 PUMS data as an option. Click \"Continue to Sample Selection.\" On this page I chose a rectangular file in SPSS format and checked the box for \"2000 1% Census PUMS.\" Click \"Continue to Variable Selection.\" This is where you'll choose the information in which you are interested. For my statewide migration story I chose STATEFIP (state FIPS code), PERWT (the weighted number of people for a given variable), AGE (I also checked the \"case selection\" box to the right), BPL (birthplace – general), EDUC99 (educational attainment), MIGRATE5 general (migration status five years ago) and MIGPLAC5 (State or country FIPS code of residence five years ago). I also was interested in income so I chose INCTOT (total personal income) and INCWAGE (wage and salary income). Click \"Continue to Case & Flag Selection.\" Here I chose \"Include only those persons meeting case selection criteria.\" Because I marked the case selection box for AGE on the previous screen, the Web site now gave me the option of selecting the ages I wanted. I chose 25 through 34. Click \"Continue to Extract Request Summary.\" Users are given the chance to make changes and write a brief description of the data before clicking \"Submit Extract Request.\" The data will be delivered to your e-mail account. It's been my experience that the data usually is delivered within 10 minutes. The e-mail includes a link to where the data can be downloaded. The files come zipped, so you'll need a program to expand them. I easily opened my data in SPSS. Because I'm not yet proficient in that program I then saved the data as dBASE file, which I imported into Microsoft Access. I ran a query to extract from the EDUC99 field those with at least a bachelor's degree. To find how many college-educated 25- to 34-year-olds left Indiana between 1995 and 2000, I filtered out Indiana in the STATEFIPS column but used it as the criterion in the MIGPLAC5 column. Summing the PERWT column gave me 58,887. To find how many entered the state, I did just the opposite. I used Indiana as the criterion in the STATEFIPS column but filtered it out of the MIGPLAC5 column. MIGPLAC5 assigns \"0\" to those not applicable folks. After filtering them out, summing the PERWT column gave me 41,782. I did this for every state, placed the results in Microsoft Excel and sorted on the difference. Indiana's estimated net loss of 17,105 was the largest. I used STATEFIPS (where people lived in 2000) and MIGPLAC5 (where they lived in 1995) and BPL (birthplace) to track the migration of native Hoosiers for a sidebar. I was able to compare education and income levels of native Hoosiers who still lived in Indiana with those who left the state. With the numbers crunched I was able to find native Hoosiers living in other states by tapping into alumni clubs listed online.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-january-february-readmetxt","slug":"2004-january-february-readmetxt","url":"https://dwillis.github.io/uplink/article/2004-january-february-readmetxt.html","year":2004,"month":"January • February","monthNum":1,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A detailed guide to understanding PUMS (Public Use Microdata Sample) data is available through IRE's Resource Center, with free access for members and paid options for others.","keywords":["PUMS","data","journalism"],"topics":["Data & Databases","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"For a detailed introduction to PUMS read Tipsheet No. 1614 by Richard O'Reilly of the Los Angeles Times, available at no cost to members at the IRE Resource Center Web site at www.ire.org/resourcecenter. Others can order a copy of the tipsheet by contacting the Resource Center at 573-882-3364 or rescntr@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-march-finding-missing-kids","slug":"2004-march-finding-missing-kids","url":"https://dwillis.github.io/uplink/article/2004-march-finding-missing-kids.html","year":2004,"month":"March","monthNum":3,"headline":"Finding 'missing' kids","kicker":"LAW ENFORCEMENT","author_name":"Marc Chase","author_title":"Reporter","affiliation":"The Times (Munster, Ind.)","summary":"A data investigation revealed that at least 10% of children listed as missing in a circulation area were not missing at all due to police oversight. This led to uncovering flaws in the state's system.","keywords":["missing","children","investigation"],"topics":["Investigative Methods","Data & Databases","Crime & Justice"],"technologies":["Excel","Internet","World Wide Web","Access"],"full_text":"At first glance, the Indiana Miss- ing Children Clearinghouse list showed that our circulation area led the state in the number of children reported missing by law enforcement agencies. The list also held the potential for a number of compelling stories about children who had been listed as missing for several years. But, as we tried to track down those compelling stories, we found the real story sur- passed our initial impressions. We found that at least 16 chil- dren, or 10 percent of those listed as missing from our re- gion, were not missing at all. And the city of Hammond likely had more children on the state's list than Indianapolis, which is larger, because Hammond de- tectives had failed to follow up with parents in those cases. What began as an attempt to turn around a quick computer- assisted reporting project led to a three-month investigation that revealed serious flaws in a statewide tool that is supposed to help locate missing children. The investigation began in July, with just a simple search of data made available on the Internet through the state's Web site. A link to the Indiana State Police Web site led to the Clear- inghouse list, containing the names of 467 children supposedly missing 30 days or more. The list, which was in fixed-width format, was easily imported into a Microsoft Excel spreadsheet for analysis. Four columns of information contained the names of the missing children, their birthdays, the dates they were reported missing and the agen- cies handling their cases. First, I sorted the data and kept only the records involving agencies within our circulation area, placing those in a separate spreadsheet. I used Microsoft Access to count the number of cases handled by each law enforcement agency in the state. That search showed two of the major cities in our readership area - Hammond and Gary - led the list in terms of the number of children re- ported missing by single agencies. At the very least, we could run a story showing that Hammond, with 56 cases, and Gary, with 60 cases, each had nearly twice as many children on the list as the state's largest city of India- napolis, which had 32 cases. Next, I created new columns in the statewide and regional spreadsheets and wrote formulas to calculate the number of years or months each child had been listed as missing. I also cal- culated their current ages and the ages at the time they disappeared. The re- sults showed cases of children who had been on the list for a number of years, including a Hammond boy listed as missing for 10 years following an alleged abduction in a custody dispute. After looking at the data, two lines of reporting emerged: one on why cit- ies in our region topped the list and the other on the stories of children who had been missing for a long time. Those areas of focus changed slightly following my first interview with the su- pervisor of the Hammond Police Department's juvenile division. During the interview, the police sergeant agreed to track down more information on the cases of Leslie Richardson, re- portedly missing for 10 years, and of Talion Mansfield, who had been listed as missing for two years. I was also told that if I wanted copies of the origi- nal missing children's reports filed by parents or guardians, I would have to file an open records request through the city's legal department. After the interview in Hammond I was driving back to our main office in Munster when my cell phone rang. The sergeant was on the other end and he explained to me that after making a couple of phone calls in the Richardson case, he deter- mined that the boy -reported missing at the age of 4 and now 14 - was no longer missing. It appeared, the sergeant said, that the boy's parents had never called back to report that the custody dispute leading to the report had been resolved. The sergeant added that parents have the responsibility to contact the authori- ties once children are found and police merely take the reports. Shortly after I returned to the office my cell phone rang again. This time the sergeant reported that the other case had inquired about involved a child who also was no longer missing and had not been for quite some time. Again, the sergeant blamed the original com- plainant on the report for failing to close out the case with the police. At this point, red flags regarding the list's accuracy and effectiveness be- gan waving furiously. In a ground- breaking series of stories that ran nearly 20 years ago, The Denver Post debunked inflated claims national child abduction totals. (For a copy of the Pulitzer Prize-winning stories contact the IRE Resource Center at 573-882- 3364 and ask for Story No. 1061). I filed public record requests for more than 100 missing children reports from Hammond and Gary and within three weeks began sifting through each one, gleaning contact numbers for parents and calling them one by one. I used people-finding tools on the Web to track down some of the parents who had moved or whose phone lines had been disconnected. That helped me find another 14 cases from the region involving children who had been listed as missing for years or months but who had actually only been missing for days or hours. In each case, parents said that the police had never contacted them for follow up after the parents filed their initial report. And in six of these cases, parents said they had called the police to tell them their chil- dren had been located. I also called parents whose children re- mained missing. Those parents reported that police had also failed to follow up in the cases of their children. We even fol- lowed one woman who walked the streets of Chicago's South Side, tack- ing posters of her daughter to utility poles because she was frustrated the police did nothing to help locate her daughter. Combing through all of the original re- ports helped identify other angles for the series. Runaways accounted for most of the cases on the list. That be- came the main angle for part two, which outlined the problems and profiled three chronic runaway children. For three, we focused on a Hammond woman whose daughter had been ab- ducted by her ex-husband and taken to Egypt in 1993, never to return. The impact of the story was felt al- most immediately. Before the series even ended, local state legislators called upon the governor and the state police for a probe of the Clear- inghouse list and its inaccuracies. The matter remains under review, and a northwest Indiana congress- man and a state lawmaker are con- sidering legislation that would require police agencies to regularly follow up on missing children's cases. Contact Marc Chase by e-mail at mchase@nwitimes.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-survey-helps-explain-heart-attack-survival","slug":"2004-march-survey-helps-explain-heart-attack-survival","url":"https://dwillis.github.io/uplink/article/2004-march-survey-helps-explain-heart-attack-survival.html","year":2004,"month":"March","monthNum":3,"headline":"Survey helps explain heart attack survival","kicker":"SPOTLIGHT: EMERGENCY RESPONSE","author_name":"Robert Davis","author_title":"","affiliation":"USA Today","summary":"An investigation into varying cardiac arrest survival rates across cities highlighted how some save substantially more victims due to differing emergency response capabilities.","keywords":["heart attack","survival","emergency"],"topics":["Health & Environment","Investigative Methods"],"technologies":[],"full_text":"In the early 1990s, a discrepancy between big city survival rates for cardiac arrest cases began to come into focus in medical journals - some cities saved 10 times more victims than others.\n\nAs a back-burner issue project that I pursued amid daily stories, I began looking for ways to detail those differences. Whenever I spoke with an emergency room physician, fire department official or cardiology researcher for a story, I asked them about the discrepancies.\n\nI began to see that the officials explained away the differences with reasons they usually said were out of","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-march-making-the-connection-between-doctors-ods","slug":"2004-march-making-the-connection-between-doctors-ods","url":"https://dwillis.github.io/uplink/article/2004-march-making-the-connection-between-doctors-ods.html","year":2004,"month":"March","monthNum":3,"headline":"Making the connection between doctors, ODs","kicker":"PRESCRIPTION DRUGS","author_name":"Fred Schulte","author_title":"Reporter","affiliation":"South Florida Sun-Sentinel","summary":"A series exposed Florida doctors prescribing large quantities of narcotics to low-income Medicaid users, leading to high financial costs and overdose deaths.","keywords":["prescription","overdose","doctors"],"topics":["Data & Databases","Health & Environment","Investigative Methods"],"technologies":["Access","CD-ROM"],"full_text":"In the recent newspaper series \"Drug- ging the Poor,\" I tried to expose not only the human tragedy of prescription drug abuse, but also the huge financial cost. The series revealed that a small group of Florida doctors have pre- scribed large quantities of narcotic painkillers and other addictive drugs to low-income people on Medicaid, costing taxpayers hundreds of mil- lions of dollars and adding to a tor- rent of overdose deaths in the state. The project linked some of the state's top prescribing Medicaid doctors to overdose deaths or histories of pro- fessional misconduct. It also uncov- ered millions of dollars in suspect bill- ings for drugs, including claims gen- erated using the license numbers of doctors who were dead or otherwise ineligible to write prescriptions. Several databases, some created from paper. records readily available to the public and others from data made public for the first time, provided key information for the reporting. First, the paper records. I obtained nearly 3,000 drug over- dose autopsy case files, most of them from 2001 and 2002, from Florida's 24 medical examiner dis- tricts. I examined each file, looking for the name of the treating physician of the deceased. Some files con- tained \"medication inventory\" sheets, which logged all of the pills found at the death scene with the name of the doctor, drug, the pharmacy that filled the prescription and the date filled, but others didn't. I typed the names of victims, doctors and drugs into a Microsoft Access da- tabase on my laptop. It was often as frustrating as it was time consuming. In many cases, pills were found at the death scene in bottles, but no- body recorded the name of the doc- tor who had prescribed them, for in- stance. After visiting morgues for several months my death file grew and a pat- tern emerged: each examiner's dis- trict had a handful of doctors with ab- normally high numbers of deaths that had gone unnoticed. I wound up with more than 40 doc- tors with four or more patient deaths for 2001 and 2002. Sixteen doctors had prescribed pills found at the scenes of eight deaths or more, mostly in that same two-year period. Some had deaths spanning several medical examiner districts. Nobody had done this type of re- search before, even though state medical licensing officials believe a pattern of overdose deaths in a doctor's medical practice warrants investigation. Nor had anyone matched doctors with high numbers of patient deaths against Medicaid prescription drug billing data. I got the chance to do that after Florida At- torney General Charlie Crist agreed to make the billing records public at no cost. After some negotiation, Medicaid fraud unit investigators sent me a Microsoft Access database on CD- ROM. It contained data for 8.8 mil- lion Medicaid pharmacy claims, which amounted to more than $346 million in drug billings for 2000-2002. The database contains two tables, one that was 1.49 GB, the other slightly smaller. The data included sales of all \"controlled\" drugs - 844 varieties of narcotics and other medications. Each drug has a five-character code that serves as a unique identifier. The drugs range from brand name painkillers like OxyContin that cost Medicaid more than $7 a pill to generic tranquilizers and sleeping pills that cost just pen- nies per dose. I spent many hours rummaging through this huge data set. The database contains fields for the name of the prescribing doctor, license number of doctor, year of the prescrip- tion, county of the doctor, amount paid, number of Medicaid recipients treated and the pharmacy that billed for the drugs. Initial queries using Access showed fewer than 3 percent of the state's doctors were prescribing two- thirds of the narcotics dispensed to people on Medicaid. Five doctors had prescribed $2 million or more worth of narcotics over three years, much of it OxyContin. I spent many hours rummaging through this huge data set, doing queries and sorting results to find other patterns. Scrolling through, I spotted the name of a doctor I knew had died years ago and that gave me the idea to see if others who were deceased or had their licenses re- voked were still generating phar- macy bills. They were. When I matched the top billing phar- macies with prescribing doctors, I no- ticed that a small group of pharma- cies had derived most of its Medic- aid billings from prescriptions written by one or two doctors, a situation state officials later said raised the possibility of kickback schemes. But the most significant finding came from checking the top prescribing doctors against the overdose data- base I created from autopsy and po- lice reports. It turned out that many, though not all, of the highest Medicaid prescribing doc- tors were the same ones with abnormally high numbers of patient deaths. For ex- ample, 16 of the top prescribers had a combined 61 patient overdoses result- ing in death in two years. Many of these doctors also had been disciplined previously by the state for offenses that included drug traffick- ing and trading drugs for sex. I used the Florida Department of Health's online physician profiles and old binders with documents summa- rizing disciplinary actions taken against doctors. The profiles include a cat- egory for \"deceased,\" and helped to identify dead doctors who were still generating bills. The profiles and the CD order form can be found on the Web at www.doh.fl.us/MQA. I also used the Internet-based Medi- care and Medicaid exclusion list and compared by hand every Florida doc- tor who had been kicked out of Med- icaid but still generated billings. (See http://exclusions.oig.hhs.gov/ search.html.) Another data set that proved invalu- able came from the Florida Agency for Health Care Administration, which oversees Florida's $12 billion- a-year program. I e-mailed a list of the codes for all of the controlled drugs to the agency's Medicaid pharmacy man- ager, and he had the state's Medicaid contractor provide at no cost a Microsoft Excel spreadsheet with the drug, doses purchased, cost of drug and number of units for 2000 through September 2003. This data showed that dosage levels of certain drugs, in- cluding Xanax bars and methadone, had soared. A project like this can be done in states that grant access to autopsy reports, though getting the Medicaid data might take some tough negoti- ating and even a lawsuit. It's worth the expense and effort. Pre- scription drug overdose deaths are soaring nationwide and in many states authorities believe Medicaid helps fuel the illegal street market in these pills. States facing huge cost increases in Medicaid simply can't afford to allow fraud schemes to go unchecked. Here are some tips for taking the re- porting beyond the data analysis: Contact the families of people who die. While many won't welcome a call from a reporter, others will help you get documentation such as pharmacy receipts showing the dates victims were prescribed pills and the amounts. Seek the advice of legitimate pain management doctors and pharma- cists at their offices or stores. Many will be helpful. Whenever possible, review the complete file from a medical examiner's death investigation. Many times an investigator's notes will have the phone number or comments from the doctor who prescribed the drugs. Phone numbers also are criti- cal in making sure you have the right physician. Fred Schulte can be contacted by e- mail at fschulte@sun-sentinel.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-bits-and-bytes","slug":"2004-march-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2004-march-bits-and-bytes.html","year":2004,"month":"March","monthNum":3,"headline":"Bits & Bytes","kicker":"INSIDE NICAR","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists can access updated CPSC injury databases and explore new investigative techniques like social network analysis, with examples including consumer product safety investigations and Medicaid fraud in pediatric dentistry.","keywords":["database","investigation","journalism"],"topics":["Data & Databases","Investigative Methods","Journalism & Newsroom"],"technologies":["Access","FoxPro","9-track tape"],"full_text":"Fresh consumer product data\n\nHow many people did space heaters or turkey fryers hurt in the past year? Journalists can use the U.S. Consumer Product Safety Commission Injury and Death database, recently updated by the IRE and NICAR Database Library, to investigate these and other products.\n\nThe database consists of four tables provided by the commission's National Injury Information Clearinghouse. The CPSC maintains this data so it can alert the public to potential hazards associated with consumer products. Fields include age, race and gender of the victim, state and city of the incident, product type involved and more. To order the database, call the Database Library at 573-884-7711 or go online to www.ire.org/datalibrary/databases/cpsc.\n\nTangled webs\n\nIRE and NICAR are beginning to more deeply explore social network analysis for visualizing and diagramming relationships between individuals, businesses and institutions. It's a new approach for journalists, but business consultants, law enforcement agencies and public health investigators have used this approach for decades. In 2003, The Washington Post used social network analysis to report on a group of Los Angeles-area language schools that appeared to be \"visa mills\" for foreign students. For more information and a list of Web links and tipsheets on this emerging technique, go to www.ire.org/sna.\n\nBaby dentistry\n\nThe IRE and NICAR Database Library worked with WCNC-Charlotte, N.C. to analyze Medicaid data for a story that aired in December. The Database Library transferred tapes of Medicaid billing records to Microsoft Access and FoxPro and identified pediatric dentists spending millions of dollars of taxpayer money on unneeded root canals in babies. The analysis found that the main metropolitan Medicaid centers in North Carolina treated fewer than 10 percent of the kids on Medicaid in the state, but performed almost half of the baby root canals. To view the story, go to www.wcnc.com/news/topstories/stories/wcnc-121803-al medicaid.b4d99017.html. The series won a Scripps Howard Foundation national journalism award for excellence in electronic media, large market television/cable. For more information on how IRE and NICAR can help your news organization with data analysis, contact the Database Library at 573-884-7711.\n\nNews resources\n\nWhen news breaks, journalists can find help at IRE's \"In the News\" Web pages. The special sections list useful databases, publications and Internet sites that can add depth to deadline reporting. When a tanker truck fell from an overpass above I-95 near Baltimore, IRE and NICAR suggested several databases from the U.S. Department of Transportation containing truck accident and hazardous material release reports. To learn more about this and other resources available, visit www.ire.org/inthenews_archive","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-march-fresh-consumer-product-data","slug":"2004-march-fresh-consumer-product-data","url":"https://dwillis.github.io/uplink/article/2004-march-fresh-consumer-product-data.html","year":2004,"month":"March","monthNum":3,"headline":"Fresh consumer product data","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists can access the U.S. Consumer Product Safety Commission's Injury and Death database through IRE and NICAR to investigate injuries caused by consumer products like space heaters and turkey fryers. The database includes detailed victim information and incident data to help identify product-related hazards.","keywords":["database","injuries","products"],"topics":["Data & Databases","Investigative Methods"],"technologies":["CD-ROM"],"full_text":"How many people did space heaters or turkey fryers hurt in the past year? Journalists can use the U.S. Consumer Prod- uct Safety Commission Injury and Death database, recently updated by the IRE and NICAR Database Library, to investigate these and other products. The database consists of four tables provided by the commission's National Injury Information Clearinghouse. The CPSC maintains this data so it can alert the public to potential hazards associated with con- sumer products. Fields include age, race and gender of the vic- tim, state and city of the inci- dent, product type involved and more. To order the database, call the Database Library at 573- 884-7711 or go online to www.ire.org/datalibrary/data- bases/cpsc.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-tangled-webs","slug":"2004-march-tangled-webs","url":"https://dwillis.github.io/uplink/article/2004-march-tangled-webs.html","year":2004,"month":"March","monthNum":3,"headline":"Tangled webs","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE and NICAR are exploring social network analysis as a tool for journalists to visualize relationships between individuals, businesses, and institutions, a technique long used by law enforcement and other professionals. The Washington Post demonstrated its effectiveness in 2003 when investigating visa mill language schools.","keywords":["analysis","relationships","journalism"],"topics":["Investigative Methods","Technology"],"technologies":["World Wide Web"],"full_text":"IRE and NICAR are beginning to more deeply explore social network analysis for visualizing and diagramming relationships between individuals, businesses and institutions. It's a new approach for journalists, but business consultants, law enforcement agencies and public health investigators have used this approach for decades. In 2003, The Washington Post used social network analysis to report on a group of Los Angeles-area language schools that appeared to be \"visa mills\" for foreign students. For more information and a list of Web links and tipsheets on this emerging technique, go to www.ire.org/sna.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-march-getting-set-for-election-coverage","slug":"2004-march-getting-set-for-election-coverage","url":"https://dwillis.github.io/uplink/article/2004-march-getting-set-for-election-coverage.html","year":2004,"month":"March","monthNum":3,"headline":"Getting set for election coverage","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"This article guides journalists on using computer-assisted reporting (CAR) to cover 2024 elections, highlighting campaign finance data analysis as a key resource. It promotes IRE's campaign finance resources, training opportunities, and a comprehensive beat book to help reporters uncover compelling political stories.","keywords":["elections","campaign","reporting"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["World Wide Web"],"full_text":"This is a big year for election news, with the U.S. House, Senate and presi- dential races going full force. Oppor- tunities abound for using computer-as- sisted reporting. If you're new to CAR the elections are a great time to sink your teeth into cam- paign finance data and see where the candidates draw their financial support. If you've been doing CAR for a long time, there are plenty of compelling stories that look deeper into campaign finance, voter histories and election results. Starting with this issue of Uplink, we'll offer articles that will help you cover elections and politics throughout the election season. Check out page 3 for an account of how Robert David Sullivan of CommonWear magazine in Boston created a map of distinct U.S. political areas. We'll have even more in the May-June Uplink. As the campaign unfolds, go online to IRE's Campaign Finance Information Center at www.campaignfinance.org for resources, data and links to stories. If you want to get a jump start on your elections and politics coverage, you'll want to get \"Unstacking the Deck: A Reporter's Guide to Campaign Fi- nance,\" an IRE Beat Book by veteran journalists Michael A. Weber, Aron Pilhofer and Derek Willis. For more information about the book visit www.ire.org/store/books/campaign.html You can also sit in on campaign finance panels and get hands-on CAR training June 17-20 in Atlanta. Keep an eye on www.ire.org/ training/atlanta0410 more information. Contact David Herzog by e-mail at dherzog@nicar.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-upcoming-hands-on-car-training","slug":"2004-march-upcoming-hands-on-car-training","url":"https://dwillis.github.io/uplink/article/2004-march-upcoming-hands-on-car-training.html","year":2004,"month":"March","monthNum":3,"headline":"Upcoming hands-on CAR training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer computer-assisted reporting (CAR) training programs including weeklong boot camps in Columbia, Missouri and hands-on classes at their annual conference, designed for journalists at all levels to learn data analysis and electronic information gathering skills.","keywords":["training","journalism","databases"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet"],"full_text":"IRE and NICAR have several train- ing opportunities for journalists seeking hands-on instruction in using computer-assisted reporting. There are two weeklong CAR boot camps in Columbia, Mo., for jour- nalists who want to learn how to acquire electronic information, use spreadsheets and databases to ana- lyze the information and to translate that information into high-impact sto- ries. IRE and NICAR provide follow- up help after participants return to their news organizations. The boot camps are May 16-21 and Aug. 1-6. Editors can take advantage of an abbreviated version of the boot camp April 23-25 in Columbia, Mo. Participants will learn how to make CAR successful in their newsrooms and will get just enough hands-on experience to understand what their reporters are tackling and en- vision what is possible with CAR. Hands-on classes will be offered at the IRE Annual Conference June 17-20 in Atlanta. Those registering for the classes will be able to learn how to use spreadsheets and da- tabase managers to analyze data.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-mapping-a-new-look-at-presidential-results","slug":"2004-march-mapping-a-new-look-at-presidential-results","url":"https://dwillis.github.io/uplink/article/2004-march-mapping-a-new-look-at-presidential-results.html","year":2004,"month":"March","monthNum":3,"headline":"Mapping a new look at presidential results","kicker":"ELECTIONS","author_name":"Robert David Sullivan","author_title":"","affiliation":"CommonWealth","summary":"CommonWealth magazine created 'Beyond Red & Blue,' an interactive report that maps 10 distinct political regions across the United States by analyzing county-level election data and Census information. The project involved compiling presidential election results into Excel spreadsheets and using GIS mapping software to identify voting patterns that cross traditional state boundaries.","keywords":["elections","mapping","analysis"],"topics":["Government & Politics","Data & Databases","Technology"],"technologies":["Excel","ArcView"],"full_text":"\"Beyond Red & Blue,\" an exclusive re- port on CommonWealth magazine's Web site, takes a fresh look at presi- dential elections by identifying 10 dis- tinct political regions that cross state boundaries. Drawing the new politi- cal map of the U.S. required elections and Census data, mapping and spreadsheet analysis and some judg- ment calls. The project evolved from \"The Lay of the Land,\" a story about the political regions of Massachusetts. That story, by comparison, was a no-frills affair. I typed election results for all of the 351 cities and towns in Massachu- setts into Microsoft Excel, and then assigned each city and town to one of 10 regions with almost equal vot- ing strength. For \"Beyond Red & Blue,\" I did the same for the 3,117 counties in the United States and used ESRI ArcView 3.3 geographic information system (GIS) to create detailed col- ored maps that later were turned into graphics for the report on our Web site. (See www.massinc.org/com- monwealth/new_map_exclusivel beyond_red_blue.html). Data into Excel As I built an Excel master data file with the names of the U.S. counties I added elections data collected from Web sites and printed sources. in each state, the Secretary of State's office - or which- ever agency oversees voting - posts this information on the Internet. In many cases you can download or cut and paste data directly into Excel. (See www.politics1.com/states.htmfor links to all 50 states.) Other great sources of data are Dave Leip's Atlas of U.S. Presidential Elec- tions (www.uselectionatlas.org) and the \"America at the Polls\" books ed- ited by Richard M. Scammon. The first column of the spreadsheet listed the county and state like this: \"Jefferson, AL,\" \"Jefferson, AR,\" to dis- tinguish between counties having the same name in different states. In the adjacent columns, I entered how each county voted in every presiden- tial election since 1976. (This was not as time-consuming as it sounds. ] of- ten used Excel's \"fill handle\" to mark all the counties in a state as Demo- cratic or Republican, then went back and noted the exceptions, which were usually few.) I then sorted the results and used for- matting tools to color the counties ac- cording to which party carried them in the close presidential elections of 1976 and 2000. I ended up with four catego- ries: Democratic both times (blue), Republican both times (red), Demo- cratic then Republican (yellow), and Republican then Democratic (green). That was a good starting point, and I used it to get a preliminary picture of the 10 regions. I added another col- umn to the spreadsheet and assigned each county to one of the 10 regions, basing the decision not only on the voting history but also on the results of certain state elections, and geo- graphic factors. To come up with 10 regions that had an equal numbers of voters (as far as that was possible), I added another column to the spreadsheet that in- cluded the total number of votes cast in each county during 2000. Using Excel, I was able to automati- cally add up all the votes cast in every county labeled as being part of, for example, the Appalachia region that I had defined. That way, whenever I re- assigned a county from one region to another, I immediately saw how the change affected the totals for both re- gions. By keeping an eye on which re- gions were significantly larger or smaller than 10 percent of the elector- ate, I was able to keep redrawing the boundaries so that none of them got too large or small. At this point, I got greedy for more data. I wanted to use demographic factors to further refine the 10 regions. So I created separate Excel spreadsheets for criteria such as age, family income, and population growth. I downloaded county-level data for all of these fac- tors from the U.S. Census Web site (www.census.gov). Smart linking I also linked all of these new spread- sheets to my original spreadsheet through the VLOOKUP function. Thus, each row of data in my Census spreadsheets included a county name and state, and I told Excel to find the equivalent county name and state in my master data file, then plug in the right region names in the new spreadsheet. As in the master data file, I used the SUMIF function - and simple calcula- tion formulas to get raw Census num- bers and percentages for each region. Because all the spreadsheets were linked to the master data file I only had to reassign a county from one region to another on my original spreadsheet to change all the spreadsheets. If I looked at the spreadsheet for edu- cational attainment and discovered that the demographics in an Appalachian county more closely matched the Farm Belt region, I would change the county's designation in the master data file, and it would automatically change in all the other Excel files, producing new regional totals in each of them. Because of this system I didn't have to make any final decisions as to which counties were in which regions until the closing days of the project. I could keep tweaking the regions as I discovered new data. Mapping regions I turned to ArcView late in the project, and I did not use the mapping soft- ware to make any new calculations. I began by using a blank map that showed boundaries for all of all the counties in the United States, a map that came with ArcView. That map includes a data (or \"attributes\") file that includes standardized codes (known as FIPS codes) assigned to each county. I then added the FIPS codes to the master data file, which I saved as a dBASE file in order to import it into ArcView. Using a method in ArcView similar to Excel's VLOOKUP function, I then linked the dBASE file to the at- tributes file in such a way that all of the counties were automatically as- signed to the proper regions. Then I created a legend that assigned a dif- ferent color to each region, and I used the \"unique value\" function to pro- duce, with just a couple of keystrokes, a map that resembles the final \"Be- yond Red & Blue\" map. I wasn't quite done, however, be- cause the ArcView map exposed cer- tain aesthetic flaws, such as regions with fingers and tentacles made up of sparsely populated counties. I then went back and made some final ad- justments to the master data file, making the regions look a bit more coherent. I decided that those odd shapes and extensions are justified only when significant numbers of vot- ers are involved. (For example, I pushed the El Norte region up to in- clude San Antonio's Bexar County be- cause it's a major urban area with some 400,000 voters. I did not expand the Big River region to include several Democratic-leaning counties on the Upper Peninsula of Michigan because they include too few voters to be much of a factor in statewide elections.) Af- ter I made these final corrections to the master data file, I saved it as a dBASE file and imported it into ArcView. To create separate ArcView maps for each region, I simply copied the na- tional map 10 times. Each time, I changed the legend so that only one region appeared up in color, while the other nine were left invisible. I also imported other ArcView data files to add state lines and major cities. The project took a couple of months while 1 worked on regular duties. The finished product, posted on our Web site in mid-December, includes close- up maps for each region and a set of tables showing the political and demo- graphic characteristics for each. Next comes what I consider to be the fun part. When the results come in for the 2004 election, I hope to redraw the \"Beyond Red & Blue\" map to reflect any changes in voting patterns. And all of my old data will be automatically up- dated to reflect these changes, with very little effort on my part - leaving more time for writing. Contact Robert David Sullivan by e-mail at rsullivan@massinc.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-baby-dentistry","slug":"2004-march-baby-dentistry","url":"https://dwillis.github.io/uplink/article/2004-march-baby-dentistry.html","year":2004,"month":"March","monthNum":3,"headline":"Baby dentistry","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR's Database Library partnered with WCNC Charlotte to analyze Medicaid data, uncovering that pediatric dentists were performing unnecessary root canals on babies at disproportionately high rates in major metropolitan areas. The investigation won a Scripps Howard Foundation national journalism award for excellence in electronic media.","keywords":["Medicaid","dentistry","investigation"],"topics":["Data & Databases","Health & Environment","Investigative Methods"],"technologies":["Access","FoxPro"],"full_text":"The IRE and NICAR Database Library worked with WCNC- Charlotte, N.C. to analyze Med- icaid data for a story that aired in December. The Database Li- brary transferred tapes of Med- icaid billing records to Microsoft Access and FoxPro and identi- fied pediatric dentists spending millions of dollars of taxpayer money on unneeded root ca- nals in babies. The analysis found that the main metropoli- tan Medicaid centers in North Carolina treated fewer than 10 percent of the kids on Medicaid in the state, but performed al- most half of the baby root ca- nals. To view the story, go to www.wcnc.com/news/topstories/ stories/wcnc-121803-al medicaid.b4d99017.html. The series won a Scripps Howard Foundation national journalism award for excellence in electronic media, large market television/ cable. For more information on how IRE and NICAR can help your news organization with data analysis, contact the Database Library at 573-884-7711.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-news-resources","slug":"2004-march-news-resources","url":"https://dwillis.github.io/uplink/article/2004-march-news-resources.html","year":2004,"month":"March","monthNum":3,"headline":"News resources","kicker":"","author_name":"Art Golab","author_title":"","affiliation":"Chicago Sun-Times","summary":"IRE's 'In the News' web pages provide journalists with useful databases, publications, and internet resources to add depth to breaking news stories, such as DOT databases for reporting on accidents like the tanker truck incident near Baltimore.","keywords":["journalism","databases","resources"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":["Internet","World Wide Web"],"full_text":"When news breaks, journalists can find help at IRE's \"In the News\" Web pages. The special sections list useful databases, publications and Internet sites that can add depth to deadline reporting. When a tanker truck fell from an overpass above I- 95 near Baltimore, IRE and NICAR suggested several da- tabases from the U.S. Depart- ment of Transportation contain- ing truck accident and hazard- ous material release reports. To learn more about this and other resources available, visit www.ire.org/inthenews_archive ","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-first-venture-principals-cash-in-on-sick-pay","slug":"2004-march-first-venture-principals-cash-in-on-sick-pay","url":"https://dwillis.github.io/uplink/article/2004-march-first-venture-principals-cash-in-on-sick-pay.html","year":2004,"month":"March","monthNum":3,"headline":"FIRST VENTURE Principals cash in on sick pay","kicker":"","author_name":"Art Golab","author_title":"Education reporter","affiliation":"Chicago Sun-Times","summary":"Chicago Sun-Times reporters investigated how Chicago Public Schools teachers and principals could legally collect up to 315 days of unused sick pay in their final year, with one principal earning $247,000 in his last year through this practice. The reporters used computer-assisted reporting techniques learned at an IRE/NICAR boot camp to analyze 159,000 salary records and uncover this little-known contractual provision that also allowed sick days to be counted toward pension calculations.","keywords":["salary","investigation","pension"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Access","CD-ROM"],"full_text":"Elementary school principal Anthony Forst Jr. was rarely, if ever, sick in his 35-year career with the Chicago Public Schools.\n\nBut he collected $129,000 in sick pay his last year, bringing his total salary to $247,000 - significantly more than the mayor or governor makes. And it was all legal.\n\nThat $247,000 figure appeared on the top of one of the first sorts of teacher salary data Sun-Times education reporter Rosalind Rossi and I received from the state.\n\nIt led us to a story detailing how, under a little-known provision in their contract, Chicago teachers and principals can collect up to 315 days of unused sick pay in their last year of service, a practice unheard of in private industry.\n\nWe also found that 244 of those sick days could be added to their service time for pension purposes, a double-dip that was unheard of even in government. As a result, principals could get bigger pensions.\n\nRossi and I were relatively new to this sort of thing, as our paper had just lost both of our computer-assisted reporting specialists and decided to hurl us into the breach.\n\nFortunately, the paper sent us to an IRE and NICAR Boot Camp where we learned the tricks of the CAR trade. We were much better prepared to tackle the story than if we had tried to teach ourselves.\n\nAs a warm-up, we did the annual school report card, which involved mostly importing school test score data into Microsoft Access and formatting it for a special section. It was mind-numbing work, importing and boiling down 11,000 columns of data for 3,900 schools. But it helped hone our CAR skills.\n\nWe had to learn to select and import large chunks of delimited text files and make sure we assigned the correct heading to each column of data. Then we found ways to check its accuracy. We also learned to link the tables we had generated to tables that had been created by another reporter the year before.\n\nWe ran queries that calculated the difference in school scores between the years for several grade levels in several subjects.\n\nIn the course of all this work and while we were going back and forth with the State Board of Education on discrepancies in the data (the boot camp had prepared us well to expect this), Rossi thought to ask them if the latest teacher salary data was available.\n\nIt was, and it arrived on a CD by courier the next day, free of charge. When we finished the report card, we felt ready for anything, and decided to tackle the teacher salary data, which was technically a lot simpler.\n\nNow the challenge was to get something interesting out of this data containing 159,000 records of the salaries of every teacher and school administrator in the state.\n\nThe teacher pay story had already been done to death.\n\nThe previous year our paper had written about the best-paid high school teachers and what they did to earn their salaries.\n\nA Sun-Times investigative reporter no longer with the paper took it up a notch, doing the definitive story on school superintendent pay. He revealed that top-paid superintendents were getting retirement incentives that had the potential to cripple the taxpayer-backed state pension fund.\n\nOur competition, the Chicago Tribune, borrowed that angle and applied it to elementary teachers.\n\nSo Rossi suggested we look at principals.\n\nOur data had the name, salary, position and district information for all school employees in the state, but the state board had omitted the names of their schools.\n\nWe asked them to provide another disk with the school name information, which they did, and forged on, running simple queries for salaries of high school, junior high and elementary principals. To do that we selected the codes used for each position, which were provided in a separate text file that also contained the record layout.\n\nThe top ten ranged from $179,000 to $247,000, which was definitely very interesting. But the question arose, how do they make the money? Most were at or close to retirement age. Was it just another case of retirement incentives inflating pay?\n\nThe raw numbers only gave us total salaries.\n\nWe had to get the breakdowns the hard way, by calling each district. We made our jobs easier by focusing only on the top ten salaries.\n\nBut the district superintendents and business managers at first took advantage of the fact that we didn't know which questions to ask. They gladly broke out retirement incentives because they could make a case that this saved them money by enabling them to hire less-experienced, lower-paid replacements.\n\nBut accumulated sick pay, vacation pay and other benefits that we had no idea existed all were lumped into one category.\n\nGradually, by comparing notes between districts, we learned to ask for more specific breakdowns from everyone and kept calling until we got them.\n\nGetting the numbers is sometimes the easiest part of the story.\n\nAs a result, the Microsoft Excel spreadsheet we used to keep track of the salary breakdowns kept growing wider.\n\nThree facts quickly emerged. All of these principals had retired, and in many cases retirement incentives had pushed them into the top pay brackets.\n\nBut only suburban principals got the retirement incentives.\n\nChicago principals got a different going away present. They were allowed to cash out nearly a year of sick days and apply a large portion of those days to their service time, thereby boosting their pensions.\n\nWe still weren't sure this was a story.\n\nTo put it in perspective, we called school districts in New York and Los Angeles. After wading through their complex retirement packages (nothing is simple in educational bureaucracies) we learned they were far less generous with accumulated sick pay than Chicago.\n\nAnd a compensation expert at the Illinois Chamber of Commerce told us such benefits were unheard of in private industry.\n\nSo we finally had a story.\n\nSchool and union officials defended the system, saying it provides an incentive for teachers (who eventually become principals) to show up every day and thus saves money spent on substitute teachers. It also rewards Chicago teachers who stick it out until retirement and don't make as much money as suburban teachers.\n\nAnd it serves as a de facto retirement incentive, allowing quicker replacement of top-scale teachers and principals with lower-paid beginners.\n\nBut other experts pointed out that the Chicago teachers and principals also get another perk, that few, if any, in private industry get - a pension that pays 75 percent of their base retirement salary.\n\nThis project confirmed what Rossi and I had learned in the CAR boot camp. Getting the numbers is sometimes the easiest part of the story. Figuring out what they mean, putting them in context and then bringing those numbers to life is the hard part.\n\nContact Art Golab by e-mail at agolab@suntimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-march-response-times-fall-short","slug":"2004-march-response-times-fall-short","url":"https://dwillis.github.io/uplink/article/2004-march-response-times-fall-short.html","year":2004,"month":"March","monthNum":3,"headline":"Response times fall short","kicker":"MAPPING","author_name":"Enric Volante","author_title":"Reporter","affiliation":"Arizona Daily Star","summary":"The Arizona Daily Star investigated ambulance response times in Tucson using geographic information system software and public records data, revealing that emergency services frequently failed to meet the eight-minute response goal recommended by industry standards.","keywords":["ambulance","response","investigation"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["ArcView","Access","Excel"],"full_text":"Manager Shawn Bell called for medical help when a man collapsed in a diabetic seizure one night at Lil Abner's Steak House. Then Bell waited 10 minutes for help - even though an ambulance station is one minute down the road. No wonder Bell's interest was keen when I sat down at the steak house bar the next day and showed him a map that compared how long ambulances took to re- spond to emergencies in his neigh- borhood and others. Thousands of people peered at that map a few days later. Last April the Arizona Daily Star published \"Res- cue Roulette, a three-day series that revealed that the Tucson Fire Department ambulances failed more than half the time to reach emergencies within the eight- minute goal recommended by a national industry group. The stories also showed how other local ambulance operators often missed the target. And many - cluding the company that re- sponded to the diabetic man in the steak house - operated under li- censing requirements less stringent than Tucson's. We analyzed emergency response data using ESRI's ArcView 3.2 geo- graphic information system (GIS) software to find out where emer- gency medical workers were going and how long they took to respond. Getting dispatch data If you want to analyze EMS re- sponse, you'll need to get the CAD, or computer-aided dispatch, data- base. CAD data typically includes all the information found on tradi- tional ambulance logs, such as time dispatched and time arrived, as well as information on the inci- dent, patient, property and ve- hicles that responded. Our city serves as dispatcher for the area's largest ambulance op- erator, the Tucson Fire Depart- ment, and for several fire districts and private ambulance compa- nies. I requested the city's data for all agencies under the Arizona Public Records Law. Tucson officials balked. In addition to patient names, they refused to release addresses to which ambu- lances responded, saying that might identify patients. They cited medical privacy concerns and the new restrictions posed by the Health Insurance Portability and Accountability Act of 1996 (HIPAA). They also withheld all medical data, such as whether the call involved cardiac arrest or an asthma attack. My colleague Rhonda Bodfield and I met with city officials to ne- gotiate which fields of data should be released. We would have liked to follow the lead of journalists at The Hartford(Conn.) Courant, who joined a database of EMS responses with a public data- base of deaths to identify patients who died after long waits for medical aid. (For more information see \"Car- diac crapshoot\" in the July/August 1999 Uplink.) But death certificates are not public in Arizona, despite a fairly broad public-records law that hasn't been tinkered with much since territorial days. So we gave up the address fields on condition that the city substitute plat numbers from a grid of quar- ter-mile land sections. Unable to argue that patients would be iden- tified, the city then also gave up the medical data for each ambu- lance call. Lacking names and addresses, we found patients to interview the old- fashioned way. We hit the court- house to review civil lawsuits and we asked sources in the EMS busi- ness to name people who had waited a long time for help. The city provided the database re- luctantly and slowly, but at no cost. We ended up with a text file of 270,968 records of dispatch calls over 18 months that included fires, routine medical transports and medical emergencies. By querying in Microsoft Access we found 92,400 were emergency am- bulance calls. We deleted calls in which an ambulance was diverted to another scene. Sometimes in the heat of an emergency, drivers sim- ply forgot to record when they ar- rived, so we also threw out calls with imprecise arrival times. That left 89,818 records. Then we tabulated how long it took from the time an ambulance was dis- patched to the time its crew reported arriving at the scene. We flipped the results into Excel to chart a compari- son of the different agencies. We mapped the calls in ArcView using a quarter-mile grid layer downloaded from the county trans- portation department's Web site, along with shapefiles of streets, fire stations and subdivisions that helped us see routes to the most populated areas. We used ArcView's legend editor to create a color-graduated grid of response times within each cell of the grid. Slower response times on the edges of the metropolitan area im- mediately jumped out, particularly in the northwest, where a once-ru- ral fire district was coping with a growth boom. But, like many parts of the coun- try where fire departments and pri- vate ambulance companies have fought for turf, the Tucson region is a patchwork of competing pro- viders. So we also asked the county's largest ambulance opera- tor outside the city for its CAD da- tabase. Rural/Metro Corp. denied the request. That lack of account- ability to the public became part of the story. We knew the Arizona Department of Health Services had paper logs of ambulance calls submitted by Rural/Metro as part of its three-year license renewal. We ordered cop- ies. Typing the data into Excel and double-checking it took three days, but it was worth it. Unlike the city, the state had no problem releasing the patient ad- dresses in those logs. We used ArcView's geocoding function to dis- play about 7,500 addresses as points on a street map from the county transportation department. As we queried the data in ArcView, we saw long waits of 15 minutes or more had occurred in many places, not just distant areas. Dennis Joyce, assistant managing editor for news, had us run side-by-side maps show- ing different wait times. While Arizona doesn't release death certificates, it was one of the first states to make campaign finance records available by FTP download. We found Arizona-based Rural/ Metro kicks off the legislative ses- sion each year by spending close to $20,000 on a bash for lawmak- ers at places such as Phoenix's Ritz-Carlton hotel. As one ex-legis- lator told Bodfield, \"I always graded the party by the shrimp - and they would have shrimp.\" After we mapped the data, I visited some neighborhoods with long re- sponse times. I spoke to a few people on the street and then walked into Lil Abner's Steak House. When a cook mentioned that a man had been convulsing and kicking tables the night before while the manager phoned for an ambu- lance, I knew I'd brought the map to the right place. You can read the stories online at www.azstarnet.com/specialreports/ emsresponse.html Contact Enric Volante by e-mail at volante@azstarnet.com Would you be willing to share a mapping example with fellow journalists? Send an electronic copy of the map along with details to David Herzog at dherzog@nicar.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-sprawl-outpaces-private-ambulances","slug":"2004-march-sprawl-outpaces-private-ambulances","url":"https://dwillis.github.io/uplink/article/2004-march-sprawl-outpaces-private-ambulances.html","year":2004,"month":"March","monthNum":3,"headline":"Sprawl outpaces private ambulances","kicker":"SPOTLIGHT: EMERGENCY RESPONSE","author_name":"Jeff Sonderman","author_title":"","affiliation":"Missouri School of Journalism","summary":"A journalist describes the process of investigating emergency response times in their community by obtaining and analyzing three years of 911 dispatch data, overcoming HIPAA privacy restrictions, and using mapping and database tools to identify patterns and performance issues.","keywords":["emergency","response","data"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["ArcView","Excel","Access","CD-ROM"],"full_text":"Investigating emergency response times takes a lot of upfront reporting and data chasing before you know if you have a story. So how do you know if you should jump into doing this story in your com- munity? There are a couple of ways. If you notice specific cases where people died waiting for an ambulance, that's an obvious red flag. But you don't need to wait for such a clear signal. In any community with sprawl and growth, journalists may wonder whether key public services, including emergency medical services (EMS), are keeping up. That's what sparked my interest and led to \"A Matter of Time,\" a November 2003 story in the Columbia Missourian. The story was based on the analysis of Boone County computer-aided dis- patch data using ESRI's ArcView 8.2, as well as Microsoft Excel and Access. | obtained three years of computer-aided dispatch (CAD) records of emergency medical calls, 39,744 total, from Public Safety Joint Communications, a city agency that operates the 911 service. The agency's database manager was cooperative and told me what data was available. Because of privacy concerns under the Health Insurance Portability and Accountability Act of 1996 (HIPAA), the agency would only release the data to the hospitals, which then handed it to me on CD. The data came in a comma- delimited text format at no cost because it required no special programming. HIPAA also challenged our desire to map response performance. The hos- pitals refused to provide addresses that were logged for each call because the information could easily identify pa- tients. I talked to the database man- ager about alternatives, and he said the dispatch agency logs the general lo- cation of each call using a grid with cells of about one-half square mile each. He e-mailed me an ArcView shapefile of the geographic grid system that day. The response data was relatively clean; 911 system computers, not people, had entered much of the data automatically. The biggest problem was that response times were not properly logged on some calls. The response time is re- corded when the dispatcher presses a button at the time of dispatch and again at the arrival on scene. In some cases, the dispatcher missed one of those events, and the response time was re- corded as zero. These records were excluded because the errors seemed to affect calls in a random pattern that did not skew the data. This anomaly, as well as all of my analysis, was ex- plained to the readers in a sidebar. We imported and analyzed the data into Access and exported the results of some of the queries in Excel. The analysis included looking at over- all response time, as well as grouping in Access to see response times by am- bulance, specific location, city or town, and time of day. I looked at the division of call volume among various ambu- lances and locations and times of day. To analyze performance relative to the eight-and six-minute response time stan- dards of the hospitals, I created two fields to indicate whether the unit had reached the scene within each of those two time frames. I also created new fields to dis- play the response time for each call in seconds, based upon the start and end times included in the data. In ArcView, I used the grid to map re- sponses by grid section, which pro- duced a color-coded mosaic showing the percentage of calls in each sector that were reached within the eight- minute goal. To do this, I first used an Access query to group by the coordi- nates for each grid sector. I then summed the number of calls reached within the goal and counted the total number of calls in the area. I imported the result of that query into Excel to figure out the percentage of success- ful response times for each grid sec- tor. Then I imported that simple table just grid coordinates and the percent- age of success into ArcView and ap- plied it to the shapefile that used the same grid coordinates. Talking to the people who kept the data also helped us to extract useful data from two of the fields in the database. The grid coordinate field of each record corresponded to a single cell on the grid map, but it also contained a letter at the end of the four or five numbers (the \"B\" in section 14041B, for ex- ample) that indicated the general lo- cation of the call. Each of the smaller cities and towns in the county had its own assigned letter. I parsed the grid coordinate field in Access and placed its letter into a new field so I could eas-ily examine all the calls in those towns using grouping and counting queries. The index field that assigns a unique identifier to each dispatch also included characters that indicate which ambu- lance responded, for example \"M31\" for Medic 31. We parsed those refer- ences into a new field, which allowed us to look at response times and emer- gency call volume for each ambulance. The two ambulance services that cover Boone County are owned and oper- ated by the two major local hospitals. The good news for residents is that they don't pay a tax for a public ambu- lance service. But this also means the ambulance services are departments of large institutions that have to watch their financial bottom lines. Emergency medical services tend to lose money, and that puts the opera- tors in a sticky position. How much money are they willing to lose on an ambulance service to provide better patient care? This is an especially important issue in small but rapidly growing regions, such as Columbia and Boone County, which has a population of about 135,000 people, with about two-thirds of them in the city of Columbia. But Columbia is sprawling outward, away from the ambulance stations, and populations are growing in other cities around the county. Once I had the story, it was time to make it come alive with field reporting. I went on several ambulance ride- alongs, arranged by one of the hospi- tals, and shadowed the paramedics when they responded to calls. This firsthand experience helped me understand the system and the num- bers I was writing about. It also gave me extended access to paramedics in their work environment, where they were talkative and comfortable. The time I spent on ride-alongs was incred- ably useful. The paramedics are on the front lines of EMS, so they often pro- vided the most insightful comments on the data I showed them. Paramedics can also help answer the big question: How much is better pa- tient care worth? Paramedics almost always want more ambulances and more resources. They would take an ambulance on every street corner if they could get it, because their biggest concern is patient care. Here you might find, if not a whistleblower, at least an advocate for change in the system. Contrast that with the concerns of man- agers and hospital administrators. Many of them have never worked as paramedics, and their focus is more on the financial costs of adding ambu- lances or other resources. So in most EMS systems, you will prob- ably find some amount of tension be- tween what the EMS workers want and what the administrators think is nec- essary or practical. Each side will be ea- ger to convince you, and you can use information from both sides to develop a balanced story. Contact Jeff Sonderman by e-mail at jeffsonderman@yahoo.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-readmetxt","slug":"2004-march-readmetxt","url":"https://dwillis.github.io/uplink/article/2004-march-readmetxt.html","year":2004,"month":"March","monthNum":3,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"CAR Tools highlights software useful for computer-assisted reporting, featuring recent reviews of mapping, analytics, and text parsing tools for journalists.","keywords":["software","journalism","tools"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["ClusterSeer","SAS","ParseRat","OS X","ArcView"],"full_text":"CAR Tools focuses on software that's useful to journalists doing computer-assisted reporting. Recent issues of Uplink have taken a look at:\n\nClusterSeer analytic mapping tool. (January/February 2004)\nSAS System Viewer (November/December 2003)\nParseRat text parser (September/October 2003)\nos X operating system (May/June 2003)\nRematch extension for ArcView 3.x (March/April 2003)\n\nIf you have a favorite CAR tool that Uplink readers should know about, drop us a line at uplink@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-march-asksam-tames-the-data-clutter","slug":"2004-march-asksam-tames-the-data-clutter","url":"https://dwillis.github.io/uplink/article/2004-march-asksam-tames-the-data-clutter.html","year":2004,"month":"March","monthNum":3,"headline":"askSam tames the data clutter","kicker":"CAR TOOL","author_name":"Bonnie Britt","author_title":"Freelance","affiliation":"Freelance","summary":"askSam is a flexible archiving and database program that allows journalists and investigators to organize and search large amounts of structured and unstructured data without forcing information into rigid fields, making it ideal for managing research and evidence.","keywords":["database","journalists","search"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["askSam","Access","Excel","Nexis"],"full_text":"I worked in a newsroom where a clut- ter-averse assistant managing editor periodically pushed a big trash bar- rel through the aisles expecting re- porters to toss documents. Packrats like me did not share his enthusiasm for a tidier newsroom. Then I learned about askSam, an archiving and free-form database program that allows journalists to squirrel away information electroni- cally - instead of letting it pile up on their desks. The program is a natural friend to journalists, historians, archi- vists, investigators, and packrats of every stripe. Using askSam is better than living with paper clutter because nothing gets lost or buried. And the program does not force you to shoehorn data into fields with set lengths. In fact, you can choose whether or not to import data into fields. You can search, sort, and sift massive amounts of data (up to 16 terabytes), structured or unstructured, from an ar- ray of formats. Its speed and flexibility explain why askSam is popular among intelligence and law enforcement agencies whose personnel often think and work like journalists. Using it The U.S. General Accounting Office uses it with other software to collect and organize survey information. Senate investigators pursuing the Iran-contra scandal employed askSam's lightning speed to good ad- vantage in the mid-1980's to ask in- formed questions of Oliver North in real time. The program enabled them to organize, catalog and present the volumes of evidence they had gath- ered during their investigation and subsequent Senate hearings. A forensic document analyst at the U.S. Postal Inspection Service built a central repository in askSam of articles relating to death threats sent through the mail, forged signatures, counterfeit passports and related topics. While askSam is valuable to people who toss unstructured notes into files, then do simple or Boolean searches for com- binations of words or phrases, it also of- fers detailed reports based on field data queries. For example, you can set askSam to automatically recognize e- mail fields such as, To, From and Sub- ject, or you can import e-mail without specifying any field. The database is searchable either way, but if you import e-mail with fields, you can ask for a re- port on all e-mail, say, from Ken Lay be- tween two dates when the subject con- tains a particular word or phrase. You can search dates, fields, proximity, or explore numeric values and relationships, or do a global search and replace. You can search through multiple askSam databases to keep track of projects or to compare statements of public officials over time. The program is easy to use once you've figured out what you want from it, and it's smart enough to adapt for use with Microsoft Access, Excel or LexisNexis. Why would anyone want to import a spreadsheet into askSam? To more easily view and page through rows; to view it differently so as to spot errors; or to query it differently. Viewing a long row of data in Excel is easier to read when laid out on one page. Each suc- ceeding row can be viewed, page by page, by pressing F9 for next (or ALT- F9 for previous.) Stories captured from LexisNexis can be searched in a variety of ways, such as for combinations of names, along- side Web pages, stories cut and pasted from today's news, annotated screen shots, and other tidbits gathered and thrown into the mix. The search results are displayed in a split window that is easy to navigate. The results show the number of hits per document, context, and the first line of data. Click on any row to see the en- tire corresponding document. Getting started To begin using askSam, you open an empty database, name it and save it. You can immediately begin building a database in any of four ways: Drop documents into askSam by cut- ting and pasting them as plain text or, using the \"paste special\" option, as HTML or rich text. (You add graphics separately.) Import all or some documents, one file type at a time, from a Windows folder. You can opt to import e-mail at- tachments. Link any file in its native format. Linked files retain their formats and can be opened in askSam and edited. Attach a file in any format. An at- tached file becomes a part of the askSam database. You can customize askSam data- bases using fonts, colors and other formatting. HTML and rich text docu- ments will retain the formatting of the original files; text documents will re- flect your choices. You can also customize each database to control how dates are displayed and what search results are shown. The biggest complaint of longtime askSam users is that table columns in HTML and Word do not align prop- erly. Users have found a work- around: pasting table text into askSam to facilitate searching, and then attaching the original document to the database so it can be viewed with columnar data properly aligned. modify - existing databases. The ability to share askSam data- bases with colleagues is a new fea- ture. Last year, the company, based in Perry, Fla., began offering a free viewer, which is similar in function to the Adobe Acrobat Reader. Col- leagues can use the free viewer to read and search - but not add to, or The standard askSam program retails for $149 and is powerful enough for importing e-mail, or keeping track of the elements of a nonfiction book or news project. For power users who want more speed and flexibility, there is a professional version, which in- cludes full-text indexing, for $395 re- tail. Indexing speeds searches of big databases. askSam Professional also includes an ActiveX control that can be accessed from Visual Basic and other programming languages to cre- ate customized applications. The program's features include a tool to pack databases to keep them lean. The program also allows document deletion and creation, and remembers which documents and searches you've marked for recall. For more about the program, visit www.asksam.com, where you can also download a 30-day evaluation version. Contact Bonnie Britt by e-mail at editor@editorsforum.org. CAR Tools focuses on software that's useful to journalists doing computer-assisted reporting. Recent issues of Uplink have taken a look at: ClusterSeer analytic mapping tool. (January/February 2004) SAS System Viewer (November/December 2003) ParseRat text parser (September/October 2003) os X operating system (May/ June 2003) Rematch extension for ArcView 3.x (March/April 2003) If you have a favorite CAR tool that Uplink readers should know about, drop us a line at uplink@nicar.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-readmetxt-2","slug":"2004-march-readmetxt-2","url":"https://dwillis.github.io/uplink/article/2004-march-readmetxt-2.html","year":2004,"month":"March","monthNum":3,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"CAR Tools highlights software useful for computer-assisted reporting, featuring recent reviews of mapping, analytics, and text parsing tools in the Uplink newsletter.","keywords":["software","journalism","tools"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["ClusterSeer","SAS","ParseRat","ArcView","OS X"],"full_text":"CAR Tools focuses on software that's useful to journalists doing computer-assisted reporting. Recent issues of Uplink have taken a look at:\n\nClusterSeer analytic mapping tool. (January/February 2004)\nSAS System Viewer (November/December 2003)\nParseRat text parser (September/October 2003)\nos X operating system (May/June 2003)\nRematch extension for ArcView 3.x (March/April 2003)\n\nIf you have a favorite CAR tool that Uplink readers should know about, drop us a line at uplink@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-march-xandros-switching-from-windows","slug":"2004-march-xandros-switching-from-windows","url":"https://dwillis.github.io/uplink/article/2004-march-xandros-switching-from-windows.html","year":2004,"month":"March","monthNum":3,"headline":"Xandros: Switching from Windows","kicker":"OPEN SOURCE","author_name":"Aron Pilhofer","author_title":"Journalist","affiliation":"Center for Public Integrity","summary":"A journalist frustrated by Windows security vulnerabilities, particularly the Mydoom virus, decides to switch to Linux as their primary operating system and evaluates various Linux distributions, ultimately considering Xandros as a user-friendly option designed for Windows users.","keywords":["virus","Linux","Windows"],"topics":["Technology","Journalism & Newsroom"],"technologies":["Linux","Windows","Microsoft Outlook","Microsoft Office","OS X"],"full_text":"Mydoom was the last straw. It was Tuesday, Jan. 27, just past 9 p.m. I was staring at dozens of Microsoft Out- look e-mail messages infected with cop- ies of the Mydoom virus, trying to figure out how, despite security precautions bordering on paranoia, one of those little buggers had managed to infect my Win- dows XP system. I was firewalled and patched. My virus software was up to date. I had never used the Outlook preview pane, read suspect messages or opened unknown attachments. Yet, I had the virus. It was then and there that I decided I was done with Windows. I was going to make the switch to Linux as my main desktop once and for all. I have been working with Linux for years now - first as a newsroom intranet and data server, and then as a secondary operating system on my home computer. But even though I had adopted open source software for much of my CAR- related work, I never did make the per- manent switch to a Linux desktop. A few years ago, I was close to the brink of sanity trying to deal with yet another utterly inexplicable Windows ME crash and burn. But then Microsoft released the truly excellent Windows XP, which was fast, easy and (most importantly) stable. Windows XP, however, isn't secure and I'm tired of having to hold my breath every time I hit the \"send/receive\" but- ton in Outlook. A properly maintained Linux box, on the other hand, is just as stable as XP and virtually immune to the kinds of attacks plaguing Windows users. I have considered other options, such as Apple OS X (which is based on a core operating system similar to Linux). But even though I love the soft- ware, I am not willing to pay a premium for new hardware. Thus, my transition to Linux began. Which distribution? There are literally thousands of ver- sions of Linux, some are well known but most are fairly obscure. Some are produced by corporations, such as Mandrake, SUSE or Red Hat. Others, such as Debian, Slackware or Gentoo, are designed by enthusiasts as a la- bor of love. Xandros lives up to its promise and then some. I had experimented over the years with most of the versions I've mentioned, so I was pretty familiar with their re- spective strengths and weaknesses. Because I was going to be using Linux for both home and work, I had some fairly specific criteria in mind. I needed a distribution that is: Easy to administer and maintain Able to run a wide range of software, both open source and proprietary Well-documented and well-maintained My first choice was Red Hat (See a re- view in the September/October 2003 Uplink), but the company recently de- cided to focus on corporate desktops, and handed over development of its desktop distribution (now called Fe- dora) to volunteers. I tried Fedora, but the installation froze not a good sign. I then read about a distribution called Xandros, which was designed to appeal to former Windows users. The deluxe version even comes packaged with software that allows users to install and use Microsoft Of- fice and other Windows applications. Best yet, Xandros is based on the Debian version of Linux, one of the old- est, most respected and reliable ver- sions available. It also features a slick software management tool called apt. I paid $89 for the deluxe version and downloaded the installation discs and documentation. A few hours later, I was on my way. Installation Xandros has one of the smoothest in- stallations available. Xandros recog- nized nearly all my computer hardware during the installation - something that was a first in all my previous operating system installations. I wanted to add Xandros while preserv- ing both my existing Windows installa- tion and another version of Linux I had already put on the machine. I also had just added some fairly cutting edge hardware, which can cause installation problems for any operating system, but especially Linux. To my surprise and delight, Xandros handled my hardware without a hiccup. My New Home Booting Xandros for the first time re- vealed a Linux environment that has been tweaked as much as possible to look and feel like Windows. Double clicking opens a directory or launches a program. Xandros automatically loads additional hard drives or parti- tions - including Windows partitions into a top-level directory named \"disks.\" Xandros goes to great lengths to hide its Linux underpinnings. When boot- ing into or shutting down from Xandros, you see a graphic, and not the typical stream of gobbledy-gook messages Linux usually spews onto the screen. The last, and biggest, difference in the Xandros deluxe version is the in- clusion of Crossover Office, which is a set of utilities that trick Windows ap- plications into running under Linux. They are optimized, as the name sug- gests, to work best with Microsoft Office, and my copies of Microsoft Access, Excel and Power Point 2000 installed flawlessly. One criticism of the desktop is that it does not look as smooth, crisp and clean as Red Hat or Windows XP. In very limited testing, I can report that all three seem to work nearly as well under Linux as they do under Windows. The only obvious differ- ences are aesthetic. Menus seem a little darker under Linux, and the menu fonts seem smaller. One criticism of the desktop is that it does not look as smooth, crisp and clean as Red Hat or Windows XP. Fonts can appear blocky, but that is a minor gripe. Simplicity In terms of software management, Xandros is outstanding. The distribu- tion has a very simple, but powerful, package manager. It can be used to install updates from the Xandros Web site, or add new software from any number of sources, including stan- dard Debian repositories on the Internet. Think of it as the Windows Update and Add/Remove Software utilities all in one. The manager operates in simple and expert modes. In simple mode, users can install only the software that has been packaged for the Xandros in- stallation as well as any security or software updates the company makes available on its Web site. This is probably more than enough for most users, but there are a few soft- ware packages (PostgreSQL, Webmin) that were excluded from the standard installation. Switching into expert mode provides access to software stored in reposito- ries across the Internet, and the ability to add more of your own. Installing an application is usually a simple matter of a single mouse click. Not only does Xandros download the application, but also it automatically handles the problem of figuring out what other programs or libraries are required and downloading those as well. Most of the popular Linux distributions use a package management system based on one developed by Red Hat a few years ago (the Red Hat Pack- age Management system, or RPM). RPM, to be blunt, stinks. Installing ap- plications other than those packaged with the distribution under RPM can be a long, frustrating, and often-fruitless experience. Xandros's apt-based package man- ager is amazing. I cherry-picked more than a dozen non-standard applica- tions - a recipe for disaster for RPM- based distributions. Xandros dutifully and flawlessly installed each one with- out a hitch. That is impressive. Final Thoughts Xandros lives up to its promise and then some. It is Linux for those who want to wade rather than dive into a new operating system. Some Linux purists will not like the modifications Xandros has made. But for folks who don't come from a Unix background, and don't have a com- puter science degree, Xandros is just possibly the best of both worlds. Contact Aron Pilhofer by e-mail at apilhofer@publicintegrity.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-march-curl-ing-apart-web-documents","slug":"2004-march-curl-ing-apart-web-documents","url":"https://dwillis.github.io/uplink/article/2004-march-curl-ing-apart-web-documents.html","year":2004,"month":"March","monthNum":3,"headline":"Curl-ing apart Web documents","kicker":"Tech tip","author_name":"Derek Willis","author_title":"Database Editor","affiliation":"Center for Public Integrity","summary":"Curl is a free command-line utility that helps journalists efficiently download large datasets from websites by automating batch downloads of pages with consistent URL patterns, eliminating the need for manual clicking through multiple pages.","keywords":["curl","download","database"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["curl","ASP","CGI","HTML","Windows","Unix","Linux","OS X"],"full_text":"Here's a pet peeve you might recognize: you find a database you need on the Web, but can't download it with a single click. Instead, the records are contained on separate pages, sometimes behind a search form that lets you see only a few entries at a time. Hitting Control-S several hundred times isn't my idea of a great day.\n\nBut there's good news: there are tools that can pry loose all that data without much trouble or effort on your part. My favorite is curl, a command-line utility for Windows, Unix, Linux and Mac OS X. With curl, I can download batches of files with a consistent naming pattern, such as those pesky ASP or CGI files that contain numbers in the URL string.\n\nCurl is a free command-line utility, meaning that you run it from the terminal in Linux, Unix or Mac OS X, or from the DOS prompt in Windows. If your system doesn't have curl installed, you can get it from http://curl.haxx.se. Be sure to bookmark curl's basic documentation (http://curl.sourceforge.net/docs/manpage.html); it comes in handy.\n\nHere's an example of using curl: a few months ago, Jeff South, a Virginia Commonwealth University professor and former database editor at the Austin American-Statesman, posted to the NICAR-L listserv asking if anyone had an easier way to put together a list of California lobbyists.\n\nThe official state Web site (http://calaccess.ss.ca.gov/Lobbying) has a list of current lobbyists but only shows one page for each letter of the alphabet, meaning that you'd have to save each one and then put them together before beginning the task of separating the records from the HTML around them.\n\nCurl was built for just this kind of task, since it offers powerful ways to automate downloads based on a consistent naming pattern. The URL for lobbyists with a last name beginning with","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-first-venture-finding-racial-tilt-in-arrests","slug":"2004-june-first-venture-finding-racial-tilt-in-arrests","url":"https://dwillis.github.io/uplink/article/2004-june-first-venture-finding-racial-tilt-in-arrests.html","year":2004,"month":"June","monthNum":6,"headline":"FIRST VENTURE Finding racial tilt in arrests","kicker":"","author_name":"Dave Moore","author_title":"Reporter","affiliation":"Columbia (Mo.) Daily Tribune","summary":"Dave Moore explores the racial imbalance in juvenile arrests in Columbia, Mo., finding that African American youths are disproportionately arrested compared to their white counterparts.","keywords":["racial","juvenile","arrests"],"topics":["Demographics & Census","Crime & Justice","Investigative Methods"],"technologies":["CD-ROM","Perl","Excel","Access"],"full_text":"For more than a decade, state and federal government agen- cies have issued reports and held meetings about the dispro- portionate number of minorities in jails and prisons. But by the time people land in jail, they're often too far gone for help. That's a heartbreaking notion, especially when you're talking about teenage children. When Congress passed the Juvenile Justice and Delin- quency Prevention Act of 2002 it intended to address the ar- rest imbalance for minority- group juveniles by reducing the number of children who get in trouble with the law. For that reason, I decided to look at the racial composition of the children arrested in Colum- bia by analyzing the police department's arrest data, us- ing the lessons I learned at a recent NICAR Boot Camp. I found the proportion of minor- ity children had swelled over eight years. In 1995, 60 percent of all juve- nile arrests in the city involved white children. By 2003, 60 percent of all ju- venile arrests involved black children, who make up 18 percent of all kids in the city. My story, \"The Color of Justice,\" didn't start out as an examination of the ra- cial composition of juvenile arrests. I had just been assigned to cover city hall when the city council member who represents central Columbia was con- sidering whether to push for a curfew law. One of the opponents said the law would be enforced primarily in the cen- tral city, where the city's black popula- tion is concentrated, and act as a sort of \"Jim Crow law.\" From my previous reporting experi- ence, I knew many police departments use computers to map where crimes and arrests occur. Sure enough, Co- lumbia possessed the same technol- ogy. I asked the police to map the gen- eral locations of curfew-hour arrests. It turned out the curfew opponent was right on the mark - most curfew-hour arrests occurred in the central city. I negotiated for as many fields as I could get. But when I showed the map to my edi- tor, he wasn't convinced there was a story yet. That's when my lessons of the NICAR boot camp kicked in. I knew that I needed to talk to the keeper of the database at the police department to determine what other information was available. I negotiated for as many fields as I could get. Because juvenile records are protected, I had little leverage. What I received from the city on CD- ROM was messy. I could have sworn that I requested the data in delimited text format, but when I opened the file, I found that the data ran together and had inconsistent breaks. Appropriately, the city's com- puter technician labeled the CD \"Datadump.\" The IRE and NICAR Data Library wrote a Perl program that cleaned the file for the most part, though there were still some problems. Later when I sought additional data, the city's computer tech produced the data in an Excel spreadsheet, which he could have done from the start. Many fields in the more complete data set were inconsistently filled, such as the officer, whether a weapon had been involved and time of the arrest. However, I found that the data in- cluded the following columns of infor- mation: uniform crime reporting code, age, year, sex and race. That's what I stayed with. I was dealing with about 15,000 records, so it was easy for me to im- port the data from Excel to Access for querying. I used Access to count the number of arrests by race and year; soon I had totals for 1995 through 2003. I then moved those numbers to an Excel spreadsheet, where I created a line graph that showed a dramatic shift toward more black arrests than white arrests. Now all I had to do was get the police department to explain why. I showed my analysis and the curfew- hour arrest map to the police chief and his captain and they had no ex- planation. \"You can make numbers do anything you want them to,\" the captain replied. My editor prompted me to find other cities and groups that have addressed the racial imbalance in arrests and pris- ons and I eventually learned about how Portland, Ore., Santa Cruz, Calif., and Seattle had reduced juvenile arrests and incarcerations. A local community outreach center put me in touch with a family that had run-ins with the police and probation department. One of the children ended up serving as a focal point for my story. After the story was published, the dis- proportionate minority arrests emerged as an issue in the mayoral election. Here are some lessons I learned re- porting the story: Nearly 70 percent of all police depart- ments track and report their arrests by race to the federal government. It's still uncertain whether all departments will be required to report the racial composition of their arrests, as the Juvenile Justice and Delinquency Prevention Act evolves. However, because courts are funded by a separate pool of money than police agencies, police don't have the same in- centive to report arrest statistics as court offices do with inmates. Because police organizations are hi- erarchical, you'll have to start some- where at the top and find your way to the department's database guru. If you ask the police whether they keep computerized records for the ar- rest of each child, and then ask to see a record layout (which shows the fields in the database, such as name, date of birth, race, location of arrest and charge), the police will often point you in the direction of the person who keeps the database. Be willing to negotiate for fields you want after you see which are available. Because juveniles' identities are highly protected under the law, you will have little leverage for negotiating. Ask for as many years of data as possible so you can observe trends. I had eight years of data and was able to see the shift toward the arrests of black children. Consider the mapping possibilities. During my reporting, I learned that the police department uses mapping to mark locations of crimes and target patrols. While police may not divulge the specific locations of arrests, they might reveal the general block group or area of arrest. I simply asked police to map juvenile arrests by block group. When assembling juvenile justice data, check out the U.S. Justice Department's Office of Juvenile Justice and Delinquency Prevention Web site at www.ojjdp.ncjrs.org. The site has national statistics on juvenile arrests, which can be compared on a county- by-county basis. You can read \"The Color of Justice\" on the Web at http://archive. columbiatribune.com/2004/Feb/ 20040208Feat051.asp. Contact Dave Moore by e-mail at dmoore@tribmail.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-june-analysis-exposes-attacks-on-volunteers","slug":"2004-june-analysis-exposes-attacks-on-volunteers","url":"https://dwillis.github.io/uplink/article/2004-june-analysis-exposes-attacks-on-volunteers.html","year":2004,"month":"June","monthNum":6,"headline":"Analysis exposes attacks on volunteers","kicker":"PEACE CORPS","author_name":"Russell Carollo","author_title":"reporter","affiliation":"Dayton Daily News","summary":"An investigation uncovers the increasing rate of violent attacks, including rapes, against Peace Corps volunteers, predominantly young women serving overseas.","keywords":["Peace Corps","attacks","volunteers"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["Paradox"],"full_text":"The idea wouldn't go away. For more than 10 years, I have rou- tinely scanned inspectors' general re- ports to Congress for database and story ideas. Thick reports from these internal government watchdogs about large agencies get nearly all of my at- tention. Still, I usually scan shorter re- ports from smaller agencies such as the Federal Maritime Commission and Office of Personnel Management, just to make sure I have looked at them all. Peace Corps reports were the very last places I thought I would ever find tips about computer databases for a major project. But beginning about four years ago, I kept noticing vignettes about rapes and other attacks on volunteers. These brief, graphic descriptions told of violent attacks, mostly on young women serving overseas. As a citi- zen, I was shocked this was going on at all; as a journalist, I was sure oth- ers would be shocked, too. Year after year, the reports kept coming. I lacked the proof but got the feeling that the number of vignettes - and the problem - was increasing. So I put the idea on a story list, not as an idea for a major project but just a suggestion for a single story that someone else might consider. Years later, during a meeting to discuss project ideas in early 2002, projects reporter Mike Wagner con- vinced Daily Newseditors to pursue the idea as a major project, asserting that most Americans would be shocked. Following that February 2002 meet- ing, I started filing the first of more than 75 federal Freedom of Informa- tion Act requests and appeals with the Peace Corps, including one for the agency's main database of assaults, the Assault Notification Surveillance System. The database contains records for nearly 3,000 attacks on volunteers since 1990. Two months later, I appealed for lack of proper response, and in May the agency re- leased the first version of the data- base at no cost. What we got was virtually worthless. The Peace Corps denied us the unique number assigned to the assault, the country of the assault, report date, time, day and date of assault, type of place where the assault occurred, a brief description of the attack, whether local authorities were notified, and whether the victim was medically evacuated. All this information, according to the Peace Corps, was being withheld to \"protect the privacy interest of individu- als.\" Since we weren't demanding the names of the victims and since just about every law enforcement agency in America releases the same informa- tion, I appealed and challenged the agency to defend how such things as the name of a country would violate someone's personal privacy. In the meantime, we continued to pur- sue the story, by interviewing hundreds of people in 11 countries and collect- ing thousands of pages of paper records, many of them overseas. Our newspaper sued the Peace Corps in federal court in November 2002, and the agency suddenly became much more cooperative. Underreporting also appeared to be a special problem for the Peace Corps. By July 2003, we had a copy of the database with nearly all of the fields. We also were able to get two other databases: the Epidemiologic Surveil- lance System, containing monthly re- ports of burglaries, alcohol-related in- incidences, pregnancies, STD cases, and other problems encountered by volunteers; and the Deaths in Service, which provided details about how more than 200 volunteers had died since 1962. I analyzed the data using Para- dox database manager. Our analysis and other records found: The number of all assaults increased more than 200 percent 1990-2002, while the number of volunteers in- creased about 29 percent. We found a 300 percent increase in aggravated assaults, the most serious type of physical assaults. In 2002, there was an assault or robbery of a volunteer every 23 hours. Many assaults were linked to a de- cades-old practice the agency refuses to change: Sending young volunteers - the majority of them women - just out of college to live alone with virtu- ally no supervision in remote regions of some of the most dangerous coun- tries in the world. Volunteers frequently arrive at their sites fresh out of training without adequate housing or a job that keeps them busy. Some turn to drink- ing, using drugs, traveling to unsafe areas or engaging in other activities that put them in danger. Volunteers who were alone frequently were the targets of assaults, and young women appeared the most vulnerable. In 62 percent of the more than 2,900 assaults, the victim was identified as being alone, and in 59 percent of all assault cases, the victim was identified as a woman in her 20s. Alcohol was identified as a factor in nearly one in three assaults since 1999, although the assailant was the sole person drinking in some of those cases. Alcohol was also linked to nearly one in six deaths since 1962. Families of dead volunteers were un- aware of crucial details in the data- bases and other records we obtained. One mother didn't know her daughter had attempted suicide with aspirin in Ecuador before lapsing into a coma and dying. The brother of another vol- unteer said the family had never been told that his 22-year-old sister had writ- ten a resignation letter the day she dis- appeared in Africa or that the Peace Corps suspected suicide. Her body was found five days later floating in a river, and the family was led to believe that a crocodile had killed her. As helpful as the databases proved to be, there were a number of problems with the way the agency collected in- formation. The Peace Corps failed to collect worldwide crime statistics until 1990, 28 years after it first sent volunteers overseas. Certain crimes still weren't specifically identified in the assault database. Abductions, kidnappings and even murder usually were recorded as assaults in the data. Some crimes that should have ap- peared in the data weren't there at all. We couldn't find two known murders in the database. In January 2001, weeks before a vol- unteer disappeared in Bolivia, Mike Mulvaney and another volunteer were abducted in a taxi, the second such abduction of a volunteer within 11 months in that country. Mulvaney was beaten during the incident but managed to escape. Both the Peace Corps country director in Bolivia at the time and the agency's worldwide security director confirmed that Mulvaney spoke to them about the abduction, but the incident wasn't recorded anywhere on the assault database. A written response from the Peace Corps says, \"Not having a record of this alleged assault was not an error.\" Mulvaney, it says, should also have reported it to the medical office. Underreporting also appeared to be a special problem for the Peace Corps, since volunteers risked be- ing moved out of the country or even out of the Peace Corps by simply making an assault report. Govern- ment auditors estimated that underreporting was as high as 50 percent. When we asked Elliot Jaspin of the Cox Newspapers Washington Bu- reau, and founder of what is now NICAR, to review our findings before publication, he found something even more troubling: The crime rate the agency was making available to Con- gress and the public was based on incidents, not the number of victims. After three women were raped in El Salvador in 1996, the incident was recorded as a single rape. About one in every five assaults in- volved more than one volunteer; but it was impossible to determine what hap- pened to the other volunteers in each case by using the data we had. At a meeting in Washington, Jaspin, reporter Mei-Ling Hopgood and projects editor John Erickson and I confronted Peace Corps officials, but they insisted their data collection methods were the best way to make volunteers safer. They also said the U.S. General Accounting Office, dur- ing an earlier audit, had validated the Peace Corps' data collection meth- ods. The GAO later said it was un- aware that the Peace Corps failed to count all victims. Problems with the way the agency collected and interpreted data was one of several reasons the extent of dangers facing Peace Corps volun- teers had been disguised for de- cades. Another reason is that the at- tacks occurred thousands of miles away. Still another reason is that the agency made little effort to publicize them or deliberately kept people from finding out - while emphasiz- ing only the positive aspects of Peace Corps service. Contact Russell Carollo by e-mail at rcarollo@coxohio.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-june-using-car-to-cover-the-vote","slug":"2004-june-using-car-to-cover-the-vote","url":"https://dwillis.github.io/uplink/article/2004-june-using-car-to-cover-the-vote.html","year":2004,"month":"June","monthNum":6,"headline":"Using CAR to cover the vote","kicker":"SPOTLIGHT: ELECTIONS","author_name":"Megan Clarke","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists are preparing to use computer-assisted reporting (CAR) techniques like spreadsheets, database analysis, and mapping to provide deeper coverage of the 2004 election beyond standard campaign finance reporting.","keywords":["election","reporting","analysis"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["spreadsheet"],"full_text":"It's that time again. Election 2004 is big news already and many newsrooms are preparing to use computer-assisted reporting for stories that go beyond the basics of who-donated-how-much campaign finance coverage.\n\nTo get you thinking how you can use CAR between now and election day, here are several examples of how journalists have used spreadsheets, database analysis and mapping to report on voters and election results.\n\nSt. Paul Pioneer Press CAR Editor Janet Roberts reported on how particular precincts and types of voters can\n\nIf the last presidential election is a precursor, then 2004 promises to be interesting and full of opportunities for new CAR techniques and stories.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-updated-audits-files","slug":"2004-june-updated-audits-files","url":"https://dwillis.github.io/uplink/article/2004-june-updated-audits-files.html","year":2004,"month":"June","monthNum":6,"headline":"Updated audits files","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library updated their Single Audit database covering federal awards exceeding $300,000 annually from 1997 to January 2004, providing journalists with detailed information on federal grants to nonprofits and public agencies.","keywords":["database","federal","grants"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["CD-ROM"],"full_text":"The IRE and NICAR Database Library recently updated the Single Audit database from the Federal Audit Clearinghouse of the U.S. Census Bureau. The database, which covers 1997 to January 2004, contains information on expenditures of federal awards greater than $300,000 a year. Journalists will find this useful for looking at federal grants used by nonprofits and public agencies.\n\nThis data contains more detail than is provided by the federal Consolidated Federal Funds Report and the Federal Award Assistance Data System. The audit data looks at spending and includes audits of indirect recipients.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-discovering-car-again-and-again","slug":"2004-june-discovering-car-again-and-again","url":"https://dwillis.github.io/uplink/article/2004-june-discovering-car-again-and-again.html","year":2004,"month":"June","monthNum":6,"headline":"Discovering CAR, again and again","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"Editor","affiliation":"NICAR","summary":"The article highlights the growing adoption of computer-assisted reporting (CAR) among journalists across various media outlets, emphasizing the learning opportunities and excitement generated at the IRE Annual Conference's CAR day in Atlanta.","keywords":["journalism","reporting","data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":[],"full_text":"One of the benefits of putting Uplink together six times a year is getting to see all the great stories journalists are doing using computer-assisted reporting on the beats and on the big projects.\n\nThere's so much going on and so much to get excited about as journalists discover the power (and even fun) of analyzing data for news stories.\n\nAs this issue of Uplink goes to the printer, hundreds of journalists from television, newspapers, magazines - you name it - will be gathering in Atlanta the day before the IRE Annual Conference officially begins to pick up CAR tips and story ideas.\n\nAside from the informative panels, the optional CAR day provides a great place for newcomers to hang out with the longtime CAR jockeys and pick up the tricks of the trade.\n\nIt's always edifying to see the informal learning that goes on during the CAR day, and how many of the newcomers leave for their newsrooms back home, excited about the possibilities.\n\nThen, as those journalists use CAR in their reporting and turn out some fantastic stories, all of us get to discover the power of computer-assisted reporting all over again.\n\nContact David Herzog by e-mail at dherzog@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-sex-offenders","slug":"2004-june-sex-offenders","url":"https://dwillis.github.io/uplink/article/2004-june-sex-offenders.html","year":2004,"month":"June","monthNum":6,"headline":"Sex offenders","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR Database Library","summary":"KIRO-Seattle investigated sex offenders living near day-care centers in Washington using mapping analysis to identify registered offenders within one city block of licensed facilities.","keywords":["offenders","daycares","mapping"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["ArcView"],"full_text":"KIRO-Seattle in February aired a series of stories about sex offenders living in or close to day-care centers in Washington. The IRE and NICAR Database Library conducted the analysis for KIRO, using ESRI ArcView to map the registered home addresses of sex offenders and the addresses of licensed day care centers. The analysis identified sex offenders living within one city block of the centers. Read the story and view maps online at www.kirotv.com/investigations/2825037/detail.html.\n\nTo learn more about how IRE and NICAR can help your news organization with data analysis projects, contact the Database Library at 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-upcoming-hands-on-car-training","slug":"2004-june-upcoming-hands-on-car-training","url":"https://dwillis.github.io/uplink/article/2004-june-upcoming-hands-on-car-training.html","year":2004,"month":"June","monthNum":6,"headline":"Upcoming hands-on CAR training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering hands-on computer-assisted reporting (CAR) training this summer, including a Boot Camp in Columbia, Missouri and a specialized GIS mapping workshop, with follow-up support provided to participants.","keywords":["training","journalism","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet","GIS"],"full_text":"Upcoming hands-on CAR training\n\nIRE and NICAR have several training opportunities for journalists seeking hands-on instruction in using computer-assisted reporting.\n\nThis summer marks the last chance this year for in-depth CAR training. There is one remaining CAR Boot Camp in Columbia, Mo., for journalists who want to learn how to acquire electronic information, use spreadsheets and databases to analyze the information and to translate that information into high-impact stories. IRE and NICAR provide follow-up help after participants return to their news organizations.\n\nThe Boot Camp is Aug. 1-6. The Boot Camps will be offered again four times next year, along with other specialized workshops.\n\nJournalists interested in learning how to use geographic information systems (GIS) in their reporting can register for the Mapping Data for News Stories mini-boot camp Aug. 20-22 in Columbia, Mo. The sessions will include address geocoding, thematic mapping, overlaying and spatial analysis.\n\nThose who attend the mapping mini-boot camp will be eligible to purchase GIS software at a discount. In many cases, the cost of the training session and discounted software is less than the full price of the software alone.\n\nFor more information on the mapping workshop and other training opportunities offered by IRE and NICAR visit www.ire.org/training:","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-getting-a-handle-on-federal-campaign-cash","slug":"2004-june-getting-a-handle-on-federal-campaign-cash","url":"https://dwillis.github.io/uplink/article/2004-june-getting-a-handle-on-federal-campaign-cash.html","year":2004,"month":"June","monthNum":6,"headline":"Getting a handle on federal campaign cash","kicker":"SPOTLIGHT: ELECTIONS","author_name":"Aron Pilhofer","author_title":"","affiliation":"The Center for Public Integrity","summary":"Tracking campaign finance has become increasingly complex due to legal changes and the rise of political nonprofit organizations that operate outside traditional disclosure requirements, requiring journalists to use sophisticated investigative techniques and available federal databases to follow the money trail.","keywords":["campaign","finance","disclosure"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["Internet","FTP","spreadsheet"],"full_text":"Getting a handle on federal campaign cash\n\nThere was a time when tracking campaign cash was a straightforward affair. But as a result of actions by Congress and the courts, and the endless creativity of political leaders over the last decade, the money trail has never been more twisted and difficult to follow.\n\nThat's particularly true in this election, the first since passage of the Bipartisan Campaign Reform Act of 2002.\n\nGone are the days when journalists covering state races could blissfully ignore the federal system and vice versa. There is no such thing as a purely federal or purely state election any more.\n\nBoth parties, along with a growing number of outside interest groups, have devised sophisticated national strategies and fund-raising networks to put money into races at all levels, from the state house to the White House.\n\nThe bulk of campaign cash continues to flow through traditional fund-raising sources, such as party committees and PACs. But a significant amount of money comes under the radar, most often through political nonprofit organizations that do not disclose their activities in the usual places.\n\nOver the past four years, the Center for Public Integrity has tracked at least $500 million in activity by these groups, including just over $100 million in the past 15 months alone. These groups do not disclose their activities to any state or federal election authorities, if at all. This makes them ideal conduits for contributors who wish to have an impact on an election with as little fanfare as possible.\n\nAs an unfortunate result, journalists today must be just as sophisticated as those they are covering. They must play the part of attorney, accountant, politico and sleuth to get a clear picture.\n\nFortunately, there is a wealth of data available either in a downloadable, electronic format or searchable on the Web that can help journalists paint a true picture of who gives to whom and why.\n\nInvestigative Reporters and Editors is developing workshops on how to use campaign finance data to report on these political nonprofit organizations.\n\nDownloadable data\n\nRaw campaign finance data for most federal races can be downloaded from two Federal Election Commission Internet sites: the electronic filing site (http://herndon2.sdrdc.com/dcdev) or its FTP server (www.fec.gov/finance/newftp1.htm). The exception is Senate candidates and party caucus committees (specifically, the Democratic Senatorial Campaign and National Republican Senatorial committees), which are exempt from filing electronically. The only source for electronic data for these Senate candidate and party committees is from the commission's FTP site.\n\nBoth sources have their pluses and minuses. The advantage to the electronic filing site is speed and convenience. Reports appear almost as soon as they are submitted by the respective campaigns, making deadline reporting easier than ever. Also, electronically filed forms include expenditures made by the committee, the FTP data does not.\n\nElectronic filings are viewable online or they may be downloaded in a comma-delimited format, which can be easily imported into a spreadsheet or database manager.\n\n(For more information about working with the electronically filed data, see the July-August 2002 Uplink. For further reference see www.claptrap.org/ire/ElectronicFilingandtheFECPart1.pdf and www.claptrap.org/ire/ElectronicFilingandtheFECPart2.pdf).\n\nIn contrast, the FTP data is much harder to work with because, among other things, the files can be quite large and require quite a bit of cleanup. (The IRE and NICAR Database Library provides a cleaned version of this database to journalists at a nominal cost. See www.ire.org/datalibrary/databases/fec for more information).\n\nThe biggest drawback to using the FTP data is the lag time, which can range for a few days for electronically filed reports to as much as a month for those Senate filings that must be manually entered.\n\nOn the other hand, the data is the official electronic record. Electronic filings are considered preliminary because FEC officials have not examined them. Furthermore, the FTP data is designed to be maintained over the long term. Each record in the database is uniquely identified, which allows the data to be updated correctly when filings are later amended (a frequent occurrence).\n\nNICAR has collected literally dozens of tipsheets covering campaign finance data. You can search for tipsheets on the Web at www.ire.org/resourcecenter/initial-search-tipsheets.html.\n\nOther resources\n\nIn addition to raw campaign finance data, there are some other sources journalists should know about. Under the new law, funds used to pay for what are called \"electioneering communications\" must be fully disclosed within 24 hours, even if the group sponsoring is a nonfederal entity or even an individual.\n\nElectioneering communications are a subset of what has come to be known as \"issue ads\" - commercials that are virtually indistinguishable from regular campaign ads, but do not specifically call for the election or defeat of a candidate. These ads proliferated over the past decade because they could be bought with unregulated soft money. The new law tries to reign in some issue ads by restricting some groups from airing them within 30 days of a primary election or 60 days of a general election, and requires full disclosure from other groups or individuals who buy such ads.\n\nThese filings are posted on the FEC Web site at www.fec.gov/pages/electioneering.html.\n\nThe FEC also has launched a searchable database of enforcement activities, which can be found at http://eqs.sdrdc.com/eqs/searcheqs. The interface allows users to search for cases in any number of ways - by respondent, by case number or even a fuzzy word-matching scheme.\n\nUnfortunately, the system only includes newer cases - those made public since January 2002. But it includes all the documents associated with the case in a searchable format, even if they were not specifically submitted in that format. For more information about the system and its limits, see http://eqs.sdrdc.com/eqs/eqs_info.html.\n\nPolitical nonprofits\n\nA more recent development has been the rise of political nonprofit organizations, which are generally (though not entirely accurately) referred to as Section 527 organizations. The name refers to the section of the federal tax code under which these groups and all political committees achieve tax-exempt status. But the difference between 527s and other political committees is enormous.\n\nFor one, the 527s don't file with the FEC or any state elections regulatory agency (unless that state's law specifically requires it). Prior to 2000, they didn't submit any financial disclosure documents of any kind. Since then, they have filed with the Internal Revenue Service, and those forms are available at http://eforms.irs.gov.\n\nA few words of caution: because of widespread confusion about which committees are required to file, there are literally tens of thousands of erroneous reports on the IRS site, most of them from state and local candidates who already file with their state elections authorities. The Center reviewed these filings and found about 500 committees that fit the definition of a true 527 out of more than 20,000 filed with the IRS.\n\nFor about the past year, most of these committees have been required to file electronically with the IRS. Those filings can be downloaded in a comma-delimited format from the IRS Web site. The download page includes a very detailed document that explains the contents and layout of the data file, which is a bit confusing.\n\nThe database we created at The Center for Public Integrity is searchable and includes data back to 2000. (See www.publicintegrity.org/527). Other groups have tracked 527 activities. The Center for Responsive Politics (www.crp.org) also has posted a searchable database of filings from last year forward and includes the largest 527 organizations active in federal elections. PoliticalMoneyLine (at http://politicalmoneyline.com) has tracked 527 activity for several years, but details of its database are restricted to subscribers and its information is not as comprehensive as that offered by the Center for Responsive Politics or The Center for Public Integrity.\n\nContact Aron Pilhofer by e-mail at apilhofer@publicintegrity.org.\n\nContact Derek Willis by e-mail at derek@thescoop.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-finding-data-for-state-elections","slug":"2004-june-finding-data-for-state-elections","url":"https://dwillis.github.io/uplink/article/2004-june-finding-data-for-state-elections.html","year":2004,"month":"June","monthNum":6,"headline":"Finding data for state elections","kicker":"SPOTLIGHT: ELECTIONS","author_name":"Derek Willis","author_title":"","affiliation":"Center for Public Integrity","summary":"State campaign finance reporting varies dramatically across the country, with some states offering easily accessible electronic databases while others provide only paper filings or poorly formatted data. Journalists researching state political money must navigate inconsistent formats, missing records, and incomplete disclosure practices to piece together a comprehensive picture of campaign spending.","keywords":["campaign","finance","disclosure"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel","Access","HTML","ASCII"],"full_text":"State elections account for hundreds of millions of dollars in activity every two years, but tracking the money that fuels those races can be a cumbersome process. Nearly every state has its own method of reporting campaign finances and most don't offer the same kind of data that's available at the federal level.\n\nState data comes in a variety of formats, but electronic access has been increasing lately. Some states provide data in electronic format or display the records on a Web site in a manner that could be imported into a database, while other states provide filings in Adobe Portable Document Files or other image files, or on paper.\n\nDuring a recent project on state political parties, the Center for Public Integrity found at least 17 states that offered campaign finance data in a database, HTML or ASCII text format. Among the best are Ohio, which has comma-delimited text files for individual candidates and for all party organizations, and New York, which offers a single file containing contributions and expenditures during a given filing period. Washington, Idaho and Colorado also provide downloadable files.\n\nSeveral other states are going the route of the searchable Web database, where a user can enter search terms or date ranges and get back text files that can easily be put into Microsoft Excel or Microsoft Access. States such as Florida and Maryland do this. Others offer data on a filing-by-filing basis - not exactly the best when you want to download everything - but it's better than nothing. Pennsylvania, Illinois and Rhode Island are examples of the latter. Typically, such files can be batch-downloaded if you know the filenames.\n\nStates that offer HTML versions of their data that can be put into a database with a little bit of work include Arizona, North Dakota, North Carolina and Nebraska. In some instances, you can simply save the HTML and use Access import utility, which will convert HTML tables into Access pretty easily. In other cases, it may make sense to develop a script that retrieves the pages or automatically \"scrapes\" the information from them.\n\nOther states offer what can be called \"pseudo-Web disclosure,\" providing only summary information but not the itemized records.\n\nIt should come as no surprise that such practices are wildly inconsistent in terms of the information provided and whether electronic records match paper filings. Maine, for example, offers text filings on its Web site purporting to contain campaign finance information, but they very often are missing thousands of records from party committees. Before revamping its system, Indiana offered downloads that included all filings by certain committees, meaning that amended records were simply piled on top of the original data with the potential for significant overlap.\n\nExpenditure data, while often overlooked, can be a valuable resource. But it can have pitfalls, particularly when it comes to the issue of \"subvendors,\" the breakdown of payments made by credit card, for example. Check with your state to see if it requires such itemization on its reports and whether any data available makes it easy to separate the lump sum payments from the itemized records (in most cases, you want the latter).\n\nHere are some tips for handling state campaign finance data:\n\nKnow what the files, and thus the data, are supposed to look like. It makes sense to grab a copy of a blank report, even if you get all-electronic data.\n\nRemember that a number of states have some form of public financing, and those records often appear in the general contributions. While it may be technically accurate to list the state government among the largest contributors to a candidate, you probably will want to remove those records from your query results.\n\nIf you are interested in the activities of political parties be sure to check your state's laws to see what kinds of accounts parties can maintain. A handful of states permit parties to have operating accounts for political use and administrative accounts for overhead and generic expenses. Sometimes these administrative accounts have little or no disclosure. South Carolina's parties will begin disclosing contributions to their administrative accounts at the end of the year. Other parties have building funds or housekeeping accounts that file separate reports, so be sure to check or you might be missing some of the picture.\n\nDon't forget the small stuff: states treat small-dollar contributions very differently. Some states require lump sum reporting of contributions below a certain threshold, while others, like Nebraska, don't require any reporting of contributions that are not itemized. In smaller states, such figures could seriously affect your numbers.\n\nContact Derek Willis by e-mail at derek@thescoop.org","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-campus-crime-data","slug":"2004-june-campus-crime-data","url":"https://dwillis.github.io/uplink/article/2004-june-campus-crime-data.html","year":2004,"month":"June","monthNum":6,"headline":"Campus crime data","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library now offers Clery Act Campus Crime data from 2001-2002 containing 19 tables of crime information reported by campus police, which journalists can use to investigate criminal activity at educational institutions.","keywords":["crime","campus","data"],"topics":["Crime & Justice","Data & Databases","Education"],"technologies":["CD-ROM"],"full_text":"New data from the IRE and NICAR Database Library can help journalists find campus crime stories. The Clery Act Campus Crime data for 2001 and 2002 contains 19 tables with information about crimes reported to campus police or local law enforcement agencies.\n\nOffenses such as homicide and sex crimes are included even if they were not prosecuted. There are separate tables for hate crimes and arrest reports, which include totals for liquor and drug law violations. This data is compiled with information campuses are required to publish and distribute as an annual campus security report.\n\nThe Rocky Mountain News in Denver recently used this data to draw attention to drug and alcohol violations at the University of Colorado at Boulder. Read the story on the Web at www.rockymountainnews.com/drmn/local/article/0,129 DRMN_15 2712250,00.html.\n\nTo order the campus crime data, call the Database Library at 573-884-7711 or 573-884-7332.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-fatal-crash-data-shows-truck-dangers","slug":"2004-june-fatal-crash-data-shows-truck-dangers","url":"https://dwillis.github.io/uplink/article/2004-june-fatal-crash-data-shows-truck-dangers.html","year":2004,"month":"June","monthNum":6,"headline":"Fatal crash data shows truck dangers","kicker":"TRANSPORTATION","author_name":"Kurt Rogahn","author_title":"reporter","affiliation":"The Gazette","summary":"A Cedar Rapids Gazette investigation into fatal accidents on Interstate 80 revealed that nearly half of 133 fatal accidents from 1994-2001 involved semi-trailer trucks, leading to changes in driver behavior and demonstrating the value of data-driven journalism.","keywords":["accidents","trucks","investigation"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["FARS","Access","CD-ROM","Internet"],"full_text":"Fatal accidents on Interstate 80 in Iowa are seen in a different light these days.\n\nHere's how another reporter at the newspaper I work for closed a routine news story about an accident that killed two people in a four-vehicle accident that include a semitrailer truck:\n\n\"A Gazette investigation published in November showed that nearly half the fatal accidents on I-80 from 1994 through 2001 involved semi-trailer trucks. Of the 133 fatal accidents, 62 involved semis. In all, 84 people died in the crashes involving semis.\"\n\nIt was rewarding to see that paragraph.\n\n\"Danger Zone,\" the package of stories it referred to, resulted from months of research, database analysis, more than a few dead ends and a fair amount of uncertainty. The package included stories from KCRG, our affiliated television station, and an online package with animated graphics.\n\nThe stories explained how increasing traffic volumes, basic physics and the way drivers of cars behave behind the wheel all contribute to fatal accidents involving semis. Big trucks can't stop as quickly as a car, nor can semi drivers steer out of trouble.\n\nSome people have told me they've changed their driving habits. That's even more rewarding than seeing the conclusions of \"Danger Zone\" repeated in a short story about a fatality.\n\nMy reporting started with a suggestion from a retired Gazette reporter who had just returned from a long trip: Look into the way cars and trucks interact on the road. He thought we might learn something from truck inspection reports. But that wasn't as helpful as examining FARS, the federal government's Fatality Analysis Reporting System.\n\nThe Gazette bought the data covering 1994 to 2001 on CD-ROM from the IRE and NICAR Database Library. I set out to find a story - whatever it might be.\n\nI began by examining outdated reports based on FARS from the Insurance Institute for Highway Safety, and localizing them for Iowa. I looked at how other news organizations had studied cars, trucks and roads. I tried to get trade groups and federal agencies to put me in touch with their researchers, to get coaching and to test some of the queries I created in Microsoft Access.\n\nOne of those queries turned up a hot spot with more fatal accidents than any other part of Interstate 80 in Iowa: rural Cedar County, about 40 miles southeast of Cedar Rapids. County traffic volumes have increased to 33,000 vehicles daily from 22,000 in a little more than a dozen years. Every third vehicle there is a heavy truck.\n\nMy Internet searches led me to a question about highway traffic deaths that I thought I could resolve: Who's to blame for fatal crashes involving trucks and cars? The Los Angeles Times reported that truckers are at fault in about half of all California large-truck crashes. Yet an American Automobile Association study fixes more blame on automobile drivers. Advocacy groups fall on both sides of the argument.\n\nMy contact with other journalists across the country led me to Daniel Blower of the Transportation Research Institute at the University of Michigan. He warned that the increase of accidents in Cedar County - 15 of 19 fatal accidents on I-80 in eight years had involved semis - could be the result of chance alone. He also pointed out that the FARS data does not disclose who is to blame.\n\nBut from studying more detailed accident data, Blower has concluded that actions by passenger vehicle drivers contribute more often to fatal crashes than do the actions of the truck drivers.\n\nSo I returned to the computer and ran more queries, this time concentrating on particular roads. By comparing a database containing only fatal accidents in which semis were involved (developed with help from our corporate trainer, Steve Cope) against the entire FARS database for Iowa, a major finding finally jumped out of the numbers. Nearly half the fatalities on I-80 in the eight-year study period involved semitrailer trucks. No other rural interstate in Iowa had such a high proportion of accidents or such a high percentage of use by trucks.\n\nAs I told KCRG reporter/anchor Renee Chu, \"The accidents involving trucks occur where the trucks are.\" Chu interviewed me on camera about my findings for one of her stories.\n\nDuring my reporting I found a former truck driver who had left the business after having been involved in a fatal accident four years ago on I-80 in Cedar County. I also found the family of a woman who in January 2000 was killed on another segment of I-80 in Cedar County. I wanted to tell about an accident from the viewpoint of the trucker, and from the viewpoint of the family of a victim.\n\nBoth the Gazette and KCRG gained from teaming up for the \"Danger Zone\" project.\n\n\"The joint project allowed the station to have access to concrete data that we would typically not devote the time and resources to uncover,\" Chu said.\n\n\"It also allowed us to convey relevant safety information that affects every person who drives.\"\n\nThe televised report received positive responses, primarily from people whose loved ones worked as truckers and those who are tired of drivers who disregard trucks on the highway.\n\nThe Gazette, meanwhile, gained from the exposure on television.\n\nBut Iowans may have gained the most from the stories that showed how traffic on part of the interstate system had neared its capacity.\n\nContact Kurt Rogahn by e-mail at Kurt.Rogahn@gazettecommunications.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-broadcast-other-winners-rely-on-data-work","slug":"2004-june-broadcast-other-winners-rely-on-data-work","url":"https://dwillis.github.io/uplink/article/2004-june-broadcast-other-winners-rely-on-data-work.html","year":2004,"month":"June","monthNum":6,"headline":"Broadcast, other winners rely on data work","kicker":"IRE AWARDS","author_name":"Megan Clarke","author_title":"","affiliation":"IRE and NICAR","summary":"IRE's annual investigative reporting contest received 595 entries, with many using computer-assisted reporting techniques like databases and tracking software. The article highlights award-winning television investigations that employed CAR methods, including exposés on industrial safety violations, port security gaps, and sexual assault cover-ups at the Air Force Academy.","keywords":["investigation","database","awards"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":["Excel"],"full_text":"IRE received 595 entries this year in its annual investigative reporting con- test. Many of the entries used a wide variety of computer-assisted report- ing techniques, including building original databases and using soft- ware programs to track shipments by sea. Here is an overview of television, other media and special category win- ners and finalists that used CAR. See the May-June IRE Journal for a full list of winners. You can order copies of these stories from the IRE Resource Center. Con- tact the center at 573-882-3364 or rescntr@ire.org. Television Network-Certificate PBS Frontline/New York Times Tele- vision/Canadian Broadcasting Corpo- ration: \"A Dangerous Business\" by Neil Docherty, David Rummel, Lowell Bergman, Linden MacIntyre, David Barstow, Robin Stein, James, Sandler, Nelli Kheyfets, Remy Weber, Jason Maloney, Lynda Baril, Louis Wiley Jr., David Fanning, Ann Derry and Lawrie Mifflin. This hour-long documentary, aired in conjunction with a series of articles in The New York Times, revealed that cast iron pipe-maker McWane Inc. had accumulated more safety violations than all of its major industry competi- tors combined. The investigation in- cluded an analysis of Occupational Safety and Health Administration (OSHA) and Environmental Protection Agency (EPA) data that uncovered dangerous working conditions. The documentary also has been awarded a Columbia duPont Award, a Goldsmith Prize for investigative re- porting, the George Polk Award and a Peabody Award. Network-Finalist Primetime Thursday, ABC News: \"Nuclear Smuggling,\" by Brian Ross, Rhonda Schwartz, David Scott, Yoruba Richen and Gerilyn Curtin. On the two-year anniversary of the Sept. 11 terrorist attacks, Primetime Thursday aired a story that revealed gaps in U.S. port security. As a ship- ping client, ABC was able to track a uranium shipment using the tracking system on the shipping company's Web site. The tracking system was cru- cial in informing ABC when the ship- ment passed certain ports. This report also won second place in the 2004 National Headliner Awards for investigative reporting and the 2004 Excellence in Media Silver An- gel Award. Top 20 Market-Certificate KMGH-Denver: \"Honor and Betrayal: Scandal at the Academy,\" by Jeff Har- ris, John Ferrugia, Kurt Silver, Jason Foster and Byron Grandy. KMGH's investigation exposed the U.S. Air Force Academy system that deterred and even punished female cadets for reporting sexual assaults. This 10-month project yielded more than 100 stories, including a one-hour documentary, and prompted several investigations and the removal of Academy leadership. The station created its own database in Microsoft Excel of current and former female cadets in order to research and locate victims. The investigation also won a Colum- bia duPont Award, a Peabody Award, the National Headliner Award - Best of Show, a Society of Professional Jour- nalists award, and a CINE award. Top 20 Market-Finalist WFAA-Dallas: \"State of Denial,\" by Brett Shipp, Mark Smith and Kraig Kirchem. WFAA's seven-month investigation exposed Texas' crippled workers' compensation system. In many cases, state regulators failed to en- force labor laws and overlooked in- surance companies that unjustly de- nied benefits to injured workers. WFAA built an Excel spreadsheet with hundreds of patients' names, insurance carriers, physicians and status of their benefits and medical treatment. The series also won the Texas AP Broadcasters Award for investigative journalism. Top 20 Market-Finalist WESH-Orlando: \"Building Homes: Building Problems,\" by Stephen Stock, Michelle Meredith, Shannon Hori, Dave McDaniel, Kathy Marsh, Travis J. Sherwin, Jim Payne, Peter A. Delis, Mark Rice and Jason Morrow. WESH joined forces with the Orlando Sentinel to inspect 406 central Florida homes built in 2001. The investigation found that 80 percent of the homes had at least one major building flaw. The station obtained a database of in- spection records from each municipal- ity and used Microsoft Access to ana- lyze its own database of inspectors' daily reports. The 14-story series also has won a Peabody Award. Below Top 20 Market-Finalist KAKE-Wichita: \"Lake Kahola: A City Manager's Ethical Abyss,\" by Paul Aker and Craig Lee. KAKE uncovered a series of no-bid consulting contracts authorized by the Wichita, Kan., city manager and issued to his close personal friend and seller of lakefront property. The four bids to- taled $40,000 and KAKE found the manager's friend never completed the majority of the work required. KAKE used a proprietary database that included a property's appraisal and its address that it used for prelimi- nary information and verification. Below Top 20 Market-Finalist . WITI-Milwaukee: Still Drunk, Still Driv- ing,\" by Bob Segall and Michele Murray. WITI's investigation found thousands of drunken drivers with multiple driv- ing while intoxicated convictions who were still driving vehicles that should have been seized by law enforcement. WITI created a database of court in- formation for the repeat drunken driv- ers. The station also analyzed the number of cases in which judges or- dered vehicle seizures. The report also won a regional Emmy and a Milwaukee Press Club Award. Other Media Magazine-Certificate Time: \"U.S. Energy Policy,\" by James B. Steele and Donald L. Barlett. A nine-month investigation revealed that the 30-year failure of the United States to adopt and stick with a na- tional energy policy will force the na- tion to struggle with recurring oil and gas crises in the future. Using Lotus 1-2-3 and Excel, Barlett and Steele constructed spreadsheets that charted energy trends containing information from oil wells drilled to gal- lons of oil refined. They also exten- sively used the Department of Energy's Energy Information Admin- istration, which has an online store- house of energy data. Magazine-Finalist The Atlantic Monthly. \"Texas Clem- ency Memos,\" by freelancers Alan Berlow and Toby Lester. After two years of research, Berlow found that George W. Bush, while gov- ernor of Texas, had approved the ex- ecutions of 57 criminals without \"care- fully\" and \"thoroughly\" reviewing their cases as he publicly promised. Berlow reviewed hundreds of pages of reports and communications between Bush and his chief legal counsel, Alberto Gonzales, now White House counsel. Berlow used Microsoft Access to sort through data to find documents relat- ing to executions and to filter out cor- respondence to Gonzales. Radio-Certificate WNYC-New York Public Radio: \"Hand- shake Hotels,\" by Andrea Bernstein and Amy Eddings. WNYC's seven-month investigation un- covered how New York City spends more than $180 million a year to house the homeless in hotels without contracts. WNYC created a database using Microsoft Excel and Access to track the biggest operators of hotels for the homeless and calculate how much the city was spending. The series also won the Society of Pro- fessional Journalists Sigma Delta Chi Award, Heywood Broun Award, Re- gional RTNDA Edward R. Murrow Award, Newswomen's Club of New York Front Page Award and the New York State Associated Press Broad- casters Association Award. Online-Certificate Center for Public Integrity: \"The Water Barons,\" by William Marsden, Maud Beelman, Daniel Politi, Erika Hobbs, Bill Allison, Andre Verloy, Laura Peterson, Samiya Edwards and Aron Pilhofer. A yearlong investigation by the Inter- national Consortium of Investigative Journalists, a project of the center, found that public drinking water sys- tems worldwide are being privatized. The investigation included the analy- sis of five databases, including one from the World Bank containing data about contracts for drinking water projects. ICIJ used several programs including Microsoft SQL Server, Ac- cess, Excel and ESRI ArcView. Online-Finalist Center for Public Integrity: \"Well Con- nected,\" by John Dunbar, Bob Williams, Scott Singleton and Morgan Jindrich. Ten months of research found that Fed- eral Communications Commission of ficials were \"showered with nearly $2.8 million in travel and entertainment over the past eight years, most of it from the telecommunications and broadcast in- dustries the agency regulates.\" The series involved the analysis of data from the commission's Cable Opera- tions and Licensing System. \"Well Connected\" also won the Online News Association's award for enter- prise journalism. Book-Medal \"Perfectly Legal: The Covert Cam- pain to Rig Our Tax System to Ben- efit the Super Rich - and Cheat Ev- erybody Else\" by David Cay Johnston and published by Penguin Group USA. Johnston's book unravels the com- plexities of the U.S. tax system, and shows how the middle class subsidizes the rich and the income gap is greater than previously reported. Johnston used Excel to create hun- dreds of spreadsheets using IRS sta- tistical reports and similar data sources. He also used Access. Special Citation for International Entry The Sydney Morning Herald. \"Public Land, Private Profit,\" by Gerard Ryle and Brian Robins. A seven-month investigation revealed how public property across Australia was being leased to corporations, wealthy individuals and political donors at rates far below market value. Journalists for the newspaper built sev- eral databases using Cyber. The pa- per also compiled in Excel spread- sheets a list of 18,000 leases from the Department of Lands. Student Work-Certificate Daily Tar Heet. \"Raising the Cap,\" by John Frank, Jamie Dougher, Matt Hanson, Joe Rauch, Suzanne Presto and Lynne Shallcross. The independent newspaper used poll and admissions data to counter the University of North Carolina at Chapel Hill's introduction of a policy to increase out-of-state enrollment. After publication, the university decided to put the debate on hold indefinitely. Excel was used to analyze admissions data and retention surveys. The paper also used SPSS to look at poll data. Student Work-Finalist Capital News Service: \"Effluent Es- calation,\" by Dan Wilcock, University of Maryland. The investigation revealed that over- flows of raw sewage from municipal sys- tems into Chesapeake Bay increased more than 1,200 percent in the past three years. The reports exposed errors in the Maryland Department of Environment's data and reporting system. Wilcock used Excel to analyze three years of sewage overflows. FOI Award-Medal Argus Leader, Sioux Falls, S.D.: \"Governor Bill Janklow's pardons and commutations,\" by David Kranz, Patrick Lalley, Stu Whitney, Terry Woster and Jon Walker. After dealing with inadequate open record laws in South Dakota, the Argus Leader published several articles on the hidden activities of former Gov. Bill Janklow. The paper's six-month investigation uncovered that Janklow commuted more prison sentences than any other governor in the nation. He sealed about 60 pardons and didn't inform the Board of Pardon and Paroles. Using Excel, the Argus Leadercreated its own spreadsheet and calculated the average age of inmates, average length of commutation and prevalence of serious crimes. Associated Press Managing Editors' Freedom of Information Award. The series also won the Scripps Howard Foundation's public service award and FOI-Finalist The Sacramento Bee. \"Liberty in the Bal- ance,\" by Sam Stanton and Emily Bazar. The investigation uncovered how the war against terrorism is eroding civil liberties of Americans and immi- grants. The four-part series focused on immigrants jailed indefinitely on minor charges because of unwar- ranted suspicions of terrorist activity. The paper used Zoomerang to cata- log the results of surveys completed by California libraries being asked for patron information by federal officials. The paper also compared immigra- tion statistics from each year. \"Liberty in the Balance\" won an inves- tigative reporting award from the As- sociated Press News Executives Council, second place National Head- liner Award for public service and third place in the Best of the West contest. Contact Megan Clarke by e-mail at megan@ire.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-june-spotting-sex-offenders","slug":"2004-june-spotting-sex-offenders","url":"https://dwillis.github.io/uplink/article/2004-june-spotting-sex-offenders.html","year":2004,"month":"June","monthNum":6,"headline":"Spotting sex offenders","kicker":"MAPPING","author_name":"Tisha Thompson","author_title":"","affiliation":"WMAR-Baltimore","summary":"A Baltimore newsroom used MapPoint software to map over 3,500 sex offenders from the state registry, discovering clusters near schools, though they faced technical challenges incorporating the Windows-based maps into their Mac-based graphics system.","keywords":["mapping","offenders","software"],"topics":["Investigative Methods","Technology","Crime & Justice"],"technologies":["Access","Excel","MapPoint","ArcView","CD-ROM","Internet","PDF"],"full_text":"Nobody wants sex offenders in his or her backyard, and most people lack the technology to see if any live nearby. It's easy enough to go to a sex offender registry on the Web, type in an address and see whether any live on your street. But, since most people are not cartographers, they may not be aware of sex offenders living on streets that wind behind their homes or a block away from a school.\n\nWe initially wanted to see if there were any sex offender clusters (large groups of registered offenders living together in the same neighborhood or at the same address) in Baltimore. We filed a public information request with Maryland's Department of Public Safety and Correctional Services for a CD-ROM containing the state Sex Offender Registry. (See the May/June IRE Journal for more about reporting on sex offenders).\n\nEven though that information is readily available on the Internet in a Portable Document Format file, the department initially didn't want to give us the information. It claimed that because the registry changes every day, any information burned onto a CD would immediately be outdated. But when I responded that I wanted to map the information, the department suddenly became super helpful. \"Our computer guys want to know what you can do with the data,\" they said. And I promised to verify the address of any sex offender we specifically referred to in our story.\n\nI imported the comma-delimited text data into Microsoft Access and Excel. I used Access to find the clusters by sorting and grouping and counting specific addresses and ZIP codes. I used Excel to import the data into MapPoint, a Microsoft mapping program.\n\nWe bought MapPoint instead of ESRI ArcView because it's less expensive ($299 list price) and easier to use. It took me about 10 minutes to somewhat master the software. MapPoint imported the more-than 3,500 sex offender rows from Excel within two minutes, including name, address and offense of each offender. The software already contains the addresses of places such as police departments, libraries, museums and other public buildings. I asked MapPoint to map all school buildings and quickly saw there were clusters located near school buildings.\n\nThe MapPoint address matching process did have some shortcomings. I had to manually approve some addresses because a ZIP code didn't match, a street name was slightly off and so forth. I also noticed a few school names were out of date, even though the addresses were correct. And I realized the software would only mark each location once, so there is no way to tell if several people live at one address without first grouping and summarizing the data.\n\nOur biggest hurdle came when we tried to incorporate the maps into the story. Like many television stations, our graphics department uses Macintosh computers and MapPoint runs on Windows. Our computer guru tried everything he knew, but we were working on deadline. We never were able to figure out how to import the maps I created into the graphics computers.\n\nWe ended up simply shooting the PC screen as I zoomed in and out of the maps in MapPoint. Surprisingly, the video ended up being much more effective than any graphic we could have made. (It wouldn't have worked if our photographer hadn't managed to get rid of that annoying computer flicker). We started with a map of Maryland, then zoomed in to show how densely packed the sex offenders are in Baltimore, zoomed in again to a ZIP code that had more than 100 offenders, and then zoomed in once again to a street where almost a dozen offenders lived in the same residence.\n\nThe maps showed sex offenders living all around Baltimore.\n\nWe did a lot of old-fashioned reporting - verifying addresses, getting court records and finding people. But the red dots on the maps told the story best. Even experienced reporters at our station were surprised at what they saw on the computer screen. Maryland looks like a big, red morass. After zooming in, Baltimore became a glaring red dot. I'm convinced that no single piece of video could have illustrated the statement that \"we found more than 100 living in the same neighborhood\" as well as the map that showed the dots lining a road.\n\nThe maps showed sex offenders living all around Baltimore. Practically everyone in the newsroom - as well as some viewers - asked me to type in their addresses so they could see how many dots were in their neighborhood.\n\nAnd the department's computer staff? They are trying to figure out if they can incorporate the mapping technology onto the Sex Offender Registry's Web site. All of which convinced me that if you have the right kind of story, maps will not only help research, but dramatically enhance storytelling as well.\n\nContact Tisha Thompson by e-mail at Thompson@wmar.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-scholarship-program-takes-from-the-poor","slug":"2004-june-scholarship-program-takes-from-the-poor","url":"https://dwillis.github.io/uplink/article/2004-june-scholarship-program-takes-from-the-poor.html","year":2004,"month":"June","monthNum":6,"headline":"Scholarship program takes from the poor","kicker":"EDUCATION","author_name":"Maurice Tamman","author_title":"Reporter","affiliation":"The Atlanta Journal-Constitution","summary":"Georgia's HOPE scholarship program, funded by lottery revenues, has distributed $2.1 billion to 700,000 students but faces financial sustainability challenges and criticism for disproportionately benefiting wealthy, white communities while being subsidized by lottery players in poor and minority neighborhoods.","keywords":["lottery","scholarship","inequality"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Georgia's lottery and HOPE college scholarship program recently cel- ebrated its 10th birthday. Until recently, pols have been able to sing its praises by noting that 700,000 Georgians have had their college education funded by the programs and that HOPE has dis- tributed about $2.1 billion. In a recent Journal-ConstitutionlZogby America poll, nine out of 10 Georgians applauded, saying HOPE is a good use of lottery dollars. Despite all the adulation, elected state leaders are faced with the reality that without changes to the HOPE pro- gram, the cash paid out to Georgia's college students will likely outstrip lot- tery sales as soon as 2007. After the spring legislative session, Gov. Sonny Perdue signed a budget that froze the money students can get from HOPE and tightened how the 3.0 grade point average is calculated. The budget also limited the number of credit hours for which technical school students can receive HOPE money. But, for the first time, part- time private college students would be eligible for the scholarship. Wholesale changes to the program, however, could be treacherous. The HOPE program most benefits those in politically powerful and wealthy com- munities, usually with the best high schools. In many ways the program had become an entitlement for middle- and upper-class, largely white, Georgians. The challenge for the newspaper in examining the HOPE program was to find data that would measure the pro- grams' successes and shortcomings. These, however, are not heady days for public data-gatherers. State edu- cation bureaucrats and lottery officials refused to share any individual student financial and academic information or store lottery sales and winner data. As a result, we were forced to employ the \"dead flesh\" method made famous by Victor Frankenstein when building his monster. When we were done, however, we had documented issues many would have preferred the villagers not gaze upon: The HOPE programs are largely sub- sidized by lottery players living in pre- dominately black and poor ZIP codes. HOPE scholarships were awarded in the highest numbers in wealthier, white ZIP codes. Some proposed changes to the HOPE program likely will result in fewer poor and minority students qualifying for the scholarship. Some high schools had lowered stan- dards so more students initially quali- fied for HOPE but were entirely unpre- pared for college academics. Less than 40 percent of students could keep their HOPE scholarships past their freshman year. (Generally, a B average qualifies a student for a scholarship.) Many of the students who received HOPE would have gone to college re- gardless of the scholarship. And a ma- jority of HOPE scholars at several of the state's largest universities came from families with income exponentially higher than the cutoff for federal needs- based assistance. The paper's director of computer-as- sisted reporting, David A. Milliron, and I split the universe in half. He focused on academic issues, such as grade in- flation and how proposed changes to HOPE would alter who qualified as a HOPE scholar. I focused on the eco- nomics and social cost of HOPE and the lottery. The lottery folks gave us data for ticket sales and winners of amounts over $600 aggregated by ZIP code and year since the lottery began. We would have preferred the lottery data by store and winner, but the legislature specifically exempted the lottery from the state open records law. Having the winners data, however, proved to be critical. We had looked at the sales data in the past and it was impossible to know if sales were driven by those living in a ZIP code or those driving into or through a ZIP code. (ZIP codes on interstates and major road- ways, especially those near the state line tended to have disproportionately higher sales.) Given the millions of tickets sold, the number of winners in each ZIP code seemed to be a fair predictor of how popular the lottery was there. We married that data with the HOPE awards aggregated by ZIP code from the Georgia Student Finance Commis- sion and Census demographic data, What we found, unsurprisingly per- haps, is that ZIP codes with the high- est number of lottery winners had rela- tively few HOPE scholars while those with highest number of HOPE schol- ars had relatively few lottery winners. The lottery-winning ZIP codes were poor, largely black; the HOPE schol- arship ZIP codes were significantly wealthier and predominately white. Consider this: The ZIP code with the most lottery win- ners was 7 percent white, 87 percent black, and 6 percent other. Median household income was $36,000 in 2000. The state paid out $1.3 million in lottery winnings. There were 369 HOPE schol- arships totaling about $900,000. The ZIP code with the most scholarship recipients was 79 percent white, 8 per- cent black, and 13 percent other. Me- dian household income was $71,000 in 2000. The state paid out $391,000 in lot- tery winnings. There were 1,331 HOPE scholarships totaling $3,850,094. This income redistribution was best il- lustrated by a University of Georgia student working at an Athens, Ga., convenience store. Reporter Matt Kempner wrote: The 25- year-old economics major says the scholarship money he got for four years was nice, but it had little to do with his decision to go to college. \"All the time, moms come in and cash their welfare and child support checks and buy tickets,\" he says. \"Bums hand me a quarter, three dimes, a handful of pennies, whatever they've got on them so they can play. They're just buying a dream.\" [The student] says he feels bad for the players and tries to make sure they know they're appreciated. \"I look them in the eye and tell them thanks for paying my tuition.\" Some teachers and educational profes- sionals questioned the basic qualifica- tion for a HOPE scholarship - a B aver- -especially for incoming freshman. Once again, examining this with data wasn't easy since the state doesn't re- lease student level grade data. Instead, Milliron looked at aggregate grade results and post-secondary data compiled by the school systems, the College Board, the Board of Regents and Georgia col- leges and universities. At Cobb County's Walton High School and DeKalb County's Towers High School, the average GPA for students headed to state public colleges in 2000 was a B, reporter Patti Ghezzi reported. Walton, however, averaged 1117 on the SATs in 2000 com- pared to 833 at Towers. Perhaps predictably, we found Walton graduates had an average GPA of 2.9 at the end of their freshman year. Tow- ers graduates, however, had an aver- age GPA of 2.2. Clearly, some school systems were softer than others. Looking at data col- lected at the college level, we found a disproportionate number of DeKalb graduates who needed remedial work in college. For example, at Lithonia High School in Gwinnett County, 41 percent of graduates who landed at in-state public colleges in fall 2001 needed re- medial courses. Yet two-thirds of the graduating class had high enough grades to qualify for HOPE. One of the proposals now being con- sidered to restrict HOPE is connecting HOPE scholarships to SAT scores. By looking at data from a number of metro Atlanta school districts, however, Milliron was able to show that doing so would disproportionately affect poor, often black students. For instance, 53 percent of DeKalb County graduates reportedly earned a B average in 2002 and thus were eligible for HOPE. But Milliron's analysis showed that if the state were to require a mini- mum 1000 on the SAT, nearly 48.2 per- cent of the minority graduates in DeKalb would be ineligible for HOPE in a school district that is 90 percent minority. Unfortunately, it is impossible to know the race of HOPE recipients because no one collects that data. In fact, most colleges and universities stopped collecting students' family in- come information in recent years. (A federal financial aid form is not needed for HOPE applicants.) For those that did have data, the State Board of Regents, which oversees the state public university system, directed the schools not to give out individual student records, even if they scrubbed the data to completely obscure stu- dents' identities. We battled with the board over this is- sue early in project. Like petrified vil- lagers, however, the board armed themselves and the targeted schools with the federal Family Educational Rights and Privacy Act (FERPA) and refused to budge. So we focused on three fairly represen- tative institutions. We asked them to look at the data from the last year they collected income data and aggregate the number of students based on ranges of family income that we defined and the average income in each range. (Almost all schools had stopped collect- ing this data earlier this decade.) We found what the ZIP code analysis had suggested: HOPE scholars tended to come from higher income families. During the 1999-2000 school year at Georgia Southern University, for example, HOPE scholars' family income averaged $67,000, about $25,000 more than the state's median income that year and $10,000 more than students who didn't qualify for HOPE; 46 Georgia Southern Hope scholars' families earned more than $250,000 a year, averaging $453,000. Further, using data from the Georgia Financial Aid Commission we were able to determine that three-quarters of HOPE scholars come from families who either have incomes too high to qualify for federal needs-based aid program or who don't bother to apply for it. The writers capably and carefully constructed project from these var- ied data elements and presented reasoned findings. Even HOPE founder and former Gov. Zell Miller could only muster this criti- cism in a letter to the editor: You fought the lottery tooth and nail from the first day I mentioned it in 1988. And now, you're going after HOPE like a vulture picking apart roadkill.\" Given Miller's choice of words, I think it's time to abandon the Frankenstein analogy before it's too late. Contact Maurice Tamman by e-mail at mtamman@ajc.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-june-making-a-point-with-geocoding","slug":"2004-june-making-a-point-with-geocoding","url":"https://dwillis.github.io/uplink/article/2004-june-making-a-point-with-geocoding.html","year":2004,"month":"June","monthNum":6,"headline":"Making a point with geocoding","kicker":"Tech tip...","author_name":"Holly Hacker","author_title":"","affiliation":"IRE and NICAR","summary":"A KIRO-Seattle investigation using GIS analysis and geocoding discovered that over 600 sex offenders in Washington state live within 100 yards of licensed day-care centers, violating state law. The article explains how geocoding technology converts street addresses into geographic coordinates to map and analyze spatial relationships between offenders and childcare facilities.","keywords":["geocoding","offenders","investigation"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["ArcView","shapefile"],"full_text":"In Washington state, child molest- ers cannot live within a city block of day-care centers. At least that's what the law says. In reality, more than 600 sex offend- ers live within 100 yards of child- care centers, KIRO-Seattle reported in February. The IRE and NICAR Database Li- brary assisted KIRO with its inves- tigation by analyzing offender and sex offender data with geographic information system (GIS) analysis. KIRO provided files with the ad- dresses of the state's 15,000 con- victed child molesters and 9,000 li- censed day-care centers. The da- tabase library mapped the files, and then found child molesters and day- care centers within 100 yards of each other. We used the geocoding function to transform the street addresses con- tained in the tables into geographic points. In a nutshell, geocoding takes a street address and converts it into latitude and longitude, so the location can be displayed on a map. Common GIS programs - like the program we used, ESRI's ArcView 8.3 - can geocode addresses. The process can save hours when jour- nalists want to map crimes, schools, businesses, or anything else with street addresses and, preferably, ZIP codes. To geocode street addresses, you need a map file of streets for the area you're interested in. To map day-care centers and sex offend- ers in Washington state, we down- loaded the files in ESRI shapefile for- mat for the entire state from ESRI's Web site at www.esri.com/data/down- load/census2000_tigerline. The files are based on data from the 2000 Census. Each street is represented in the map file by a number of line seg- ments and records in the map data table. Each street segment has two address ranges reported in the data table: one for each side of the seg- ment. The beauty of geocoding, at least in ArcView, is that the program can map locations even when the address in the data table fails to exactly match what's in the street file. For example, there might be a seg- ment for East Broadway that in- cludes the address ranges 100-150 and 101-159. ArcView's geocoding tool attempts to match the day-care and offender street addresses to the street segment address ranges. So if there is a day-care center at 125 East Broadway (whose address range falls exactly midway between 100 and 150), the geocoding tool will place the point at the physical mid-point of that line segment. The beauty of geocoding, at least in ArcView, is that the program can map locations even when the ad- dress in the data table fails to ex- actly match what's in the street file. Whether a day-care center is listed at 203 W. Maple St., 203 West Maple Street or even 203 Maple St. West, ArcView will do its best to match it to the corresponding street segment. Each match receives a score from 0 to 100, with 100 being a perfect match. That also means many matches are less than perfect. And you can't accept or reject them based solely on the score. Take two cases from the sex offender data- base. ArcView matched an of- fender at E. 2909 37 to 2901- 3099 East 37th Ave. (identical ZIP codes) with a score of 63. But with the same score, it matched an of- fender at 225 South Warren to 201-299 North Warren Ave., both in ZIP code 99156. So either the sex offender's address is wrong, one of the ZIP codes is wrong, or South Warren Avenue doesn't show up in ArcView's street files. (It turns out there is a street seg- ment for South Warren, but it doesn't include an address range.) It's one thing if you are geocoding a few businesses in your town and you need to verify them by driving by the businesses or calling to check their locations. But what about when you're mapping 15,000 anonymous sex offenders across an entire state that you don't live in? (KIRO received the list of all child molesters but without the names.) We didn't want to mistak- enly place molesters - or day-care centers, for that matter - in the wrong location. Because we had to select molesters within 100 yards of day-care centers, putting either in a bogus spot would com- promise our results. That's where the geocoding options in ArcView came in handy. You can set the minimum score for a match. The default is 60, so any scores of 60 or more will result in a valid match. We upped that threshold to 80. You can also tailor your results with the Spelling Sensitivity and Mini- mum Candidate Score options. You can lower the Spelling Sensitivity (the default is 80 out of 100) if lots of addresses are misspelled. The Minimum Candidate Score deter- mines how closely an address matched the street file to even be considered a match. There's no easy way to determine which settings to use - just experi- ment with your data. For both the molesters and day-care centers, we compared the number of valid matches we got using a minimum match score of 60 versus one of 80. We lost 257 centers using a score of 80 instead of 60. That sounds like a lot, but it was only 3 percent of all day-care centers. Similarly, we lost 843 molesters by changing a mini- mum score of 60 to 80 - that's 5 percent of all offenders. Using that tougher standard, we still mapped 82 percent of all day-care centers and 69 percent of all molesters in the original address files. Based on those results, we came up with 605 cases of molesters living near daycare centers. By tightening the geocoding criteria, we erred on the conservative side in our analysis. Had we used a lower minimum match score we could have picked up more valid cases of mo- lesters living near day-care centers - but with so many cases to check, we could have ended up with bad matches too. Even with our higher match criteria, we spot-checked matches on the Web using MapQuest and other sources to make sure we had valid matches. See the stories on the Web at www.kirotv.com/investigations/ 2825037/detail.html. Contact Holly Hacker by e-mail at holly@ire.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-june-upcoming-car-workshops","slug":"2004-june-upcoming-car-workshops","url":"https://dwillis.github.io/uplink/article/2004-june-upcoming-car-workshops.html","year":2004,"month":"June","monthNum":6,"headline":"Upcoming CAR Workshops","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering Computer-Assisted Reporting boot camps and Better Watchdog workshops to train journalists in using electronic data, databases, and investigative techniques to produce impactful stories. Workshops are scheduled throughout 2004-2005 with fellowship opportunities available for minority and small-news organization journalists.","keywords":["workshops","reporting","training"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet","GIS"],"full_text":"Upcoming CAR Workshops\n\nComputer-Assisted Reporting Boot Camps\n\nThese unique seminars, taught by IRE and NICAR's experts, train journalists to acquire electronic information, use spreadsheets and databases to analyze the information and to translate that information into high-impact stories. In addition, IRE and NICAR provide follow-up help when participants return to their news organizations.\n\nAug. 1-6-Columbia, Mo.\n\nSeveral 2005 dates will be announced soon. Check the IRE Web site: www.ire.org/training/bootcamps.html\n\nAug. 20-22 - Columbia, Mo. Mapping Data for News Stories Boot Camp Learn how to uncover interesting news stories by mapping data with geographic information system software.\n\nWhat previous Boot Campers have said about the experience:\n\n\"They were able to talk in a real way about issues that reporters face when practicing computer-assisted reporting skills - Analisa Nazareno, San Antonio Express-News \"The intense, small group workshop helped me understand how powerful database research can be. I now consider CAR one of the most powerful tools to expose wrongdoing.\" - Holly Whisenhunt, WOAI-San Antonio \"I realized I hadn't been fully tapping my beats, and you don't have to be a computer nerd to do computer-assisted reporting.\" - Teresa Taylor Williams, Muskegon Chronicle\n\nFellowships\n\nA limited number of minority and small-news organization fellowships are available for IRE and NICAR workshops. Visit www.ire.org/training/fellowships.html for more information and an application.\n\nBetter Watchdog Workshops\n\nInvestigative Reporting on the Beat\n\nIRE and SPJ, with funding from the SDX Foundation, The workshops, specifically for journalists at small- the Chicago Tribune Foundation and the Ethics and to medium-sized news organizations and those in Excellence in Journalism Foundation, have joined bureaus of larger organizations, will emphasize the forces to offer a series of workshops focused on use of FOI laws and address juggling a beat while doing investigative reporting while covering a beat. producing investigative and enterprise pieces.\n\n\"You'll learn enough in the first 15 minutes to keep you busy for a month.\" - Kevin McGrath, The Wichita Eagle\n\nWorkshops are scheduled for:\n\nAug. 4, 2004 - Washington, D.C. In conjunction with the UNITY Conference\nSept. 18-19, 2004 - Richmond, Va.\nSept. 25, 2004 - New York, N.Y.\nNov. 6-7, 2004 - Oklahoma City, Okla.\nFeb. 5, 2005 - Orlando, Fla.\n\nFor more information, visit www.ire.org/training/betterwatchdog To request a workshop for your area, contact Executive Director Brant Houston at watchdog@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-readmetxt","slug":"2004-june-readmetxt","url":"https://dwillis.github.io/uplink/article/2004-june-readmetxt.html","year":2004,"month":"June","monthNum":6,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"This article highlights several investigative journalism projects that analyzed U.S. Department of Transportation fatality data to examine traffic safety issues across different regions and contexts, including hit-and-run accidents, intersection dangers, and seat-belt compliance.","keywords":["traffic","safety","accidents"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["FARS"],"full_text":"\"Danger Zone\" is available in the IRE Resource Center. To order the story, contact the center at 573-882-3364 or rescntr@ire.org and ask for Story No. 21020.\n\nLooking for news stories that have used the U.S. Department of Transportation's Fatality Analysis Reporting System data? You can find some in the IRE Resource Center or on the Web.\n\nNewsday analyzed three years of FARS data as part of a 12-part series, \"Death on the Roads.\" The paper found that Suffolk County's traffic fatality rate ranks among the worst for large urban areas nationwide and is the worst in the New York metropolitan area. (Story No. 20601 in the IRE Resource Center)\n\nThe San Francisco Chronicle examined eight years of FARS data and found that California leads the nation in fatal hit-and-run accidents. Hit-and-runs accounted for 7.8 percent of the state's fatal crashes in 2001, more than twice the national average. Traffic safety experts cited the high number of drivers with a suspended or revoked license as one explanation for the state's problems with hit-and-run accidents. Read the story at www.sfgate.com/cgi-bin/article.cgi?file=/c/a/2003/07/271MN292165.DTL.\n\nReader's Digest looked at fatal accident data from 1998 to 2001, focusing on incidents at intersections and removing crashes blamed on driver error or impairment. The magazine found that more than 24,000 people were killed. One-third of them were at intersections, where confusing lanes, blind spots and inadequate signs can be deadly. Read the story on the Web at www.rd.com/common/navindex jhtml?channelld=5&subChannelld=21&articleld=9526014&wtGroup=SPLASHMITI&wtID-SPLASTHISI003.\n\nThe Indianapolis Star used three years of FARS data to track fatal pickup truck accidents on Indiana roads in which victims were not wearing seat belts. The paper's analysis found that more than 90 percent of motorists who are thrown from the truck cab during an accident die or suffer a disabling injury. Indiana doesn't require adults in pickup trucks to wear seat belts. The death toll from these accidents is rising, but efforts to change the seat-belt law have failed. Read the story online at www.indystar.com/articles/6/135115-5226-092.html.\n\nTo order any of the stories in the IRE Resource Center, call the center at 573-882-3364 and provide the story number.\n\nFor more information about the FARS database see www.nicar.org/data/fars. Journalists and journalism organizations can order the data by calling the IRE and NICAR Database Library at 573-884-7711 or 573-884-7332.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-june-readmetxt-2","slug":"2004-june-readmetxt-2","url":"https://dwillis.github.io/uplink/article/2004-june-readmetxt-2.html","year":2004,"month":"June","monthNum":6,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"The Dayton Daily News won an IRE Award certificate in 2003 for their investigative series 'Casualties of Peace' in the medium newspaper category. Information on ordering the story and details about other IRE Awards winners are provided.","keywords":["investigation","award","journalism"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"The Dayton Daily News series \"Casualties of Peace\" won a certificate in the 2003 IRE Awards for medium newspapers (100,000-250,000 circulation).\n\nTo order a copy of the story and a contest entry questionnaire that details the reporting, contact the IRE Resource Center at 573-882-3364 and ask for Story No. 20926.\n\nFor more information about other winners and finalists in the IRE Awards, visit the IRE Web site at www.ire.org/contest/past/03winners.html.\n\nThe 2004 edition of Winning Investigations details how winners and finalists in the 2003 IRE Awards reported their stories. To order, go to www.ire.org/store.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-targeting-voting-wrongs","slug":"2004-july-august-targeting-voting-wrongs","url":"https://dwillis.github.io/uplink/article/2004-july-august-targeting-voting-wrongs.html","year":2004,"month":"July-August","monthNum":7,"headline":"Targeting voting wrongs","kicker":"ELECTIONS","author_name":"Benjamin Lesser","author_title":"Reporter","affiliation":"The Record","summary":"An investigation into voting rights violations in Teaneck, New Jersey, revealing racially polarized voting patterns and the potential for federal intervention.","keywords":["voting rights","Teaneck","investigation"],"topics":["Government & Politics","Investigative Methods","Demographics & Census"],"technologies":["ArcView","Excel"],"full_text":"Last year reporters at our news- paper learned that the U.S. Department of Justice was in- vestigating possible voting rights violations in Teaneck, a small New Jersey suburb of New York. This January one of our reporters, Scott Fallon, ap- proached me about doing something more substantive than the typical he said-she said story about the probe. After some discussions with our editors we decided to give it a go. Our plan was to conduct a parallel investigation to the Jus- tice Department's. A few months later we published a story that detailed for our read- ers what the Justice Depart- ment investigation involved, what conclusions the investiga- tors may draw and ultimately what the investigation could mean for the town. The Justice Department began its investigation after receiving an anonymous complaint from a local resident. Investigators were try- ing to determine whether Teaneck's at- large voting system should be replaced with a ward system that would help promote the election of black candi- dates to the town council and school board. Teaneck was 28 percent black but none of the members of the town council and only one member of the school board were black. In fact, there hadn't been a black resident on the town council since 2000. We worked the story on two tracks. Fallon, through his sources within the town, began retracing the steps of the investigators from the Department of Justice. He requested the same records from the town and interviewed the same town officials and residents. Meanwhile, I began to work on the sta- tistical analyses the investigators were likely to conduct. We contacted Edward Still, an Ala- bama attorney who specializes in vot- ing rights cases. Still explained what the investigators would be looking at and how we could mimic their analy- sis. I also asked two university profes- sors to act as expert advisers. We learned that the investigators would be seeking to answer three key questions. First, was there racially polarized voting? Put simply, do whites vote predominately for white candidates and blacks vote predomi- nately for black candidates? Second, do black candidates routinely run for office and lose? And, third, can a ward system be created that includes at least one ward in which minorities would be dominant? So, while Fallon did the shoe-leather reporting, I dug into the numbers. To answer the first question I needed to conduct a regression analysis. Sounds frightening, doesn't it? It is. At least that was my initial thought. But just before I ran away screaming, I pulled out notes from a weeklong IRE and NICAR statistics boot camp that I had attended in Chapel Hill, N.C., where I learned statistical analysis. I said to myself, \"You can do this.\" So what is regression analysis? It is simply an attempt to determine the pre- dictive value of the relationship that exists between two variables. In this case, our variables were the demo- graphics of the voting age population and voting results for each of the 21 voting districts within Teaneck from several recent elections. My first step was to collect the data. Determining the racial makeup for each of the voting districts became a long and tedious process. First, I grabbed block level race data from the 2000 Census and then mapped that informa- tion out using ESRI's ArcView 3.2a. That sounds simple enough, but here's where the real work started. Teaneck did not maintain an electronic version of the voting districts map. Fortunately, the borders of the voting districts and those of the Census blocks were, in all but one case, identical, So I placed a street map of the town over my map of the Census blocks and fol- lowed the streets to determine which blocks were within which voting district. I then created mapping shapefiles for each of the 21 voting districts and also exported the underlying data to a Microsoft Excel spreadsheet. Now that I had the demographic data and the voting results data I was ready to conduct a regression analysis. (NOTE: It would have been better to use the racial breakdown of registered voters rather than the racial break- downs in the Census voting age popu- lation data but that information was un- available.) I conducted the initial regression analy- sis in Microsoft Excel after my expert advisers, Bernard Grofman at the Uni- versity of California-Irvine and David Epstein at Columbia University, recom- mended the program. I built a spread- sheet containing both the racial break- down and voting results for each of the 21 voting districts. Then, with a few clicks, Excel produced a scatter plot graph and several statistical measures of the data to indicate what - if any - relationship existed between the per- cent of the population that was black and the performance of black candi- dates within each district. Excel's regression routine does not report statistics that tell whether the any relationship is meaningful. I repli- cated the analysis in SPSS, a statisti- cal program that has the tools, and also ran the results by Grofman and Epstein. I analyzed five recent Teaneck elec- tions in which a black candidate ran and found at least some level of ra- cially polarized voting in four of those. I then asked Epstein to double-check my analysis, and he came to the same conclusions. However, Epstein was quick to point out that the pres- ence of racially polarized voting is not uncommon. Complicating matters further was the success of Gordon Johnson, a black state assemblyman, who had great success within Teaneck during primary and general elections. So, while it appeared that the town did suffer from racially polarized voting that didn't necessarily mean the at-large voting system needed to be replaced. But it did indicate that the Justice De- partment investigation was not a waste of time, and that the town had some difficulties. The second and third questions were much easier to answer. We found that since 1990, seven of the 22 black can- didates (32 percent) for the township council and school board won while 56 of 90 white candidates (62 percent) ran successfully. So black candidates did have a history of running and losing. By using ArcView and Excel I was able to determine that the population in the northeast section of the town was 65 percent black so the creation of a ward where minority group members would dominate would not be difficult. At the end of the analysis we were able to paint a statistical portrait of Teaneck for our readers that helped explain what had happened in recent elections and also gave them insight into what was probably happening inside the federal investigation. One important piece of advice I might offer is that you don't need a federal investigation to do a story like this. Many elected bodies in towns and cit- ies all across the country look nothing like the communities they represent. Does that mean those towns or cities are violating federal law? No, but it is worth the time and effort of reporters to find out why and, in the process, perhaps begin a constructive discus- sion within those communities about how best to ensure elected officials represent all segments of a community. Contact Benjamin Lesser by e-mail at lesser@northjersey.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-july-august-examining-evacuation-during-nightclub-fire","slug":"2004-july-august-examining-evacuation-during-nightclub-fire","url":"https://dwillis.github.io/uplink/article/2004-july-august-examining-evacuation-during-nightclub-fire.html","year":2004,"month":"July-August","monthNum":7,"headline":"Examining evacuation during nightclub fire","kicker":"MODELING REALITY","author_name":"Paul Edward Parker","author_title":"","affiliation":"The Providence Journal","summary":"The Providence Journal used computer simulation software to model the crowd evacuation during the 2003 Rhode Island nightclub fire, demonstrating how computer models can be valuable tools in investigative journalism for visualizing events and exploring alternative scenarios.","keywords":["simulation","evacuation","journalism"],"topics":["Investigative Methods","Technology","Public Safety"],"technologies":["Simulex","STEPS","CAD","AVI","Screen Movie Studio"],"full_text":"Computer simulations, or models, are commonplace in the scientific world. They are used to predict hurricanes, forecast the spread of epidemics and calculate how long it would take people to leave a crowded building. Computer models are considered scientifically reliable, yet they are a rarity in the world of computer-assisted reporting.\n\nIn the course of covering the Rhode Island nightclub fire that killed 100 people in February 2003, The Providence Journal developed a database of people who were inside the building when it caught fire. This database was used for stories that documented how many people were inside the building, and for a narrative of the six minutes from when the fire started until the last survivor escaped. We also used it for a computer model that examined how the crowd fled the nightclub and how things could have been different with a few minor changes.\n\nCrowd evacuation simulation software is well accepted in the building design and fire safety industries. In most jurisdictions, such software can be used to demonstrate that a building complies with codes governing emergency evacuations. An owner whose building might otherwise fail to meet fire codes that prescribe, among other things, the number and size of exits, can still pass inspection if an evacuation simulation shows building occupants would reach safety in time.\n\nThe Station nightclub fire was the ideal situation for a newspaper to use a computer model. The newspaper, through interviews with more than 200 survivors, had excellent data regarding where people were when the fire broke out and how they got out of the building. Because of those interviews and because the first minute of the fire was videotaped from inside the building by a television camera operator, the newspaper had a good idea of what actually happened, providing an excellent test of the computer model against the real world.\n\nA model would produce worthwhile results, both in terms of visualizing what happened and in exploring \"what ifs\", such as what if the exits had been configured differently.\n\nThe Journal researched several evacuation simulation software packages online before selecting two - Simulex and STEPS - for evaluation.\n\nThough the software is normally expensive, the developers of each package, after learning what the newspaper wanted to do, offered free licenses and technical support. Both packages were similar in their features and calculations, though they had substantial differences.\n\nSTEPS produced excellent three-dimensional color animations of the evacuation, allowed the user to have somewhat greater control over the movements of building occupants and easily rendered the animation in AVI video format.\n\nSimulex was easier to set up, the movements of individual people in the model appeared more lifelike, and - most important for the Journal - the overhead, two-dimensional animation of the evacuation was better suited for conversion to a newspaper graphic, though it had to be done by hand by a graphic artist.\n\nScreen-grab shareware Screen Movie Studio was used to generate .AVI animations that were posted on the newspaper's Web site because Simulex could only create video in a proprietary format.\n\nSimulex, as does STEPS, starts with a building floor plan, which is created in a Computer-Aided Design (CAD) program and imported into Simulex. The Journal had used public records requests to obtain excellent floor plans of the nightclub on paper that were recreated using CAD software by the Journal employee who oversees remodeling of the newspaper's building.\n\nNext we defined the exits by entering their widths and graphically placing them on the floor plan. Then we entered information about the people inside the nightclub, to account for gender, body sizes, walking speeds and alcohol consumption.\n\nWe placed the people on the floor plan, along with the exit they would use and a defined delay factor. The delay factor was critical in The Station fire. Because rock band Great White started its show with fireworks and many in the crowd had been drinking, few were alarmed to see flames shooting up the stage walls. The television video showed it took more than 30 seconds for the crowd as a whole to start moving toward the exits.\n\nAfter we placed the people, we ran the simulation, which records an animation of the evacuation, and provides statistics that are updated 10 times a second.\n\nThe Journal employed a few tricks in getting Simulex to mimic what happened the night of the fire. Because the program is designed to calculate how long it takes to evacuate a building, it does not account for the \"crowd collapse\" that led to people piling up in the front door.\n\nCrowd collapse can occur for many reasons. In The Station fire, people at the back of the crowd were the first to feel the effects of heat and smoke, and rushed forward to get nearer the exit. Meanwhile, people at the front of the crowd slowed down as they reached the outer doors, a normal reaction once safety is reached in an evacuation.\n\nThis difference in speed led to people in front being knocked over, and, as the crowd continued to surge forward, more people tripped and fell on top of them. This pileup effectively closed the front door of the club.\n\nBut the Simulex software was not designed to model doors that close during an evacuation. The Journal tricked Simulex into doing that by programming several large extra people who tried to push their way into the door at the moment the crowd collapsed. Having people trying to go both ways through the same door effectively jammed it.\n\nIn another instance, we had to compensate for a window broken after the start of the fire. We did that by adding some other people into the model and setting their delayed reaction to the time that the window broke. These people were then placed right in the window, making that area impassable until the delay had elapsed.\n\nIn the case of both tricks, the extra people were electronically removed from the graphics published in the paper and the video published on the Web.\n\nThe model showed that, in addition to the pile of people at the front door, fire victims were trapped deeper inside the club, at the beginning of a hallway leading to the front door.\n\nThe model also showed that if the club had a rear exit if, the hallway at the front exit had not been built or if fewer people were inside the club, which was over capacity, virtually everyone would have had a chance to escape.\n\nThe stories developed from the computer models were part of The Journal's package that was recognized as a finalist for the Pulitzer Prize for public service and in the IRE Awards.\n\nCrowd evacuation models such as Simulex and STEPS could be useful for a variety of computer-assisted reporting projects, such as evaluating existing or proposed buildings or even, with a few tweaks, examining outdoor venues.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-property-data-analysis-finds-extra-tax-credits","slug":"2004-july-august-property-data-analysis-finds-extra-tax-credits","url":"https://dwillis.github.io/uplink/article/2004-july-august-property-data-analysis-finds-extra-tax-credits.html","year":2004,"month":"July-August","monthNum":7,"headline":"Property data analysis finds extra tax credits","kicker":"SPOTLIGHT: REAL ESTATE","author_name":"Kevin Corcoran","author_title":"","affiliation":"Indianapolis Star","summary":"An Indianapolis Star investigation found that over 9,300 taxpayers illegally claimed multiple homestead property tax credits on properties they didn't live in, costing Marion County over $6 million in fraudulent tax breaks.","keywords":["fraud","taxes","homes"],"topics":["Data & Databases","Business & Economy","Investigative Methods"],"technologies":["CD-ROM","Excel"],"full_text":"Thousands of Indianapolis taxpayers cashed in last year on multiple homestead property tax credits even though this valuable break was intended to reduce taxes only on a homeowner's primary residence.\n\nAfter analyzing computer records for nearly 210,000 properties that received the break for owner-occupied homes we reported that more than 9,300 people benefited from as many as 11,465 homestead tax credits they weren't entitled to.\n\nThese credits cut taxes on homes taxpayers no longer lived in - or had never lived in. Some taxpayers got five, 10 - even 20 - credits at once.\n\nIndiana's credit is a line-item reduction in each homeowner's tax bill. The state subsidizes the tax cut by writing checks to local governments that depend on property taxes.\n\nThe Star calculated the cost of the credit and deduction errors uncovered in Marion County at more than $6 million. No one in state government could say how widespread the problem was, but, after checking the newspaper's findings, budget and tax officials said they were concerned it could be occurring elsewhere in Indiana.\n\nThe news story ran March 28 and highlighted the 10 county taxpayers who had received the most credits. James E. Chalfant, who received unwarranted credits worth $3,512 on 19 of his residential properties, topped the list. He received related homeowner deductions worth another $1,000 or so.\n\nThe idea for the story emerged while I was cleaning a larger database of nearly 400,000 county property records. I wanted to evaluate the effects of a wrenching court-ordered overhaul of Indiana's system of valuing property for tax purposes. The Indiana Supreme Court had declared the old system unconstitutional and ruled that taxes on homes must be more closely tied to market prices.\n\nThe General Assembly, fearing hordes of angry homeowners at the ballot box, had passed a tax-relief package in mid-2002 that raised Indiana's sales tax to 6 percent from 5 percent to cover tax revenue losses that resulted from an increase in the homestead credit and a related deduction.\n\nBut in early 2003 legislators found a 17-year-old mistake in how the homestead credit was calculated. They chose to correct the error, which had favored homeowners. The action diminished the value of the credit and generated hundreds of millions of dollars for a state budget that was running in the red. Months later, in the summer of 2003, thousands of Indianapolis homeowners learned their property tax bills would be double or more as a result of the court-ordered reassessment.\n\nI had originally sought the tax data so I could recalculate Marion County's tax bills and figure out how much of the relief taxpayers had been promised had been taken back. (The Star had previously reported they had paid an extra $30.2 million.)\n\nThe county's computer vendor burned a copy of the data onto CD-ROM in delimited text format for me at no cost, at the request of a township assessor. The data had some inconsistent owner names and I spent a lot of time cleaning the data to create uniform names. In addition, some townships had combined a number of parcels into one, making it impossible to compare the tax bills over the years. So I created a field called \"combo\" that indicated whether the parcel was created by combining other parcels.\n\nIn the course of cleaning the tax data in Microsoft Excel, I noticed during sorts on taxpayer names that many had more than the single homestead credit allowed by law.\n\nWe already knew about owners getting more than one credit. Earlier in the year, two reporters for the Star, Vic Ryckaert and Matthew Tully, had disclosed that Marion County's prosecutor had more than one homestead credit in 2003.\n\nMy data analysis showed the extent of the problem in Marion County and put a price tag on it. Inside the newsroom, Neill Borowski, the Star's assistant managing editor for news, championed the story. I largely credit him for the Sunday A1 play the story received.\n\nGoing into the story, I had assumed the multiple homestead credits were a result of homeowners applying for more than their fair share of tax breaks. (You can read the story and search a database of property records at www.indystar.com/news/politics).\n\nBut I began to question that assumption after taxpayers with the most homestead credits began calling me back and vehemently denying they had applied for them. I reached six of the top 10 before the stories ran. I reached two others by sending them letters via Federal Express at their tax-bill addresses. One contacted me after the story ran and I never did hear from one. They all told the same story, and upon checking paper land records, I found they were telling the truth. They had not applied for the breaks; they had inherited them.\n\nThe errors had turned out to be a result of poor record keeping by the county auditor, the local official responsible for administering this state-funded program. Problem No. 1: State officials were relying on local officials to make sure tax relief went to the right people, but local officials had no reason to care about the data's accuracy, because money to pay the credit was coming from the state's bank account.\n\nCounty officials told me that it would be difficult, if not impossible, to pull data off the county's old mainframe computer and put it into a format that I could use. Problem No. 2: State and local officials usually have no idea how computers work.\n\nThe data was made available at no cost after I persisted and found a township official who asked the contractor to provide it as an ASCII text file on CD-ROM.\n\nThe Star used a snapshot of property tax data taken Aug. 18, 2003, to determine which taxpayers had received more than one homestead tax credit. The database was narrowed to include just the 209,767 land parcels with these credits. I knew I had the correct parcels because the sum spent on homestead credits in my database matched, to the penny, the amount of the tax relief check the state had to write Marion County last year.\n\nUsing Microsoft Access, I analyzed the data to determine which taxpayers with the same first and last names received more than one credit by grouping and counting them. Because the county's data does not include unique identifiers such as Social Security numbers, I eliminated people with common first and last names from the analysis. Admittedly, that required making some judgment calls.\n\nI assumed the most valuable homestead credit for each taxpayer with more than one credit was valid. I also assumed credits on properties near those with the largest credit each taxpayer had received were legitimate, because in Indiana the homestead credit applies to a primary residence and up to an acre of adjacent land. Any additional credits the remaining property owners received were totaled to determine the cost to taxpayers of paying for undeserved credits.\n\nThis analysis found 9,315 taxpayers with 20,780 credits; the 11,465 extra credits cost taxpayers an additional $1.11 million.\n\nIn addition, a homestead credit comes with a related property tax deduction of up to $35,000 per home.\n\nTo determine the effect of giving county taxpayers extra homestead deductions, I assumed that the largest deduction for each property owner with more than one deduction was the valid one.\n\nFor each invalid homestead deduction, property value was removed from Marion County's tax rolls, effectively increasing tax rates. I did not attempt to re-compute county tax rates for the county's 60 taxing districts, which would have left room for error and added weeks to the project.\n\nInstead, after talking to property tax experts, I used a proxy measure of the likely effect of property owners getting more than the one homestead deduction they were entitled to by law.\n\nTo calculate this effect, the property value these undeserved deductions had eliminated from tax rolls was totaled ($171 million), divided by 100, and multiplied by $2.9421, the weighted average tax rate per $100 for homes in Marion County last year. This proxy measure indicated the extra deductions had shifted the burden of paying $5.03 million of property taxes, mostly onto businesses and homeowners with just one homestead credit.\n\nI queried Access for a list, grouped by name, of each taxpayer with more than one deduction. In addition, I asked the program to create fields totaling the values of these deductions for each taxpayer and listing the highest-valued deduction for each taxpayer. Then I exported the data to Excel, where I subtracted the highest deduction from the deduction total to get the invalid amount of deductions. I added the 6,146 undeserved deductions to arrive at the $171 million total.\n\nI found the unwarranted homeowner tax credits and deductions cost state and local taxpayers least $6.14 million.\n\nSeveral months before the story ran, I provided a list of property owners with duplicate credits to the county auditor so she could review them. She told me her own study, performed after the prosecutor's homestead credits had drawn scrutiny, found 9,000 more duplicate credits than the Star's conservative estimate had.\n\nIn addition, the auditor's aides double-checked the list of the two dozen people with the most credits and removed more than 210 homestead credits from their properties. This list became the basis for the Top 10 list the paper published.\n\nI also walked through my methods and results with state property tax analysts at the State Budget Agency, Department of Local Government Finance and nonpartisan Legislative Services Agency. They reviewed my methodology and did not dispute my findings. After the story ran, the budget agency assigned an analyst to check some other counties. During a cigarette break outside the Statehouse he told me that he'd found a public school district in suburban Hamilton County with six homestead credits.\n\nWhen I contacted property owners with the most homestead credits, one local real estate agent getting too many credits asked, \"How much are you going to cost me?\"\n\nThe answer: At least $3,000 a year after the auditor canceled his credits.\n\nNot everyone was so upset. Chance L. Felling, No. 8 on the Star's list, wrote a check to pay the $1,691 owed for 11 invalid credits.\n\n\"Regardless of who made the mistake,\" Felling said, \"if I owe money, then I should pay.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-bits-and-bytes","slug":"2004-july-august-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2004-july-august-bits-and-bytes.html","year":2004,"month":"July-August","monthNum":7,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library has released expanded mortality data from 1988-2001 and OSHA inspection records from 1972-2004 to help journalists investigate public health and workplace safety issues. The article also announces upcoming training opportunities and highlights examples of investigative reporting using these databases.","keywords":["databases","journalism","investigation"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["ArcView"],"full_text":"The IRE and NICAR Database Library now has Multiple Cause-of-Death Public Use (Mortality) data covering 1988-2001 and Occupational Safety and Health Administration (OSHA) data covering 1972-2004.\n\nThe 2001 mortality data contains over two million death certificate records from the United States and its territories. Mortality data has generated several stories about death trends in the United States. Combined with U.S. Census data, the mortality database can help reporters investigate how the economic health of a particular area contributes to mortality and life expectancy. See www.ire.org/datalibrary/databases/health for more information.\n\nThe latest OSHA data contains over three million inspection records and is current as of January 2004. OSHA data can be used to investigate which organizations violate employee safety standards, as The New York Times and Frontline recently did in an award-winning joint investigation of the McWane Corp. See www.nicar.org/data/osha for more information about the database.\n\nA sure bet\n\nEver wonder how to turn complicated stories into interesting television journalism? Do you want to learn investigative skills that keep government and business accountable? Then join some of the best journalists from around the country at the IRE Regional Conference in Las Vegas, Sept. 10-12.\n\nSessions include presentations taken from the Better Watchdog Workshops sponsored by IRE and the Society for Professional Journalists, in addition to several sessions for broadcast journalists. IRE and NICAR are also offering computer-assisted reporting classes on a first-come, first-serve basis.\n\nThe conference costs $50 for IRE members and $90 for nonmembers (includes $40 discounted membership). Students may attend for $25.\n\nFor more information, including hotel rates, please see www.ire.org/training/vegas04.\n\nCampus crime\n\nThe 1990 Jeanne Clery Disclosure of Campus Security Policy and Campus Crime Statistics Act mandated all colleges report the number of crimes occurring on their campuses. But it wasn't until 2000 that these figures were centrally collected.\n\nSenior producer Lora Johnson LeSage of WMAQ-Chicago recently enlisted NICAR data analysts to examine the data and rank Illinois universities by crime totals. WMAQ reported \"the University of Illinois at Urbana-Champaign reported 390 arrests for alcohol-related infractions and 93 for drug offenses on or near its campus in 2002, ranking UIUC at the top among schools analyzed.\" The report also offers reasons for the large number of assaults on or near Northwestern University's Evanston campus.\n\nRead the story on the Web at www.nbc5.com/education/3279237/detail.html To purchase the campus crime database from IRE and NICAR or to learn more about it, see www.ire.org/datalibrary/databases/campus.\n\nUpcoming hands-on CAR training\n\nIRE and NICAR have several training opportunities for journalists seeking hands-on instruction in using computer-assisted reporting.\n\nThere are four weeklong Boot Camps in Columbia, Mo., for journalists who want to learn how to acquire electronic information, use spreadsheets and databases to analyze the information and to translate that information into high-impact stories. IRE and NICAR provide follow-up help after participants return to their news organizations. The 2005 Boot Camps are Jan. 9-14, March 20-25, May 15-20 and Aug. 7-12.\n\nJournalists interested in learning how to map data for news stories can take advantage of a mini-Boot Camp in using geographic information systems (GIS). The training, Aug. 20-22 in Columbia, will use ArcView 8.3 GIS. Participants will learn how to create basic maps, work with projections and geocode data. All participants will be eligible to purchase GIS software at discount. Additional GIS training is being planned for Washington, D.C.\n\nHands-on classes in basic CAR skills will be offered at some of the Better Watchdog Workshops sponsored by IRE and the Society for Professional Journalists. Two upcoming workshops with optional CAR training are Sept. 18-19 in Richmond, Va., and Nov. 6-7 in Oklahoma City.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-new-training-director-making-mark","slug":"2004-july-august-new-training-director-making-mark","url":"https://dwillis.github.io/uplink/article/2004-july-august-new-training-director-making-mark.html","year":2004,"month":"July-August","monthNum":7,"headline":"New training director making mark","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"David Donald has joined IRE and NICAR as training director, bringing extensive newsroom experience in computer-assisted reporting and investigative journalism. He is available to conduct customized training workshops and seminars for news organizations.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["SPSS"],"full_text":"David Donald is the newest member of the team at IRE and NICAR. As the training director, he's been on the road during the past few months. Maybe you've seen him this year at the annual computer-assisted reporting and investigative reporting conferences, at a Better Watchdog Workshop or another training event.\n\nDonald joined IRE and NICAR early this year and brings a wealth of experience from the newsroom. He oversaw the CAR and research program at the Savannah Morning News in Georgia, where he was precision editor. Before that, he covered education and served on the newspaper's projects team.\n\nHe was the lead organizer of IRE's regional conference in Savannah in 2002, and many of you have seen him speak on panels at the Annual CAR Conferences, sharing his knowledge of statistics, SPSS, surveys and building a CAR program. Donald also brings his experience teaching in high school and university classrooms to his work for IRE and NICAR.\n\nIf you'd like to arrange on-the-road training - anything - from an open workshop to a private seminar at your news organization - contact him at ddonald@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-upcoming-hands-on-car-training","slug":"2004-july-august-upcoming-hands-on-car-training","url":"https://dwillis.github.io/uplink/article/2004-july-august-upcoming-hands-on-car-training.html","year":2004,"month":"July-August","monthNum":7,"headline":"Upcoming hands-on CAR training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer multiple training programs in computer-assisted reporting, including week-long Boot Camps in Columbia, Missouri, GIS mapping mini-camps, and hands-on CAR classes at Better Watchdog Workshops throughout 2005.","keywords":["training","journalism","databases"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["ArcView"],"full_text":"IRE and NICAR have several training opportunities for journalists seeking hands-on instruction in using computer-assisted reporting.\n\nThere are four weeklong Boot Camps in Columbia, Mo., for journalists who want to learn how to acquire electronic information, use spreadsheets and databases to analyze the information and to translate that information into high-impact stories. IRE and NICAR provide follow-up help after participants return to their news organizations. The 2005 Boot Camps are Jan. 9-14, March 20-25, May 15-20 and Aug. 7-12.\n\nJournalists interested in learning how to map data for news stories can take advantage of a mini-Boot Camp in using geographic information systems (GIS). The training, Aug. 20-22 in Columbia, will use ArcView 8.3 GIS. Participants will learn how to create basic maps, work with projections and geocode data. All participants will be eligible to purchase GIS software at discount. Additional GIS training is being planned for Washington, D.C.\n\nHands-on classes in basic CAR skills will be offered at some of the Better Watchdog Workshops sponsored by IRE and the Society for Professional Journalists. Two upcoming workshops with optional CAR training are Sept. 18-19 in Richmond, Va., and Nov. 6-7 in Oklahoma City.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-new-teacher-data-boosts-narrative-project","slug":"2004-july-august-new-teacher-data-boosts-narrative-project","url":"https://dwillis.github.io/uplink/article/2004-july-august-new-teacher-data-boosts-narrative-project.html","year":2004,"month":"July-August","monthNum":7,"headline":"New-teacher data boosts narrative project","kicker":"FIRST VENTURE","author_name":"Tara McLain","author_title":"reporter","affiliation":"Statesman Journal","summary":"Data analysis reveals a significant number of older individuals entering teaching in Oregon, challenging common assumptions about new teacher demographics.","keywords":["education","teachers","demographics"],"topics":["Investigative Methods","Data & Databases","Education"],"technologies":["Excel"],"full_text":"I was several weeks into my narra- tive project that involved following a college student through 10 weeks of student teaching. The student was 54 years old and teaching was his third career. Not a normal situation - so I thought. The data told me otherwise. After attending an IRE and NICAR Boot Camp this year, I figured I could bolster my story with some statistics about how rare it is for older people to go into teaching. I called the Or- egon Teaching and Standards Prac- tices Commission. Just as I learned in Boot Camp, the spokeswoman didn't know the data and the data guy didn't know policy. After a lot of conversation and get- ting a few sets of data with the wrong categories, the data guy e-mailed a Microsoft Excel file containing infor- mation about the 10,000 teacher li- censes given out last year. The col- umns of data included the teacher's name, school district, date of birth, date the license was issued and the type of license. Then I got back on the phone with the spokeswoman and found out just a few of the dozens of types of licenses are given to new classroom teachers. The rest were renewals or went to princi- pals or many other variations. I used Excel to filter and sort my way to the new teachers. Then I needed to calculate their ages when they got the license. Here's where a second lesson from Boot Camp came true: The program doesn't know the difference between dates, numbers and text unless you tell it the difference. I wrote a formula for Excel to subtract the date of license from the date of birth. It gave me a huge number. Excel thinks in days, so I divided the number by 365. Then I noticed there were some doubled, tripled, and even quadrupled, entries. I started deleting a few then scrolled down to see many more. That's where a third lesson from Boot Camp helped: There's always a shortcut. There's always a shortcut. This was my first CAR story and couldn't think of a way to search and destroy the duplicate rows. So I e- mailed Jeff Porter, the IRE and NICAR Database Library Director. He pro- vided, with lightning speed, the perfect formula. He said he stole it from some- one but couldn't remember whom. With apologies to Jeff and to the person he stole it from, here it is: =IF (AND (E2=E3, F2=F3, A2=A3, D2=D3) \"dup\" \"not\") The formula tells Excel to compare in- formation in each column by row. If a last name, first name, date of birth and date of license were all the same in one row as the row before, it would answer \"dup\". Then I filtered and sorted the names by duplicates and deleted them. So a fourth lesson from boot camp came true: Cleaning the data can take longer than crunching the data. Now, down to the analysis. I made a pivot table of the count of teachers by age. I got one count of each teacher. Remember the dividing by 365? Excel was looking at the age of each teacher as a decimal, although I had rounded it up, and created a cat- egory for each. | ran another formula (with the help of the research coordinator in the marketing department of our newspa- per) that stripped everything off the number except the first two digits. So 35.4332355 in column H became 35 in column I after I entered the formula =LEFT (H2,2) I ran the pivot table again. I summed the counts into easy-to-digest age groups: 20 to 29, 30 to 39, and so forth. Here's where the fifth lesson came in: Editors love pie charts. The pie chart showed that half of the new teachers in Oregon last year were over 30. Not exactly your fresh-faced college undergrads. I got the data from five years ago, went through all the cleaning again (much faster the second time) and found that the proportion of older people (30 plus) being hired to teach had grown from to 52 percent last year from 41 percent in 1999. That blew my assumption that all new teachers were young. The officials said the growth probably was because of the stagnant Oregon economy in the past few years and loads of school budget cuts. Districts had their pick of new hires and were choosing older people with more life experience whom they could pay the same as 23-year-olds, they observed. I found out that my 54-year-old student teacher isn't as rare in Oregon as I had assumed. My data-driven sidebar complemented my narrative feature. I also found an 84-year-old teacher in my local district who just had her li- censed renewed. Contact Tara McLain by e-mail at tmclain@statesmanjournal.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-july-august-data-shows-maintenance-problems-with-planes","slug":"2004-july-august-data-shows-maintenance-problems-with-planes","url":"https://dwillis.github.io/uplink/article/2004-july-august-data-shows-maintenance-problems-with-planes.html","year":2004,"month":"July-August","monthNum":7,"headline":"Data shows maintenance problems with planes","kicker":"AIR SAFETY","author_name":"Ted Mellnik","author_title":"","affiliation":"The Charlotte Observer","summary":"A Charlotte Observer investigation reveals that airline maintenance problems have become a significant factor in fatal crashes, contributing to 42 percent of fatal accidents since 1994, with inadequate oversight and cost-cutting measures compromising safety.","keywords":["maintenance","crashes","oversight"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["Access","SQL Server","ASP","VBScript","SQL"],"full_text":"Soon after a heavily loaded U.S. Airways Express Flight 5481 crashed in Charlotte shortly after takeoff, airline maintenance appeared as a likely cause. A mechanic at a small West Virginia re- pair station had recently adjusted the plane's elevator control cables, a job he had never done on a Beech 1900D. His instructor was also his inspector. He and other mechanics worked nights; the manager usually worked days. And the FAA officer in charge of monitoring the site had visited only once since mechan- ics began working there. The Charlotte Observer set out to find how the Jan. 8, 2003, crash that killed all 21 aboard fit with industry trends. A four-part series Dec. 7-10 reported that airlines are spending less to maintain their planes. Mechanics are checking them less often. Federal oversight is stretched thin. And maintenance is in- creasingly a factor in fatal crashes. The Observer found that since 1994, maintenance problems have contributed to 42 percent of fatal airline accidents in the U.S., up from 16 percent the previ- ous decade. Faulty maintenance contrib- uted to three of the past five fatal airline crashes in the U.S., and likely played a role in a fourth, now under investigation. Airlines have invested millions to fix other serious problems such as pilot error, and overall, crashes are declin- ing. But faulty maintenance, an equally preventable problem, has never re- ceived the attention it deserves, ex- perts told the newspaper. Sources of data for the series included: National Transportation Safety Board accidents. This relational data- base, available for download in Microsoft Access format or for online query from the NTSB, includes facts about crashes, the planes involved, events and contrib- uting factors. At the time of our use, the database held details on more than 50,000 events; it's updated monthly. It was the primary source for statements about accident trends. We started with a list of U.S. fatal air carrier accidents since 1994 in which the aircraft was substantially damaged or destroyed. An intranet application built using Active Server Pages/ VBScript and Microsoft SQL Server allowed reporters to browse the list, fil- ter it with a Structured Query Language \"Where\" statement, follow a link to in- vestigative reports, as well as add and edit crash entries. The additions were for new crashes. The air accident- data is referred to as ADMSPUB data and documentation can be downloaded at www.ntsb.gov/ntsb/ query.asp. The documentation is key to understanding and using the relation- ships among accidents, aircraft, people, events and contributing factors. Staff in the NTSB Public Inquiries branch an- swered almost all data questions. PTRS inspections. The FAA's Pro- gram Tracking and Reporting Subsystem manages information about inspections of contract repair stations and nonmajor airlines, and a portion of inspections of the nation's major carriers. It was ob- tained through a Freedom of Information Act request. The data was provided in yearly text data files, sometimes cover- ing more than a million inspections. While the data structure was documented, the challenge was to learn how the inspec- tion data was created and used so we could formulate queries. We focused on more than 450,000 closed, completed airworthiness and avionics inspections from 2000 to mid-2003. An inspection was considered to have an unfavorable result if the inspector reported that it led to an enforcement investigation or re- quired follow-up action. Valuable information about PTRS can be gleaned from watchdog reports is- sued by agencies such as the Gen- eral Accounting Office. For example, in 1998 the GAO reported that PTRS records understate the incidence of problems and violations. The Observer obtained an unpublished memo that described in detail the GAO's methods and confirmed ours. Key questions about PTRS were an- swered by interviewing inspectors and others in the FAA who use the system and its data. EIS. The Enforcement Information System data is the FAA's record of its actions against people and orga- nizations in aviation. Current data used to be available online. Its distri- bution was limited and some key Web pages removed following the 9/11 ter- rorist attacks. An extract of the data became available again in mid-2003. The IRE and NICAR Database Li- brary makes the EIS available to jour- nalists. For more information see www.nicar.org/data/faae. TranStats. This online service by the Bureau of Transportation Statistics provides data on airline spending, in- cluding maintenance spending, depar- tures and passengers. It's available at www.transtats.bts.gov. Other data. Several other data- bases were used in background. These included FAA Accidents and Incidents Database, System Difficulty Reports and lists of contract repair sta- tions. These and other databases are listed on the Aviation Data Systems (AFS-620) Web page at http:// afs600.faa.gov/AFS620.htm. You can read the stories on the Web at www.charlotte.com/mld/charlotte/news/ special_packages/planes, and an archive of the newspaper pages in Adobe Portable Document File format is at http://161.188.204.190/charlotte/air. Contact Ted Mellnik by e-mail at tmellnik@charlotteobserver.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-july-august-newspaper-winners-bank-on-car","slug":"2004-july-august-newspaper-winners-bank-on-car","url":"https://dwillis.github.io/uplink/article/2004-july-august-newspaper-winners-bank-on-car.html","year":2004,"month":"July-August","monthNum":7,"headline":"Newspaper winners bank on CAR","kicker":"IRE AWARDS","author_name":"Andrea Lorenz","author_title":"","affiliation":"IRE","summary":"IRE's annual investigative reporting contest highlighted newspaper winners and finalists who used computer-assisted reporting, including investigations into pharmaceutical distribution problems, drug company consulting fees, and military mishandling of assault cases.","keywords":["investigation","journalism","databases"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["FoxPro","Access","Excel","ArcView","SAS"],"full_text":"The May-June issue of Uplink highlighted the television, special category and other media winners and finalists of IRE's annual investigative reporting contest that used computer-assisted reporting. Here we feature the daily and weekly newspaper winners and finalists that used CAR.\n\nSee the May-June issue of The IRE Journal for a full list of winners.\n\nYou can order copies of these stories and related conference tipsheets from the IRE Resource Center. Contact the center at 573-882-3364 or rescntr@ire.org and provide the number.\n\nSome of the journalists who worked on these stories also recently spoke on panels at the IRE Annual Conference in Atlanta. You can order recordings of these panels from Sound Images Inc. Contact Sound Images at 888-649-1118 or service@soundimages.net.\n\nLargest newspapers or wire service\n\n(More than 500,000 circulation)\n\nFinalists\n\n\"Pharmaceutical Roulette,\" The Washington Post, Mary Pat Flaherty and Gilbert M. Gaul\n\nWhat started as a story about Medicare drug prices turned into a widespread investigation of problems in the drug distribution system. The Post reporters uncovered an extensive network of players including doctors with sketchy histories prescribing the drugs online, felonious middlemen obtaining drugs fraudulently on the cheap, and Fortune 500 companies buying questionable drugs at bargain prices from illegal wholesalers.\n\nThe reporters built databases of Internet pharmacy sales from a variety of sources and obtained physician prescribing data.\n\nThe stories also won Best of Show in the Maryland-Delaware-D.C. Press Association Awards and first place in Investigative Reporting, Division A: Dailies over 75,000. (See Story No. 20870. To hear Flaherty speak on the Pharmaceuticals: Prescriptions for investigations panel order recording number IRE04-043)\n\n\"Stealth Merger: Drug Companies and Government Medical Research,\" Los Angeles Times, David Willman\n\nWillman investigated drug companies paying consulting fees to National Institute of Health scientists - a practice authorized by the head of the research institute. More digging revealed the consulting deals were kept private, as employees were not required to divulge payments from other sources.\n\nThe Times obtained data (13,784 pages from the NIH alone) for this series from financial-disclosure statements and other information about public employees. The information was entered by hand to create two databases.\n\nThe series also won the Worth Bingham Prize for investigative journalism. (See Story No. 20867)\n\n\"Betrayal in the Ranks,\" The Denver Post, Amy Herdy and Miles Moffeit\n\nThis series uncovered military mismanagement of sexual assault and domestic violence cases. After battling the military, The Post obtained a database of domestic abuse and sexual assault cases from the U.S. Army's Criminal Investigation Command. Along with another database compiled from U.S. Air Force sexual assault records, the Post was able to profile more than 30 cases of abuse.\n\nThe series won a Denver Society of Professional Journalists award. (See Story Nos. 20697 and 20869)\n\nLarge Newspapers\n\n(250,000-500,000 circulation)\n\nCertificate\n\n\"Crumbling Schools,\" The Miami Herald, Debbie Cenziper and Jason Grotto\n\nIn this series of articles about the Miami school district's botched construction efforts, The Miami Herald found overpriced building projects, delayed jobs, safety hazards and sketchy business deals. The reporters used a database of construction projects, the district's accounting ledger, county and city safety data, as well as paper records to build their own databases. Microsoft FoxPro, Access, Excel, ESRI ArcView and SAS helped them analyze construction costs and delays, contracts, safety violations and population statistics. (See Story No. 20155. To hear Grotto and Cenziper speak on Exposing school system failures panel order recording number IRE04-068)\n\nFinalists\n\n\"U.S. Olympians had failed drug tests,\" The Orange County Register, Scott M. Reid, William Heisel and Tony Saavedra\n\nThe reporters found that more than 100 American athletes were allowed to compete in the Olympics despite failed drug tests. Less than 40 percent of the U.S. Anti-Doping Agency's tests were given without notice though it is recommended that at least 70 percent be a surprise.\n\nReporters sorted through 10,000 paper records from 9,700 drug tests of 4,600 athletes and entered the information into Excel and Access. They matched dates of drug tests with corresponding sporting events (found through LexisNexis, fan club sites and phone calls) to find whether the athletes attended these events.\n\nThe article won second place in the Associated Press Sports Editors investigative reporting category. (See Story No. 20338)\n\n\"Damaged Lives: Lead's Toxic Toll,\" Detroit Free Press, Emilia Askari, Tina Lam, Megan Christensen, Marsha Low, Hugh McDiarmid, Jr., Dan Shine, Shawn Windsor and Wendy Wendland-Bowyer\n\nA team of reporters investigated the contamination from lead smelters and leaded gasoline. They found government agencies neglecting to clean up lead in neighborhoods and test children for lead poisoning.\n\nThey used Access to analyze a Michigan Department of Community Health database for children tested for lead poisoning and mapping software to show the results of their soil tests. (See Story No. 20369)\n\n\"Shell Game,\" The Times-Picayune, Aaron Kuriloff and Jeffrey Meitrodt.\n\nThis series investigated $2 billion in court judgments against the state in favor of oyster farmers. The reporters found the awards to be either unjustified or excessive. Using tax and harvest records of oyster farms, court documents and sales data, they built tables in Excel to show results of a project initiated by the state to rebuild the coast. They found oyster beds were doing better - although oyster farmers claimed they were destroyed - than before the project.\n\nThe series won the Louisiana Press Association investigative award, and the Associated Press/Louisiana-Mississippi story of the year. (See Story No. 20608)\n\n\"Silent Alarm,\" The Kansas City Star, Karen Dillon and Mike McGraw\n\nThis series provided an in-depth look at the government's handling of the Hepatitis C virus. They used Microsoft Visual FoxPro and Excel to work with some of the data. Spreadsheets containing information about people alive today with the virus contracted through blood transfusions showed the estimated mortality of these patients. Other spreadsheets showed the number of Hepatitis C cases since the 1980s that could have been prevented by the government with a test. They also created a database of medical journal articles about the virus. \"Silent Alarm\" won first place in the Kansas Press Association award for investigative reporting. (See Story No. 20921)\n\nMedium newspapers (100,000-250,000)\n\nMedal\n\n\"Buried Secrets, Brutal Truths,\" The Toledo Blade, Michael D. Sallah, Mitch Weiss and Joe Mahr\n\nThe investigation of atrocities committed by the Army platoon Tiger Force in the Vietnam War brought out details that were buried by the government decades ago. Going through old documents and interviewing former soldiers and Vietnamese villagers gave essential details of the case.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-housing-prices-on-rise","slug":"2004-july-august-housing-prices-on-rise","url":"https://dwillis.github.io/uplink/article/2004-july-august-housing-prices-on-rise.html","year":2004,"month":"July-August","monthNum":7,"headline":"Housing prices on rise","kicker":"MAPPING","author_name":"Matthew Waite","author_title":"Reporter","affiliation":"St. Petersburg Times","summary":"A St. Petersburg Times analysis using geographic information systems reveals that housing prices in the Tampa Bay area are skyrocketing, transforming the region from having cheap housing comparable to Wichita to prices now matching larger cities like Dallas. The newspaper created detailed neighborhood-level maps analyzing over 260,000 home sales records to track how price increases are spreading across the region.","keywords":["housing","prices","neighborhoods"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["ArcView","ArcGIS","Access","SPSS","Excel"],"full_text":"Housing prices on rise\n\nBy Matthew Waite St. Petersburg Times\n\nImagine buying a house for $60,000. Over three years you fix a bathroom, paint a few walls and put in some shrubs. And then you decide to sell it. Maybe you've heard that other homes on the block were selling well, so what the heck. And, on a whim, you make more in profit than you spent on the house in the first place.\n\nLuck? An isolated case? Hardly, a St. Petersburg Times analysis of home sales found. The Tampa Bay housing market is in the midst of a transformation that has affected many bigger cities: the price of a home is skyrocketing, changing neighborhoods house by house and, in our case, greatly eroding the area's reputation for cheap housing. In 1998, the median price of a house around Tampa Bay was the same as Wichita, Kan. Now, it's the same as Dallas, a larger city with more wealth.\n\nTo get at this change on a neighborhood level, we largely relied on geographic information systems (GIS), in this case, ESRI's ArcView 3.2 and ArcGIS 8.3. We obtained databases of home sales from the property appraiser's offices in our five-county circulation area, ending up with more than 260,000 home sale records. We also got parcel map files from each county's appraiser. In most cases, the appraisers - elected officials - were happy to provide the data or maps free of charge. Others had standard charges, and just who had them seemed hit or miss. One county gave us maps for free, but charged us for data. Another county gave us data for free, but charged us for maps. The most expensive was $85, from one of our rural counties.\n\nWith the sales data and parcel data, we then turned to the problem of analyzing the data at the neighborhood level. Some cities have GIS departments, and gladly turned over GIS boundary maps of neighborhood associations. But our area includes large, heavily populated unincorporated areas, with no defined neighborhood associations, other than developer-created subdivisions that were too small for meaningful analysis. So we created our own, using ArcGIS. We started with a Census Designated Place map - which provides reasonable boundaries for cities and areas and went from there. With help from editors, reporters and maps, we drew more than 100 neighborhoods onto our main map. In the end, we had more than 300 neighborhoods, towns, subdivisions and unincorporated areas in our analysis and maps.\n\nThat allowed us to dig much deeper than we could have with available reports from the Realtors' associations. We could compare cities, towns and neighborhoods from one to another. We could look at how one hot neighborhood spills over into the ones around it. We could get very, very local for our readers.\n\nThe GIS portion of this, while key to the whole project, wasn't particularly difficult. I used ArcGIS to perform spatial joins, which assigned our neighborhood names to parcels based upon where the parcel was located. Then, with the names joined to the parcels, I could import the parcel table into Microsoft Access and join the parcel table to the sales table. Then, with a neighborhood name given to a sale, I could group the sales by the neighborhood name and then calculate median sale prices by years using SPSS, a statistics program. I then cleaned up the median output SPSS created in Excel and calculated percent changes. By the end, I used almost every software tool in my bag.\n\nSince the Times is heavily zoned, the idea from the start was to give each zoned edition a story specific to its area. In the end, we published about 30 stories over two days, 25 of them on one Sunday. For anyone wanting to attempt this kind of story, some advice:\n\nBrace yourself (part 1): In the beginning, this didn't sound like an enormous undertaking. It is. I promise you. The folder where this project resides on my hard drive comes in just under 8 gigabytes, which does not lend itself to easy management.\n\nBrace yourself (part 2): Be prepared for what you're going to get from the property appraiser offices. Some are using Access and ArcView, making everything easy. Some use arcane data management software and unheard-of mapping applications. I was able to get maps and data from them, but it makes a great case for having both Arc 8 and Arc 3 on a machine. Arc 8 relies on a map layer having a projection assigned to it. Arc 3 does not. Data from one of our counties came from a computer-aided drawing system. The parcel file wasn't a collection of polygons - it was a giant collection of hundreds of thousands of lines that seemed to connect, but didn't. I could see what the parcel map was supposed to look like, but it was worthless to me as an analysis tool. The data, however, contained an X/Y coordinate that corresponded to the center of the parcel it was related to. To assign a neighborhood name to a parcel, I did a spatial join. The problem is, none of my layers - from the points to the parcels to the custom-made neighborhood file - had a projection attached to them. Arc 8 won't do a join without a projection, but Arc 3 will.\n\nCheck everything twice: Each appraiser's office stored data differently, and we could get only the barest details about a sale from all five counties. And we were after a very specific slice of the market: single family residential sales of houses (no vacant lots, no condos, no townhouses) that were qualified - an arm's length transaction between a buyer and a seller who didn't know each other and weren't giving the other guy a break that would unduly influence the price.\n\nSome officials understood this perfectly, and knew exactly what we wanted. Others didn't get it, and gave us bad data. To check our data, we took more than 1,000 parcel ID numbers, some from each county, and looked them up, checking the sales history of each parcel for sales we had and sales we didn't have but should have. In one county, the appraiser's office gave us three dirty data sets before getting it right.\n\nWrite everything down: The last thing I did should have been the first. As one of the last things I did for the project, I wrote out, in excruciating detail, exactly what I did from the start. Much of it was cobbled together from handwritten scribbling on printouts and scratch paper. It was therapeutic almost to write it all out. In the future, I plan to start by writing out what I am going to do, then keeping an electronic journal on what I've done since then. It will come in very handy when editors begin asking questions about why this was done and that was done when it's been four months since you did them. I'm designing an intranet page whose whole purpose will be to track major data projects like this, from data acquisition to analysis to deadlines for stories.\n\nThe reaction to this story has been great - my editor called it the \"feel-good investigative project of the year.\" People who own homes are awed by what they might be able to get for their home. I've received a couple of e-mail messages from house hunters thanking me for showing them that they weren't alone. People were buying the Sunday paper at our front counter days after the story ran. The folks who sell them told me they didn't have many left on Wednesday after it ran.\n\nAnd, our editors have said, the story resonated with people more than anything we've done in recent memory. My thought is it resonated because we wrote about something people care deeply about: home. And that made all the work well worth it.\n\nThe complete package, including interactive features, can be found at: www.sptimes.com/2004/webspecials04/homeprices\n\nContact Matthew Waite by e-mail at waite@sptimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-finding-faults-in-new-homes","slug":"2004-july-august-finding-faults-in-new-homes","url":"https://dwillis.github.io/uplink/article/2004-july-august-finding-faults-in-new-homes.html","year":2004,"month":"July-August","monthNum":7,"headline":"Finding faults in new homes","kicker":"SPOTLIGHT: REAL ESTATE","author_name":"Katy Miller","author_title":"Database Designer","affiliation":"Orlando Sentinel","summary":"An investigative journalism project by the Orlando Sentinel and WESH-Orlando uncovered widespread construction quality problems in central Florida's new homes, leading a reporter to develop a complex Microsoft Access database with 17 linked subforms to analyze inspection data from thousands of homes built in 2001.","keywords":["construction","database","homes"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["Access","Excel","Monarch"],"full_text":"One of the most expensive purchases a consumer can make is his or her home. For buyers of new homes, one benefit is not having to worry about repairs. However, an examination of house construction quality in the booming central Florida building market by the Orlando Sentinel and WESH-Orlando, found thousands of problems with the new homes.\n\nOur series, \"Building Homes: Building Problems,\" uncovered a serious decline in craftsmanship in central Florida's new homes. Among the major problems we found were a myriad of heating, ventilation and air conditioning issues, wall and foundation cracks and improperly installed windows. In addition, we found everything from major construction failures to simple poor workmanship.\n\nDon Tracy, a senior projects reporter, asked me to help him build a Microsoft Access database. We had worked together on an Access database in the past, and I knew he had a working knowledge of the program. I didn't realize the extent of project we would be undertaking.\n\nTracy was doing a quantitative study of new home quality in central Florida. As part of the project, Ron Resch, a local building inspector, was selected to train engineering students at the University of Central Florida's home constructability lab to inspect homes built in 2001. My responsibility was to build the database for the inspections information and analyze the results.\n\nThe newspaper acquired a list of homes built in 2001 from local property appraisers. Clerks working for the newspaper and television station called homeowners and asked them whether they would be willing to participate in our study. Those names and phone numbers were submitted to the inspectors to set up appointments.\n\nWith the assistance of Resch, Tracy created a paper copy of an inspection report for the students. It had more than 800 categories of information. That should have been my first tip that Access was the wrong tool for this project because Access tables are limited in the number of fields. But at the time it seemed like the best choice, and it was a database the UCF students were familiar with.\n\nI had two challenges in designing the database. First, I needed to make a form that somewhat resembled the inspection report used by the students for data entry. To achieve this I created multiple tables and linked them together using subforms.\n\nI created tables that corresponded to the distinct sections of the inspection report. For example, there was a bathroom subform and a kitchen subform. Each table contained check boxes that indicated whether the student inspectors spotted a certain problem. There was a total of 17 subforms linked to the master form.\n\nWhile the UCF students inspected the houses, Tracy obtained residential inspection data from the six surrounding counties for the past three fiscal years. We also acquired inspections conducted by the city of Orlando. The goal was to determine the pass-fail rate of the official inspections by county.\n\nThe cost for the data ranged from free to under $100, varying by county. The city of Orlando provided a Microsoft Access database, but it was read-only. We exported the tables into Excel and created a new database. Polk County provided text files. However, the data was in print format and looked like screen shots of a report. I used Monarch to parse the data into fields and export it as a delimited text file.\n\nOsceola County at first refused to provide anything other than a printout; the press officer claimed that the database was not available in electronic format. After we spoke to someone in the technical department, we learned this was untrue and obtained the data in Excel. The remaining counties provided Excel spreadsheets that we easily imported into Access.\n\nUnfortunately, the data sets had a couple of problems. Although we requested the past three years of data, the counties did not provide the exact same time periods. However, I felt there were enough records to show an inspection pass-fail rate.\n\nThe second problem with the data was that every county had provided some duplicate records. I used a select distinct query in Access to create a table with the unique values.\n\nSELECT DISTINCT Inspections SITEADDRS Inspections JOBDESCR, Inspections.CONTRTNAME Inspections.CONTRTCOMP Inspections INSPECTOR, Inspections.INSPCODE, Inspections CODEDESCR, Inspections.inspecdate, Inspections RESULTCODE INTO Deduped\n\nFROM Inspections\n\nAfter I compiled the results, I attempted to verify our findings with the building inspection departments of each county. Only one of the counties had calculated pass-fail rates, and the results matched ours to the decimal point.\n\nIn Access, I ran queries to determine the number of inspections daily and a failure rate by inspector. I then generated a list of inspections by day for each inspector. This helped find inspectors who appeared to pass a majority of the houses.\n\nWe found an inspector who logged more than 90 inspections in one day. Using MapInfo geographic information system, I mapped the daily route for an inspector in each county, illustrating how little time was spent on each inspection because of travel time.\n\nAfter UCF started entering data I needed to transfer it between UCF and the Sentinel to monitor the inspection results. Our technology department set up an FTP site and UCF uploaded the newest version of the database, with accompanying photos and videos, every Monday.\n\nWith the assistance of Resch, Tracy classified the problems found into three categories of severity: worth noting, concerns and priority problems. At this point I realized the restrictions of working in Access. I needed to group the problems into these categories using data from multiple tables.\n\nThe first hurdle was to convert the checkboxes on the forms into a data value. Even after converting the data to a value, I was unable to run queries on the data. I solved this problem by converting the values into new tables, and used the new tables when running a query. To avoid re-creating my efforts, the UCF students downloaded a copy of the database I had created with the queries and added data to that database.\n\nWhen the inspections were almost completed, the editor for the project decided to put the database on OrlandoSentinel.com, our newspaper's Web site. I knew that this would take an enormous amount of programming, and using Structured Query Language (SQL) was the best choice.\n\nIn an effort to be fair to the builders, every problem found was not published in the series. Before converting the data from Access in SQL, I asked Tracy and WESH producer Travis Sherwin which problems should be included in the Web version of the database. They refined the original list of problems down to about 200. (See the database online at http://extra.orlandosentinel.com/buildingproblems)\n\nAfter having a pared down list of problems, I sat down with a programmer from the Orlando Sentinel's technology department assigned to build the Web database. Together we assigned each problem a location in the house (i.e., bathroom), a severity ranking, a system (HVAC), and a system element (i.e., ductwork) for classification purposes.\n\nAfter the inspections were completed, we exported the tables from Access into the Microsoft SQL Server database. There were more than 120 builders, more than 400 homes, and more than 200 potential problems in each home.\n\nOn the inspection subforms, I created a comments box for problems not listed on the inspection sheet. To include this data in our database, I sorted through more than 800 separate comments to be classified. In many cases I had to create a new field for problems that had not been anticipated when the form was created. For example, there were 37 homes with mold on the HVAC unit.\n\nI was put in charge of all the data analysis and statistics for the series. I signed off on every number and kept a spreadsheet listing every stat I checked and the date so I could easily refer back to what I had done when. This was vital because the numbers were flying everywhere, and there were several instances in which preliminary numbers made their way into the final versions of stories.\n\nOne of the main difficulties I encountered was the reporter and editor changing the severity of a problem several times. Unlike the data gathered from the inspection reports, the severity rating was the most subjective part of the series. For example, a problem originally classified as a \"priority problem\" was changed to a \"concern\" the afternoon before a segment was scheduled to air on WESH. At 3 p.m. I was scrambling to help Sherwin with the updated numbers for the segment running at 6 p.m.\n\nI also worked closely with the graphics department, checking off on every chart or graphic. This also paid off. A chart for the pass-failure rate of inspections was mistakenly labeled New Home Inspections, when it was residential inspections of homes of all ages, a category that includes renovations of older homes.\n\nThanks to this diligence, there were not any corrections or errors in the data I was responsible for. There were a couple of corrections because data was entered incorrectly into the database by students, despite our efforts to compare written reports to the electronic database.\n\nThe response to the series has been very positive. The Web page devoted to the database generated an incredible number of hits and the message boards are full of homeowners reporting similar problems. Tracy had to set up a separate voice mail account to handle all the phone calls in response to the series. A state senator has said he will push for tougher standards on new home construction, and the local Home Builders Association has started to expand a homeowner advocacy program.\n\nContact Katy Miller by e-mail at kmiller@orlandosentinel.com\n\nInterested in seeing how others are using computer-assisted reporting to cover news about real estate? Check out this and other recent stories that appeared on IRE's Extra! Extra! (www.ire.org/extraextra).\n\nRobert Gebeloff and Maura McDermott of The (Newark, N.J.) Star Ledger examined tax assessment data for three northern New Jersey counties and found that residential property tax bills are off as much as $1,000 because of tax rolls that towns haven't fully updated in many years.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-readmetxt","slug":"2004-july-august-readmetxt","url":"https://dwillis.github.io/uplink/article/2004-july-august-readmetxt.html","year":2004,"month":"July-August","monthNum":7,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Computer-assisted reporting is being used by journalists to investigate election-related issues, including analysis of Florida's voter purge practices and local politicians' voting records.","keywords":["database","elections","analysis"],"topics":["Investigative Methods","Government & Politics","Demographics & Census"],"technologies":[],"full_text":"Interested in seeing how others are using computer-assisted reporting to cover news about the elections? Check out these recent stories that appeared on IRE's Extra! Extra! (www.ire.org/extraextra).\n\nChris Davis and Matthew Doig of the Sarasota Herald Tribune examined Florida voter data and found that a quirk in the state's drive to clear convicted felons from the voter rolls would spare Hispanics, who tend to support Republicans, in greater numbers than other races.\n\nMark Greenblatt of WBBH-Fort Myers, Fla., built and analyzed a database of the voting records of 34 local politicians and found that a handful voted less than half the time since 1990.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-readmetxt-2","slug":"2004-july-august-readmetxt-2","url":"https://dwillis.github.io/uplink/article/2004-july-august-readmetxt-2.html","year":2004,"month":"July-August","monthNum":7,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Computer-assisted reporting is being used by journalists to uncover electoral irregularities, including analysis of Florida's voter roll purging that disproportionately affects certain racial groups and investigation into local politicians' poor voting attendance records.","keywords":["elections","data","voting"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Interested in seeing how others are using computer-assisted reporting to cover news about the elections? Check out these recent stories that appeared on IRE's Extra! Extra! (www.ire.org/extraextra).\n\nChris Davis and Matthew Doig of the Sarasota Herald Tribune examined Florida voter data and found that a quirk in the state's drive to clear convicted felons from the voter rolls would spare Hispanics, who tend to support Republicans, in greater numbers than other races.\n\nMark Greenblatt of WBBH-Fort Myers, Fla., built and analyzed a database of the voting records of 34 local politicians and found that a handful voted less than half the time since 1990.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-vigilant-program-tracks-web-changes","slug":"2004-july-august-vigilant-program-tracks-web-changes","url":"https://dwillis.github.io/uplink/article/2004-july-august-vigilant-program-tracks-web-changes.html","year":2004,"month":"July-August","monthNum":7,"headline":"Vigilant program tracks Web changes","kicker":"CAR TOOL","author_name":"Sarah Cohen","author_title":"","affiliation":"The Washington Post","summary":"Website Watcher is a monitoring tool that helps journalists track changes to websites, such as FEC campaign finance filings, by ignoring irrelevant updates like timestamps and alerting users only to substantive changes.","keywords":["journalism","monitoring","automation"],"topics":["Government & Politics","Technology","Investigative Methods"],"technologies":["Website Watcher","Perl"],"full_text":"As any journalist who has to deal with presidential campaign finance raw data can tell you, the 20th of each month this year poses a competitive nightmare. Candidates must report their monthly contributions and expenditures to the Federal Election Commission by midnight of that day. But some report a day or so early, and others miss the deadline by a few minutes.\n\nThis means that journalists might have a day to analyze and report the results or may still be waiting, at deadline, for the data. Knowing when the data is available is crucial and getting the filings as soon as they are posted on the Web becomes a competitive obsession.\n\nEnter Website Watcher. The utility is available for purchase and download at www.aignes.com. A single-use commercial license costs $99. There are several similar tools on the market. I chose Website Watcher because it was the first one I found that had a few features I needed.\n\nBasic use of the program is simple: You type a Web address for the program to check and give it a name.\n\nYou can keep adding pages. For those you want to check routinely, add them to a HotSite. For those you just want to keep track of once in a while, you can just keep it manually.\n\nAfter you enter the sites, set them to \"Autowatch,\" choose how often they should be checked and you can walk away. You can set Website Watcher to alert you of changes by playing a sound, opening the page in a browser, e-mailing a copy of the page or even running a program.\n\nThere are a few features that make this program especially good for the kind of work we journalists do.\n\nThe most important feature is the ability to ignore changes that don't matter. For example, the FEC electronic filing system re-generates the page each time it's checked. This means that technically, the page always changes.\n\nThat little phrase, \"Generated Thu May 20 11:37:54 2004\", in the page would create a new version each time the page was checked. But Website Watcher can ignore all changes in dates, ignore a phrase or pattern of characters that you define, or to do the opposite: Only count it as a change when a certain pattern appears. Here is how I've told the program to ignore this little \"Generated\" phrase.\n\nI've told Website Watcher to ignore links and images, since I know they won't include a new filing. I've also told it to ignore internal coding. But I didn't want it to ignore \"all typical date strings\", because that might be included in a new filing. Instead, I gave it a pattern to look for by checking \"Ignore userdefined strings\", then typing in this pattern.\n\nThis expression means look for the word \"Generated\", then anything, then four digits in a row. If this is the only thing that changed in the page, then Website Watcher won't consider the page changed.\n\nAnother feature I like is that the program archives the most recent copy of the page. When there is a change, Website Watcher alerts you and then saves a new copy with the changes highlighted in yellow. In this case the phrase \"(As of March 31, 2004)\" has been highlighted and formatted as italic in my viewer.\n\nFor the Bush-Cheney fund-raisers, called Pioneers, it highlights new last names. It can also show you what has been deleted from a page at the end.\n\nThere are many more advanced features I haven't explored. Among them are options to e-mail copies of the changed page and run programs, such as a Perl script, when a change is identified.\n\nBut I have to go. Website Watcher just beeped to let me know that the Bush campaign filed an amendment to its April report.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-pdf-to-text-using-xpdf","slug":"2004-july-august-pdf-to-text-using-xpdf","url":"https://dwillis.github.io/uplink/article/2004-july-august-pdf-to-text-using-xpdf.html","year":2004,"month":"July-August","monthNum":7,"headline":"PDF to text using XPDF","kicker":"Tech tip","author_name":"Derek Willis","author_title":"","affiliation":"Center for Public Integrity","summary":"Xpdf is a free command-line utility that allows journalists to efficiently extract text and tables from PDF files, converting them to formats like Excel in seconds rather than hours of manual copying and pasting.","keywords":["PDF","extraction","tool"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["XPDF","Excel","DOS","Windows","Linux","Unix"],"full_text":"Here's a question that should have a familiar ring for CAR veterans: How do I get text out of a PDF file?\n\nIf your experience has been anything like mine, the answer is likely to be \"painfully, if at all.\"\n\nThe existence of Adobe Portable Document Files has been a boon for reporters because governments and agencies everywhere have been able to make documents broadly available over the Internet. It's hard not to love the idea of a file that looks the same no matter what kind of computer you have.\n\nBut if you've ever tried getting tables out of a PDF document - and we all have - the results usually aren't worth the effort. Until now. (See \"Slicing and dicing those pesky PDFs\" in the July-August 2003 Uplink.)\n\nIf you're ready to leave behind the annoyance of copying and pasting from PDF files, a free command line utility called Xpdf will save you time and aggravation. It will, in most circumstances, enable you to go from PDF to Microsoft Excel in a matter of seconds, rather than minutes or hours. Did I mention that it's free?\n\nAlthough Xpdf comes with many versions of the Unix or Linux operating systems, Microsoft Windows users will need to download and install the small application in order to run it.\n\nXpdf can be downloaded from www.foolabs.com/xpdf/download.html, and it comes in packages for Windows and Linux/Unix. Look for the precompiled binaries for your operating system. A DOS version is available, but unless you run DOS as your primary operating system, the Windows version should be your choice. The Windows version does not have all the features of the Linux/Unix version, but it has all you'll need for converting PDFs to text tables.\n\nFirst, save the download file (a zip file) to your computer and then unzip it into a directory (c:\\xpdf works fine). Then, and this is the technical bit, when you have a PDF you want to get tabular text from, simply place the PDF in that directory, open a command prompt, then navigate to c:\\xpdf and type:\n\npdftotext layout pdfname.pdf\n\nDepending on the size of the PDF file, your output text file (with the same name as the original) will be in the same directory in a matter of seconds.\n\nLet's go through the command line syntax. First, the command \"pdftotext\" is required for this process, and \"pdf2text\" won't work. The \"-layout\" tag tells Xpdf that you want to preserve the layout that the PDF file uses, which keeps the text in those nice, clean tables. And you need to have the full name of the file (I recommend a single-word name, even though Windows supports filenames with spaces). That's it. You would be forgiven for thinking that it can't be that easy, but it really is.\n\nThe resulting text file will be the entire text of the PDF, meaning that you may have to wade through pages of text in order to get to your tables. The preservation of the PDF's layout means that if a page contained two tables side-by-side, that's the way they will look in the text file, too.\n\nXpdf doesn't work in all instances; specifically, it won't convert PDFs that have been locked by their creators. Don't bother asking the author of Xpdf, either, as he has posted a message on his Web site indicating that he will not add that ability.\n\nFor dealing with most government documents, Xpdf can be a huge time-saver and allow you to spend more time actually analyzing data rather than trying to free it from the confines of the PDF.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-july-august-winners","slug":"2004-july-august-winners","url":"https://dwillis.github.io/uplink/article/2004-july-august-winners.html","year":2004,"month":"July-August","monthNum":7,"headline":"Winners","kicker":"IRE AWARDS","author_name":"Andrea Lorenz","author_title":"Contributing Editor","affiliation":"IRE and NICAR","summary":"Three award-winning investigative journalism projects are highlighted, including investigations into lead contamination in Detroit, oyster farm compensation in Louisiana, and government handling of Hepatitis C virus cases. The articles showcase how reporters used data analysis tools and databases to uncover government negligence and questionable spending.","keywords":["investigation","data","contamination"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["Access","Excel","FoxPro","GIS"],"full_text":"Winners\n\nThe article won second place in the Associated Press Sports Editors investigative reporting category. (See Story No. 20338)\n\n\"Damaged Lives: Lead's Toxic Toll,\" Detroit Free Press, Emilia Askari, Tina Lam, Megan Christensen, Marsha Low, Hugh McDiarmid, Jr., Dan Shine, Shawn Windsor and Wendy Wendland-Bowyer\n\nA team of reporters investigated the contamination from lead smelters and leaded gasoline. They found government agencies neglecting to clean up lead in neighborhoods and test children for lead poisoning.\n\nThey used Access to analyze a Michigan Department of Community Health database for children tested for lead poisoning and mapping software to show the results of their soil tests. (See Story No. 20369)\n\n\"Shell Game,\" The Times-Picayune, Aaron Kuriloff and Jeffrey Meitrodt.\n\nThis series investigated $2 billion in court judgments against the state in favor of oyster farmers. The reporters found the awards to be either unjustified or excessive. Using tax and harvest records of oyster farms, court documents and sales data, they built tables in Excel to show results of a project initiated by the state to rebuild the coast. They found oyster beds were doing better - although oyster farmers claimed they were destroyed - than before the project.\n\nThe series won the Louisiana Press Association investigative award, and the Associated Press/Louisiana-Mississippi story of the year. (See Story No. 20608)\n\n\"Silent Alarm,\" The Kansas City Star, Karen Dillon and Mike McGraw\n\nThis series provided an in-depth look at the government's handling of the Hepatitis C virus. They used Microsoft Visual FoxPro and Excel to work with some of the data. Spreadsheets containing information about people alive today with the virus contracted through blood transfusions showed the estimated mortality of these patients. Other spreadsheets showed the number of Hepatitis C cases since the 1980s that could have been prevented by the government with a test. They also created a database of medical journal articles about the virus. \"Silent Alarm\" won first place in the Kansas Press Association award for investigative reporting. (See Story No. 20921)","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-september-october-purged-voter-problems","slug":"2004-september-october-purged-voter-problems","url":"https://dwillis.github.io/uplink/article/2004-september-october-purged-voter-problems.html","year":2004,"month":"September • October","monthNum":9,"headline":"Purged voter problems","kicker":"ELECTIONS","author_name":"Chris Davis and Matthew Doig","author_title":"","affiliation":"Sarasota Herald-Tribune","summary":"Reporters discovered a fatal flaw in Florida's felon voter purge list where Hispanics were systematically excluded due to inconsistent race categorization. The analysis led to the scrapping of the list, which could have significantly impacted the 2004 presidential election results.","keywords":["voter purge","Florida elections","felon list"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel"],"full_text":"Our analysis of a controversial database that Florida election officials were using to purge felons from the voter rolls exposed a fatal flaw that led to the list's demise. The analysis was simple, though the story was anything but ordinary. We imported the state's data into Microsoft Excel. With the help of a pivot table, we found how many of the 48,000 people on the list were white, black or Hispanic. Excel's pivot table function allows users to create cross-tabs to categorize and summarize data. The check was a basic step any journalist should take after he or she first gets new data. Immediately, the problem appeared in the results – 24,000 of the potential felons were white, 22,000 were black, and 61 were Hispanic. We knew that Hispanics in Florida make up a key Republican voting bloc that had been instrumental in getting Gov. Jeb Bush and the President elected. Roughly 17 percent of Florida's population is Hispanic, the largest minority population in the state. It didn't seem logical that there would only be 61 on the list. After we double-checked our work to make sure we hadn't made a mistake, it left us with three possibilities to explore: 1. Are Hispanics less likely to commit felonies and less likely to register to vote? Neither turned out to be true. Hispanics make up about 11 percent of the state's registered voters, about 11 percent of Florida's prison population and 11 percent of the state's voting-age population. Given those numbers, it didn't make sense that less than 1 percent of the people on the state's purge list was Hispanic. 2. Were Hispanics hidden on the list under different race classifications? To answer this, we took a sample of 200 names from the list and examined them one by one, looking for Spanish surnames. We settled on a sample of 200 after talking with Steve Doig, a journalism professor at Arizona State University and Matthew's father. The sample size gave us an acceptable error margin of just about 7 percent. We obtained our sample by dividing 200 into 47,763, the total number of names on the list. We took the result, 239, and used Excel to number each name from 1 to 239 and repeated that process all the way through the list Then we employed the highly scientific method of asking someone in the newsroom to shout out a number between 1 and 239. When we had our number, we used Excel's AutoFilter tool to display only those names with that number beside them. We found a list of common Hispanic surnames on the Internet, and we enlisted a Hispanic colleague at the newspaper to help us identify people who could be Hispanic felons but listed under another race category. We repeated this process three times to convince ourselves we hadn't made an error. Each time we determined that, at best, 3 percent of the people on the state's purge list were Hispanic. That fell far short of the 11 percent we expected to see. 3. Did Florida's election officials botch their list somehow? Given what happened during the 2000 presidential election, this was something we'd considered from the start. So when we ruled out the other potential explanations, we started investigating how the state had built their felon list so we could spot where a data flaw might have been introduced. Our Hispanic colleague helped again when he reminded us that Hispanics traditionally have trouble being placed into a particular racial or ethnic category. An official with the U.S. Census Bureau agreed with his observation. The perfect example of this problem would be a man born in Cuba whose skin is black. That person might think of himself as Hispanic, and he could identify himself that way when registering to vote. But if someone else, such as a police officer, attempted to categorize him, he could easily be considered black. We interviewed an official with Florida's Department of State who explained that they merged two data sets – one from the Florida Department of Law Enforcement and another from the Division of Elections – to create a list of people who were convicted felons registered to vote. Florida is one of a handful of states in which a felon's voting rights aren't automatically restored. If you've used Microsoft Access to join two tables, you know an identifier common in both databases is needed to pull off the feat. And because the data might not be uniform in the tables, you also know what a headache it can be to make that match. To ensure that John Smith the felon was also the same John Smith registered to vote, state officials used every identifier available to make their match. This included first and last names, date of birth, sex and race. If any of those fields didn't match across the tables, that person would not have been included in the purge list. Because state officials used race for the matching process, we knew that's what had caused so many Hispanics to be left off the list. We presented what we found to the state's elections chief, and she conceded that problems in classifying Hispanics into a single race category had probably resulted in their exclusion. Despite the admission, officials said they had no plans to fix the Hispanic felon flaw. But apparently it's a lot harder to resist something published in The New York Times. After seeing the Herald-Tribune story, Times reporter Ford Fessenden learned that there was more to the problem than ambiguity in race reporting. It turned out that the FDLE database didn't even offer Hispanic as a race category. Later, the Herald-Tribune reported that prior to 1994, many supervisors of elections did not offer Hispanic as a race category. That virtually guaranteed that Hispanic felons wouldn't end up on the purge list. And it meant that Florida election officials had botched three felon purge lists since 1998. Three days after the Herald-Tribune first reported the Hispanic voter flaw, Fessenden's story ran in The New York Times. Later that day, Florida Secretary of State Glenda Hood announced she was scrapping the list. The flaw has cost the state millions of dollars and diminished its credibility, especially with Democrats and African-Americans who claim the list is designed to minimize their political power. The current purge list had only 9,500 Republicans, compared to about 28,000 Democrats. The demise of the list could have a huge impact on the November presidential election, which polls show could be as close as the one in 2000. With another quick Excel analysis, we were able to illustrate the potential impact. We began with the premise that the presidential election in Florida would be as close in 2004 as it was when it was decided by a 537-vote margin in 2000. We wanted to see how many people on the purge list would have to vote along party lines to reverse the outcome of a hypothetical election that was as close as the Bush-Gore race. What we found is that even if every Republican on the purge list voted for Bush, less than 36 percent of the Democrats would need to vote for Kerry to give Florida's electoral votes to Democrats. And if every Republican and Democrat on the list voted for their party's candidate, the outcome would be an 18,000-vote victory for the Democrats. That little bit of math let us write this lead: 'If the 2004 presidential election ends up being as close as the last one, Gov. Jeb Bush may have cost his brother his job.'","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-guiding-the-project-from-start-to-finish","slug":"2004-september-october-guiding-the-project-from-start-to-finish","url":"https://dwillis.github.io/uplink/article/2004-september-october-guiding-the-project-from-start-to-finish.html","year":2004,"month":"September • October","monthNum":9,"headline":"Guiding the project from start to finish","kicker":"SPOTLIGHT: EDITING & MANAGING","author_name":"Tim Ragones","author_title":"","affiliation":"IRE and NICAR","summary":"Veteran project editor John Ullmann offers essential guidance for newsroom managers on how to effectively lead and edit computer-assisted reporting projects.","keywords":["project management","newsroom editors","investigative reporting"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Working on a computer-assisted reporting project can mean dealing with slow record requests, dirty data and uncooperative software. But untrained editors may pose as much an obstacle to projects as these other hurdles. That's what John Ullmann, IRE's first full-time executive director and the current executive director of the World Press Institute in St. Paul, Minn., told a room of newsroom managers at a CAR boot camp for editors earlier this year in Columbia, Mo. If project reporting is a team sport, Ullmann said, editors are its weakest members. Ullmann, tapping into his experience as a former project editor, provided guidance to the editors and news managers at the boot camp on how to build better computer-assisted stories from conception to publication. Ullmann served as the assistant managing editor for projects at the Star Tribune in Minneapolis for six years. He supervised projects that won more than four dozen awards. In 1990, a series showing how members of the St. Paul, Minn., fire department profited from arson won the Pulitzer Prize for investigative reporting. In Ullmann's experience, the typical computer-assisted or investigative project lasts three to six months. Ullmann liked two-person investigative teams because he could partner people with different skills, and they would finish more quickly. Ullmann said good stories come from good editors who are well-read and good reporters working out in the field. While data is a valuable resource for finding story ideas, it's just a starting point, he said. Reporters should not be too proud of themselves when they get the data because the data is rarely the story. Journalists are in the business of storytelling, not describing numbers. Stories are always about victims and protectors, and it's important to look beyond the data to find them. When a reporter pitches a prospective story, Ullmann said an editor should ask two questions. First, what is the best-case scenario of the story? Second, would the reporter read the story if his or her name wasn't on it? Once a news organization approves a computer-assisted project and the reporting begins, Ullmann said it's important to give meaning to the data. That means the reporters need to find people who can reflect what the data shows and identify experts who can help interpret conclusions. Ullmann recommended transcribing soon after the interview. He said reporters should throw out any notes they took during the interview because these are not as accurate as the transcribed tape. Numbers can mean many things; Ullmann said it's important to include numbers that can support other interpretations in the story. He suggested working just as hard to disprove a story's thesis. Show the findings to the 'enemy' early in the reporting to avoid surprises after publication or broadcast. Reporters and editors should meet frequently for story conferences. When he was an editor, Ullmann said he barred computer printouts and notebooks from these meetings so numbers would not rule reporters. Reporters should never wait until they finish reporting before starting to write, Ullmann said. He liked to see drafts from his reporters every few weeks. Reporters should write what they know as they report. This strategy allows the news organization to publish or broadcast earlier than planned if necessary. In addition, it gets the reporters to focus on the story, not on each individual document or source. Ullmann suggested that reporters distribute their drafts to editors around the newsroom to find out if they get excited about the story. Ullmann said it's important to allow substantial time for editing and rewriting. That may involve shortening the story, but not always. Ullmann said long stories work if readers and viewers care about them. Charts, graphics and pictures are one way to increase a story's pace by cutting number-heavy material out. Ullmann said editors should include the photo and graphic staff from the beginning. They are partners in the presentation of the project, and they are critical to making it work. When the project is ready for publication or broadcast, Ullmann stressed one final step: line-by-line editing. Reporters should do this with their data. When Ullmann supervised projects, he wanted to see the source of every number in the story, whether it was in a document or came from a formula. He asked his reporters to redo calculations in front of him to make sure they were correct. If editors follow these steps, Ullmann said they would not be stuck on the bench; instead they will become valuable members of the team.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-starting-small","slug":"2004-september-october-starting-small","url":"https://dwillis.github.io/uplink/article/2004-september-october-starting-small.html","year":2004,"month":"September • October","monthNum":9,"headline":"Starting small","kicker":"","author_name":"David Donald","author_title":"","affiliation":"IRE and NICAR","summary":"David Donald provides a strategic list of ten 'dos' and ten 'don'ts' for journalists attempting to establish computer-assisted reporting practices in smaller newsrooms.","keywords":["small newsrooms","CAR techniques","professional development"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["CD-ROM","Internet","World Wide Web"],"full_text":"What's next? That's the question I face most often when I finish teaching an introductory session of computer-assisted reporting to journalists from small news organizations. What they're often really asking is simple: How do I go back to a small- or medium-sized newsroom that has never done CAR? Sad to say, even today some larger newsrooms exist without doing much CAR. But for reporters in smaller organizations, the thought is intimidating. They know that with CAR, as with most skills, it's use it or lose it. The real trick is changing that mind set. When working in a small- to medium-sized newsroom, you must not use size as an excuse, even if you have fewer CAR converts and resources than your counterparts in larger newsrooms. Easier said than done. But journalists in some smaller newsrooms are doing outstanding CAR work. A good place to see this in action is in Uplink's First Venture feature, which provides useful lessons for journalists in smaller newsrooms. A recent example: Dave Moore of the Columbia (Mo.) Daily Tribune wrote about his analysis of racial disparity in juvenile arrests in the May/June 2004 issue. When Dave got the records on a CD, they were a mess. He could have given up. But with some help from NICAR, he got the data straightened out and a good story into the paper. When I worked in a smaller newsroom, I had some rules to keep on track. What follows are 10 of those dos and 10 don'ts. Cut out the ones that make sense to you and paste them on your computer. Do: 1. Have reporters sit next to you while you do your analysis. 2. Insist your reporters gather all documents electronically if possible. 3. Attend news meetings at which brainstorming for story ideas happens. 4. Get to be the 'numbers person' in your newsroom. 5. Start small. Editors adore daily copy that makes their lives easier. 6. Broaden your definition of CAR. 7. Get to know your graphic artists. 8. Bookmark 10 Web sites you can use to find data. 9. Carve out a neglected niche in your newsroom that is CAR-intensive. 10. Stay passionate and love what you do. Don't: 1. Take no for an answer when asking for hardware or software. 2. Be a recluse. Keep networking with CAR folks. 3. Let public officials win when they refuse to give you data digitally. 4. Stay in a newsroom that has absolutely NO appreciation for CAR. 5. Forget the IRE Resource Center when trolling for CAR ideas. 6. Get totally caught up with the computer and forget about stories. 7. Be afraid to think big. 8. Fall for Stockholm Syndrome in your newsroom. 9. Try to make everyone in your newsroom a CAR believer. 10. Get discouraged. You're doing work vital to journalism and those you serve.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-hazardous-site-data-available","slug":"2004-september-october-hazardous-site-data-available","url":"https://dwillis.github.io/uplink/article/2004-september-october-hazardous-site-data-available.html","year":2004,"month":"September • October","monthNum":9,"headline":"Hazardous-site data available","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"Journalists investigating environmental issues can now access a cleaned and enhanced EPA hazardous waste database through the IRE and NICAR Database Library, which includes additional tables on contaminants and responsible parties not available from the EPA website.","keywords":["database","hazardous","journalists"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["CERCLIS"],"full_text":"Journalists investigating environmental issues in their community can use a database about hazardous waste sites now available in the IRE and NICAR Database Library. The database contains information on potential and confirmed hazardous waste sites at which the U.S. Environmental Protection Agency Superfund Program has some involvement.\n\nMaintained by the EPA, the Comprehensive Environmental Response, Compensation and Liability Information System (CERCLIS) database has been cleaned by NICAR data analysts for easier use. It includes two tables about contaminants and responsible parties that are not available for download from the EPA Web site. The data is current as of April 2004.\n\nTo order the CERCLIS database, contact IRE and NICAR at 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-september-october-pollution-beat-book","slug":"2004-september-october-pollution-beat-book","url":"https://dwillis.github.io/uplink/article/2004-september-october-pollution-beat-book.html","year":2004,"month":"September • October","monthNum":9,"headline":"Pollution beat book","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE published a new environmental pollution beat book in collaboration with the Society of Environmental Journalists that provides journalists with tools for quick investigations, including federal database searches and tips from experienced reporters on interviews and writing.","keywords":["pollution","investigation","journalism"],"topics":["Health & Environment","Investigative Methods","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"IRE recently published an environmental pollution beat book, \"Covering Pollution,\" in collaboration with the Society of Environmental Journalists. The book shows journalists how to get to the heart of an investigation quickly and without waiting months for Freedom of Information requests to be fulfilled. Included is information about how to use Web-based, federal database searches and tips from veteran reporters about how to handle investigative reporting aspects such as interviews and writing. To buy the book or to search for another IRE beat book, visit www.ire.org/store/books or call 573-882-3364.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-september-october-putting-a-focus-on-the-editing","slug":"2004-september-october-putting-a-focus-on-the-editing","url":"https://dwillis.github.io/uplink/article/2004-september-october-putting-a-focus-on-the-editing.html","year":2004,"month":"September • October","monthNum":9,"headline":"Putting a focus on the editing","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"Director","affiliation":"NICAR","summary":"This article highlights the critical role of editing in the computer-assisted reporting process and details training opportunities designed specifically for newsroom managers.","keywords":["editing","CAR training","news management"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Here at NICAR and in the pages of Uplink, we tend to focus on computer-assisted reporting. It makes sense, given that the majority of journalists who are doing CAR are reporters. But reporting is only one part of the process, and the reporting side is no more important than the editing. So with this issue of Uplink we provide guidance for editing CAR stories and address some of the unique challenges these stories present. Also, you'll find articles about how to start CAR in smaller newsrooms – something that's on the mind of many journalists in these days of fewer resources. Editors and news managers have specific needs for hands-on CAR training. That's why IRE and NICAR regularly offer a mini-Boot Camp for editors. Ten editors from newspaper, television and radio attended the last session. They learned how to make CAR successful in their newsrooms, negotiate for electronic records, and understand what their reporters are doing. Next year's Editor Boot Camp is April 8-10 in Columbia, Mo. You'll find registration materials at www.ire.org/training/april05EditorCamp.html. If you're an editor or news manager who wants to understand more about CAR and allow it to blossom, this is essential training.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-upcoming-hands-on-car-training","slug":"2004-september-october-upcoming-hands-on-car-training","url":"https://dwillis.github.io/uplink/article/2004-september-october-upcoming-hands-on-car-training.html","year":2004,"month":"September • October","monthNum":9,"headline":"Upcoming hands-on CAR training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR are offering computer-assisted reporting training programs in 2005, including week-long Boot Camps in Columbia, Missouri, specialized GIS mapping courses, and hands-on classes at annual conferences in Hollywood and Denver.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["ArcView","spreadsheet","GIS"],"full_text":"Looking for intensive computer-assisted reporting training in the coming year? IRE and NICAR have scheduled several introductory and specialty hands-on CAR seminars.\n\nThere are four weeklong Boot Camps in Columbia, Mo., for journalists who want to learn how to acquire electronic information, use spreadsheets and databases to analyze the information, and to translate that information into high-impact stories. IRE and NICAR provide follow-up help after participants return to their news organizations. The 2005 Boot Camps are Jan. 9-14, March 20-25, May 15-20 and Aug. 7-12.\n\nJournalists interested in learning how to map data for news stories can take advantage of mini-Boot Camps in using geographic information systems (GIS). The training sessions, Jan. 14-16 and Aug. 19-21 in Columbia, will use ArcView GIS. Participants will learn how to create basic maps, work with projections, and geocode data. Additional GIS training is being planned for Washington, D.C.\n\nHands-on classes will be offered at the Annual CAR Conference March 17-20 in Hollywood, Calif., and at the IRE Annual Conference June 2-5 in Denver.\n\nFor complete information about IRE and NICAR training visit www.ire.org/training.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-september-october-scraping-away-for-useful-data","slug":"2004-september-october-scraping-away-for-useful-data","url":"https://dwillis.github.io/uplink/article/2004-september-october-scraping-away-for-useful-data.html","year":2004,"month":"September • October","monthNum":9,"headline":"Scraping away for useful data","kicker":"INSIDE NICAR","author_name":"Derek Willis and Jeff Porter","author_title":"","affiliation":"The Center for Public Integrity / IRE and NICAR","summary":"IRE and NICAR are launching the Scraping Project to help journalists use programming scripts to extract and format data from public government websites.","keywords":["web scraping","data collection","Perl"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["Perl","PHP","Python","spreadsheet","Internet","World Wide Web"],"full_text":"Been scraping lately? Toss out the mental images of home improvement, and instead think about databases on the Internet. The rise of government Web sites provides a great opportunity for CAR-savvy newsrooms. As with many developments involving governments, the promise is somewhat hidden, but IRE and NICAR are planning a project to change at least part of that. More and more public databases are being placed on the Web, but often they are hidden behind a search form, or they require hours of repetition to save each record. And then there's the time spent separating the data from the HTML that surrounds it. The Scraping Project is an answer to these problems. With a small dose of programming (think scripting languages like Perl, PHP or Python), news organizations can \"scrape\" the information directly from a Web site and save it in a format that spreadsheet and database programs can use. Many state and federal agencies now provide data in this way, even if the underlying data can be obtained through public record requests. The advantage of scraping is that often you can get the data in a much quicker time frame. Here are some recent examples of scraping: Mark Schaver of The (Louisville) Courier-Journal wrote a script in Perl to grab information about state-registered charities from the Kentucky attorney general's Web site. In another instance, John Perry of The Oklahoman used Perl to scrape docket information off a state court Web site. Other IRE members have done – or will soon do – similar tasks, and the Scraping Project will become a central resource for either the code or the data (or both, depending on the situation). Journalists would identify data sources in their states or cities that are not available as a single download, and then either write a script themselves or ask for assistance from fellow scrapers. Depending on the nature of the data and competitive factors, the script – and perhaps the data – would be made available to other journalists through IRE's Web site. For news about the Scraping Project, keep checking the main IRE Web page (www.ire.org). Developments will also be announced on the NICAR-L listserv.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-hazmat-updated","slug":"2004-september-october-hazmat-updated","url":"https://dwillis.github.io/uplink/article/2004-september-october-hazmat-updated.html","year":2004,"month":"September • October","monthNum":9,"headline":"Hazmat updated","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR's Hazardous Materials Incident Reporting System (HMIRS) database provides journalists with detailed incident reports on hazardous material releases across all transportation modes from 1970 to 2004, enabling investigation of regional safety impacts. The resource includes examples of investigations, such as a Cleveland Plain Dealer story about FAA monitoring failures.","keywords":["hazardous","database","investigation"],"topics":["Investigative Methods","Health & Environment","Data & Databases"],"technologies":["Hazardous Materials Incident Reporting System (HMIRS)"],"full_text":"New data in IRE and NICAR's Hazardous Materials Incident Reporting System (HMIRS) database can help journalists investigate the impact of accidents in their region. The database includes incident reports about hazardous material releases for all types of transportation (air, highway, railway and water).\n\nThe main table, Hazmat, provides incident details including location, carrier, shipper and consigner's identity, origin, and destination. Data covers the period between 1970 and June 3, 2004.\n\nThe IRE Resource Center offers examples of hazardous material stories including one by the The (Cleveland) Plain Dealer that found the Federal Aviation Administration had failed to monitor hazardous materials transported on airplanes (Story No. 15476). For sample stories, visit the IRE Resource Center at www.ire.org/resourcecenter. For more information or to order the HMIRS database, contact IRE and NICAR at 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-september-october-editors-note","slug":"2004-september-october-editors-note","url":"https://dwillis.github.io/uplink/article/2004-september-october-editors-note.html","year":2004,"month":"September • October","monthNum":9,"headline":"Editor's note","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"A correction to a previous Uplink article about home construction faults, clarifying that senior projects reporter Dan Tracy should have been credited, with references to his related work in the IRE Resource Center.","keywords":["correction","construction","journalism"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"In the last issue of Uplink, \"Finding faults in new homes\" by Katy Miller of the Orlando Sentinel included an incorrect name. It should have referenced senior projects reporter Dan Tracy. Construction stories in which Tracy took part can be found in the IRE Resource Center (www.ire.org/resourcecenter) under Story No. 21240 and 20682.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-september-october-cobbling-together-data-for-public-lands","slug":"2004-september-october-cobbling-together-data-for-public-lands","url":"https://dwillis.github.io/uplink/article/2004-september-october-cobbling-together-data-for-public-lands.html","year":2004,"month":"September • October","monthNum":9,"headline":"Cobbling together data for public lands","kicker":"CRIME","author_name":"Judd Slivka","author_title":"","affiliation":"The Arizona Republic","summary":"Reporter Judd Slivka details the difficulties of normalizing crime data from five different federal agencies to investigate public safety in Western parks.","keywords":["crime data","public lands","federal agencies"],"topics":["Investigative Methods","Data & Databases","Crime & Justice"],"technologies":["Excel"],"full_text":"Our story about crime in parks along the Arizona-Mexico border started after a park ranger remarked to me, 'It's getting so dangerous out here that I won't even take my family camping anymore.' At first, my newspaper wanted to measure crime on public land in Arizona and the West, but we quickly ran into obstacles. One big problem: Five federal agencies dominate land ownership in the Southwest, and none kept their data the same way. The reporting got complex, even for what seemed like simple questions. Working with BLM's public affairs and law enforcement staff, we finally limited the scope to properties in the National Resource Conservation System. Even then, there was so much public land that we ended up limiting the scope even further by focusing on the region west of the Rocky Mountains. We had to deal with five agencies, none of which kept its data in the same way. The federal agencies did not record their crimes using the FBI Uniform Crime Report (UCR) format. The USDA Forest Service, for example, has a new system for reporting crimes that uses a violation code that does not conform with UCR categories. It took days to figure this out by building an Excel spreadsheet for each of the 40 forests in the West. The National Park Service has a standard form that its parks are supposed to use, but we discovered that not all parks use the form. Now that we had the data, we had to figure out how to measure it. The statistical part was simple; we would use a common benchmark, crimes per 10,000 visitors. We also chose to measure acres per law enforcement officer. After we compiled the data and sorted it in Excel, we faced a new problem: dissimilar data. The numbers looked the same, but they were not comparable. We used the combined list internally to show us where to focus the reporting. Our database guided us to a story on how crime on the Arizona-Mexico border was drawing law enforcement officers of the agencies away from their assigned stations. As a result, crime was going unnoticed and unpunished on vast tracts of recreational land. The database wasn't perfect. But it was a great guide for us to write a story on a subject close to the hearts of our readers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-plotting-firehouse-coverage","slug":"2004-september-october-plotting-firehouse-coverage","url":"https://dwillis.github.io/uplink/article/2004-september-october-plotting-firehouse-coverage.html","year":2004,"month":"September • October","monthNum":9,"headline":"Plotting firehouse coverage","kicker":"MAPPING IT OUT","author_name":"Thomas Gaumer","author_title":"reporter","affiliation":"The Cleveland Plain Dealer","summary":"The Cleveland Plain Dealer used GIS mapping to reveal potential redundancies in fire station locations, suggesting that consolidation could save taxpayers millions.","keywords":["mapping","GIS","government efficiency"],"topics":["Technology","Investigative Methods","Public Safety"],"technologies":["ArcView","Excel","Adobe Illustrator"],"full_text":"As part of a regionalism series, Plain Dealer reporter Martin Stolz decided to explore fire services. A central question was whether greater Cleveland had duplicative fire departments, equipment or stations. He quickly realized the fire stations element was a story best told visually. Cuyahoga, the county around Cleveland, has a total of 103 fire stations. Stolz wanted to see if the county really needed that many, or if perhaps consolidation might save taxpayers millions of dollars without affecting firefighting abilities. The result was a map of the location of every fire station in Cuyahoga County with a one-mile radius around it. The Plain Dealer was making the switch from ESRI ArcView 3.3 to ArcView 8.3 and this was the first chance to do a major map with the new software. Stolz had created a Microsoft Excel spreadsheet with every station in the county. On our first pass through geocoding, ArcView plotted all but 15 of the 103 stations. That's when the hard part started. For several addresses, ArcView had the street name, but not an exact address, and we were able to force ArcView to geocode those addresses using a nearby address. We found one street that was not even in ArcView. It was barely more than an alley, yet it was the legal address of the fire station. We deliberately gave some stations the wrong address from the standpoint of the post office, but perfectly fine for ArcView. In the case of SOM Center Road, the letters were officially separated by periods, as in 'S.O.M.,' but ArcView saw 'S.' as 'South.' Once we found the problem it was easy enough to fix. It was easy to draw the one-mile radius around each station using ArcView's buffer wizard and telling it we wanted a buffer of one mile. At first, ArcView made circles appear as ovals, which signaled that we needed to pick a better map projection. After we selected a projection for northern Ohio the ovals turned into circles. We exported the map in the Adobe Illustrator format and our graphics staff was able to open it with Illustrator and change colors and reposition numbers and names as they wanted.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-wheres-the-fire-station","slug":"2004-september-october-wheres-the-fire-station","url":"https://dwillis.github.io/uplink/article/2004-september-october-wheres-the-fire-station.html","year":2004,"month":"September • October","monthNum":9,"headline":"Where's the fire [station]?","kicker":"","author_name":"","author_title":"","affiliation":"The Plain Dealer","summary":"Cuyahoga County's 53 independent fire departments operating 103 stations cost taxpayers over $300 million annually with significant overlap in service areas, prompting calls for consolidation to reduce duplication and costs.","keywords":["consolidation","duplication","inefficiency"],"topics":["Public Safety","Data & Databases","Government & Politics"],"technologies":[],"full_text":"Cuyahoga County is home to 53 municipal fire departments with 103 fire stations. This map and accompanying chart show that most stations have potentially overlapping service areas, but each department operates independently of its neighbors. The combined cost to taxpayers is more than $300 million. What's wrong with that picture? Some leaders say that it shows needless and costly duplication and offers a compelling reason to consolidate all of the departments into one.\n\nFire station location and area within one mile of the station.\n(Number refers to station on the chart below.)\n\nHow many stations for three communities?\n\nIn a research paper last year, Cleveland State University urban planning student Archie Tse mapped how much area firefighters could cover from fire stations in Warrensville Heights, Highland Hills and North Randall in the minutes after a 9-1-1 call. The calculation:\n\nWithin four minutes: The stations could cover a lot of the same ground.\n\nWithin eight minutes: Any of the stations could reach\n\nDATA COMPILED AND ANALYZED BY MARTIN STOLZ, PATRICIA O'DONNELL AND THOMAS GAUMER. GRAPHIC BY BILL MARSHALL AND JOEL HURWIT, THE PLAIN DEALER.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-september-october-mapping-hidden-patterns-in-crime-ridden-city","slug":"2004-september-october-mapping-hidden-patterns-in-crime-ridden-city","url":"https://dwillis.github.io/uplink/article/2004-september-october-mapping-hidden-patterns-in-crime-ridden-city.html","year":2004,"month":"September • October","monthNum":9,"headline":"Mapping hidden patterns in crime-ridden city","kicker":"FIRST VENTURE","author_name":"Brad Branan","author_title":"projects reporter","affiliation":"Tucson Citizen","summary":"Reporter Brad Branan used GIS mapping to disprove local official perceptions about crime locations in Tucson, leading to significant law enforcement resource increases.","keywords":["crime mapping","ArcView","police reporting"],"topics":["Crime & Justice","Investigative Methods","Technology"],"technologies":["Excel","ArcView"],"full_text":"I found an investigative story last spring before I even moved to Arizona to work as a projects reporter at the Tucson Citizen. The Tucson metro area had the nation's highest crime rate, Sperling's BestPlaces found in a study of FBI data. Department officials blamed the FBI's Uniform Crime Reports for providing a distorted picture of crime. Using Microsoft Excel, I crunched the FBI numbers to come up with rankings for violent crime. The violent crime rate for the Tucson area was 55th out of 280 metro areas. I needed to find out where the violent crime was concentrated. The skills I learned at an IRE and NICAR mapping mini-Boot Camp this year helped me answer those questions. Go to the Center for Advanced Spatial Technology Web site (www.cast.uark.edu), click on 'Starting the Hunt,' and you'll find a list of GIS sites nationwide. Tucson police provided me with an electronic file of the reports but wouldn't give me exact street addresses. I created a spreadsheet of the crime reports in Excel and then opened the file in ArcView. To find the areas with the greatest number of crimes, I needed to count the reports within smaller geographic areas. I attached the geocoded crime reports to a map of 38 police patrol areas. While I was putting together the map, editors and reporters speculated that the highest violent crime rate would be found on Tucson's south side. The perception proved to be wrong. The area with the highest violent crime rate was on the other side of town – the north side. The north side had long been the site of the city's red-light district, attracting prostitutes and drug dealers. By placing a map of city neighborhood associations over the crime clusters, I knew who to talk to in each community. The ability to downplay violent crime was diminished after the story was published. In June, the Bureau of Alcohol, Tobacco, Firearms and Explosives announced a violent crimes task force for the city. A city official said my story was the reason the task force was formed. Despite a budget crunch, the Tucson City Council approved the police department's request for $4 million to hire 71 more officers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-lobbyist-data-rich-with-trips-food-gifts","slug":"2004-september-october-lobbyist-data-rich-with-trips-food-gifts","url":"https://dwillis.github.io/uplink/article/2004-september-october-lobbyist-data-rich-with-trips-food-gifts.html","year":2004,"month":"September • October","monthNum":9,"headline":"Lobbyist data rich with trips, food, gifts","kicker":"ELECTIONS","author_name":"Kit Wagar","author_title":"Reporter","affiliation":"The Kansas City Star","summary":"Kit Wagar analyzed 20,000 lobbyist gift records to expose how Missouri legislators received millions in gifts, highlighting a trend of using caucuses to hide spending.","keywords":["lobbyists","Missouri","ethics"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel","Access"],"full_text":"The entire trip – the private flight, food and drink after the game, and tickets to see the St. Louis Rams lose on a last-second field goal – was worth $4,136.61, as reported by two lobbyists who picked up the tab. The trip came to light two years later, after The Kansas City Star launched a series of articles about lobbyist spending in the Missouri statehouse. The series is based on the analysis of data about more than 20,000 gifts from January 2001 through July 2003. The value of the gifts totaled $2.4 million. The analysis found the major difference was that new members under term limits tried harder to cover up the gifts they received from lobbyists. Work on the project began in September 2003, when I requested the entire database of lobbyist gifts from the Missouri Ethics Commission. The number of ways a single name could be entered was astounding. Michael Gibbons, the Senate majority leader, was entered as Mike Gibbons, Sen. Michael Gibbons, Senator Gibbons, and more. All those had to be standardized because the computer sees each variation as a different person. I spent parts of three weeks cleaning up the data and putting it into usable formats. I divided the data into four broad areas: individual elected officials; groups of lawmakers; staff, family and friends; and lobbyists. Once the data was cleaned up, I used Excel to perform calculations and exported the information into Access to aggregate the gifts. After Republicans took control of the House in January 2003, lobbyist spending on the Social Services Appropriation Committee fell 90 percent, while spending on tax- and business-related panels rose 141 percent. In 2003, lawmakers expanded the use of caucuses as a way to disguise their sponging off of lobbyists. While spending on individual lawmakers fell 34 percent, spending on private caucuses rose 80 percent. The total numbers give backbone to a project like this. But what riles readers is the nature of individual gifts. The database, however, described most gifts vaguely as food or beverage, travel, lodging or entertainment. Armed with my results, I was able to approach lobbyists and lawmakers from a position of strength. The closest I came involved a Harvard seminar. Of the 10 lobbyists who paid for the legislator to attend, eight were insurance lobbyists. The lawmaker happened to be vice chairman of the House Insurance Committee.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-tapping-into-mysqls-advanced-features","slug":"2004-september-october-tapping-into-mysqls-advanced-features","url":"https://dwillis.github.io/uplink/article/2004-september-october-tapping-into-mysqls-advanced-features.html","year":2004,"month":"September • October","monthNum":9,"headline":"Tapping into MySQL's advanced features","kicker":"OPEN SOURCE","author_name":"Aron Pilhofer","author_title":"","affiliation":"Center for Public Integrity","summary":"Aron Pilhofer explores the advantages of using MySQL for data analysis, highlighting specific features like regular expressions, GROUP_CONCAT, and ROLLUP.","keywords":["MySQL","SQL","database features"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["MySQL","SQL"],"full_text":"I have extolled the virtues of MySQL in Uplink because the program is insanely fast, rock-solid stable and free of cost. But over the past few months, I have discovered this open-source database server has a few more tricks up its sleeve. MySQL supports an extremely advanced set of regular expression functions that make searching databases easier and more precise. I recently investigated this feature when I was trying to find a way to efficiently join two tables of data by conditionally matching all or part of a company name. I wanted a way to search for various combinations of 'Kellogg, Brown and Root' in a single query, and that's exactly what regular expressions allowed me to do. The REGEXP operator can be used in the place of a LIKE clause in SQL, including in joins. Regular expressions offer a great deal more precision over a wildcard search. GROUP_CONCAT is a new function for MySQL as of version 4.1, and I have found it particularly useful for aggregating non-numeric data. Say you have a number of ZIP codes and a number of towns within each ZIP, and you want to know in one query all the towns contained in each ZIP code. MySQL also includes a useful function that allows users to get not only aggregates but also subtotals. This function is a modifier to the GROUP BY clause, called ROLLUP. The records with the NULL value are subtotals, and the final record is a grand total. MySQL includes 95 percent of the features CAR folks use 95 percent of the time, as well as a few features the other ones lack. With version 5.0, MySQL will add stored procedures. Even if it does not become your platform of choice for CAR work, it is worth investigating as an alternative to the high-priced database servers.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-scaping-for-data-the-perl-way","slug":"2004-september-october-scaping-for-data-the-perl-way","url":"https://dwillis.github.io/uplink/article/2004-september-october-scaping-for-data-the-perl-way.html","year":2004,"month":"September • October","monthNum":9,"headline":"Scaping for data the Perl way","kicker":"Tech tip...","author_name":"John Perry","author_title":"","affiliation":"The Oklahoman","summary":"John Perry explains how to use Perl scripts and specific modules like WWW-Mechanize to automate the collection of data from court docket websites.","keywords":["Perl","web scraping","automation"],"topics":["Technology","Investigative Methods"],"technologies":["Perl","WWW-Mechanize","HTML-TokeParser","PHP","Python","Curl"],"full_text":"In the March/April edition of Uplink, Derek Willis wrote about using the free utility Curl to automate the task of downloading pages from Web sites. Sometimes you need a smarter tool that cannot only find Web pages, but also looks at the content and decides what to do with it. That's where Perl – and other scripting languages such as PHP and Python – come to the rescue. Two modules, WWW-Mechanize and HTML-TokeParser, are godsends for Web scrapers. WWW-Mechanize acts like a virtual Web browser and hides the technical intricacies of HTTP requests. HTML-TokeParser is one of several tools Perl provides for pulling useful information from Web pages. Web scraping came in handy during a recent investigation of the Oklahoma County District Court and the court clerk's office. We wanted to look at all Oklahoma County cases in which the bond had been forfeited. The Supreme Court's Web search result pages are simple and seem designed for easy parsing. With a Perl script using the Mechanize module, I grabbed the docket info page for each of the cases and saved them in a directory. I then wrote a script that used the TokeParser module to search for key words in the docket text. The first step in Web scraping is to find the information about the search page that you'll need. To find and save all the cases in our list of bond forfeitures, we had to accomplish these tasks: get the search form page; set the 'db' and 'number' fields; click the search button; save the page; and return to the form. A one-second sleep is necessary because servers protect themselves from denial of service attacks. TokeParser turns a Web page into a stream of tokens. To get to the docket text, we start at a comment, then step down rows and cells. This is a relatively simple example of the capabilities of Perl and its collection of Web scraping tools.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-mapping-candidate-support","slug":"2004-september-october-mapping-candidate-support","url":"https://dwillis.github.io/uplink/article/2004-september-october-mapping-candidate-support.html","year":2004,"month":"September • October","monthNum":9,"headline":"Mapping candidate support","kicker":"ELECTIONS","author_name":"Ron Campbell","author_title":"Reporter","affiliation":"The Orange County Register","summary":"Ron Campbell used FEC data and GIS mapping to visualize the geographic differences in donor support between George W. Bush and John Kerry in the 2004 election.","keywords":["campaign finance","GIS mapping","2004 election"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["ArcView"],"full_text":"While reporting on presidential campaign fundraising, Ron Campbell of The Orange County Register used Federal Election Commission campaign contribution data and ArcView 8.3 to show where President Bush and Democratic Party challenger John Kerry were drawing financial support. Campbell prepared these maps (Bush on top left, Kerry on top right) showing the locations of individual donors who gave the maximum $2,000 contribution to each candidate. The maps show that Bush was successful in attracting contributions from a wide area. Kerry's big financial support, in contrast, has come mostly from metropolitan areas. For both candidates, Campbell queried FEC data of individual contributions to count all $2,000 donations by five-digit ZIP code. He then joined those tables to a ZIP code map in ArcView and created a national map (bottom left) and one for southern California (bottom right).","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-september-october-readmetxt","slug":"2004-september-october-readmetxt","url":"https://dwillis.github.io/uplink/article/2004-september-october-readmetxt.html","year":2004,"month":"September • October","monthNum":9,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"Journalists from top news organizations share expertise on computer-assisted reporting projects through panel sessions, tapes, and tipsheets available from the IRE Annual Conference in Atlanta, covering topics from story selection to data analysis and final publication.","keywords":["journalism","data","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":[],"full_text":"Some of the best journalists in the business recently spoke at the IRE Annual Conference in Atlanta about putting together a computer-assisted reporting project from start to finish. You can purchase tapes of these panels and related tipsheets.\n\nOrder tapes from Sound Images Inc. by providing the Session ID number. Contact Sound Images at 888-649-1118 or service@soundimages.net. A complete list of the available sessions can be found at www.ire.org/store/tapes/atlanta04.pdf.\n\nIRE members can search for the tipsheets online and view them online at www.ire.org/resourcecenter/initial-search-tipsheets.html. You may also order the tipsheets by contacting the IRE Resource Center at 573-882-3364 or rescntr@ire.org and providing the tipsheet number.\n\n\"CAR from start to finish: Choosing stories.\" John Erickson of the Dayton Daily News and Douglas Pardue of The (Charleston, S.C.) Post and Courier. (Session ID No. IRE04-009). Pardue provides advice on looking for cues to good stories in Tipsheet No. 2065. Erickson's Tipsheet No. 2001 also provides ideas for identifying stories while on the beat.\n\n\"CAR from start to finish: Fighting for data.\" Russell Carollo of the Dayton Daily News and Jennifer LaFleur of The Dallas Morning News (Session ID No. IRE04-014). LaFleur's Tipsheet No. 2095 shows how data restrictions have hampered reporting.\n\n\"CAR from start to finish: The data is in – now what?\" Ted Mellnik of The Charlotte Observer and David Milliron of The Atlanta Journal-Constitution. (Session ID No. IRE04-019). Tipsheet No. 2132 offers advice about looking at data before analyzing it.\n\n\"CAR from start to finish: Putting it all together.\" Rose Ciotta of The Philadelphia Inquirer, Shawn McIntosh of The Atlanta Journal-Constitution and George Papajohn of the Chicago Tribune. (Session ID No. IRE04-024). The panelists share their advice in Tipsheet No. 2099.\n\nFor an overview of the lessons learned in two decades of CAR, order \"Blending traditional reporting with data analysis: What we've learned in 20 years\" (Session ID No. IRE04-099) with Sarah Cohen of The Washington Post, John Erickson of the Dayton Daily News, and Deborah Nelson of the Los Angeles Times. Nelson has a checklist to use at the start of a CAR project (Tipsheet No. 2092). Erickson provides guidelines for editing the CAR project (Tipsheet No. 2087). Cohen shares advice for working in teams (Tipsheet No. 2128).\n\nThe IRE Resource Center also has tipsheets from earlier conferences:\n\nMark Nichols of The Indianapolis Star provides ideas on how editors can help guide the CAR story. (Tipsheet No. 2049)\n\nBarbara Vobejda of The Washington Post provides advice for putting together the CAR project. (Tipsheet No. 1945)\n\nDebbie Wolfe of the St. Petersburg Times provides a quick checklist of questions for editors so they can be CAR-savvy through every step of the reporting process. (Tipsheet No. 1828)\n\nGriff Palmer of the San Jose Mercury News offers tips about editing and organizing a large data-heavy project, from getting a strong start and helping it grow to making it relevant for readers. (Tipsheet No. 1557)","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-november-drilling-for-lease-deals","slug":"2004-november-drilling-for-lease-deals","url":"https://dwillis.github.io/uplink/article/2004-november-drilling-for-lease-deals.html","year":2004,"month":"November","monthNum":11,"headline":"Drilling for lease deals","kicker":"ENERGY","author_name":"David Pace","author_title":"Reporter","affiliation":"The Associated Press","summary":"An investigation into oil and gas leasing on federal lands, highlighting the major players and examining if industry rules are followed.","keywords":["oil","gas","leasing"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["SQL Server","Enterprise Manager","Access","World Wide Web"],"full_text":"Earlier this year The Associated Press decided to investigate oil and gas leasing on federal lands, as the Bush administra- tion pushed to open more gov- ernment land to exploration. We wanted to identify the major players, track the leasing pro- cess to find out if the rules were being followed and determine if the administration's friends in the oil and gas industry were benefiting disproportionately from their leasing decisions. We began with the U.S. Bureau of Land Management's LR2000 database, which the agency's field offices use to record and track all actions related to oil and gas leasing on federal lands. The database can be queried from the BLM's Web site (www.blm.gov/lr2000) but using this proved too limited and time- consuming for our purposes. We needed the entire database so we could structure our own queries. The database was easy to ob- tain but difficult to decipher. While journalists have used LR2000 infrequently, oil and gas industry insid- ers rely on it for information that helps them prepare for oil and gas lease auc- tions. As a result, the BLM's Denver office, which administers LR2000, pulls a complete extract of the database each month and makes it available to the public for a nominal fee. We used two extracts in our analysis, one pulled from the database on Feb. 15, and the other on March 15. Each cost about $45 and covered BLM actions since 1982. Each extract consisted of 11 compressed files. The extract totals more than 11 gigabytes and is organized by state, with each state file (the Eastern states are combined into one file) composed of a series of compressed tables. There are 11 case recordation tables in each state file, although most of the essen- tial information can be found in four of them: the case table, listing the essen- tial details about each lease; the ac- tion table, detailing all actions taken by BLM on each lease; the customer table, listing details of each lease owner; and the land table, providing details on the location of each lease. We wanted to look at leasing decisions on all federal lands, so we unzipped the tables and uploaded the data from each state in Microsoft SQL Server, a process that took the better part of a day. We used Enterprise Manager to manage the tables. The largest, which listed all BLM actions on various leases, had more than 9 million records. The next task was cracking the BLM's complicated coding system. The Den- ver office provided a database schema, a spreadsheet with definitions of the hundreds of action codes the agency uses for oil and gas leases and a dic- tionary explaining how each of the data elements is used, both now and his- torically. The documentation helped, but it took many telephone calls to BLM database administrators before the data began to make sense. Finding leaseholders We set out initially to identify those in- dividuals or companies with the most federal acres leased for oil and gas development. That proved to be quite a task, since the BLM database identi- fies unique leaseholders only within each state. And the agency makes no effort to associate legal subsidiaries with their corporate parent, even though the law limiting the number of federal acres that can be leased by a company in any state makes it clear that a corporation's holdings include those of its subsidiaries. To get around this problem, we created a table of the top leaseholders in each state and manually assigned codes to each that would enable us to calculate total hold- ings by the same company and its sub- sidiaries across the country. Using Microsoft Access, we created a master table of companies by assigning the same code to the various names used in the state tables to identify the same company and its subsidiaries. This involved researching Securities and Ex- change Commission filings to verify com- pany subsidiary names. Then we joined that master table to each state table on the company name field and summed up the acres owned by each company using the assigned code field to group by. The same technique was used to calculate each company's total holdings across the country. In state after state, we noticed that one address in Artesia, N.M., kept appear- ing among the top leaseholders. The one thing they all had in common was the name Yates. A little digging uncov- ered the fact that the Yates family of Artesia, through nearly three dozen companies, individuals and trusts op- erating out of the same building in Artesia, controls 2.7 million acres of oil and gas leases on public lands, far more than any other company or indi- vidual, Because leases owned by af- filiated companies or individuals aren't counted together under BLM rules, unless the companies are legal sub- sidiaries, the Yates family has been able to accumulate far more leases than any of the big oil companies. Our analysis found that the Yates fam- ily and a dozen large oil companies - including one formerly headed by Com- merce Secretary Don Evans - now control a quarter of all federal lands leased for oil and gas development outside Alaska, despite the acreage cap law that was supposed to prevent such concentration. Exploration data After identifying the largest lease- holders, we began looking at how the leases were being used. LR2000 proved less useful here, because the BLM tracks most oil and gas explora- tion activity on federal lands in a separate database, the Automated Fluid Minerals Support System, or AFMSS. But LR2000 does include coding that enabled us to differenti- ate between leases that are produc- ing oil and gas and those that aren't. We found that nearly three-fourths of the 40 million acres of federal lands leased for oil and gas development in the lower 48 states aren't produc- ing any oil or gas. Using the Freedom of Information Act, we obtained an extract from the AFMSS database that detailed drilling activities on federal leases for $41. We had negotiated the format of the AFMSS data with BLM by including in the FOIA request a detailed listing of the types of information we were seeking. After the request was filed, we answered questions posed by AFMSS data managers who were writing the queries to respond to the FOIA request. Ultimately, we received four tables in delimited text format - one detailing information on all ap- plications for drilling permits, one on closed inspections, one on wells started or drilled and one on well in- spections. The data covered the same time frame as the LR2000 data, from 1982. The well inspections table was the largest, with 1.9 million records. Using SQL Server to match that data against the LR2000 files of non-pro- ducing leases, we found that 98 per- cent had never had a single well drilled and 97 percent had never had a single application filed for a permit to drill. We joined the tables on the lease identifi- cation number. While industry officials complained that regulatory delays were the main cause for the lack of exploration, the data showed that it took only 61 days on average during 2000 for the BLM to make a final decision on each drill- ing permit application. The average decision time has declined steadily since the Bush administration came into office. In opposing new federal oil and gas leases in sensitive areas, environmen- talists had long argued that industry was not exploring many of the leases it already had. They claimed that these undeveloped leases, available for an- nual rents of $2 an acre or less, were being used more to boost oil com- pany share prices than to increase oil and gas production. Our analysis was the first to use the BLM's own data to show that two-thirds of exist- ing oil and gas leases have never been explored. Industry officials said large inventories of undeveloped leases are both normal and neces- sary to protect their investment in energy exploration. They blamed the lack of exploration on a shortage of drilling equipment and skilled person- nel, regulatory delays and pipeline infrastructure deficiencies. Both the LR2000 and AFMSS data- bases offer a wealth of information for further analysis. AFMSS, for ex- ample, tracks the inspection history of all oil and gas wells on federal lands and details the violations found and penalties paid. It also includes the decision timeline for key stages in the processing of drilling permit ap- plications. LR2000 includes data on environmental reviews required for most federal oil and gas leases, rental payments on leases and ap- peals of BLM lease decisions, among other things. Contact David Pace by e-mail at dpace@ap.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-diving-deep-into-marine-park-problems","slug":"2004-november-diving-deep-into-marine-park-problems","url":"https://dwillis.github.io/uplink/article/2004-november-diving-deep-into-marine-park-problems.html","year":2004,"month":"November","monthNum":11,"headline":"Diving deep into marine park problems","kicker":"ANIMAL HEALTH","author_name":"John Maines","author_title":"reporter","affiliation":"South Florida Sun-Sentinel","summary":"A look into issues faced by marine mammals in captivity at U.S. marine parks, revealing deaths and inadequate care.","keywords":["marine","animals","captivity"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Marine Mammal Inventory Report","MMIR"],"full_text":"Brandie the sea lion died after plung- ing into her 27-foot-deep swimming pool at the Aquarium of the Pacific in Long Beach, Calif. Sadly for Brandie, the pool had been drained for cleaning. Fanny and Brigitte, bottlenose dolphins and star performers at the Montreal aquarium, starved to death after their regular trainers joined a picket line in a strike. The animals refused to take food from anyone but their trainers, and both died after 38 days. The official cause of death for Taffy the sea lion, dead at age 21 at the Morro Bay Aquarium in California, was suc- cinct but not very clinical: \"Love sick.\" In May, the Sun-Sentinel in Fort Lau- derdale gave readers a detailed look into the world of marine mammals in captivity in the five-part series \"Marine Attractions: Below the Surface,\" by se- nior writer Sally Kestin. Although the billion-dollar U.S. marine park industry thrills millions of visitors each year with slick choreographed performances by dolphins, seals, sea lions, whales and other animals, our behind-the-curtain look showed a far less pretty picture of life in captivity. Our primary research tool was the Ma- rine Mammal Inventory Report (MMIR), a database kept since 1972 by the U.S. government's National Marine Fisher- ies Service. The MMIR was designed as a tracking and research tool to fol- low every major event in the life of a marine mammal in captivity - from its birth or capture, to its transfer between marine parks, its death or release into the ocean. Among our findings: More than 3,850 marine mammals have died under hu- man care, many of them young. Of nearly 3,000 whose ages could be de- termined, a quarter died within a year of birth, and half were dead by age 7. fabout 2,400 deaths in which a spe- cific cause is listed, one in five marine mammals died of uniquely human haz- ards or seemingly avoidable causes in- cluding capture shock, stress during transit and poisoning. The tip that led to the series came from a local animal rights activist, who contacted Kestin urging her to look into the death of an orphaned dolphin calf found stranded near Kennedy Space Center and named Rocketman by rescuers. What intrigued Kestin in the federal records about the young dolphin was not what the activist saw - that the dol- phin had died as a result of a bad deci- sion by the federal government. Kestin instead focused on the reasons the government had ruled out several Florida marine attractions that wanted Rocketman. One had a herpes outbreak among its dolphins. Another had a \"history of los- ing calves, maybe due to a viral dis- lease.\" Another had inexperienced staff and problems with veterinary care. The conditions described in those federal documents did not seem to match up with the happy image por- trayed by marine attractions. Florida, the birthplace of the marine park in- dustry, seemed the perfect place to launch an investigation of the industry's origins and history. Antiquated database The creators of the MMIR database some 32 years ago probably imag- ined they were creating a reporting system that by 2004 would be a wealth of information for animal bi- ologists and zoo caretakers. Instead, we found a creaky old database that wasn't even really understood by the agency that keeps it. In fact, it had never even analyzed the information in the MMIR. More than 3,850 marine mammals have died under human care. The fisheries service headquarters in Silver Spring, Md., is based out of the headquarters of the National Oceano- graphic and Atmospheric Administra- tion (NOAA). There, fisheries service officials at first refused to honor Kestin's Freedom of Information Act request seeking the database. They said they wouldn't give it to us because they didn't know how. They could only give us printouts, which translated to more than 800 pages of legal-sized paper for the 9,678 records in the MMIR. When the Sun-Sentinel threatened to sue, fisheries service officials contin- ued to plea that they had no way of getting the database out of their com- puters. It was a DOS program, they said, and they barely knew how to use it beyond simple data entry and re- trieval. They said they had no instruc- tion manual, and the company that designed the database interface in 1995 was no longer in business. All this was hard to believe coming from an agency that's part of NOAA, the same people who run our weather satellites and use supercomputers to study glo- bal climate change. We tried walking the fisheries service folks through the process of using dif- ferent variations of the DOS copy-to- file command. They said they tried it, and it didn't work, and their bosses didn't want them spending time mess- ing with it. We hammered at them for weeks, and it looked like we were headed for court. Rosemary Armao, the investigations editor at the paper, came up with the solution - she proposed sending me to Silver Spring to look at the database and try to retrieve it. The cost was a bargain compared to hiring lawyers: $140 for a round-trip ticket on JetBlue, and $85 for a room at a run-down mo- tel in Silver Spring. We also expected the fisheries service to balk at the idea of a stranger messing with their com- puters. But they liked the idea. When government officials say they can't make the computer work, they're not necessarily lying. For the trip to Maryland, I took a laptop loaded with several hexadecimal edi- tor and disassembler tools such as Hackman, expecting that I'd need to try to crack into PPIMS, the proprietary program that ran the MMIR. But I never had to. It turned out that the PPIMS interface was linked to 104 dBASE files on a fisheries service server, and agency technicians knew where those files were located. The only challenge was to find which ones I really needed to re-create the MMIR reports, which include vital statistics such as name, age, species, birth or capture date, date and cause of death, and the current and previous locations of the animal. I used a three-step process to find the tables I needed. Of the 104 possible choices, many were junk files. These appeared to be remnants of long-for- gotten research: queries that had been run years earlier but were still trapped in the system Weeding out the tables I needed turned out to be simple. First I picked the larg- est ones. Then I selected the ones with names that sounded important. Finally, I asked the fisheries service people if they recognized any of the table names from their use of PPIMS. It only took three hours to identify the correct tables (datashet.dbf, inventory.dbf, species.dbf, and phf.dbf), import them into Microsoft Access and join them into a working copy of the MMIR. Data gaps For anyone who wants to take a shot at the MMIR, a warning: The tattered old database is filled with problems that make analysis difficult. There's missing data for many animals (the entries go back to the 1950s, appar- ently typed in from old paper records), but there are also many surprises. For example, of the 7,121 animals in the database, more than 900 vanish with- out any explanation. Usually, this hap- pens in older records from the 1970s and 80s, and occurs when the ani- mals are transferred out of a facility. They are never shown as arriving anywhere. They're just gone. Another headache - and this applies to any research on zoo animals - is the issue of reporting how long ani- mals live in captivity. Animal rights activists often look at only the aver- age age at death of captive-born ani- mals, which can be very low because of the high mortality rate in the first year of life. For example, the MMIR tells us that the average age at death for bottlenose dolphins that are born in captivity is only 2.5 years. But if the analysis is done a different way, look- ing at only animals that survive at least a year after birth, the average age at death climbs to 6.3 years. The marine park industry cringes at either formula. They argue that aver- age-age-at-death calculations are un- fair because they do not include the age of animals that are now living. They prefer the use of \"life tables,\" similar to actuarial tables used by in- surance companies to predict human life expectancy. The American Zoo and Aquarium Association's life-table analysis provided to us for the series said that its own analysis of the MMIR shows that bottlenose dolphins now in captivity have a life expectancy of 20 years. Of the 875 dolphins that have died in captivity with a known birth date, only 17 percent have lived to age 20 and beyond. And 20 years is a long way from the 2.5 years and 6.3 years we found in the MMIR, and nothing close to the life expectancies reported by some marine parks at shows and on their Web sites - up to 30, 37 and 45 years. The association says it will take another 30 years of data collection before they have an accurate picture of how long bottle- nose dolphins live in captivity. Despite checking with experts around the country, the best we could do for our series was point out the different sets of numbers, which unfortunately didn't clarify the lifespan question. The MMIR also suffers from under- reporting, particularly in the past de- cade. In 1994 at the urging of the in- dustry, the marine fisheries service was stripped of enforcement power over marine parks. The job was turned over to the U.S. Department of Agriculture, which keeps inspec- tion records of facilities only on pa- per and in Adobe Portable Document Files, and only for three years before they're tossed. In the years before the 1994 agency switch, the MMIR shows that 30 per- cent of marine mammals born in cap- tivity died within a year. But after 1994, the rate drops to 15 percent. This remarkable improvement is not from advances in veterinary care. Marine parks simply stopped report- ing stillborn deaths or the deaths of newborns to the government. To dem- onstrate this, we used LexisNexis to find newspaper accounts of the deaths of newborn animals, which are often publicized because the birth of a calf or pup is a big public relations event. Many deaths were not reported in MMIR. In a few hours, library re- searchers found 26 deaths in recent years that had never made it into the database. And the MMIR might not have much of a future. In Washington, a bill passed by the House Committee on Resources in the fall of 2003 and an amendment introduced in April would cut back on the parks' reporting responsibilities. Opponents fear it would lead to the elimination of the MMIR, and could take away the public's ability to find out what happens to their sea stars. The MMIR has data on every marine park in the country, so it's possible to do analysis on local parks. The con- tact person at the marine fisheries service is Jennifer Skidmore, at jennifer.skidmore@noaa.gov or 301- 713-2289. Contact John Maines by e-mail at jmaines@sun-sentinel.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-data-helps-probe-death-on-the-tracks","slug":"2004-november-data-helps-probe-death-on-the-tracks","url":"https://dwillis.github.io/uplink/article/2004-november-data-helps-probe-death-on-the-tracks.html","year":2004,"month":"November","monthNum":11,"headline":"Data helps probe death on the tracks","kicker":"SPOTLIGHT: ROADS AND RAILS","author_name":"Steve Orr","author_title":"Reporter","affiliation":"Democrat and Chronicle","summary":"An examination of rail-crossing safety following a fatal train accident, using data to reveal maintenance lapses.","keywords":["rail","safety","accidents"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["Excel","Access","Internet","World Wide Web"],"full_text":"The Democrat and Chronicle's cover- age of rail-crossing safety began ear- lier this year after a retired couple were struck and killed by a CSX freight train as they drove onto a crossing in a Roch- ester suburb. Within a few days, it emerged that CSX personnel had purposely deactivated circuitry that controls the gates and warning lights at that crossing after re- peated complaints that the gates had become stuck in the down position. The circuit was off-line for a full week without repairs made. During that time, crews were given orders to stop their trains before entering the crossing, dis- mount, make sure car traffic had stopped and then proceed. On the morning of the accident, the crew of a 110-car freight train failed to do SO. I was among several staff members asked to follow the story. As I did, I made frequent use of railroad safety data that is available online from the Federal Railroad Administration. While the data has the usual accu- racy and timeliness limitations, it is as useful as any government cache I've seen for doing quick-and-dirty research. It's much like the Federal Aviation Ad- ministration airplane accident and maintenance data that many journal- readme.txt The IRE Resource Center offers copies of stories by journalists who have use computer-assisted reporting for transportation sto- ries. Here are some examples: A Kansas City Star investigation found that the problem of fatigued driving among truck drivers is per- vasive. The Star analyzed truck- ing databases for the series. (Story No. 11812). An investigation by The (Cedar Rapids, Iowa) Gazette found semi-trailer trucks accounted for nearly half of all fatal accidents on Interstate 80 in Iowa. The Gazette analyzed the Fatality Analysis Reporting System (FARS) for the story. (No. 21020). To order these stories and others, contact the Resource Center at 573-882-3364 or rescntr@ire.org and provide the story number. ists have used. The FRA online data can be tapped quickly to obtain back- ground in the aftermath of a train de- railment or highway crossing accident. It also can be employed to craft follow- up stories or to take a more thorough look at railroad safety. The data is available through the FRA Office of Safety Analysis Web site, http://safetydata.fra.dot.gov. In contrast to the agency's main Web site (www.fra.dot.gov), which is largely be- reft of content useful to journalists, this site has nine separate sections rich with facts. I don't profess to be an expert on this data, but here's some of what you can find: information on highway crossing accidents, as well as the crossings themselves; train accidents (derail- ments, collisions of trains, etc.); inju- ries and deaths of employees and non- employees; and FRA inspections of equipment and crossings. In our case, we-used the data to find other accidents on the rail line where the couple died, to look at the results of federal inspections of crossings in New York state and to pull out various nuggets of information about crossing accidents nationwide, Search and download Using the online query forms, you can pull off records of accidents or derailments in a few minutes' time. You can search by railroad, by state and county, by particular crossing and so on. Some sections have 30 years' worth of data, though you'll find in some cases that you have to search a month at a time, which is highly annoying. There are literally dozens of different searches you can perform. The re- sults from these on-the-fly queries come mainly in the form of text docu- ments or pre-made text tables and charts. It's easy enough to paste such a table into Microsoft Excel or Ac- cess, but the text documents require a great deal of manipulation or data entry to get them into a database. You can get some of this information by downloading entire databases, (see below). The summary reports of individual accidents that these queries produce are vague and at least occasionally inaccurate. The online summary of the Feb. 3 accident in suburban Rochester, for instance, continues to say that the \"crossing was protected by gates, cantilever flashing lights, bells (audible),\" but doesn't note that those devices failed to activate in a timely fashion. You can obtain an Adobe Portable Document File of the crossing-acci- dent reports filed by the railroad by using the \"Query and Generate Crossing Accident Reports\" link. Hav- ing the ID number of the crossing in hand will help. You'll find that these more detailed reports also can be in- accurate. This summer, for example, I wrote about a 2001 fatal crossing accident that occurred in the Roch- ester area. The railroad report I down- loaded indicated that the crossing sig- nals worked as intended. It turned out that wasn't true, as demonstrated rather vividly by a security-camera video of the accident scene around which we built our story. The same information that is included in the PDF accident reports is available in a full database that can be down- loaded from the FRA Safety data Web site. Other databases of train accidents, casualties and highway crossings also can be obtained. The databases are available in six for- mats, including Access, Excel and comma-delimited text. The files come compressed or uncompressed. I used the Access 2000 format, and the download was flawless. I chose the uncompressed version; it took about a minute to get the 1.5 MB da- tabase onto my machine, using a high-speed Internet connection. That 1.5 MB file, though, represented a year of crossing accidents nation- wide. That's the primary problem you can download only one year at a time. I spent about an hour download- ing and merging the tables to as- semble a 22-year collection of high- way-crossing accident data for New York state. That compilation has about 1,400 records in it; five years of crossing accidents nationwide has about 16,000. Each record contains 100 fields of data about a given ac- cident, though there is a fair amount of repetition. I worked mostly with the crossing-ac- cident database, and found it helpful for several stories. You can sort not only for railroad or location or time frame, but also for the reported cir- cumstances of the accident: car hits train or train hits car; type of crossing equipment; gender of driver; reported speed of train; and so on. I used the crossing-accident data in several ways. I separated all the cross- ing accidents on the 20-mile-long rail spur where the Feb. 3 accident OC- curred, which was useful for a story about the claim that the spur had too many crossings. Finding problems For a story on CSX's maintenance record, I used data about FRA inspec- tions of freight railroads in New York state. I found CSX had been cited for crossing defects during inspections much more often than its competitors. The inspection data isn't in a downloadable database so I did some on-the-fly queries, and pasted the re- sults into a spreadsheet. I also pulled data on crossing accidents in New York, and reported that CSX had a higher number of accidents than other freight railroads. For a longer story about CSX's cross- ing maintenance practices and how they contributed to the Feb. 3 double- fatal crash, I used the database to find how many accidents nationwide had occurred at crossings protected by gates and lights, and in how many of those accidents there had been alle- gations that the crossing equipment malfunctioned. For a two-part story on the 2001 fa- tal accident, I was able to pull data showing how many accidents OC- curred when motor vehicles drove into the side of trains, and how many occurred at night. Both factors played a role in the accident I was chroni- cling, which occurred when the gates and lights didn't deploy promptly and the young motorist plowed into the side of a locomotive, possibly with- out ever seeing it. There are literally dozens of different searches you can perform. Understanding data Before I worked with the data, I had to understand it by looking over the data- base file structure document, available for download from the Web site's down- load section. As I went through that file structure, I found myself a bit confused on sev- eral key points, and it wasn't until I also downloaded a second document that they made sense. This document is a blank copy of the agency's \"High- way-Rail Grade Crossing Accident/In- cident Report\" form, which can be found at the very bottom of the main query page under FRA Forms. The blank form contains some of the codes used in the database, which are not replicated on the file struc- ture sheets. One example is the type of crossing equipment. Another and this is the one I was really after - is whether that equipment was working as intended when the acci- dent occurred. So how do you find this in the data- base? I'm getting a bit arcane, but it took me days to puzzle out on my own, and it might be worth filing it away just in case you need it. In the main crossing-accident data- base are two fields entitled \"Signal\" and \"Sigwarnx.\" If the crossing in question has automated equipment such as gates and lights, then the \"Signal\" field should include a digit from 1 to 7. But what do those digits represent? If you consult the instruction page that comes with the accident/incident re- porting form, you'll discover that those digits represent the reporting railroad's statement about whether the gates and lights worked properly. \"1\" indicates everything worked well; anything else suggests an alleged or confirmed problem. Go one layer deeper and you'll find that if the railroad reported \"5,\" \"6\" or \"7,\" which stand for \"confirmed\" sig- nal problems, they also must fill out the \"Sigwarnx\" field. This field, which uses letters instead of numbers, gives 19 possible explanations for the fail- ure of the crossing equipment. The explanations include things such as \"devices down for repair,\" \"vandalism,\" or a train going too fast for the cir- cuitry to keep up. (Here's a story tip for someone: Look in the database and see how often \"insulated rail ve- hicle\" is the explanation for a cross- ing-signal failure. It turns out that some railroad maintenance vehicles are made in such a way that they won't activate gates and lights.) Admittedly, crossing activation prob- lems are rare - or, if you believe the rail critics I spoke with, railroads only report such problems rarely. But if you're writing about a crossing signal failure as I did in two local cases, it is immensely helpful to be able to pull out other accidents that might have occurred under similar cir- cumstances. Contact Steve Orr by e-mail at sorr@democratandchronicle.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-upcoming-training","slug":"2004-november-upcoming-training","url":"https://dwillis.github.io/uplink/article/2004-november-upcoming-training.html","year":2004,"month":"November","monthNum":11,"headline":"Upcoming training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer intensive computer-assisted reporting (CAR) boot camps and training sessions in Columbia, Missouri throughout 2005, covering data analysis, mapping, and newsroom integration. Participants receive follow-up support to implement their skills at their news organizations.","keywords":["training","journalism","data"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["GIS"],"full_text":"Looking for intensive computer-assisted reporting training in the coming year?\n\nThere are four weeklong Boot Camps in Columbia, Mo., for journalists who want to learn how to acquire electronic information, use spreadsheets and databases to analyze the information, and to translate that information into high impact stories. IRE and NICAR provide follow-up help after participants return to their news organizations. The 2005 Boot Camps are Jan. 9-14, March 20-25, May 15-20 and Aug. 7-12.\n\nJournalists interested in learning how to map data for news stories can take advantage of mini-Boot Camps in using geographic information systems (GIS). The training sessions are Jan. 14-16 and Aug. 19-21 in Columbia.\n\nNews managers interested in learning to better integrate CAR into their newsrooms can attend the April 8-10 mini-Boot Camp in Columbia.\n\nFor complete information about IRE and NICAR training visit www.ire.org/training/","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-november-scrape-away","slug":"2004-november-scrape-away","url":"https://dwillis.github.io/uplink/article/2004-november-scrape-away.html","year":2004,"month":"November","monthNum":11,"headline":"Scrape away","kicker":"INSIDE NICAR","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR have launched the Scraping Project to help journalists efficiently extract data from online public databases by sharing and collaborating on web scraping scripts that can be modified and reused.","keywords":["scraping","data","journalists"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["World Wide Web"],"full_text":"IRE and NICAR are seeking contributors to the Scraping Project, which was formed to address the concerns of jour- nalists who need to quickly pro- cess public data on the Web. An increasing number of public da- tabases are available online, but they are often stored on multiple pages accessible only by lengthy search strings. This project is an effort to help jour- nalists share scripts that \"scrape\" data from Web sites for use in spreadsheet or database programs. To begin the project, IRE and NICAR are collecting any scraping scripts, explanatory notes and databases, either simple or complex. The Scrap- ing Project will make these scripts available for journalists to modify and use, provided they also share their modified scripts and documentation. For more information about the scraping project see the October- November 2004 issue of Uplink. If you would like to submit a script, contact Jeff Porter at jeff@ire.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-changes-for-the-database-library","slug":"2004-november-changes-for-the-database-library","url":"https://dwillis.github.io/uplink/article/2004-november-changes-for-the-database-library.html","year":2004,"month":"November","monthNum":11,"headline":"Changes for the Database Library","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"NICAR and Missouri School of Journalism","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library introduces its new staff members, including several master's students and visiting journalists specializing in investigative reporting, data analysis, and digital journalism. The organization also highlights recent departures of staff members who have moved on to positions at major news organizations.","keywords":["staff","journalism","database"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"The new staff at the IRE and NICAR Database Library is settling in, so it's a good time to introduce our talented roster of data analysts. Brian Hamman is a first-year master's student focusing on new media and investigative journalism. He spent the previous two years at Washington Uni- versity in St. Louis working on digital archives projects. Catherine Rentz Pernot is a first-year master's student specializing in news reporting. She's currently reporting for the Columbia Missourian. Before joining IRE and NICAR, she analyzed business and government finances in Houston. Megan Clarke returns to the Database Library as a second-year master's stu- dent. She also reports for KOMU-Co- lumbia and produces special projects for Missouri Public Radio and KMOX radio in St. Louis. Byung C. (Peter) Lee is a visiting jour- nalist and scholar from South Korea, where he is working on a doctorate in journalism. He also is an award-win- ning reporter for the Busan Daily News in Busan and is interested in covering crime, poverty and the environment. Matt Wynn is a junior and worked at a weekly newspaper in Omaha, where he covered municipal government and spot news. Last summer he worked at the Omaha World-Herald, where he worked on special sections, spot news and investigative stories. We've also had some departures from the Database Library. Master's student Christina Caron left at the end of the summer to take the Kaplan Fellowship, working with ABC News in Washington, D.C. Jaimi Dowdell recently received her master's degree and, after filling in as acting director of the Resource Center, is continuing her work for IRE. Beth Kopine has joined IRE as director of the Resource Center. Dowdell and recent master's recipient Aaron Kessler shared a byline with Mark Morris of The Kansas City Star for an article in the newspaper about a Colum- bia-based Islamic charity whose assets have been frozen by the U.S. govern- ment. Dowdell used social network analysis software to visualize the government's allegations of how the charity helped support Osama bin Laden and other terrorists. Kessler is a former IRE Resource Center staff member. Holly Hacker received her master's de- gree and now is the higher education reporter for The Dallas Morning News. Andrea Lorenz, departs the Database Library in January for The Kansas City Star, where she will work on her master's degree professional project. Tim Ragones also received a master's degree and is reporting for WSAZ- Charleston, W. Va. It's great to work with such enthusias- tic and bright journalists. We hope you can get to know the new staff as they assist you. Our Annual Computer-Assisted Report- ing Conference is coming up soon (March 16-20 in Hollywood, Calif.) and we'd like your ideas for panels, panelists and dem- onstrations. So please send any ideas to confideas@ire.org. Remember to in- clude your contact information. Contact David Herzog by e-mail at dherzog@nicar.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-reservation-deaths-uncovered-with-data","slug":"2004-november-reservation-deaths-uncovered-with-data","url":"https://dwillis.github.io/uplink/article/2004-november-reservation-deaths-uncovered-with-data.html","year":2004,"month":"November","monthNum":11,"headline":"Reservation deaths uncovered with data","kicker":"CHILDREN","author_name":"Brent Walth","author_title":"Reporter","affiliation":"The Oregonian","summary":"An investigation into the high death rates of children at the Warm Springs Reservation in Oregon, using state databases.","keywords":["deaths","reservation","children"],"topics":["Investigative Methods","Public Safety","Demographics & Census"],"technologies":[],"full_text":"In Oregon, there is a place where chil- dren die at more than three times the statewide average. A place where ba- sic protections, such as seat belt use for kids, are ignored. A place where children die anonymously in a secre- tive welfare system. A state deaths database helped us find it. But it took more than a year of report- ing by our team, including reporters Kim Christensen and Julie Sullivan and photographer Rob Finch, to under- stand why the deaths continued with no public outcry, and why public offi- cials were not being held accountable. The place is the Warm Springs Reser- vation, where the U.S. government forced three tribes to relocate in the 19th cen- tury. Today, about 3,800 people live on the 1,000 square mile reservation in the central Oregon desert. About half the residents are age 19 or under. As with many reservations, poverty is a big problem, and Warm Springs offi- cials say their community struggles with the loss of history and culture and with a continuing tide of fractured families, alcohol abuse and domestic violence. We soon found, however, that many people in Warm Springs did not see their circumstances as hopeless. Instead, they shared stories of tragedy and loss to show many child deaths could have been prevented and that tribal leaders had too often turned a blind eye to the problem. By operating successful businesses, the Warm Springs tribes have been more pros- perous than tribes at other reservations. Many people pointed us to reforms tribal leaders have declined to take, even though they have proven to save children's lives elsewhere. What's more, we found the secrecy surrounding tribal government includ- ing a censored newspaper - meant that those leaders had never been held accountable. Here is what we found: Traffic accidents killed kids more than any other cause, but only four of 10 Warm Springs infants and toddlers are properly belted in. Tribal police sporadi- cally enforced seat belt and child re- straint laws and ranked the problem as a low priority. Five children had died violently af- ter the tribal welfare system allowed them to remain with dangerous adults. The cases included two un- solved child killings that Warm Springs residents had not been told about until our series appeared. The tribes' adolescent alcohol and drug programs fail at twice the rate of other programs in Oregon, and tribal leaders have ignored counselors' pleas for improvements that have been proven to work on other reservations. The tribal council has cut programs that help children even as it has con- tinued to spend $147,000 annually on a luxury skybox at Portland's Rose Garden sports arena. The response to the series, \"A Place Where Children Die,\" was swift. (You can read the series online at www.oregonlive.com/ special/warmsprings). Shaken by our findings, tribal leaders launched a \"top to bottom\" review of tribal programs to reduce child death rates and are examining every child death case, something they had not done before. They also ordered police to follow a zero- tolerance policy for anyone allowing a child to ride unsecured in a car. Meanwhile, the U.S. Indian Health Ser- vice took the extraordinary step of form- ing its own watchdog team to make sure tribal child-welfare officials acted quickly on neglect and abuse complaints. The story began in the summer of 2000, when 4-year-old Andres Saragos died of heat exhaustion after his tribal court-appointed guardian locked him in a car for nine hours under a pounding July sun. The Oregonian's investiga- tion at the time, written by Courtenay Thompson, found the Warm Springs child welfare authorities sometimes ig- nored reports of neglect and abuse. Two years later, the newspaper de- cided to go back to see if tribal lead- ers' promises to improve child welfare had been fulfilled. At the outset, I wanted to find out just how deadly a place Warm Springs was for kids. The task seemed impossible. Tribal records are confidential and not sub- ject to any state or federal disclosure laws. That meant the traditional report- ing avenues, such as court files, po- lice reports and budget records, were closed to us. We soon found this veil of secrecy isn't just for outsiders; the se- crecy works against members of the Warm Springs tribes, too. Even parents who had lost kids to violent deaths had been kept in the dark by tribal police. After a lot of phone calls, I discovered the state of Oregon kept a database of every death in the state going back more than 20 years. State officials had always refused the newspaper's requests for the data, citing confidentiality concerns. I eventually won access to the data - about 700,000 records of Oregon deaths over a 20-year period - by ne- gotiating a way to obscure the exact dates of death, allowing the decedents to remain anonymous. In turn we got key details, such as how old they were, how they died and where they lived, including county and ZIP code. The ZIP code was key: The entire Warm Springs reservation has a unique ZIP. So we could calculate with precision, how many Warm Springs children had died and what had killed them. But were the death rates high? I used Microsoft Access and Microsoft Excel to calculate and compare mor- tality rates for Warm Springs kids ages 19 and under to those across Oregon. I did it the same way state and federal officials set mortality rates: by calcu- lating the number of deaths per 100,000 residents. (For exact methods on calculating mor- tality rates and making sure they are sta- tistically sound, I used formulas pub- lished each year by the state of Oregon's Center for Health Statistics and Vital Records. You can find a straightforward explanation of how the formulas work in the appendix of the center's annual re- ports, located at www.dhs.state.or.us/ publichealth/chs/vol2.cfm.) To calculate a death rate, you need the number of deaths within a certain age group, and population for the same age group, for a specific period of time. The database gave us the number of deaths. I obtained population data from the U.S. Census Bureau's American FactFinder (http://factfinder.census.gov) Web site. Using Excel, I lined up the number of Oregon children's deaths by ZIP codes and counties with the populations. When I ran the numbers, no place in Oregon had a child death rate that came close to that in Warm Springs. We had to take special care, because the U.S. Census Bureau typically undercounts residents of Indian reservations. Warm Springs of- ficials said their own counts of the res- ervation population showed that the U.S. Census usually missed between 10 percent and 15 percent of residents. To be safe, we increased the Census tally of Warm Springs by 25 percent twice the estimated undercount. This adjustment lowered the mortality rate for Warm Springs. But it didn't matter: Even with this conservative population estimate, we found that Warm Springs children die at a rate 3.4 times those in Oregon. We could now call Warm Springs the deadliest place for kids in Oregon. But my reporting had shown that, tragi- cally, mortality rates for Native Ameri- can children are higher than those for the United States as a whole. I worked with Indian Health Service statisticians for a year and found Warm Springs' child mortality rates were nearly double those for Native Americans nationwide. I ran my calculations past state vital records officials and IHS statisticians to make sure they were statistically sound. The numbers checked out. To understand why child death rates were so high at Warm Springs, we catalogued 20 years' worth of the tribal newspaper, the Spilyay Tymoo, for key players, major events and deaths. Again, I used Excel to organize the in- formation by name, age of death, cause of death and other key informa- tion we gleaned from obituaries. We also collected documents wherever we could: the tribes' grant proposals to; state and federal agencies; state alco- hol treatment reports on Warm Springs; and state police highway death investi- gations. We obtained the approval of parents to release children's medical records, which often included police and autopsy reports. The most important reporting we did was spending time with the people of Warm Springs. We conducted more than 300 interviews with people in Warm Springs, including teachers, physicians, fire and rescue crews, po- lice officers, and tribal judges and pros- ecutors. Rather than finding a closed society, we found people eager to talk about their lives and their struggles to keep kids safe. Many shared stories of their lives and tragedies faced by their families. Warm Springs residents allowed us to observe elk hunts, sacred ceremonies, school activities and fu- nerals, and even take part in a tradi- tional sweat lodge ritual. We found there was no substitute for the time we devoted: The people at Warm Springs opened up when they saw we wanted to shed light on the solutions that had been ignored by tribal leaders. Their stories - not the data - became the heart of our series. Contact Brent Walth by e-mail at brentwalth@news.oregonian.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-extra-extra","slug":"2004-november-extra-extra","url":"https://dwillis.github.io/uplink/article/2004-november-extra-extra.html","year":2004,"month":"November","monthNum":11,"headline":"Extra! Extra!","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"Extra! Extra! is an online resource from IRE that catalogs investigative and computer-assisted reporting stories, allowing journalists to search by category and discover relevant reporting techniques for their beats.","keywords":["investigative","journalism","database"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":["Internet","World Wide Web"],"full_text":"For some inspiration, check out Extra! Extra!, an online guide to investigative and computer-as- sisted reporting stories. The Web page, at www.ire.org/extraextra, carries summaries of recent in- vestigative pieces and links to the stories. Journalists can search for stories by category, from health and homeland security and politics to nonprofits. With nearly 1,000 entries, it's more than likely that you will find stories related to your beat or whatever topic you're cur- rently covering. The story listing is just one more effort to help IRE members stay up to date on the latest in inves- tigative stories and techniques. Readers are encouraged to sub- mit stories and links by sending them to extraextra@ire.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-debunking-some-myths-about-speeders","slug":"2004-november-debunking-some-myths-about-speeders","url":"https://dwillis.github.io/uplink/article/2004-november-debunking-some-myths-about-speeders.html","year":2004,"month":"November","monthNum":11,"headline":"Debunking some myths about speeders","kicker":"FIRST VENTURE","author_name":"Holly Whisenhunt Stephen","author_title":"Investigative Producer","affiliation":"WOAI-San Antonio","summary":"A journalist describes their first computer-assisted reporting project analyzing 1.6 million municipal court records from Bexar County, Texas, focusing on data import procedures, database management in Microsoft Access, and isolating speeding violations from 400,000+ records.","keywords":["data","database","analysis"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["CD-ROM","Access","SQL"],"full_text":"My first computer-assisted reporting story started shortly after I arrived at WOAI last year. The data involved more than 1.6 million municipal court records from Bexar County, Texas. The investigative producer I was re- placing had already put the wheels in motion to get the data. It took some time to convince the court system what information was public. I'm told there was some grappling over ob- taining the information, but it wasn't the biggest fight either. I picked up the data, which cost a couple hundred dollars, a few weeks after I started the new job. During an IRE and NICAR Boot Camp, 1 learned to always make cop- ies and never work with the original data. So, after loading the informa- tion, I put the disks in a safe place where they wouldn't get back into the mix. We received 10 years of data on a CD-ROM that had 12 comma-de- limited text tables. I imported the data into Microsoft Access, making sure to bring all the fields in as text, and later changed the number fields in Access table design view. When I imported the data, I selected the advanced options and created specifications to control the importing of the data. This was very helpful and saved lots of time when it came to im- porting the remaining tables, which had the same fields and same data types. The one point I cannot stress enough is to make sure the specifications are absolutely correct, or the resulting data table will be a mess, and you'll have to clean it up or repeat the import. After importing each table, I checked to make sure all records were imported properly. Then I performed an append query to create tables with more than one year of data and joined all other tables. Before that I tallied the total number of records in the individual tables, so I could check whether the append query was successful. Each record represented a case in munici- pal court. The municipal court data table con- tained 85 fields. The complaint descrip- tion field was very helpful because it told us what the actual violation was. Other helpful fields included the viola- tion date, time and location. The data also contained the officer's name and badge number, which made it easy to find out how many tickets an officer is- sued. We analyzed five years worth of data. The information contained sev- eral violations ranging from disorderly conduct to wasting water. We chose to focus on speeding, and ended up with more than 400,000 records that I merged into one table. To separate the information, I queried the complaint description field. Because there are several different variations for speeding, I used the wildcard in my Structured Query Language (SQL) WHERE statement to make sure all speeding violations were included in the query. After isolating the information, I performed a make-table query, so I would be working with a smaller table containing the data I needed. This first project taught me the impor- tant steps of cleaning data, so it can be analyzed more easily. I performed several counts including how many tick- ets were written for speeding and speeding in a school zone. We wanted to find out where the most tickets were issued in the city. To accomplish this I needed the street address of the vio- lation in one field, which was originally spread over four. I concatenated the contents of the four fields into a new one using SQL. I must admit this is one of my favor- ite tricks because it seems like a huge task to accomplish, but with concatenating it's easy. Before doing this I made a new field called speeding fulladd to hold the new ad- dresses. Here is the SQL I used: UPDATE speeding1 SET speeding1.fulladd = trim ([speeding1] [new vioblk1] & [speeding1] - [new vioblk2] & [speeding1] [vio street1] & & [speeding1] [vio street2]) ; After concatenating the addresses, I was able to get a better look at traf- fic-ticket patterns. I ran queries to group by address and count the num- ber of times each appeared in the data. I chose not to change the names of all streets and highways. For instance, Highway 281 @ Bitters Road was also listed as Hwy 281 @ Bitters Rd. I decided that standardiz- ing all of the addresses would be too time consuming. Instead, I chose to use Access and an Excel spreadsheet to get an estimate of how many tickets were issued at specific locations. I queried the data in Access to show the streets and the count of accidents for each. I ordered the results in descending order so the streets with the highest number of ac- cidents appeared at the top. Then I noted the top streets and did another query to get the records for those streets. I put the records for each street into its own Excel worksheet and worked with the data there. Although it was not scientific, we were able to estimate ticket locations and counts. We chose to publish the speeding sites where more than 1,000 tickets were issued in the San Antonio area. Another thing we wanted to know was the average amount of time that passed from when a ticket was issued until it was resolved. I created a new field to hold the calculated times and ran an update query that inserted the number of days, as calculated using the violation and adjudication dates. Here is the SQL: UPDATE speeding1 SET ADJTIME = INT [NEWADJ DATE]-[NEW VIO DATE]) /1) WHERE [NEW VIO DATE] IS NOT NULL OR [NEWADJ DATE] IS NOT NULL; We were able to look at the data and determine some of the common \"myths\" linked with speeding tickets for instance, that red vehicles are tick- eted most often - weren't true in San Antonio. Other fun facts we were able to find in this data: The officer who issued the most tick- ets and what time of day most tickets were issued. The prime time was 3 pm. A judge stated that's when people are picking up kids from school and maybe shift workers are leaving for home. A monthly breakdown when most tick- ets are issued. We found the dates with the most tickets were at the beginning of the month (the 3rd, 2nd, and 6th). The reason this was surprising is most driv- ers talk about and assume that offic- ers strive to meet quotas at the end of the month and write more tickets then. The average age of drivers who get tickets: 31. The average speed over the posted limit that results in a ticket: 17 mph. The amount of tickets dismissed be- cause officers did not show up for court or did not remember the traffic stop: 3 percent of all speeding tickets. The color vehicle received most tick- ets. We ran into some difficulties with this because not all records indicated the color of the vehicle. The Texas De- partment of Transportation does not record vehicle color on car titles be- cause those colors can change. So we could only report on those who records included the data. White vehicles were the most-often ticketed. Overall this was a great database to work on after Boot Camp. Many of the lessons I learned during the Boot Camp helped me to clean and analyze the data. The best advice was to be me- thodical and not overwhelmed by the volume of data. My advice to others is take your time, concentrate and under- stand what you are trying to accomplish before you start trudging through steps that might not be helpful. As far as SQL goes, I now know it is impossible to learn everything you think you should know at Boot Camp. [ started compiling a list of SQL state- ments and what they can accomplish. I find this to be very helpful when working on different data that has similar challenges. Another great re- source is the NICAR-L listserv, where journalists doing CAR assist each other. Whenever I've had any ques- tions (especially on deadline) I just post it online and usually receive sev- eral answers. Contact Holly Whisenhunt Stephen by e- mail at HollyWhisenhunt@woai.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-unmasking-murder-myths","slug":"2004-november-unmasking-murder-myths","url":"https://dwillis.github.io/uplink/article/2004-november-unmasking-murder-myths.html","year":2004,"month":"November","monthNum":11,"headline":"Unmasking murder myths","kicker":"MAPPING","author_name":"Mark Houser","author_title":"Reporter","affiliation":"Tribune-Review (Pittsburgh, Pa.)","summary":"A Pittsburgh journalist investigates the city's record homicide rate in 2003, discovering that murders are concentrated in specific neighborhoods and disproportionately involve young Black victims, and uses data mapping to present these findings clearly without reinforcing stereotypes.","keywords":["homicide","Pittsburgh","mapping"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["Excel","dBase","ArcView"],"full_text":"When Peguese, 26, was shot in his car in Pittsburgh three days after Christmas, his death helped set a record. By the time the coroner was done counting, Allegheny County had 125 homicides in 2003. Pittsburgh's murder rate of 21 per 100,000 residents last year was on par with Miami and Chi- cago. Suburbanites lamented they could no longer go downtown at night for a play or a concert with- out risking their lives. I was assigned to do an investiga- tive report on the rising murder rate, and I was skeptical about the conventional wisdom. From my shifts on the weekend rotation, I knew there was a good chance on any given Saturday that I would be chasing down a murder in the projects. Like the Peguese case, these usually involved a young black man shooting another young black man. Often, there were no witnesses. The incident would get a couple paragraphs the next day, and we moved on. Although only 11 percent of Al- legheny County residents are black, they accounted for three- quarters of homicide victims last year. Statistics also show that black victims were usually killed by blacks, while whites were killed by whites. Fewer than one in 10 ho- micides involved a white person kill- ing a black person, or vice versa. When I looked at homicides going back to 1990, I found another trend. Both in the early '90s and now, when the rate of killings rose, it was pushed by an upsurge of black vic- tims. Police say this phenomenon tracks with the influx of first crack cocaine and now heroin. The casu- alty tally for whites hovered near 30 for each year through the decade, but for blacks it fluctuated from 36 in 1991 to 91 in 2003. If we were going to run a big story about murder, then we needed to be clear about the problem. Down- town Pittsburgh isn't Disneyland, but neither is it a nest of armed mug- gers waiting in alleys to rob and kill their victims. The problem is local- ized in a few outlying neighborhoods most people don't venture into, and the law-abiding people who live there are highly unlikely to be vic- tims. Residents of Homewood, the city neighborhood with the most shootings, repeatedly told me they felt safe there. It concerned me that using these stark statistics could lead some to think I was playing up racist stereo- types. I needed to put the numbers in a way nobody could argue with, so readers could absorb this detail fully and move on to thinking about why it happens, what it means, what can be done, and why they should care. The best way to do that was with a map. First, I supplemented a Microsoft Ex- cel spreadsheet our crime reporter was keeping on each homicide with information from the coroner's log book. In the end, I had a database showing the address of where the body was found for every homicide in 2003, along with demographic infor- mation such as age, gender and race for each victim and for those arrested for the crimes. I saved the Excel sheet in dBASE IV format, imported it into an ESRI ArcView 3.2 project and geocoded the crime scene addresses to create murder location points. The results showed the clustering of killings. Because the crime addresses were often vague, without even a general- ized street number, I had to plot about 50 of the 125 dots by hand. Then used the Legend Editor to color the dots based on whether the victim was coded \"B\" for black or \"W\" for white in the database. (Unlike most U.S. cit- ies, Pittsburgh has a minuscule His- panic population and they didn't fig- ure in the statistics.) Finally, I used Add Theme to pull in a Census 2000 map of Census block groups shaded by racial demographics to show the minority neighborhoods. series the newspaper published in July. (You can find it in the \"Special Reports\" section of our main page at www.triblive.com.) The final product was featured in \"Murder in the Streets,\" a two-day The map made clearer than words could why the stories focused ex- clusively on black murders. Most of the voices I used were black, from the veteran homicide detective from the projects to the accused pusher worried about his safety on the streets. Or that of two teenage boys in a Pittsburgh ghetto and their uncles, one of whom sits in prison for killing the other in a gang shoot- ing a decade ago. The headline, \"Black Death,\" was coupled with a moving photo of a mother in a funeral home caressing the chest of her dead son, a crack dealer who was shot off his bike. The map inside revealed the numbers, but the impact of the front page re- inforced the stories main purpose, which was to report the news we often overlook and remind readers that many victims are not innocents. Each was somebody's child. Contact Mark Houser by e-mail at mhouser@tribweb.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-nicar-has-transportation-data-for-journalists","slug":"2004-november-nicar-has-transportation-data-for-journalists","url":"https://dwillis.github.io/uplink/article/2004-november-nicar-has-transportation-data-for-journalists.html","year":2004,"month":"November","monthNum":11,"headline":"NICAR has transportation data for journalists","kicker":"SPOTLIGHT: ROADS AND RAILS","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library provides access to updated U.S. Department of Transportation databases that journalists can use to investigate safety issues in the trucking industry and identify dangerous roads.","keywords":["databases","trucking","safety"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"The IRE and NICAR Database Library maintains several recently updated U.S. Department of Transportation da- tabases relevant to the safety of U.S. roadways and the trucking industry. To order these and other databases, con- tact the Database Library at 573-884- 7711 or 573-884-7332. Journalists have used these databases to identify dangerous roads and uncover prob- lems with semitrailers on the highways. The Department of Transportation's truck census contains records for each com- pany that has commercial interstate ve- hicles weighing more than 10,000 pounds. This group includes buses, semitrailers and shippers of hazardous materials. Useful information includes the company's census number, name, address, number of vehicles owned or leased and the type of cargo it carries. Although the DOT, citing national secu- rity concerns, has recently redacted much of the hazardous-material informa- tion, the IRE and NICAR Database Li- brary offers an earlier version of the data that provided detailed hazardous-mate- rial information for carriers. See www.nicar.org/data/trkcensus for details. The truck accidents database consists of records of accidents that involve commercial vehicles weighing more than 10,000 pounds. The vehicles in- cluded in this database are buses, semitrailers, moving trucks and rental trucks, among others. The data can be used as a starting point to see how many vehicles have been in accidents for different carriers. You can also link this data to the truck census database to find out details about the motor carrier. See www.nicar.org/data/truck for details. The Hazmat database contains the incident reports of unintentional re- leases of hazardous materials for all modes of transportation (air, highway, rail and water). Journalists can ana- lyze the data to learn more about chemical spills on highways, including which hazardous-material carriers have the most accidents and in which states. The chemicals most often re- leased and accident details are also re- corded. See www.ire.org/datalibrary/ databases/haz for details. The Fatality Analysis Reporting Sys- tem database contains details about all fatal accidents, since 1975, on public U.S. roads. This data contains information about the location and conditions of each accident, the ve- hicles' make and model, the state that issued the driver's license and the status of any commercial license, prior speeding and DUI convictions, involvement in prior accidents, and more. See www.nicar.org/data/fars for more details. Contact Jeff Porter by e-mail at jeff@ire.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-readmetxt","slug":"2004-november-readmetxt","url":"https://dwillis.github.io/uplink/article/2004-november-readmetxt.html","year":2004,"month":"November","monthNum":11,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"IRE Resource Center","summary":"The IRE Resource Center provides copies of journalism stories that used computer-assisted reporting to investigate transportation safety issues, including studies on truck driver fatigue and fatal accidents involving semi-trailer trucks.","keywords":["transportation","investigation","databases"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["FARS"],"full_text":"The IRE Resource Center offers copies of stories by journalists who have use computer-assisted reporting for transportation stories. Here are some examples:\n\nA Kansas City Star investigation found that the problem of fatigued driving among truck drivers is pervasive. The Star analyzed trucking databases for the series. (Story No. 11812).\n\nAn investigation by The (Cedar Rapids, Iowa) Gazette found semi-trailer trucks accounted for nearly half of all fatal accidents on Interstate 80 in Iowa. The Gazette analyzed the Fatality Analysis Reporting System (FARS) for the story. (No. 21020).\n\nTo order these stories and others, contact the Resource Center at 573-882-3364 or rescntr@ire.org and provide the story number.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2004-november-automate-common-tasks-with-foxpro-programs","slug":"2004-november-automate-common-tasks-with-foxpro-programs","url":"https://dwillis.github.io/uplink/article/2004-november-automate-common-tasks-with-foxpro-programs.html","year":2004,"month":"November","monthNum":11,"headline":"Automate common tasks with FoxPro programs","kicker":"Tech tip","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library automated routine tasks like updating Federal Election Commission and FAA data, and developed a system to automate state-sliced data extraction from large databases, learning important lessons about handling inconsistent data structures across different government agencies.","keywords":["automation","databases","data"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["FoxPro","SQL","CD-ROM"],"full_text":"Automation can be a good thing. You can turn a burdensome chore into a quick-and-easy routine. You can update information more often. You can set up a scheme to do a task consistently. To that end, the IRE and NICAR Da- tabase Library has automated quite a few tasks over the past year. We've set automated weekly updates of Federal Election Commission cam- paign finance data, Federal Aviation Administration accident and service difficulty reports databases, and monthly updates of databases of air- craft, airmen and Internal Revenue Service nonprofits. Such automated tasks, via pro- grams written for Visual FoxPro, keep data fresh and make down- loads for users a snap. Not a FoxPro user? You can program automated tasks in almost any software. One automated process, which we run almost daily, benefits lots of us- ers of our databases, especially those with smaller news organiza- tions seeking a state slice of data. Some state slices are particularly advantageous. Occupational Safety and Health Administration inspec- tion data, for example, is voluminous for the entire U.S. but records for one state would likely fit on one CD In years past, state-slicing was a manual process. From a copy of the data on CD, we queried the data, copied the results into a folder, then copied the documentation. The big- ger the database - for example, the truck inspection database con- sumes seven CDs - the more labo- rious the process. The end result was, for the most part, that copying the entire country would be easier than slicing out a particular state. So we set out on a project to auto- mate state slices, and those lessons learned can be applied to many types of tasks that can be automated. First, we inventoried what we wanted to offer by state. Our order form (www.ire.org/datalibrary/orderform) lists several databases available via state slice. Then we examined the da- tabases in question. Some natural inconsistencies occur: First, not all the databases have the same field names for states, and some have multiple fields listing states. For example, the federal con- tracts database contains three fields that might include a state - the lo- cation of the government office, the address of the contracting company, and the state where work was to be performed. The states in those fields might all be different. Second, government agencies don't treat the state identifiers the same. For contractors, it's a postal abbreviation. For federal assistance, it's a two-digit Federal Information Processing Stan- dards (FIPS) code. For the FBI, it's yet another series of two-digit codes. Then there are the multiple-table da- tabases. In some instances (OSHA, for example) the state code is in- cluded in every table. Others (truck accidents, for example) do not. The bottom line: When you get ready to do some automation, get very, very familiar with the data structure and documentation. Here's how we handled the inconsis- tencies: We created a table of data- bases - including a one-character code for each database, the full name of the database (strictly for human reference; our state-slicer program relies on the code, not the name) and a memo field that contains the SQL query language needed to query the data for a particular state. We created a simple SQL-written program that has three steps: 1) It poses a series of questions to the user - On which local drive do you want to save the data? Which database do you want to slice? Which state do you want? 2) It makes a copy of the table of databases, transfers the answers of the questions to the SQL language about that particular database, and copies the memo field material into a new program. 3) It then runs the just-created pro- gram, including the final step of copy- ing the data, plus documentation, to the local drive the user chose during the first step. To dissect each step: Since this is an internal tool at the moment, there's no fancy interface - a double-click on a desktop icon starts the process, and just four keystrokes later (that is, one for the drive designation, one for which database, two for the state), the state-slicing begins. The first section sets the stage and offers some direction: gcstatedir = \\compname\\datal gcstoredir = \\compname\\data\\states\\list\\4 set directory to (gestatedir) Now, the series of questions begin: ? GET READY FOR A SERIES OF QUESTIONS ? ? 'YOU WILL BE ASKED: ? 1. WHICH LOCAL DRIVE WILL YOU SAVE THE DATA? TYPICALLY, YOU WILL CHOOSE M OR N' ? 2. WHICH DATABASE ARE YOU QUERYING? ? 3. WHICH STATE ARE YOU SLICING FOR? ? ? ? YOU SHOULD HAVE IN FRONT OF YOU THE STATE SLICE BINDER' ? ? wait IF YOU ARE READY, HIT THE SPACE BAR TO CONTINUE' set alternate to (gcstatedir) + 'drive' set alternate on wait 'Which drive will choose to save the data? close alternate What's an \"alternate?\" Instead of just splashing questions and answers on the screen, the \"alternate\" command tells it to store that information into a file, in this case naming it \"drive.\" The next few steps create a temporary table, append records from the just- created file named \"drive\" and parse it into records so that we can record the local drive's letter. create cursor tempdrive; (junk c(50), drive c(1)) append from drive type sdf for not empty(junk) replace all drive with upper (substr (junk, at (\"?\" junk)+1 goto top store alltrim (drive) to curdrive We repeat the process for the other questions/answers. The \"store\" com- mand takes needed pieces of infor- mation - drive, database, state - and stores that information tempo- rarily, only long enough to complete the state-slicing task. Those tempo- rary, arbitrary, names are \"curdrive,\" \"curdbase\" and \"curstate.\" For the sake of example, let's say we choose the \"N\" drive on our ma- chine, the government database of firearms licensees (the code on that would be \"F\"), and the state of Ala- bama (the code in this database would be postal: \"AL\"). The next step is to use the table of databases, named \"tables stored\" and make a copy of the table. The program names the copy as the word \"table,\" plus the one-letter code of the database in question and the state code. So our copy of the table would be named \"tableFAL.\" Then, opening up that copy, we start plugging in the stored pieces of information into the memo field content of the record that relates to the specific database. replace all sql with strtran (sql, XX' (curstate)) for id = (curdbase) replace all sql with strtran(sql,inh (curdrive) for id = (curdbase) The field called \"sql\" is the memo field that contains the SQL program lan- guage for each individual database to slice. The function \"strtran\" is a find- and-replace tool with three arguments - the field, the character to find and the material to replace it with. For the firearms licensee data, here's some key SQL language: gcyears = '04' gcdrivedir = \\ATF_XX\\' gcstatedir = \\\\compname\\\\data\\\\states\\\\\\' gcpickupdir = \\compname\\data\\states\\pickup\\ATF_FF gcstate 'XX' Notice the \"XX\" and \"~\" that are serv- ing as placeholders for the actual state code and the drive name where we want to store the data and docu- mentation. The \"gcpickupdir\" folder already contains the documentation that goes along with the data. The query for the data, then copying the results, uses the \"gcstate\" vari- able, with the \"XX\" value replaced by \"AL\" for our example of Alabama: select from 'Ffl + (gcyears); where premst = (gcstate) or mailst = (gcstate) ; nowait copy to (gepickupdir) + 'Ffl' + (gcyears) type foxplus The last step is to make some copies of the data and documen- tation to a folder named \"N:\\ATF_AL.\" command = 'xcopy /e /v + (gcpickupdir) + (gcdrivedir) run &command This is, of course, a work in progress: The next step is to add another question and allow the user to automatically place it on our FTP server if someone needs the data right away. After that, we'll start add- ing our archival data in addition to our current copies of databases we provide year-by-year. All that can be accomplished in some not-too-diffi- cult Visual FoxPro language. We'd love to see how you've used your favorite applications to auto- mate tasks; if you've got one you're willing to share, please e-mail me. Contact Jeff Porter by e-mail at jeff@ire.org","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-building-a-no-cost-open-newsroom","slug":"2004-november-building-a-no-cost-open-newsroom","url":"https://dwillis.github.io/uplink/article/2004-november-building-a-no-cost-open-newsroom.html","year":2004,"month":"November","monthNum":11,"headline":"Building a no-cost, open newsroom","kicker":"OPEN SOURCE","author_name":"Aron Pilhofer","author_title":"","affiliation":"The Center for Public Integrity","summary":"This article introduces a series on implementing free, open-source software tools in newsrooms to create a computer-assisted reporting platform, starting with MySQL as a database server and outlining basic hardware requirements and future topics.","keywords":["MySQL","newsroom","database"],"topics":["Investigative Methods","Technology","Journalism & Newsroom"],"technologies":["MySQL","Apache","PostNuke","Subversion","SQL Server","Microsoft Office","Pentium"],"full_text":"Over the past two years, I've reviewed nearly a dozen free and open-source applications I have found useful in a newsroom setting. Starting with this is- sue of Uplink, I'm taking a different tack: I'm going to show you how to put this software to work in your newsroom. This series is intended to enhance rather than replace or repeat existing documentation. I'll point out some of the more important things to think about, as well as how to avoid pitfalls. The goal is to provide the road map for a totally free, fully functional computer- assisted reporting platform to rival com- mercial products costing tens of thou- sands of dollars. The heart of computer-assisted report- ing is data, and the best way to handle large quantities of data is with a high- end database server. That job goes to the subject of this article: MySQL, which boasts an ideal combination of speed, power and features. In future articles, I'll cover a number of other software packages, including the Apache Web server (the most popular server in the world) and PostNuke, a powerful but simple con- tent management system that will drive our intranet and deliver data to our newsroom users. I'll also discuss Subversion, a software package designed for programmers that I have found invaluable as a re- pository for important files, scripts and other documents. Subversion allows sharing of documents across a net- work. It tracks changes to the docu- ments, so users are able to revert to a previously saved version at any time. And finally I will talk about how all this software works together with the soft- ware most journalists are familiar with: Microsoft Office and SQL Server (yes, they play well together). Basic needs For the purposes of simplicity and brev- ity, I'm going to assume users have ad- equate hardware. In this case, a basic system would be a Pentium 4 desktop or server, 512 megabytes or more of memory and at least 40 gigabytes of hard disk space. One of the benefits of open-source software is that it tends to have fewer bells and whistles than other commercial software. Although I wouldn't recommend doing so, it would most likely run just fine on something with less power. MySQL is fast, stable and free. The server should have a tape drive or separate hard disks to use for back- ups. If you are going to put critical data on the server, you should have a sys- tem in place to ensure that it is safe. In terms of operating systems, I am assuming the server will be running some flavor of Windows, preferably 2000 or XP. Although I have never per- sonally tried to install these software packages on a version of Windows server, I believe they will work just fine. In all cases, you will need local adminis- trator privileges to install and configure this software, which is something you may need to negotiate with your IT staff. I do not recommend running Windows 9x/ME. It is too unstable and some of the tools simply will not work. Linux is an excellent alternative to Windows, and will run faster on older hardware, but the configuration is a bit trickier. Starting with MySQL MySQL is the world's most popular open-source database server for a number of reasons: It's fast, stable and free. MySQL AB, the Swedish company developing their namesake product, has made remarkable strides in just the past five years building a software pack- age that is second only to Oracle in speed and power, and it is closing the gap with SQL Server in terms of features. Just in the past few months, MySQL AB has been rolling out some new tools to make installing and adminis- tering a MySQL Web server easier. But let's start by grabbing the files we need from the MySQL Web site (http:/ /dev.mysql.com). The server itself comes in several flavors. The 4.0.x line is considered the \"produc- tion\" version, although I recommend the 4.1.x (4.1.5 as of this writing) variant even though it is still technically in a testing stage. MySQL AB is extraordinarily con- servative, meaning versions can be in testing for months or years before they are declared officially stable. The 4.1.x line has been in development for nearly two years and is rock-solid stable. I have never seen it crash. Plus, it has several performance and feature advantages that make it the version of choice right now. Version 5.0.x is in early testing right now and will add the last gaping fea- ture hole in MySQL: stored procedures. I have experimented with it without trouble, but believe it is still too early to use in a production environment. On the download page, you'll notice there are three versions of the MySQL server for Windows. The one to get is labeled \"Windows Essentials.\" Also a relatively new innovation for MySQL, this version includes not only an in- staller but also a configuration wizard to make the process easier. Download the installer and accept the default settings (you really can't make a mistake). The application will be installed into the following directory: C:\\Program Files\\MySQLIyySQL Server 4.1). When asked to select a server configuration, \"typical\" will suffice. Initial configuration Once the software is installed, you'll be prompted to launch the configura- tion wizard. Here, your selections do matter a bit more. At the initial screen, choose \"detailed configuration\" to be- gin setting up your new server settings. The second page will ask whether the server will be used as a development machine, a multipurpose server or as a dedicated server just for MySQL. Unless you intend to run MySQL on your per- sonal desktop computer, choose \"Server machine.\" This will allow MySQL to use a portion, but not all, of your system re- sources when under heavy loads. The third page is a bit more complicated, and delves into the various types of stor- age engines MySQL offers. In short, the server can store data in one of three pri- mary formats, each of which has cer- tain advantages for certain applications. Since we are optimizing our server for speed, select the first (default) setting, \"multifunctional database.\" The next page asks where to store data for one of the three types of tables used in MySQL (InnoDB). Because we will not use this type of table, accept the defaults. Next, the wizard will prompt you to in- dicate the number of users you expect to hit your database server at any one time. In a small- to medium-sized newsroom, chances are you can be relatively conservative and select the lowest number you feel comfortable with by using the \"manual setting\" se- lection and pull down. The next page asks how the server should be configured to communicate with users, and you're going to have to bear with me here. This can be con- fusing, but it's important. By default, MySQL is set up to oper- ate over TCP/IP, the standard address- ing system for most networks ing the Internet. Under this scheme, each computer on a network -whether it is a server or desktop - is assigned a unique IP address that allows them to talk to one another. You can see your machine's IP ad- dress on your network by going to the command line, and typing \"ipconfig\" at the DOS prompt. You should see a number that looks something like \"192.168.1.34\". This address is critical to communicat- ing over a network, especially between servers and clients. Without this ad- dress, for example, an e-mail server wouldn't have a clue where to send your e-mail. The same applies to MySQL. If you don't know what your MySQL server's IP address is, neither you nor your users will be able to log in over a network. (That's not entirely true be- cause there is another protocol for find- ing the server called a \"named pipe,\" but that's quite a bit more complicated.) In addition to the IP address, it's also important to know which port your server software communicates over. Think of the port as a channel on your radio dial. If all radio stations broadcast on the same channel, the results would be a mess of confusing and conflicting signals. The same is true for a server, which couldn't keep all the communica- tions straight if they came in over the same channel. Thus, server software is configured to run on different default ports - 3306 in MySQL's case. To get back to the con- figuration wizard, this page is prompt- ing users to indicate whether or not they wish MySQL to use TCP/IP (we do), and, if so, what port it should use. We can accept the default values in this case. (That may seem a bit anticlimac- tic after the above explanation. But it's a critical concept to understand, as you will see later). On the next two pages, regarding char- acter sets and installing MySQL as a ser- vice, select the default settings. Although if you intend to store text characters not found in English or most Western Euro- pean languages, you might wish to choose the \"best support for multilingual- ism\" option on the character set page. On the second page, the default settings will install MySQL as a Windows service where it will run in the background auto- matically upon booting the computer. The next page deals with security, spe- cifically setting a password for the de- fault administrator account (called \"root\" in MySQL parlance). This is critical, and should not be skipped. By default, MySQL ships with a blank password for the root user - which is a very bad thing. Root access should be granted only to the database administrator and highly trustworthy individuals who ab- solutely need it, because a root user has the power to alter or delete any- thing and everything - a fairly gaping security hole to say the least. Choose a good, secure password that you won't forget, enter them in the textboxes and click next. There's no need to change anything else on this page. On the final page, click the execute but- ton and, if all goes well, the configura- tion will complete successfully. If you receive an error message when it tries to install MySQL as a service and you have had a previous version of MySQL installed before, you may have to back up to the configuration page and change the name of the service to something like \"MySQL1.\" That has solved the problem for me. For more information about the new MySQL installer and configuration wizard, see this article on the MySQL Web site: : http://dev.mysql.com/tech-re- sources/articles/4.1/installer.html Final words You now should have MySQL up and running on your server. To confirm, check the Processes tab in the Win- dows task manager and you should see either mysqld.exe or myswqld- nt.exe listed there. In the next article, I will cover the ba- sics of communicating with MySQL, (including how to connect via Microsoft Access/Open Database Connectivity), importing data and making that data available to your newsroom. Contact Aron Pilhofer by e-mail at apilhofer@publicintegrity.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2004-november-readmetxt-2","slug":"2004-november-readmetxt-2","url":"https://dwillis.github.io/uplink/article/2004-november-readmetxt-2.html","year":2004,"month":"November","monthNum":11,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Journalists can use data from multiple sources like the U.S. Census and specialized databases to investigate child welfare issues and quality of life disparities in their communities. Resources and tipsheets provide guidance on accessing adoption, foster care, and other child welfare data for reporting.","keywords":["children","data","databases"],"topics":["Data & Databases","Investigative Methods","Education"],"technologies":["Adoption and Foster Care Reporting System"],"full_text":"For more information using data to report on dangers and neglect to children see:\n\n\"Tracking quality of life,\" by Dave Davis of The (Cleveland) Plain Dealer in the September-October 2003 Uplink. Davis tells how journalists at the newspaper analyzed more than a million records from a number of sources, including the U.S. Census, and found that children in Cleveland were worse off than their counterparts in many comparable big cities.\n\nTipsheet No. 1822 by Jason Method of the Asbury Park (N.J.) Press lists resources for reporting on children's issues. The tipsheet contains details about using the Adoption and Foster Care Reporting System database.\n\nTo order the tipsheet contact the IRE Resource Center at 573-882-3364 or rescntr@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-january-february-post-vote-politics-payoffs","slug":"2005-january-february-post-vote-politics-payoffs","url":"https://dwillis.github.io/uplink/article/2005-january-february-post-vote-politics-payoffs.html","year":2005,"month":"January • February","monthNum":1,"headline":"Post-vote politics, payoffs","kicker":"ELECTIONS","author_name":"Andrea Lorenz","author_title":"","affiliation":"IRE and NICAR","summary":"This article discusses how journalists can use campaign finance and voter registration data to investigate political influence and structural changes following the 2004 election.","keywords":["campaign finance","elections","voter registration"],"topics":["Government & Politics","Data & Databases","Investigative Methods"],"technologies":["Excel","Access"],"full_text":"November has come and gone. Journalists, readers and viewers don't want to hear or see the words \"red\" and \"blue\" in the same sentence as the word \"state.\" The 2004 presidential election is long over. But story ideas abound for enterprising journalists who want to use computer-assisted reporting to find out what happened in the elections and explore possible connections between campaign contributions and political favors. \"Who paid for this election, and what do they want?\" asked Ron Campbell, a staff writer at The Orange County Register. Journalists can answer these questions with the help of campaign finance data available on the Web or from the IRE and NICAR Database Library. Campaign contributions can explain why some bills pass Congress and other stall in committee. In 2001, after the previous presidential election, The Wall Street Journal investigated campaign, inaugural and recount fund contributions by industry. A yearlong series of stories by the Journal showed the GOP's top 10 contributors. Journal reporters tracked the policy interests of the contributors and found the administration and Congress supported (or opposed) legislation in line with the interests of contributors. Tom Hamburger, who was the Journal's money and politics reporter at the time, says once the data analysis by The Center for Responsive Politics was complete, Journal staffers on every Washington beat were involved in the reporting. Says Hamburger, analyzing contributions to the administration and legislators shows the public how the government works and affects lives. The problem with donation stories, Campbell says, comes from the necessity of joining three databases together. For example, to find out who gave money to which candidates in California, reporters need Federal Election Campaign data, 527 contribution data available on the IRS Web site and data from the California secretary of state. Campaign data also can help journalists look at structural changes in politics and provide context. \"Why is [billionaire George] Soros writing all these checks?\" Derek Willis of The Washington Post asks. \"It has to do with structural changes.\" Changes in tactics, laws and procedures affect donors, parties and committees. In October, The Washington Post published an article detailing the results of the paper's analysis of 527 data maintained by The Center for Public Integrity. They found six out of 10 donors to 527 groups are billionaires. The McCain-Feingold campaign finance law prohibited soft money contributions to political parties. The Post used data to show how the 2002 law affected the process. To make up for the new constraints, donors gave more money to state parties. Some of the structural changes are so minute, Willis says, that reporters have to understand campaign finance law as well as political science professors and campaign lawyers do. He points to a radio ad that aired before the election. The campaign ad criticized a proposal by Democratic candidate Sen. John Kerry. At one point, words were added to include liberals in Congress. Willis says doing this allowed the campaign to share a cost of the ad with the party. Changes that might seem insignificant to people are deliberate. \"It's got to be the data,\" Willis says. \"You can't see them in meetings making decisions about what to do, but you can see where the money goes.\" Structural changes made before the election might not be apparent until afterward. Now is a good time to figure out first what the changes are, and next how did they affect the election. It's best to have earlier data, Willis says. Compare contributions, results and policy to the last election. If power changed hands, fundraising might change. Who shifts allegiances? Check out the newcomers in the legislature. \"Okay, they got in – now what?\" Willis asks. Know who has campaign debt. Know not only how much money was spent, but know where the campaign put money. Willis suggests creating timelines in Excel or Access of when and how campaigns spent contributions. Even if the end result doesn't make it into the paper, knowing this information will help during the next campaign. Journalists can also use voter registration data to check up on registration and voting patterns. The Sarasota Herald-Tribune, like a number of other newsrooms, has a database of historical election results and Census data. Adding the 2004 results would yield a database that could show shifts in demographics and an increase in voter registration. \"We could spend a few months with that database we have,\" says Matt Doig, a reporter at the paper. Florida reporters could follow up on absentee ballot stories by using geographic information system software and absentee-ballot data to find out how many \"snow birds\" voted and where they live the rest of the year, he says. And of course, any analysis done for the 2004 elections will add context to 2006 and 2008 federal elections. If you took a breather from the red and blue state madness, now is a good time to get back; the data is waiting.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-mapping-stats-show-scope-of-racial-bias","slug":"2005-january-february-mapping-stats-show-scope-of-racial-bias","url":"https://dwillis.github.io/uplink/article/2005-january-february-mapping-stats-show-scope-of-racial-bias.html","year":2005,"month":"January • February","monthNum":1,"headline":"Mapping, stats show scope of racial bias","kicker":"SPOTLIGHT: PROBING THE POLICE","author_name":"Andy Lehren","author_title":"Producer","affiliation":"Dateline NBC","summary":"Dateline NBC investigated racial disparities in traffic stops by analyzing millions of records and using internal benchmarks to identify patterns of suspicion.","keywords":["racial profiling","traffic stops","data analysis"],"topics":["Demographics & Census","Investigative Methods","Crime & Justice"],"technologies":[],"full_text":"For a few days in April 2001, it was hard to miss news accounts about riots in Cincinnati. Daily stories outlined how police officers chased an unarmed 19-year-old black man named Timothy Thomas. It was past midnight in the Over-The-Rhine neighborhood. A dispatcher called in officers to help pursue Thomas. The voice over the police radio offered a few details: a brief description of Thomas, and that he had 14 outstanding warrants. An officer responding to the call spotted Thomas. He got out of his car and drew his gun, fired and killed Thomas. Cincinnati rioters protested the police shooting, capturing headlines across the nation. Fellow Dateline NBC producer Jason Samuels and I were asked to look into the circumstances surrounding these events. After collecting traffic ticketing data and going through court records, I came back to the dispatcher's brief call on the radio: 14 outstanding warrants. How does a 19-year-old rack up so many warrants? That question led us on a lengthy investigation that culminated with an hour-long report called \"A Pattern of Suspicion.\" With correspondent John Larson, we examined disparities in how police across the United States deal with blacks and whites. Despite its national scope, the story focused on something simple: a traffic ticket. That's because Thomas' 14 outstanding warrants all stemmed from traffic tickets. Almost all were for the tickets themselves, and the last two were for fleeing from officers for not paying the traffic tickets. There was also something intriguing about the kinds of traffic tickets that Thomas got. They were not for the usual things, like speeding or running a stop sign. Almost every citation was for an offense that is difficult or almost impossible for an officer to know about without first stopping someone. The list includes driving with a passenger not wearing a seatbelt and driving without a license. Only one ticket was for a moving violation. Police, prosecutors and criminologists have another way of looking at these tickets. Some, like speeding far above the speed limit or running a stop sign, are also known as low-discretion tickets. These are moving violations, and the offense is obvious even at a distance. Police don't have to make too many decisions before chasing those drivers. Then there high-discretion stops for driving without a seatbelt or having windows tinted too dark. Notwithstanding special crackdowns, these non-moving violations are harder to detect, and are often ignored by officers. These are usually ignored unless an officer is looking for something else. In the war on drugs, police and others told us it's perfectly legal for an officer to pull over drivers for these high-discretion violations in the hope of catching them at something worse. Advocates charge this is essentially racial profiling, violating the equal protection clause of the 14th Amendment and also inflaming already strained relations between police and minority communities. With Cincinnati's ticketing database, we were able to see Thomas was not alone in getting high-discretion non-moving violations. We obtained the data, with more than 300,000 entries, through a state open records request. It included fields such as the time, street, violation, the driver's name and demographic information like race. By calculating odds ratios, we could see that blacks were at least three times as likely as whites to get cited for driving without a seatbelt or not having a driver's license. This kind of ratio is known as an internal benchmark. It relies on information just within the ticketing database. You compare the rate for which blacks versus whites get a high-discretion ticket, like driving without a seatbelt, and divide that by the rate for which blacks versus whites get low-discretion tickets like speeding. For those interested in learning more about odds ratios, Sage publishes several handy books, and Phil Meyer includes a valuable critique in his latest edition of \"Precision Journalism.\" One advantage of internal benchmarks is that it is an accepted standard in criminal justice circles, and it does not involve population estimation. It is as precise as possible. Comparing ticketing versus overall population is known as an external benchmark, and among police, its soundness is more debatable. We looked at the data other ways. By mapping violations in ArcView, geocoding the locations where tickets were issued, we could see the disparities were not particular to Over-The-Rhine. In predominately white, low-crime neighborhoods, blacks were still ticketed far more often for non-moving violations versus moving violations. We collected various documents and reports that were useful for starting our data shopping list. A General Accounting Office (now known as the Government Accountability Office) report, among others, outlined which cities with at least 100,000 people across the nation are collecting ticketing and stop data. We sought data from as many city police departments as possible from coast to coast. This included police departments that had never had anyone request the information – that includes journalists as well as advocates and academic researchers. The problems we encountered are unfortunately typical: places like San Diego took months and months in replying; Denver rejected our initial request but later turned over the data, and St. Louis charged by far the most. In the end, we had more than 4 million traffic tickets and stops in more than a dozen cities. One hurdle we faced was that almost no city kept the data exactly like another. In some places, it was impossible to tease out moving versus non-moving violations; those cities were excluded from our analysis. Other police departments recorded stops, and noted the general category of the violation, rather than the precise violation code, that led to the stop. In those cases, we could include the data because the categorizations offered at least a window into that department. As can be expected, this kind of categorization was limiting because it prevented us from examining variations within a grouping – when a police department was pulling over blacks at a higher rate for, say, equipment violations, was the rate even more disproportionate for seatbelt offenses? In any city that included geographic information we mapped out where the violations occurred. We cross-checked both high-crime and low-crime neighborhoods. In the end, we mapped more than 1 million addresses in six cities, a process that accelerated with programming that improved standardization. We performed a logistic regression in SPSS to examine the effects of other demographic data on the disparities involving non-moving violations. Logistic regression tests the probability of an event, when there are only two possible outcomes. The results are often reported as odds or odds ratios. This underscored how race was a key factor by allowing us to see essentially how the odds of someone being black is cited for these violations, versus the impact of a person's sex or age. We constantly re-checked the data and talked to criminal justice experts. We gave each police department more than two months to respond to a written summary of our findings. We wanted to ensure that our work was accurate, and that every police agency had an opportunity to talk with us about what was in the numbers. The work we did along the way proved extremely useful in these discussions. When one department cited a Bureau of Justice Statistics survey, we could point out that the questionnaire categorized stops in a way that made it impossible to detect moving versus non-moving violations. When another department cited a NHTSA study that blacks did not wear seatbelts about a quarter of the time, we noted that the rate was virtually the same for whites. Only one police department offered a geographical argument: Richmond said the ticketing disparity was a result of heavy policing in high-crime neighborhoods, which are often black. When we pointed out that the disparity passed through to low-crime, predominately white sections, it responded with \"No comment.\" For other journalists doing this kind of story, it's important to get the data, talk with sources and get familiar with the subject of ticketing disparities. Aside from Dateline NBC's investigation, we helped affiliates with local stories. This allowed us to get out more information than what could fit in our piece. We also gave the data and worked with The Denver Post, which has an affiliation with NBC. It published a front-page story on the day we aired. And we posted additional reporting on the Dateline Web site at www.msnbc.msn.com/id/4703833.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-tallying-lost-money-through-farm-loophole","slug":"2005-january-february-tallying-lost-money-through-farm-loophole","url":"https://dwillis.github.io/uplink/article/2005-january-february-tallying-lost-money-through-farm-loophole.html","year":2005,"month":"January • February","monthNum":1,"headline":"Tallying lost money through farm loophole","kicker":"TAX BREAKS","author_name":"Paula Lavigne","author_title":"reporter","affiliation":"The Dallas Morning News","summary":"The Dallas Morning News investigated how a farm tax loophole intended for family farmers is being exploited by large corporations and land developers to save millions in taxes.","keywords":["tax exemptions","property records","land development"],"topics":["Business & Economy","Data & Databases","Investigative Methods"],"technologies":["SQL Server","dBase","FoxPro","Access","Excel","CD-ROM","FTP"],"full_text":"Right across the street from our office building is a field. Every once in a while we'll notice farm equipment rolling onto the property, tending to the growing rows of various crops. It's prime real estate, almost 110 acres near the intersection of two major highways. The owners are members of one of Dallas' most notable old money families who, among other pursuits, are land developers. They are not farmers. Yet, by leasing their land to a farmer who reaps a nominal harvest, they get an agriculture tax break that saves them about $545,000 each year in taxes. Their savings is lost money for cities, schools and county governments. Last year, two Morning News county reporters (Ed Housewright and Kevin Krause) and I decided to investigate just how much tax money schools, cities, and counties lost through these agricultural exemptions. We also wanted to know which landowners profited the most from the tax break. Were they really farm families for whom the exemption was created almost 30 years ago? Or were they corporations, developers and land speculators looking to save millions in taxes by leasing the land for farming? Our goal was straightforward. We wanted to measure the taxable value of the agriculturally designated land and its market value. Then we had to calculate taxes based on the agricultural value versus the market value to find the total revenue lost to exempt property. Our data came from property records from our four main counties: Dallas, Denton, Collin, and Tarrant. In Texas, entities called appraisal districts assess properties for the various taxing entities. The Tarrant Appraisal District had all of its data available for free download on its Web site. The Denton district sent its data to us on CD. We had previously paid for Collin County data, but the appraisal district didn't charge us for this year's update, downloaded via FTP. Dallas district officials initially said they were going to charge us for our download, but never did. The format of each district's data varied. Dallas' was in Microsoft SQL Server format. Collin's was in dBASE. Denton's was comma delimited text. Tarrant's initially came with special translation software, but it was too cumbersome so the technician there ended up sending a comma delimited file. The number of records ranged from about 270,000 for Collin to about 900,000 for Dallas, with a maximum of 30 fields. I talked to every database manager before coming up with my methodology to make sure I was using the right categories. After I finished, I outlined my step-by-step process and e-mailed it to them asking them to try to poke holes in it. They responded that my methods were sound. The calculations were simple. Using Microsoft FoxPro, I created four separate smaller tables from the original large datasets (more than 1 million records in one case). I chose records in which the amount in the agricultural value field was greater than zero. I created a Microsoft Access database and imported the smaller files so I and the other reporters could work on them. For each table, I ran a sum query on the market and agriculture (or taxable) value fields. The difference between market value and agriculture taxable value provided the value of taxable property exempt or \"lost\" to the taxing districts. I put these LOST numbers in a separate column. But it's not the amount of lost tax revenue yet. That takes another step. Each property record in an appraisal district database includes one or several fields to indicate that property's taxing entities. I did a GROUP BY on each entity field and a SUM on the newly created LOST field. That gave me the total value lost by each entity. I had previously downloaded Excel or text files from the districts' Web sites with each taxing jurisdiction's tax rate. I plugged my LOST totals into those files. The result: The total amount of tax revenue each taxing entity lost this year because of the agriculture tax break. One district in a fast-growing suburb north of Dallas was losing more than $20 million in revenue. Speculators and businesses with valuable land were paying a pittance in taxes because they either ran a farming operation on it themselves or leased it to a farmer to keep it agriculturally active. After we saw who the big losers were, we wanted to find the big winners. I started to calculate this by doing a GROUP BY on the taxpayer name field and a SUM on the LOST field. I also grouped by taxpayer ID and by the taxpayer address field. I struggled to definitively say that a certain proportion of property owners claiming the exemption were big businesses or land speculators. So, I dumped my taxpayer file that I recently sorted into Excel. My intent was to find an easy way to prove that most – or more than half – the value of the ag exemption was held by major corporations or land speculators. I started by sorting descending by the companies with the largest amount of taxable value deducted because of the exemption. I did a rolling sum of those values until the total sum equaled more than half the total sum of the exemption values. Through some calculations, and some eyeballing, I determined that most of the value was attributable to large corporations and property development companies. We also decided to map the parcels receiving the tax break using ESRI ArcView 8.3, and the parcel shapefiles the districts gave us for free. Farmers and ranchers were the most vocal proponents of the system. Even though they're not benefiting directly from the tax break, they are able to lease cheap land and keep farming in an area where vacant farmland is disappearing.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-felon-hunters","slug":"2005-january-february-felon-hunters","url":"https://dwillis.github.io/uplink/article/2005-january-february-felon-hunters.html","year":2005,"month":"January • February","monthNum":1,"headline":"Felon hunters","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"A database matching investigation revealed that nearly 1,000 felons in Missouri held hunting licenses despite federal laws against felon firearm ownership.","keywords":["felons","hunting licenses","database matching"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"Matching government databases can produce high-impact reporting. Investigative reporter Steve Chamraz of KCTV-Kansas City recently reported on Missouri felons with hunting licenses who were discovered with database matching, and state lawmakers took note. KCTV asked the IRE and NICAR Database Library to compare state hunting licenses and felon data tables. Database Library analysts obtained the data from the state corrections and conservation departments, cleaned the data and formatted the tables to make database matching possible. Then the analysts matched the two tables. The results: nearly 1,000 felons –13 of them convicted killers – in Missouri had hunting licenses. It is against federal law for felons to own or even hold a gun. However, it is not against Missouri state law to sell firearm-hunting permits to convicted felons. As a result of the report, some state legislators are considering changing the state law.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-disaster-aid","slug":"2005-january-february-disaster-aid","url":"https://dwillis.github.io/uplink/article/2005-january-february-disaster-aid.html","year":2005,"month":"January • February","monthNum":1,"headline":"Disaster aid","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"The SBA Disaster Loan database provides a resource for journalists to track government assistance provided to businesses and individuals following natural disasters.","keywords":["disaster aid","SBA","government loans"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"Journalists interested in discovering which businesses or people have received government loans relating to a disaster can use the recently updated Small Business Administration (SBA) Disaster Loan database from the Database Library. The database logs every approved loan. Each loan record in the database includes the amount and the related disaster. Loans to businesses also indicate whether the loan has been paid. Current as of September 2004, the data includes more than 805,500 records of loans and nearly 7,800 disasters. The (Fort Worth) Star-Telegram used the data in the aftermath of tornadoes in Fort Worth to see who received the promised federal assistance.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-network-news","slug":"2005-january-february-network-news","url":"https://dwillis.github.io/uplink/article/2005-january-february-network-news.html","year":2005,"month":"January • February","monthNum":1,"headline":"Network news","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists can access resources for social network analysis to visualize complex connections between people and organizations in investigative reporting.","keywords":["social network analysis","SNA","investigative software"],"topics":["Journalism & Newsroom","Technology","Data & Databases"],"technologies":[],"full_text":"IRE and NICAR offer journalists resources for using social network analysis (SNA) software to analyze data for news stories at www.ire.org/sna. The resources list includes links to beginning guides to SNA and to recent news stories that have used the software to display connections between political contributors and political nonprofit committees, and between an Islamic charity and groups associated with Osama bin Laden. The Web site also has a wealth of tipsheets for journalists who want to learn more about SNA, the available software and potential applications to stories.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-politics-and-elections-an-evolving-story","slug":"2005-january-february-politics-and-elections-an-evolving-story","url":"https://dwillis.github.io/uplink/article/2005-january-february-politics-and-elections-an-evolving-story.html","year":2005,"month":"January • February","monthNum":1,"headline":"Politics and elections: An evolving story","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"While the presidential election is over, David Herzog emphasizes that journalists should continue using databases to investigate political power and voter registration issues throughout the year.","keywords":["election coverage","voter registration","CAR training"],"topics":["Government & Politics","Investigative Methods","Demographics & Census"],"technologies":[],"full_text":"Covering elections and politics is an ongoing enterprise. The November presidential election is over and the crush of resulting stories has lightened. Still, it's good to remember that some of the best stories coming out of the elections are the ones developing now, the ones you can do any time of the year to examine subjects that are key to understanding our democracy: political power and voter registration, to name just two. In the most recent election season, journalists from coast to coast fired up their databases and discovered interesting stories about voters registered in more than one locale, political nonprofit contributions and problems with felon-voter purge lists – all great stories. From the stories reported so far, it's clear that even more are waiting to be uncovered. This issue of Uplink, starting with the front page article by IRE and NICAR's Andrea Lorenz, presents resources for journalists who want to stay on the trail through the year ahead. It's that time of year: We're busy getting ready for the CAR Conference March 17-20 in Hollywood, Calif., putting together the best possible schedule of hands-on classes and panels on computer-assisted reporting topics. For the first time, we're offering a full day (Thursday, March 17) of special sessions tailored for advanced CAR practitioners. So mark your calendar; we'd love to see you in Hollywood.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-upcoming-training","slug":"2005-january-february-upcoming-training","url":"https://dwillis.github.io/uplink/article/2005-january-february-upcoming-training.html","year":2005,"month":"January • February","monthNum":1,"headline":"UPCOMING TRAINING","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer multiple computer-assisted reporting training programs including full Boot Camps in Columbia, Missouri, a specialized program for editors, and a GIS mapping workshop to help journalists acquire, analyze, and report data-driven stories.","keywords":["training","reporting","data"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["ArcView"],"full_text":"IRE and NICAR have several training opportunities for journalists seeking hands-on instruction in using computer-assisted reporting.\n\nThere are three weeklong Boot Camps in Columbia, Mo., for journalists who want to learn how to acquire electronic information, use spreadsheets and databases to analyze the information, and to translate that information into high-impact stories. IRE and NICAR provide follow-up help after participants return to their news organizations. The Boot Camps for this year are March 20-25, May 15-20 and Aug. 7-12.\n\nA special mini-Boot Camp designed for editors will be April 8-10 in Columbia. The training, which places an emphasis on what newsroom managers need to know, will include just enough hands-on work to provide an understanding of CAR.\n\nJournalists interested in learning how to map data for news stories can take advantage of a mini-Boot Camp in using geographic information systems (GIS). The training, Aug. 19-21 in Columbia, will use ArcView 9 GIS. Participants will learn how to create basic maps, work with projections and geocode data.\n\nFor more information on these and other IRE and NICAR training events see www.ire.org/training/otr.html.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-january-february-training-networking-at-car-gathering","slug":"2005-january-february-training-networking-at-car-gathering","url":"https://dwillis.github.io/uplink/article/2005-january-february-training-networking-at-car-gathering.html","year":2005,"month":"January • February","monthNum":1,"headline":"Training, networking at CAR gathering","kicker":"ANNUAL CAR CONFERENCE","author_name":"","author_title":"","affiliation":"","summary":"The 2005 Annual CAR Conference in Hollywood offers extensive training in database analysis, mapping, and investigative techniques for journalists of all skill levels.","keywords":["CAR conference","journalism training","data analysis"],"topics":["Journalism & Newsroom","Data & Databases","Investigative Methods"],"technologies":["spreadsheet","Internet"],"full_text":"The 2005 Annual Computer-Assisted Reporting Conference will feature an Advanced Day for CAR veterans, special tracks for broadcasters and beginners. The Advanced Day will be Thursday, March 17, and will be followed by three main days of panels and hands-on training for all levels of experience and expertise. All sessions will be at the Renaissance Hollywood Hotel in Hollywood, Calif. Practitioners in CAR will conduct more than 40 panels on nearly every beat and 60 hands-on classes on spreadsheets, database analysis, statistics, mapping and searching for and downloading data from the Internet. KNBC-Los Angeles and the Los Angeles Times are conference hosts. The Orange County Register is a sponsor. The conference Demo Room will offer informal tutorials and small workshops on specific federal and state data and analytic techniques. Among the topics will be using data analysis to report on homeland security, pollution, homicides, natural disasters, elections and campaign finance, inequities in education, exploitation of the poor, the military and the war in Iraq, and immigration border issues. Advanced topics will include social network analysis, data cleaning, open-source tools, constructing intranets for newsrooms and mining text databases. For beginning CAR reporters, the conference will include a start-to-finish track and a mini-boot camp of hands-on training. Experienced practitioners can learn to hurdle data challenges with advanced programming, sophisticated mapping techniques, scraping data from the Internet and employing more complex statistical methods.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-blocking-out-unsolved-murders","slug":"2005-january-february-blocking-out-unsolved-murders","url":"https://dwillis.github.io/uplink/article/2005-january-february-blocking-out-unsolved-murders.html","year":2005,"month":"January • February","monthNum":1,"headline":"Blocking out unsolved murders","kicker":"MAPPING IT OUT","author_name":"Doug Smith","author_title":"Reporter","affiliation":"Los Angeles Times","summary":"The Los Angeles Times used GIS mapping and database analysis to visualize the dense patterns of unsolved homicides in South Los Angeles, leading to a reallocation of police resources.","keywords":["GIS mapping","homicides","crime statistics"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["ArcView","Adobe Acrobat"],"full_text":"Sometimes a database story starts with an idea. Other times it arises from the data itself. In this case, it was both. Reporter Jill Leovy, who was covering homicide, showed up at my desk with a database of every homicide investigated by the LAPD since 1988. I quickly looked through the table and had to inform Leovy that it wouldn't answer her question about detective deployment because there was no information on the detectives' identities. She was going to each of the 18 police divisions to look through old detective rosters. I started keypunching. When she was done, I did a simple calculation that showed the caseload for every division for every year. It was obvious that detectives working South Los Angeles had consistently been burdened with more cases. Our concept began to focus on the perception and fear of violence. We shifted our focus to unsolved homicides, and I turned to mapping to find patterns. Using ESRI ArcView 8.2, I geocoded the homicides and made a subset of the unsolved ones. Geocoding turns street addresses in data tables into point maps. The first pass yielded startling results. By laying the dots over a street map, I saw that a large area had one or more unsolved homicides on almost every block. My next step was to produce theme maps to show unsolved homicides as a ratio of population and area of Census tracts. It finally hit me that the ideal base was the city block. By shading individual blocks with the number of unsolved homicides that occurred on them, I produced the dramatic pattern that was ultimately published. No rate was necessary because everyone knows what a block is. I also added an elementary school shapefile and made a table showing the number of unsolved homicides within a quarter mile of each elementary school. For the Web presentation, I wanted readers to be able to see the unsolved homicides in their own neighborhoods. Over a couple of days, I made 25 Adobe Acrobat Portable Document Files with the street grid showing lightly in the background. The Web staff turned this into an interactive display. The stories prompted the police chief to beef up detective strength in overworked bureaus.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-data-helps-uncover-ohios-dangerous-dams","slug":"2005-january-february-data-helps-uncover-ohios-dangerous-dams","url":"https://dwillis.github.io/uplink/article/2005-january-february-data-helps-uncover-ohios-dangerous-dams.html","year":2005,"month":"January • February","monthNum":1,"headline":"Data helps uncover Ohio's dangerous dams","kicker":"FIRST VENTURE","author_name":"Ron Regan","author_title":"Reporter","affiliation":"WEWS-Cleveland","summary":"Following an IRE and NICAR Boot Camp, reporter Ron Regan used the National Inventory of Dams database to expose hundreds of unsafe and uninspected high-hazard dams in Ohio.","keywords":["dams","infrastructure safety","database queries"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":["Access","SQL"],"full_text":"Our November news report that 450 of Ohio's 1,700 dams are considered unsafe provoked a nearly incredulous response from the state's dam safety chief: \"How did you run across that?\" Admittedly, that was pretty obscure information. It's information that could ultimately save lives and there was only one way for me to get it – by using a database manager to analyze dam inspections data. Before last August, when I attended an IRE and NICAR Boot Camp, I was clueless about how to do that. It started with a U.S. Army Corps of Engineers database called the National Inventory of Dams. I first laid eyes on the database during the NICAR training and quickly realized I could turn this from a classroom exercise into a more comprehensive report. So, before leaving the Boot Camp, I purchased the data from the IRE and NICAR Database Library and with some assistance from Jeff Porter, was able to whittle down the data to just Ohio dams. I used Microsoft Access to store and view data tables, and to write queries that filter data. I was able to use Structured Query Language (SQL) to write queries that showed me the name of each dam, the year its construction was completed, the most recent inspection date and the hazard level, which indicated how close the dam is to populated areas. With a few keystrokes, I discovered 1,727 dams in Ohio with 462 rated high hazard. Not only that, many of the dams appeared to have gone years without inspection. Now, here's the really big lesson in learning CAR: You can run queries in seconds, but that does not replace old-fashioned fact-checking and digging. I obtained paper records of the most recent inspections, using open-records requests. I also learned that visiting the dams and walking around them pays off. I discovered a charming 85-year-old who lived just down the street from the most dangerous dam in Ohio. In addition, I spent weeks talking with nationally recognized experts on dam safety to ensure I had interpreted the data correctly. I contacted Ohio officials early in my reporting, explaining what I was doing. I relied on them for assistance. Most state dam inspection programs are seriously under funded and under staffed. People in those agencies recognized that a story about years-long budgetary neglect could ultimately help them and those living downstream from the dams.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-analysis-finds-law-in-disorder","slug":"2005-january-february-analysis-finds-law-in-disorder","url":"https://dwillis.github.io/uplink/article/2005-january-february-analysis-finds-law-in-disorder.html","year":2005,"month":"January • February","monthNum":1,"headline":"Analysis finds 'law in disorder'","kicker":"SPOTLIGHT: PROBING THE POLICE","author_name":"Heather Ratcliffe and Trisha Howard","author_title":"","affiliation":"St. Louis Post-Dispatch","summary":"A yearlong Post-Dispatch investigation used crime data and budgets to expose systemic incompetence, lack of training, and equipment shortages across St. Louis area police departments.","keywords":["police oversight","public records","misconduct"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Excel"],"full_text":"As the police department in the small Missouri town of Kinloch crumbled from incompetence, cops began whispering to reporters. They said many of the other 123 police agencies that patrol the St. Louis area shared the same problems. Resources were spread too thin, causing dangerously inconsistent law enforcement. The St. Louis Post-Dispatch began an investigation. In flagrant violation of police licensing laws, untrained auxiliary officers were patrolling alone. There were troubled officers shuffling amoung poor departments. There were broken-down patrol cars and one department with no two-way radios. During the yearlong investigation we amassed an enormous amount of data through public information requests. Crime rates, budgets, officer salaries and other statistics were compiled into spreadsheets and compared in graphics. In the spring of 2003, we mailed open records requests to more than 140 police departments seeking budgets, staff lists, and policies for using deadly force. None of the departments provided these documents electronically. We gathered Uniform Crime Report statistics electronically from the Missouri Department of Public Safety and the FBI. Reporters downloaded the Illinois crime data from the state's Web site. We quickly learned the flaws of relying too much on self-reported crime statistics. We decided to compare only violent crimes and averaged the crime rate over three years to get a more accurate account. We used Microsoft Excel to track the vast amount of information. We entered the 5,900 officers into a spreadsheet so we could search the names and track certification status. Reporters used the budgets to calculate statistics such as cost per capita, cost per call and cost per officer. Reaction from the public was immediate. The St. Louis County Prosecuting Attorney launched an investigation into the misuse of volunteer officers. The Missouri State Auditor began a probe into the state program that investigates police misconduct.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-2005-car-training","slug":"2005-january-february-2005-car-training","url":"https://dwillis.github.io/uplink/article/2005-january-february-2005-car-training.html","year":2005,"month":"January • February","monthNum":1,"headline":"2005 CAR Training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer specialized boot camps and training seminars to teach journalists how to acquire electronic information, analyze data using spreadsheets and databases, and develop impactful stories, with follow-up support provided after training.","keywords":["training","journalism","data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["spreadsheet","GIS"],"full_text":"These unique seminars, taught by IRE and NICAR's experts, train journalists to acquire electronic information, use spreadsheets and databases to analyze the information and to translate that information into high-impact stories. In addition, IRE and NICAR provide follow-up help when participants return to their news organizations.\n\nBoot Camps:\nColumbia, Mo.\n• March 20-25\n• May 15-20\n• Aug. 7-12\n\nMapping Data for News Stories Boot Camps:\nColumbia, Mo.\nUncover news stories by mapping data with geographic information system software.\n• Aug. 19-21\n\nBoot Camps for Editors:\nColumbia, Mo.\nTailored to the needs of newsroom managers.\n• April 8-10\n\nAdvanced CAR Statistics:\nTempe, Ariz. (ASU)\nAimed at strengthening the skills of reporters who want to move beyond basic CAR.\n• Feb. 18-20\n\nFor more information: www.ire.org/training/bootcamps.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-january-february-data-shows-cops-ignore-hot-pursuit-rules","slug":"2005-january-february-data-shows-cops-ignore-hot-pursuit-rules","url":"https://dwillis.github.io/uplink/article/2005-january-february-data-shows-cops-ignore-hot-pursuit-rules.html","year":2005,"month":"January • February","monthNum":1,"headline":"Data shows cops ignore hot-pursuit rules","kicker":"SPOTLIGHT: PROBING THE POLICE","author_name":"Ian Demsky and Christian Bottorff","author_title":"reporters","affiliation":"The Tennessean","summary":"The Nashville Tennessean found that two-thirds of police chases violated department safety policies by analyzing paper records that contradicted the department's electronic database.","keywords":["police pursuits","public safety","accountability"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Access","ArcView"],"full_text":"Just because a police department says it followed all the rules doesn't make it true. After a chase in January, Nashville police killed a fleeing suspect after he crashed at a busy intersection. The policy was very clear: Officers may chase only if allowing a suspect to flee \"would present an imminent danger to human life or cause serious injury.\" We decided to examine all pursuits from 2003. We found that 67 percent of the pursuits appeared to have violated the written policy. Additionally, pursuits of suspects wanted for minor traffic violations led to the most accidents and injuries. We started with the original paper records kept by the police department. We photocopied all 229 of the pursuit report forms and narratives. We created a Microsoft Access database with the fields we wanted to track. If we had requested an electronic database from the department, we wouldn't have found the story. The database they kept of chases didn't match the paper records; it showed far fewer fast chases and far more slow chases than the records indicated. Using information from the narratives, we categorized the chases based upon what the suspect was wanted for when the chase began. The pursuit reporting forms lacked critical detail regarding duration, distance and top speed. Two things we found extremely helpful: a memo field in which we could note brief highlights, and unique identification numbers for each chase. We also kept track of the gender and race, finding that 63 percent of those pursued were black. Later, we used ESRI ArcView 3.2 geographic information system to map the starting locations and times of the pursuits. After the story ran, the new police chief announced changes to the pursuit policy. Now, supervisors have the responsibility to authorize each pursuit and grant continuing permission as conditions change.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-setting-up-stocking-your-data-warehouse","slug":"2005-january-february-setting-up-stocking-your-data-warehouse","url":"https://dwillis.github.io/uplink/article/2005-january-february-setting-up-stocking-your-data-warehouse.html","year":2005,"month":"January • February","monthNum":1,"headline":"Setting up, stocking your data warehouse","kicker":"OPEN SOURCE","author_name":"Aron Pilhofer","author_title":"Database Editor","affiliation":"The Center for Public Integrity","summary":"This guide explains how to administer a MySQL database for newsroom use and how to connect it to Microsoft Access to provide a familiar interface for reporters.","keywords":["MySQL","database security","open source"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["MySQL","Access","SQL Server","FoxPro","Navicat","MySQL Administrator","MySQL Query Browser"],"full_text":"If you followed the instructions in the November-December 2004 Uplink, a MySQL server should be running on your system or network, though you would hardly know it. The only evidence you'll see is an entry for mysqld or mysqld-nt in the Windows Task Manager processes tab.\n\nSo how do you run MySQL and work with data?\n\nMySQL is different from a standalone database manager like Microsoft Access. The programs (or clients) that you use to connect to the database engine (or server) are separate.\n\nTechnically speaking, any software interface used to connect to a database is a client – that includes Web pages or even the DOS command line. In order to set up our newsroom database system, we want something a little more specialized: a client to help manage our server, data and system users.\n\nSome database servers come packaged with an official client (Enterprise Manager, for example, in the case of Microsoft SQL Server), but not MySQL. Due to the explosion of interest in MySQL, there are plenty of excellent options.\n\nMany of the best are free, open source or very inexpensive shareware. If you have the money, Navicat (www.navicat.com) is worth it every penny of its $100 price for its unparalleled speed, stability, import filters and feature set.\n\nMySQL will import ASCII text files at mind-bending speeds\n\nWe'll continue building our system with a pair of newer no-cost entries – MySQL Administrator and MySQL Query Browser – from the company that makes MySQL. The former program is designed for administering the database and the latter for running queries. Download both from http://dev.mysql.com.\n\nAfter downloading, install the programs and accept the defaults. Launch MySQL Administrator. Enter \"root\" for the user name and then the password that you selected during the initial installation of MySQL.\n\nIf the host field is empty, enter \"localhost\" if you are working on the computer that's running MySQL. If you're not, type the IP address of the server. The port field should default to 3306, but if it's empty you can enter that. Click OK and you should be greeted with a page showing some basic information about your MySQL installation.\n\nYou can use this interface for most of the tasks necessary to keep your MySQL data warehouse up and running. Many of the functions available through the interface are helpful in enhancing the program's performance. That's outside the scope of this article, but you can learn more by reading the reference manual at http://dev.mysql.com/doc/administrator/en.\n\nUsing this interface, you can do just about anything you need to do to keep your MySQL data library up and running. Most of the functions are designed to help tweak MySQL and get the most performance possible, and much of that is beyond the scope of this article. MySQL Administrator is extremely well documented: http://dev.mysql.com/doc/administrator/en.\n\nSecurity first\n\nOne of the big advantages of moving your newsroom data from standalone Access or Microsoft FoxPro databases into a centralized MySQL database system is you can better regulate security. As newsrooms gather more and more and more reporters start doing their own data work, maintaining your newsroom database collection can become a logistical nightmare. More importantly, when anyone can open, change, delete, update and edit data, it becomes impossible to ensure the integrity of the data.\n\nMySQL, like all database servers, allows you to control who has access to which databases, and what they can do with the data. You can give certain users access to only certain databases, specific tables within those databases, or even particular fields.\n\nYou can use MySQL Administrator to easily create, manage and administer user accounts by pointing and clicking. (See Chapter 9 of the user manual for detailed instructions). You can also perform these functions with the MySQL command line (see http://dev.mysql.com/doc/mysql/en/User_Account_Management.html). You can create a user-permissions scheme to reflect the expertise groupings among your newsroom staff and assigned shared users names to these groups.\n\nConnecting users\n\nOnce you have your user scheme in place, you can connect users with a client program designed for end users. There are a number of excellent client programs available; Query Browser (above) is good for making a more robust connection to the database, though the program is still in early development. In addition, users must run queries using Structure Query Language. (For other options, read \"Putting a useful face on MySQL\" in the May-June 2003 Uplink).\n\nTo connect users using one of the many MySQL-specific clients, follow the same procedure you did with Administrator. Enter the host name of the server (usually \"localhost,\" or an IP address), as well as the user name and password – that's it.\n\nBut the easiest client for users initially is one you probably already have on your newsroom computers: Access. After going through a simple setup, users can link to data tables stored in MySQL and then write and save queries on those tables just like any other Access table. The only thing users cannot do from Access – and this is a problem with Open Database Connectivity (ODBC), not MySQL – is create or alter a table in any way. That is an operation that must be done through Query Browser or another MySQL-specific client.\n\nTo use Access as a front end, first you will need to make sure you have the proper drivers installed for each user who wants to connect to MySQL using Access. Here's how:\n\n• Open Windows Control Panel and go to Administrative tools\\Data sources.\n\n• Click on the drivers tab and look for MySQL ODBC driver. If it is there, skip the next two steps.\n\n• If the driver is not there, download it from: www.mysql.com/products/connector/odbc.\n\n• Install the driver on all PCs that will use Access to query data stored in MySQL.\n\nThen you will need to configure the ODBC driver:\n\n• Open Windows Control Panel and go to Administrative tools\\Data sources.\n\n• Double-click on the Data sources icon and then select either the user (data source name) or system DSN tab (either will work, though you may not be able to save system DSN connections, depending on Windows security settings.\n\n• Click Add and then find the MySQL driver, and double click on it.\n\n• In the dialog box that opens, give your connection a name that your users will be able to understand. Enter the IP address of the server, or \"localhost\" if MySQL is running on the local machine. Enter the name of the MySQL database that you want to connect to. By default, MySQL installs two databases: MySQL and Test. MySQL is the database that contains all the user, password and login information – never, ever, ever give anyone access to this database. Choose Test.\n\n• Enter the user name and password you would like this user to log in under (one of the accounts we created earlier) and click the test data source button to ensure the connection works. If it doesn't, you might have to go back to Administrator and grant that user account specific privileges to that database, done under the \"Schema Privileges\" tab in the user accounts section. Once you've successfully connected, your users can begin using tables within the database.\n\nNow the user can use Access' File\\Get external data menu item to link to or import the MySQL tables. For example, to import:\n\n• Select Import from the menu.\n• Select ODBC databases in the Files of type list that appears in the dialog box that opens.\n• Navigate to the DSN list and select the desired data source name\n\nData in\n\nNow that we've connected to the database, we can put data in it.\n\nThere are a number of ways to get data into a MySQL database. The easiest is to export tables from Access, for data tables with 500,000 or fewer records:\n\n• Right click on the table in Access and select Export.\n• Choose ODBC as the file export type\n• Click OK in the next dialog box\n• Select the DSN into which you wish to import\n\nMany of the clients available for download or purchase have data import fil-","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-january-february-learning-to-play-the-access-numbers-game","slug":"2005-january-february-learning-to-play-the-access-numbers-game","url":"https://dwillis.github.io/uplink/article/2005-january-february-learning-to-play-the-access-numbers-game.html","year":2005,"month":"January • February","monthNum":1,"headline":"Learning to play the Access numbers game","kicker":"Tech tip...","author_name":"MaryJo Sylwester","author_title":"","affiliation":"USA Today","summary":"This technical tip explains how to choose and configure numeric field types in Microsoft Access to ensure data like decimals and percentages are stored accurately.","keywords":["Microsoft Access","data types","database design"],"topics":["Technology","Data & Databases"],"technologies":["Access"],"full_text":"You've created a table in Microsoft Access, and you set the data type as number. But every time you type \"1.6,\" it puts \"2\" in the field. This happens because the field properties need to be configured correctly. When you create a field for numeric values, you need to set the Data Type (Text, numeric, date/time, etc.) and Field Properties (Field size and decimal places). If you need whole numbers, choose between Byte (1-255), Integer (-32,768 to 32,768), or Long Integer (up to 2 billion). For numbers with decimals, use Single (up to three decimal places) or Double (more than three). Unless you are a rocket scientist, you'll be fine with Single. Specifying the number of decimal places is important because reducing the display also rounds the stored value. If you have numbers like ZIP codes where you don't want to do calculations, set the Data Type to Text. Access also has pre-set types for Currency and Percentage. Setting as Percentage: If you have a field with percentages in decimal format (such as 0.50 for 50 percent), you can set the Data Type as Number and the Format to Percent.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-january-february-readmetxt","slug":"2005-january-february-readmetxt","url":"https://dwillis.github.io/uplink/article/2005-january-february-readmetxt.html","year":2005,"month":"January • February","monthNum":1,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR provide journalists with resources and databases for analyzing crime statistics and law enforcement data, including the Uniform Crime Report, campus crime data, and guidance on detecting potential underreporting and bias in policing.","keywords":["crime","journalism","data"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":["Uniform Crime Reports","National Crime Victimization Survey"],"full_text":"IRE and NICAR offer resources for journalists who want to use data to probe law enforcement and crime.\n\n\"Understanding Crime Statistics: A Reporter's Guide,\" by Kurt Silver is an IRE Beat Book that provides detailed guidance for analyzing crime report databases. The book covers use of the FBI's Uniform Crime Report data and the National Crime Victimization Survey. Order the book online at www.ire.org/store/books/crime.html or call 573-882-3364.\n\nJournalists can purchase Uniform Crime Report and U.S. campus crime data from the Database Library. UCR data includes data on index crimes – murder, non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny-theft, motor-vehicle theft and arson – by law enforcement agency, and detailed information about homicides. For more information see www.nicar.org/data/fbi. The Database Library has the data for 1980, 1990 and 1993-2002.\n\nThe campus crime data from the U.S. Department of Education contains information about alleged criminal incidents reported to campus or local law enforcement agencies. The data covers 2001-02 and includes data about arrests that result in campus disciplinary proceedings. For more information see www.ire.org/datalibrary/databases/campus.\n\nContact the Database Library at 573-884-7711 or 573-884-7332 to order the data.\n\nTipsheet No. 1196 by journalists from The Philadelphia Inquirer provides advice for using the Uniform Crime Report data to examine whether police are underreporting rapes.\n\nJohn Archibald of The Birmingham News lists resources for journalists who want to use crime and justice data for looking at law enforcement (Tipsheet No. 2056).\n\nKarisa King of the San Antonio Express-News provides guidance for examining traffic stop data to detect possible racial disparities. (Tipsheet No. 2094).\n\nOrder tipsheets by contacting the IRE Resource Center at 573-882-3364 or rescntr@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-january-february-readmetxt-2","slug":"2005-january-february-readmetxt-2","url":"https://dwillis.github.io/uplink/article/2005-january-february-readmetxt-2.html","year":2005,"month":"January • February","monthNum":1,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR provide resources and databases for journalists to analyze crime statistics and law enforcement data, including the Uniform Crime Report, campus crime data, and guidance on detecting issues like rape underreporting and racial disparities in policing.","keywords":["journalism","crime","data"],"topics":["Crime & Justice","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"IRE and NICAR offer resources for journalists who want to use data to probe law enforcement and crime.\n\n\"Understanding Crime Statistics: A Reporter's Guide,\" by Kurt Silver is an IRE Beat Book that provides detailed guidance for analyzing crime report databases. The book covers use of the FBI's Uniform Crime Report data and the National Crime Victimization Survey. Order the book online at www.ire.org/store/books/crime.html or call 573-882-3364.\n\nJournalists can purchase Uniform Crime Report and U.S. campus crime data from the Database Library. UCR data includes data on index crimes – murder, non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny-theft, motor-vehicle theft and arson – by law enforcement agency, and detailed information about homicides. For more information see www.nicar.org/data/fbi. The Database Library has the data for 1980, 1990 and 1993-2002.\n\nThe campus crime data from the U.S. Department of Education contains information about alleged criminal incidents reported to campus or local law enforcement agencies. The data covers 2001-02 and includes data about arrests that result in campus disciplinary proceedings. For more information see www.ire.org/datalibrary/databases/campus.\n\nContact the Database Library at 573-884-7711 or 573-884-7332 to order the data.\n\nTipsheet No. 1196 by journalists from The Philadelphia Inquirer provides advice for using the Uniform Crime Report data to examine whether police are underreporting rapes.\n\nJohn Archibald of The Birmingham News lists resources for journalists who want to use crime and justice data for looking at law enforcement (Tipsheet No. 2056).\n\nKarisa King of the San Antonio Express-News provides guidance for examining traffic stop data to detect possible racial disparities. (Tipsheet No. 2094).\n\nOrder tipsheets by contacting the IRE Resource Center at 573-882-3364 or rescntr@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-january-february-readmetxt-3","slug":"2005-january-february-readmetxt-3","url":"https://dwillis.github.io/uplink/article/2005-january-february-readmetxt-3.html","year":2005,"month":"January • February","monthNum":1,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR provide journalists with resources including databases, guidebooks, and tipsheets to analyze crime statistics and law enforcement data, such as FBI Uniform Crime Reports and campus crime data. These tools help journalists investigate issues like underreported crimes, racial disparities in traffic stops, and other law enforcement practices.","keywords":["data","crime","journalists"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["Uniform Crime Reports","National Crime Victimization Survey"],"full_text":"IRE and NICAR offer resources for journalists who want to use data to probe law enforcement and crime.\n\n\"Understanding Crime Statistics: A Reporter's Guide,\" by Kurt Silver is an IRE Beat Book that provides detailed guidance for analyzing crime report databases. The book covers use of the FBI's Uniform Crime Report data and the National Crime Victimization Survey. Order the book online at www.ire.org/store/books/crime.html or call 573-882-3364.\n\nJournalists can purchase Uniform Crime Report and U.S. campus crime data from the Database Library. UCR data includes data on index crimes – murder, non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny-theft, motor-vehicle theft and arson – by law enforcement agency, and detailed information about homicides. For more information see www.nicar.org/data/fbi. The Database Library has the data for 1980, 1990 and 1993-2002.\n\nThe campus crime data from the U.S. Department of Education contains information about alleged criminal incidents reported to campus or local law enforcement agencies. The data covers 2001-02 and includes data about arrests that result in campus disciplinary proceedings. For more information see www.ire.org/datalibrary/databases/campus.\n\nContact the Database Library at 573-884-7711 or 573-884-7332 to order the data.\n\nTipsheet No. 1196 by journalists from The Philadelphia Inquirer provides advice for using the Uniform Crime Report data to examine whether police are underreporting rapes.\n\nJohn Archibald of The Birmingham News lists resources for journalists who want to use crime and justice data for looking at law enforcement (Tipsheet No. 2056).\n\nKarisa King of the San Antonio Express-News provides guidance for examining traffic stop data to detect possible racial disparities. (Tipsheet No. 2094).\n\nOrder tipsheets by contacting the IRE Resource Center at 573-882-3364 or rescntr@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-january-february-readmetxt-4","slug":"2005-january-february-readmetxt-4","url":"https://dwillis.github.io/uplink/article/2005-january-february-readmetxt-4.html","year":2005,"month":"January • February","monthNum":1,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR provide resources and databases for journalists investigating crime statistics and law enforcement, including crime report analysis guides, FBI Uniform Crime Report data, and specialized tipsheets on topics like rape underreporting and racial disparities in traffic stops.","keywords":["crime","data","journalism"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":["Uniform Crime Reports","National Crime Victimization Survey"],"full_text":"IRE and NICAR offer resources for journalists who want to use data to probe law enforcement and crime.\n\n\"Understanding Crime Statistics: A Reporter's Guide,\" by Kurt Silver is an IRE Beat Book that provides detailed guidance for analyzing crime report databases. The book covers use of the FBI's Uniform Crime Report data and the National Crime Victimization Survey. Order the book online at www.ire.org/store/books/crime.html or call 573-882-3364.\n\nJournalists can purchase Uniform Crime Report and U.S. campus crime data from the Database Library. UCR data includes data on index crimes – murder, non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny-theft, motor-vehicle theft and arson – by law enforcement agency, and detailed information about homicides. For more information see www.nicar.org/data/fbi. The Database Library has the data for 1980, 1990 and 1993-2002.\n\nThe campus crime data from the U.S. Department of Education contains information about alleged criminal incidents reported to campus or local law enforcement agencies. The data covers 2001-02 and includes data about arrests that result in campus disciplinary proceedings. For more information see www.ire.org/datalibrary/databases/campus.\n\nContact the Database Library at 573-884-7711 or 573-884-7332 to order the data.\n\nTipsheet No. 1196 by journalists from The Philadelphia Inquirer provides advice for using the Uniform Crime Report data to examine whether police are underreporting rapes.\n\nJohn Archibald of The Birmingham News lists resources for journalists who want to use crime and justice data for looking at law enforcement (Tipsheet No. 2056).\n\nKarisa King of the San Antonio Express-News provides guidance for examining traffic stop data to detect possible racial disparities. (Tipsheet No. 2094).\n\nOrder tipsheets by contacting the IRE Resource Center at 573-882-3364 or rescntr@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-marchapril-to-share-or-not-to-share","slug":"2005-marchapril-to-share-or-not-to-share","url":"https://dwillis.github.io/uplink/article/2005-marchapril-to-share-or-not-to-share.html","year":2005,"month":"March/April","monthNum":3,"headline":"To share or not to share","kicker":"CAR TOPICS","author_name":"Brian Hamman","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists face dilemmas on whether to share data with others, balancing transparency against organizational interests.","keywords":["data sharing","journalism","transparency"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":[],"full_text":"After Matthew Waite wrote stories about rising home sale prices for the St. Petersburg Times, real estate agents lined up to buy the data he used for his reporting. Doug Haddix, projects editor at The Columbus Dispatch, turned down a request by a state agency that sought data from the newspaper. Officials said the agency would have to spend $60,000 to $70,000 to create the data itself.\n\nThese stories shine light on a dilemma that journalists doing computer-assisted reporting sometimes face: Whether to share data with others.\n\nDeciding whether to reject requests for data is not always simple, according to several CAR experts. Journalists need to weigh standards of fairness and transparency against the potential harm to the reputation or bottom line of the organization, they say.\n\nWhile most journalists agree that it's important to remain transparent when their work comes under criticism, they disagree on how much journalists need to share for the sake of fairness.\n\nTom Torok, chief database editor for The New York Times, says when journalists analyze data and present their results, they act as social scientists.\n\n\"If we do original analysis, then we have to adhere to the standards of scientific analysis, which is replication,\" he says.\n\nWhen social scientists do research, they traditionally offer the data to others in a central repository to allow others to verify their conclusions and perform additional analysis.\n\nWhile Torok says he does not necessarily advocate giving data to anyone who requests it - he might not give it to a competitor or a business whose intent is purely commercial - journalists should give data to those who challenge the results. Torok says the Times recently handed over two datasets to government agencies, after the newspaper ran critical stories about railroad accidents and the Occupational Safety and Health Administration.\n\nBill Dedman, of The Boston Globe, also says making the data available helps boost credibility. \"There's a point where if you don't give people your data, then they have no reason to believe your story.\"\n\nHow transparent?\n\nHaddix disagrees that it is necessary to release data to justify it. He says the Dispatch routinely runs its analysis methods by statisticians and other experts. \"It's not really peer-review in the scientific sense, but it works for our purposes.\"\n\nWaite also disagrees with the characterization of journalists as social scientists. \"We are not producing peer-reviewed papers here, we are in a competitive business.\"\n\nIn cases in which real questions may arise with a story, several journalists say it is better to place the data online for everyone to view, rather than provide it only to organizations that request it.\n\n\"Our obligation of transparency is to the public, not the government,\" says Brian Bowling, a reporter for the Pittsburgh Tribune-Review.\n\nSome journalists say they are concerned that offering such transparency with their data could undermine shield law protection for journalist's notes and interviews.\n\nSandy Davidson, professor of journalism and law at the University of Missouri, says journalists do not need to worry. She compared sharing data to sharing tools on a farm.\n\n\"Just because I say that you can borrow my shovel does not mean you can come and take the wheelbarrow,\" Davidson said.\n\nAdding value\n\nGenerally, however, when someone requests data from a news organization it isn't because they disagree with a story. Journalists add value to data by analyzing it, and that analysis can provide a shortcut to other organizations.\n\nHaddix says that extra value is the primary reason the Dispatch often declines to share data.\n\n\"It's material that belongs to the Dispatch. We've put a lot of analysis into the data and we just don't feel that we should give that away,\" he says.\n\nIn addition to protecting the news organization's bottom line, journalists need to be concerned about appearing to be working with government.\n\n\"As a citizen, I want government to go out and dig up its own information, carry out its own analysis and then present all that before it deliberates toward a decision,\" Bowling says. \"News media are not and never should be agents of the government.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-marchapril-analysis-shows-feds-claims-about-charity","slug":"2005-marchapril-analysis-shows-feds-claims-about-charity","url":"https://dwillis.github.io/uplink/article/2005-marchapril-analysis-shows-feds-claims-about-charity.html","year":2005,"month":"March/April","monthNum":3,"headline":"Analysis shows feds' claims about charity","kicker":"SPOTLIGHT: THE WAR AT HOME & ABROAD","author_name":"Jaimi Dowdell","author_title":"","affiliation":"IRE and NICAR","summary":"Social network analysis tools reveal links of the Islamic American Relief Agency to terror activities, highlighting investigative techniques using data.","keywords":["social network","charity","terror"],"topics":["Investigative Methods","Technology","Data & Databases"],"technologies":["UCINET"],"full_text":"For more than a year, the Islamic American Relief Agency (IARA) existed in my world as a series of documents, lines, dots and connections. When the FBI raided and the U.S. Treasury Department froze the assets of the Columbia, Mo., charity it was evident that the U.S. war on terror was active in mid-Missouri. It also provided an opportunity to test social network analysis as a computer-assisted reporting tool.\n\nSocial network analysis and IARA have been my obsession on-and-off since late 2003. I spent a good deal of that year reading about social network analysis and fiddling with UCINET for Windows software from Analytic Technologies. While journalists have traditionally used data to analyze specific details, or attributes about certain individuals or things,","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-marchapril-peace-accidents","slug":"2005-marchapril-peace-accidents","url":"https://dwillis.github.io/uplink/article/2005-marchapril-peace-accidents.html","year":2005,"month":"March/April","monthNum":3,"headline":"Peace Accidents","kicker":"SPOTLIGHT:","author_name":"Michael Fabey","author_title":"military reporter","affiliation":"Savannah Morning News","summary":"Military aviation accident data reveals insights into safety issues, stressing the importance of data analysis in investigative reporting.","keywords":["aviation accidents","military","safety"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["SPSS","Access","dBase","FoxPro"],"full_text":"When I asked for the U.S. Army's database of aviation accidents in June 2004, I was hoping to find out the accident numbers, scenarios and other details of the service's helicopter fleet.\n\nAs the military reporter at my newspaper, I cover the locally based 3rd Infantry Division, whose soldiers earned fame for their lightening-fast march from the Kuwaiti border to the heart of Baghdad during the first weeks of the 2003 invasion.\n\nThe division had suffered its first fatal Apache crash and a cursory look at the history of the helicopter promised a potential gold mine of stories.\n\nI requested a record layout of the Army's mishap database from the Army Safety Center at Fort Rucker, Ala.\n\nI had learned about the existence and availability of the database at an Investigative reporting event.\n\nBut which country had the most? During what years?\n\nI determined the worst years and places for accidents were during the Cold War in the United States, Germany and Korea.\n\nUsing fields such as the severity of accident and type, I could do an even more detailed analysis, especially with SPSS, a statistical analysis program.\n\nBut, as David Donald, a former editor for the Savannah Morning News who is now the IRE training director, explained, SPSS does not like Access database files. So I exported selected fields from the main file to a dBASE file and used Microsoft FoxPro to check it. I learned on FoxPro a decade ago and feel more comfortable with it. Also, working with FoxPro does make it easier to go back and forth with SPSS.\n\nI used SPSS to run frequencies and discovered soldiers were more likely to be involved in more dangerous ground accidents than in those in the air.\n\nThe story we printed went a long way to debunk the myth that the Army and other services were trying to perpetuate: more accidents are bound to happen during wartime as more is demanded of warriors and their machines. An analysis of the Army's own databases showed just the opposite to be true.\n\nBut the data and story did show that some safety maxims in the civilian world held true in the military one as well:\n\nFlying was safer than traveling by car or in the Army's case, by Humvee.\nNot wearing a seatbelt can have deadly consequences.\nFailure to follow regulations, ignoring simple precautionary measures and other types of human error lead to more accidents than equipment failure.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-marchapril-oreilly-joins-ire-team","slug":"2005-marchapril-oreilly-joins-ire-team","url":"https://dwillis.github.io/uplink/article/2005-marchapril-oreilly-joins-ire-team.html","year":2005,"month":"March/April","monthNum":3,"headline":"O'Reilly joins IRE team","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"Richard O'Reilly, a veteran computer-assisted reporting expert from the Los Angeles Times, has joined IRE and NICAR as a consultant to assist with special database projects and analysis for media organizations.","keywords":["databases","journalism","analysis"],"topics":["Investigative Methods","Journalism & Newsroom","Data & Databases"],"technologies":[],"full_text":"Veteran computer-assisted reporting expert Richard O'Reilly has joined IRE and NICAR as a consultant for special database projects. O'Reilly, who worked 30 years at the Los Angeles Times, will collaborate with the IRE and NICAR database library staff on analysis for broadcast and print organizations.\n\nO'Reilly was the CAR coordinator at the Times and worked on numerous projects involving data analysis. In addition, O'Reilly has assisted IRE and NICAR by speaking and teaching at training events and conferences. He will work directly with Database Library Director Jeff Porter.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-marchapril-updated-data","slug":"2005-marchapril-updated-data","url":"https://dwillis.github.io/uplink/article/2005-marchapril-updated-data.html","year":2005,"month":"March/April","monthNum":3,"headline":"Updated data","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"The FDA's Adverse Event Reporting System tracks safety issues with pharmaceuticals and biological products to guide regulatory actions and inform public health reporting. Journalists can access this database as a resource for investigating consumer medical issues and the pharmaceutical industry.","keywords":["FDA","pharmaceuticals","safety"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"The U.S. Food and Drug Administration relies on the Adverse Event Reporting system to flag safety issues and identify pharmaceuticals or therapeutic biological products (such as blood products), for further epidemiological study. It may ultimately prompt regulatory responses such as drug labeling changes, letters to health care professionals, or market withdrawals.\n\nJournalists who have used this data recommend using it to guide reporting on consumer medical issues, the FDA or the pharmaceutical industry. This data is current as of December 2003. For more information, see www.ire.org/datalibrary/databases/AERS. To order the data, call 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-marchapril-war-stories","slug":"2005-marchapril-war-stories","url":"https://dwillis.github.io/uplink/article/2005-marchapril-war-stories.html","year":2005,"month":"March/April","monthNum":3,"headline":"War Stories","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"A journalism newsletter highlights how reporters have used data analysis to investigate Iraq War-related stories, including military spending, casualties, and homeland security, while announcing plans to integrate the newsletter more closely with NICAR's website.","keywords":["data","reporting","war"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"Two years ago U.S. forces pushed across the Kuwait border into Iraq, starting the war that continues today. Since then, we've kept you posted about the best stories about the war at home and abroad in the pages of Uplink. We've also been trying to spotlight data you can use in your own reporting, and bring you tips from journalists who have already done so.\n\nWe have provided a comprehensive list of data resources (March-April 2003) and details about analyzing federal procurement data to find military contractors near you (May-June 2003).\n\nYou have responded by producing dozens of significant stories that look at war casualties, homeland security spending, border and airport vulnerability, terrorism prosecutions and immigration - all based on analysis of databases.\n\nThis issue of Uplink looks at some of those stories: the war burden in South Carolina, alleged charity-terrorist ties and noncombat deaths. I know there will be many more great stories.\n\nOn an unrelated note, in the coming months we are planning to link Uplink's printed edition more closely with NICAR's Web site (www.nicar.org). Stay tuned for further developments and be sure to send me a message if you have ideas of how we can better serve you with the newsletter and the Web.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-marchapril-upcoming-hands-on-car-training","slug":"2005-marchapril-upcoming-hands-on-car-training","url":"https://dwillis.github.io/uplink/article/2005-marchapril-upcoming-hands-on-car-training.html","year":2005,"month":"March/April","monthNum":3,"headline":"UPCOMING HANDS-ON CAR TRAINING","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer multiple hands-on training programs in computer-assisted reporting, including weeklong boot camps in Columbia, Missouri, and specialized mini-boot camps for managers and data mapping instruction.","keywords":["training","journalism","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet","GIS"],"full_text":"IRE and NICAR have several training opportunities for journalists seeking hands-on instruction in using computer-assisted reporting.\n\nThere are two weeklong Boot Camps in Columbia, Mo., for journalists who want to learn how to acquire electronic information, use spreadsheets and database to analyze the information, and to translate that information into high-impact stories.\n\nIRE and NICAR provide follow-up help after participants return to their news organizations. The boot camps are May 15-20 and Aug. 7-12.\n\nNews managers interested in learning how to better integrate CAR into their newsrooms can attend the April 8-10 mini-Boot Camp in Columbia. Participants will learn enough about spreadsheets and database managers to understand the programs and effectively manage journalists who use the programs in their reporting and analysis.\n\nJournalists interested in learning how to map data for news stories can take advantage of a min-Boot Camp in using geographic information systems (GIS) Aug 19-21 in Columbia.\n\nFor more complete information about these and other IRE and NICAR training opportunities, visit www.ire.org/training.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-marchapril-uncovering-an-age-old-scam","slug":"2005-marchapril-uncovering-an-age-old-scam","url":"https://dwillis.github.io/uplink/article/2005-marchapril-uncovering-an-age-old-scam.html","year":2005,"month":"March/April","monthNum":3,"headline":"Uncovering an age-old scam","kicker":"MAPPING","author_name":"Ron Hurtibise","author_title":"Investigative Reporter","affiliation":"Daytona Beach News-Journal","summary":"An investigation into eBay sales of Florida swampland exposes repeated real estate scams facilitated by online auctions.","keywords":["real estate","scam","eBay"],"topics":["Investigative Methods","Business & Economy","Government & Politics"],"technologies":["GIS"],"full_text":"Interested in buying some Florida swampland?\n\nIf you have any idea how much prime real estate in Central Florida sells for these days, you'd probably think twice before believing you could spend a mere $5,000 to $7,000 an acre for your future retirement home site.\n\nThat's why our newspaper's investigative team was surprised that many people are still spending thousands of dollars for soggy, inaccessible land they'll likely never be able to develop or sell at a profit.\n\nIn Volusia County, between Daytona Beach and DeLand, eBay auctioneers are amassing small fortunes by peddling lots that had been targeted for purchase by the county's wetlands acquisition program. The sales are pricing the county out of the market, creating an uncertain future for a vital water recharge area and setting up unsuspecting buyers for a bitter reality check.\n\nWe uncovered the story, in part, by analyzing property data with a geographic information system (GIS) program.\n\nThe News-Journal had played a prominent role exposing boiler room swampland scams to a national audience in the 1960s, but it took an alert reader and a few minutes on eBay to tip us off that the cycle is repeating itself in the Internet age.\n\nIn April 2003, I received an e-mail from a real estate agent in Flagler County, on the north side of our coverage area, alerting me to eBay sales of subdivision lots that could not be developed. So I checked it out: Sure enough, ads touting \"beautiful Florida land\" filled eBay's fledgling Real Estate auction pages.\n\nA call to Flagler County's zoning department confirmed that buyers of lots in a \"paper subdivision\" called Flagler Estates stood little chance of getting building permits because there were no roads or utilities available.\n\nThe situation they described was fascinating, and totally new to me: Back in the 1960s, a supposed development company bought a large tract of land, carved it into hundreds of lots, named it Flagler Estates and sold the lots as \"investment properties\" for as much as $2,000 or $3,000 apiece, plus financing charges.\n\nBuyers, usually retirees from the north, bought the parcels sight unseen on installment plans from telephone salespeople, and then often stopped paying local property taxes on the lots after realizing they'd been duped. The lots reverted to the county, which sold them at tax sales for as little as $300 an acre. Buyers of the auctioned properties then made thousands of dollars selling them on eBay.\n\nAfter we ran a story on Flagler Estates, a Volusia County resident told me that eBay sales of lots in paper subdivisions was much more widespread than we had realized or reported, involving half a dozen large tracts in the Daytona Beach area and others throughout the state.\n\nThe most blatant example was University Highlands, a 9,400-acre watery no-man's-land between Daytona Beach and DeLand. Dozens of yellowed clippings in the newspaper's archives told the story of how the subdivision was created:\n\nLax 1960s-era county development regulations enabled the Firstamerica Development Corp. to plat a subdivision with no roads, sewers, drainage ditches or utilities; set up a telephone boiler room and market the lots as investment parcels to out-of-state buyers.\n\nThe county's zoning board, alarmed by the prospect of development on what was even then recognized as an important water recharge area, tried to rezone the tract for conservation in the mid-1960s. Faced with the prospect being unable to market the rezoned lots as investment properties, Firstamerica threatened legal action, contending the zoning board had overstepped its authority. After the Volusia County Commission refused to fund any legal defense of the rezoning, all five zoning board members resigned and the rezoning was rescinded.\n\nIn a way, the mess created by Firstamerica was good for conservation. With ownership of the subdivision's lots divided among thousands of owners across the nation, no developer was interested in spending the time, money and effort to assemble enough lots to create a profitable development plan - especially with larger, cheaper, drier tracts available elsewhere. Today, according to county planners, residential development on a commercial scale would require acquisition of contiguous parcels and most likely a replatting of any portion submitted for development.\n\nBut buyers who dreamed of one day building on their Florida dream lots didn't know this.\n\nWe guessed that few of today's eBay buyers were aware that prospects for developing the area remain just as slim. We set out to change that by mapping parcel data.\n\nThe availability of local land recording and appraisal data made it possible for us to identify the most prolific swampland resellers and show that some were making as much as 1,900 percent profit selling lots.\n\nI used the Clerk of Courts' Web site (www.clerk.org) to identify who was buying and selling the most lots in University Highlands, how much they had paid for them, who the buyers were, and the selling price. I copied parcel numbers from the eBay ads and pasted them into the search field of the clerk's public records database. The database stores property transaction data and images of nearly all documents recorded at the Volusia County Courthouse since the mid-1990s.\n\nThe data revealed that many of the buyers had Asian surnames, leading us to suspect many were immigrants most likely unfamiliar with the historic risks of buying inexpensive Florida land.\n\nAt this point, we knew we had a powerful story. But I wanted to give readers a visual sense of the size of the tract and the high volume of sales activity there. From two highways that flank it, the tract looks like nothing more than undeveloped woods. Road maps show nothing there. Only a parcel record map, never seen by the average reader, could convey what the Firstamerica Development Corp. created.\n\nI set out to create the most cumbersome graphic I've ever pitched - a map of the entire 9,400-acre, 6,167-lot tract that pinpoints all lots that changed hands since its discovery by the eBay sellers.\n\nFor $30, the Volusia County property appraiser sold us a copy of its entire database - with 352,000 records - in a Microsoft Access file, burned onto a CD within 24 hours.\n\nTyping University Highlands subdivision numbers in the WHERE statement of my Structured Query Language query, I built a table of property sales in the subdivision over the past three years. I converted the Access query result to a Microsoft Excel spreadsheet table, and then exported that as a tab-delimited text file.\n\nFrom the property appraiser's Web site, I downloaded aerial photographs of the subdivision that illustrated the size of the tract and the lack of development there since the 1960s. The photo, shot by the Florida Department of Transportation and made available in a Seamless Image Database (.sid) file format, was embedded with geographic coordinates enabling its precise placement in ESRI's ArcView 8 mapping program.\n\nFrom the county's geographic information system Web site, I downloaded a shapefile that displayed the original plat and all of the lots.\n\nI overlaid the parcel map onto the aerial image, then joined the property identification number field in my recent-sales table to the property ID field in the parcel map's attribute table. I sorted the joined attribute table in descending order so the recent sales records were grouped at the top. I selected those tables so the recently-sold lots appeared highlighted in blue on my map.\n\nI created a new map layer from the selected lots, then re-sorted the attribute table of the new layer to isolate lots purchased by the county for conservation. After creating yet another map layer of just the conservation lots, I shaded them yellow and red so readers could see how the private sales, shaded in blue, were encroaching upon the taxpayer-funded conservation effort.\n\nI exported the graphic as a camera-ready .JPG file that required minimal work from the art department before it was ready for publication in an information graphic. To bolster my pitch, I created a mock-up of how the map could look as a stand-alone graphic, with its own headline and text box. Atop the map, I pasted fact boxes pointing out five examples of quick-turn sales for big profits.\n\nThe Sunday editors liked my mock-up and approved the graphic at nearly a full page inside, with color.\n\nNone of the major eBay sellers agreed to be interviewed for the story. I emailed them through their eBay accounts, and when possible, tried to contact them via addresses and phone numbers filed with the state Division of Corporations.\n\nThe state Division of Land Sales, created 30 years ago to regulate swampland sales, said it was unaware that the practice had been revived. After an initial interview in September, the division chief said he had begun an investigation and could not comment on specific examples.\n\nA retired division investigator living in southwest Florida had no such qualms. He reviewed several examples of eBay ads and pointed out ads that appeared to violate state laws barring sellers from claiming development was imminent and predicting how much a lot's value would increase in the future. And he pointed us to laws requiring sellers of undeveloped subdivision lots to register with the state and submit examples of advertising for review - laws the highest-volume eBay sellers we identified appeared to be ignoring because the state had no record of their registration. We continue to await results of the state's investigation.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-marchapril-big-bucks-flow-to-rural-areas","slug":"2005-marchapril-big-bucks-flow-to-rural-areas","url":"https://dwillis.github.io/uplink/article/2005-marchapril-big-bucks-flow-to-rural-areas.html","year":2005,"month":"March/April","monthNum":3,"headline":"Big bucks flow to rural areas","kicker":"SPOTLIGHT: THE WAR AT HOME & ABROAD","author_name":"Bert Dalmer","author_title":"","affiliation":"The Des Moines Register","summary":"Analysis of Homeland Security grants in Iowa reveals questionable allocation patterns, spotlighting discrepancies in rural versus urban funding.","keywords":["homeland security","funding","Iowa"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Excel"],"full_text":"By now, every journalist with a passing interest in federal or state government has heard the complaints out of New York and Chicago - that America's largest cities are being shortchanged by the federal government in the area of homeland security.\n\nAt a glance, the fussing seems logical. Especially when you discover that a state like Iowa, whose largest city would constitute a neighborhood in Chicago, has been pledged more than $100 million thus far to protect its populace from terrorists.\n\nFew in the media have suggested that police and firefighters don't need equipment to respond to a large-scale attack. But few have bothered to dive into the records to illustrate the problem in terms every reader can understand.\n\nThat was the challenge The Des Moines Register took on in the wake of a flurry of praiseful but cryptic news releases from Iowa congressmen and the U.S. Department of Homeland Security. After a few weeks of background study and another week of crunching the numbers, Iowa's grant decisions were revealed at best to be questionable, and at worst, laughable.\n\nInitially, the task of following the money seemed daunting. The bureaucracies were extensive; the jargon was mind numbing. Not only were we tracking budgets set in Washington, D.C., from Des Moines, but also it was nearly impossible to find anyone who could explain the grant process or to define the government's overall homeland security strategy.\n\nWhen half a dozen phone calls to the Homeland Security press office yielded little more than referrals to old press releases, it was clear that our research time would be better spent close to home.\n\nWe scheduled a pair of meetings with the state's homeland security office just to get the lay of the land. Then we made a verbal request for a county-by-county accounting of all the homeland security grants it had received.\n\nWe received a Microsoft Excel spreadsheet just a few days later.\n\nFor a number of reasons, the grants as a whole defied neat and easy explanation. Many homeland security grants are administered by federal agencies with differing purposes and grant rules. Many traditional grant programs, particularly in the Department of Justice, were taken over by Homeland Security, which offered money for a whole new set of initiatives.\n\nFor ease of explanation, we decided to concentrate our efforts on just one grant program that would be both interesting and relevant. We chose the State Homeland Security Grant Program, which has thus far funneled $28 million to Iowa's first-responders, such as police, firefighters and emergency managers.\n\nUnder this program, the state had wide latitude to decide how the money was to be allocated to its first-responders. Eventually, we learned that the counties also had wide latitude to spend their share of the grant.\n\nIn 2003, the grant's first year of existence, money was awarded in two waves. I asked the state for a further breakdown of the funds and added a column of population statistics from the U.S. Census Bureau.\n\nCurious patterns emerged in the Excel spreadsheet right away. Why were the two grants awarded so differently? Why were some grants spread evenly among counties, while others had skipped some counties altogether?\n\nWhen I asked for an explanation, state officials acknowledged that they had divvied up most of the first state homeland security grants without much prioritization or forethought. By the time of the second grants, a more complex formula was devised, based in part on population, critical infrastructure (like power plants, refineries, skyscrapers and landmarks) and agricultural production.\n\nI asked the state for a further breakdown of the second round of grants.\n\nAfter some basic Excel sorting, the numbers revealed a few eye-opening trends:\n\nAll 99 Iowa counties had received a base share of the grants, and often thousands more without regard to how vulnerable they were to a terrorist attack.\nThe state had kept almost one-third of the grants for administrative costs and statewide purchases, like 98 pairs of night-sight binoculars for park rangers.\nIowa's five largest counties, home to one-third of the state's population, received just 15 percent of the homeland security money. The state's largest county, Polk, shared only $250,000 of its $1.2 million with the state's largest city, Des Moines.\n\nA subsequent analysis of a master equipment request compiled by the state for review by the feds fleshed out the story in more tangible terms. This was the nut graf of the Register story:\n\n\"Iowa counties' first anti-terrorism requests to the federal government include $6,700 in traffic cones, signs and barricades; $21,000 for six squad-car cameras; $4,500 in night-sight equipment; a $500 underwater camera; a $370 wall clock with a built-in hidden camera; and megaphones, pagers, work gloves, rubber boots and paper shredders.\"\n\nFurthermore, money that had been awarded in the name of fighting \"agro-terror\" in one of America's most rural states was often used to buy everyday equipment like radios.\n\nAuthorities in Iowa's most prolific farm county received $114,000 specifically for its agricultural resources, but officials there said the money was too insubstantial to try to protect their crops and livestock.\n\nExperts called the Iowa grant formula politically safe but strategically silly. The state spending plan, they said, paralleled the indecision seen at the federal level, where grants were handed down to the states based only on population - not risk.\n\nA Congressional report issued in the days following the Register story showed that Iowa was among a minority of 21 states that considered some terrorist risk before parceling out federal grants to cities and counties. But Iowa was also one of 25 states that gave money to counties, regardless of the risk.\n\nSounds like Chicago might have a point after all.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-marchapril-data-for-examining-security-spending","slug":"2005-marchapril-data-for-examining-security-spending","url":"https://dwillis.github.io/uplink/article/2005-marchapril-data-for-examining-security-spending.html","year":2005,"month":"March/April","monthNum":3,"headline":"Data for examining security spending","kicker":"","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"The article provides guidance on using federal databases to track Department of Homeland Security spending across states and communities, including contracts, grants, and assistance programs that impact local jobs and funding.","keywords":["security","federal","grants"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":["Federal Assistance Award Data System","Consolidated Federal Funds Report"],"full_text":"Security = money.\n\nThe U.S. Department of Homeland Security came into existence midway through federal fiscal year 2003. Since that time, the agency has poured billions of dollars worth of contract work, assistance and pay into every state and even beyond U.S. borders.\n\nTo follow the money - and show how it impacts your community with jobs and grants - you'll need to get acquainted with several government data sources, all available from the IRE and NICAR Database Library:\n\nFederal contracts - The General Services Administration keeps a database of almost every agency's contract transactions, including Homeland Security. The database includes company name and location, dollar amount, type of work, location of work and other pieces of information. (More details: www.ire.org/datalibrary/databases/fedcontacts)\n\nFederal Assistance Award Data System - If the Department of Homeland Security provides a grant, this database shows the name of the recipient, location, the specific federal program, the amount, date and, often, a project description. (More details: www.ire.org/datalibrary/databases/faads)\n\nConsolidated Federal Funds Report - Unlike the award data system, this report, produced annually, shows the total dollars committed to a specific state, county or city. While it does not identify the recipients (as the previously mentioned databases do), it identifies the agency and specific program, and includes contracts, payroll, grants, loans and other types of transactions. (More details: www.nicar.org/data/cffr)\n\nEach database serves a different purpose. For mapping, Consolidated Federal Funds reports can show county-by-county totals by general spending categories. The Federal Assistance Award Data System can pinpoint recipients and more detailed look at type of recipient (for example, nonprofit, college or university, small business, local government) and often, details about the use of the money. The federal contracts database is a tool to show how businesses are interacting with the Department of Homeland Security.\n\nThese downloadable reports are helpful in understanding more about homeland security grants:\n\nGeneral Accounting Office report, \"Reforming Federal Grants to Better Meet Outstanding Needs,\" #GAO-03-1146T (Search for the report at www.gao.gov).\nThe United States Conference of Mayors, \"Tracking Federal Homeland Security Funds Sent to the 50 State Governments,\" available at www.usmayors.org/72ndWinterMeeting/homelandreport_012204.pdf.\nThe U.S. House Select Committee on Homeland Security, \"An Analysis of First Responder Grant Funding,\" available at http://homelandsecurity.house.gov.\nThe California Institute for Federal Policy Research, \"Federal Formula Grants and California\" www.ppic.org/main/publication.asp?i=481","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-marchapril-ire-and-nicar-resources-for-war-security-coverage","slug":"2005-marchapril-ire-and-nicar-resources-for-war-security-coverage","url":"https://dwillis.github.io/uplink/article/2005-marchapril-ire-and-nicar-resources-for-war-security-coverage.html","year":2005,"month":"March/April","monthNum":3,"headline":"IRE and NICAR resources for war, security coverage","kicker":"SPOTLIGHT: THE WAR AT HOME & ABROAD","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer databases and resources to help journalists investigate federal spending on homeland security and military programs, including tipsheets, story reprints, and access to procurement and assistance award data.","keywords":["homeland","security","databases"],"topics":["Investigative Methods","Government & Politics","Data & Databases"],"technologies":["Internet","spreadsheet"],"full_text":"IRE and NICAR resources for war, security coverage\n\nThe Iraq war and homeland security have been hot issues in news coverage. The IRE and NICAR Database Library and IRE Resource Center have resources for journalists who want do in-depth reporting about these subjects.\n\nFederal Assistance Award Data System (FAADS) contains records of federal assistance awarded to state and local governments as well as all major programs are covered under FAADS. With FAADS and CFFR you can find grants and loans that are awarded to your local or state government for homeland security purposes.\n\nFor more information, log on www.ire.org/datalibrary/databases. Contact the Database Library at 573-884-7711 or 573-884-7332 to order the data.\n\nThe Resource Center offers tipsheets and story reprints that can help guide journalists who want to report on war and security issues using databases.\n\nAdam Bell of The Charlotte Observer provides advice for using the procurement database to analyze military spending and avoid pitfalls along the way. (Tipsheet No. 1765)\n\nMaud Beelman, formerly of the International Consortium of Investigative Journalists, provides a list of Internet and printed resources for following the multinational arms trade in Tip Sheet No. 591.\n\nThe Washington Post featured a story about how the Immigration and Naturalization Service system that keeps tabs on foreign students is riddled with errors. The Post exposed the problems in the system, since scrapped, and showed the flaws can be a risk to homeland security. (Story No. 20463)\n\nThe Oakland Tribune, Los Angeles Daily News and smaller sister Media News Group newspapers in California found that officials in the state misused homeland security funds. After obtaining more than 2,000 pages of purchase order documents from area counties, the newspapers put the data into a spreadsheet. The journalists then compared the data to a master list of identified terrorism targets in California. Some of the smaller counties used the anti-terrorism funding for such minor repairs as fixing courthouse doors, instead of becoming better prepared for future terrorist attacks. (Story No. 21353)\n\nIn \"Anti-Terror Funds Buy Wide Array of Pet Projects,\" The Washington Post traced the path of the region's first wave of homeland security aid from its distribution to its final use. The Post found the path of the aid is largely unexamined by federal regulators. Reporters analyzed contracts, grant proposals and purchasing databases. Results showed millions were spent on items such as leather jackets for police officers. (Story No. 20865)\n\nYou can purchase stories and tipsheets by contacting the Resource Center at 573-882-3364 or rescntr@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-marchapril-simple-calculations-show-scs-war-burden","slug":"2005-marchapril-simple-calculations-show-scs-war-burden","url":"https://dwillis.github.io/uplink/article/2005-marchapril-simple-calculations-show-scs-war-burden.html","year":2005,"month":"March/April","monthNum":3,"headline":"Simple calculations show S.C.'s war burden","kicker":"SPOTLIGHT: THE WAR AT HOME & ABROAD","author_name":"Chuck Crumbo","author_title":"Reporter","affiliation":"The State","summary":"A journalist analyzed Pentagon casualty data to reveal that South Carolina and other small, rural states had disproportionately high Iraq War death rates compared to their populations, confirming earlier studies showing casualties concentrated among troops from economically disadvantaged areas.","keywords":["casualties","demographics","analysis"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":["Excel","World Wide Web"],"full_text":"Every day the Pentagon dispatches e-mails with the words \"DOD Identifies Army Casualty\" in the subject field. Sometimes the e-mails say the casualty is a marine, other times it's a sailor or airman. I read each one.\n\nAfter reading a few hundred of these Department of Defense e-mails it became apparent that troops from small towns in small states like South Carolina were dying in high numbers in Iraq. This, though, was not a spectacular revelation.\n\nJust months into the war and when the death toll was about 350, the Austin American-Statesman reported that 16 percent of the casualties were more likely to come from a county with lower-than-average levels of college education and below-average incomes. Those two demographic categories fit South Carolina to a \"T.\"\n\nIn August, after the 20th South Carolinian had died in the war, we decided to see where the state ranked.\n\nI discussed the idea with Chris Roberts, an assistant business editor and computer-assisted reporting specialist, and together we pulled together a story that ran on the front page of The State last summer.\n\nTo build the spreadsheet, we downloaded the death counts for each state from the constantly updated database maintained by the Associated Press on its Web site for members. We double-checked the AP numbers with the numbers that we compiled for South Carolina and found no discrepancies.\n\nUsing a Microsoft Excel spreadsheet, we copied the state names and number of casualties into two columns. In the next columns we copied state names and state-by-state population counts from Census 2000, which are available at www.census.gov/popest/estimates.php. We made sure the state names from the casualty lists and population numbers lined up, and then deleted the redundant column of state names.\n\nThe next step was math: In one column, we calculated each state's number of casualties per 100,000 residents - a simple equation of the number of dead divided by the state's population, multiplied by 100,000. (We used the deaths-per-100,000-residents calculation as the basis for comparison, because each state has at least 100,000 residents and it is a clear way to account for the wide range of state populations.)\n\nThe next step, in another column, was to use Excel's Rank function on the deaths-per-100,000-residents calculation.\n\nThe results showed that South Carolina, which ranked 26th in population, was eighth in Iraq war deaths.\n\nTwo other southern states - Mississippi and Arkansas - followed South Carolina in rounding out the top 10.\n\nOur reporting confirmed the Austin story as well as other studies done on the Iraq death toll. For example, the office of U.S. Rep. Ike Skelton, a D-Mo., found that 44.3 percent of troops killed came from hometowns with populations under 20,000. A St. Louis Post-Dispatch analysis found that 46 percent of the Americans killed in the war through May 26 came from small towns outside metropolitan areas.\n\nWhat also was interesting was that smaller, so-called \"red\" states that President George W. Bush carried in 2000 and 2004 elections, had war death rates well above the larger \"blue\" states.\n\nWe finished off the reporting by talking to members of the South Carolina congressional delegation and leaders of communities that have suffered casualties.\n\nThe story also questioned whether the disparity in deaths could prompt Congress to reinstate the draft.\n\nThis was a simple story to report, not much different than other statistical analyses we've done on dangerous intersections and highways.\n\nThe data is readily available. All you need is some Excel know-how.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-may-june-charting-test-cheaters","slug":"2005-may-june-charting-test-cheaters","url":"https://dwillis.github.io/uplink/article/2005-may-june-charting-test-cheaters.html","year":2005,"month":"May-June","monthNum":5,"headline":"Charting test cheaters","kicker":"EDUCATION","author_name":"Holly Hacker","author_title":"Reporter","affiliation":"The Dallas Morning News","summary":"An investigation into unusually high test scores at Wilmer Elementary in Texas reveals possible cheating by educators. The analysis shows that scores improved significantly compared to previous years, leading to suspicion of test manipulation.","keywords":["test cheaters","Wilmer Elementary","Texas"],"topics":["Education","Investigative Methods","Data & Databases"],"technologies":["SPSS","SAS"],"full_text":"\"We just worked real hard.\"\n\nThat's how the principal of Wilmer Elementary near Dallas explained her school's astonishing performance on the state third-grade reading test last year.\n\nWilmer finished No. 1 in Texas, ahead of more than 3,000 other schools. Amazing.\n\nA little too amazing, in fact. Standardized test scores are a big deal in Texas, as elsewhere. States and the federal government rate schools based on test scores. In some schools, bonuses or promotions hinge on good results. In past years the state had rated Wilmer low-performing or acceptable, nothing higher. So how could scores climb sky-high in a year?\n\nOur efforts to answer that question turned into a statewide analysis of test scores. We found nearly 400 schools with unusual swings from one test to another, with several cases that pointed to cheating - not by students, but by educators.\n\nIn Wilmer Elementary's case, a state investigation confirmed what our reporting found: Teachers didn't work \"real hard.\" They gave students answers to the test.\n\nWilmer Elementary belongs to the Wilmer-Hutchins school district, where finances, leadership and academics have been in chaos for decades. The district made headlines last summer when it had to delay the opening of a high school because the building had roaches, mold, mildew and a leaking roof.\n\nA host of financial and legal troubles followed. The FBI and Texas Rangers (the law enforcers, not the baseball team) got involved. And then came some good news: high test scores brought an \"exemplary\" state rating to one elementary school and \"recognized\" status to the other two, including Wilmer.\n\nMy colleague covering Wilmer, Josh Benton, didn't believe it. He had heard rumors of cheating before in the district, but nothing concrete. Pointing to Wilmer Elementary's reading scores, he wondered how we could put the dramatic rise into statistical context. Just how likely was it?\n\nEnter SPSS, the statistical software program. I had grown to love and respect SPSS (and statistics, for that matter) at my previous newspaper, the St. Louis Post-Dispatch. We used SPSS to analyze school test scores relative to family income, as several other papers have done.\n\nSPSS and similar programs, such as SAS, also have a wonderful ability to draw pictures of data. You can see stuff with charts and graphs that you wouldn't in rows and columns. So to see how Wilmer Elementary's third-grade reading scores compared to the rest of the state, I plotted each school's scores for 2003 versus the scores for 2004.\n\nEach circle represents a school in Texas that tested at least 30 students. (Smaller numbers can skew results.) The scatterplot shows a trend: Schools do about as well in 2004 as they did in 2003. But see the circle hovering high above everyone else? That's Wilmer Elementary. It went from an average score of 2,240 one year to 2,501 the next. No other school had such a jump.\n\nThen we looked at the scores another way: comparing third-grade reading in 2004 to fourth-grade reading in 2004.\n\nAgain, Wilmer stands out at the top, with the stellar third-grade score but just a so-so fourth-grade score.\n\nBut that wasn't the only school that stood out. Check out the school on the far right, with the near-perfect score for fourth grade but a lousy one for third grade. That's Harrell Budd Elementary in Dallas.\n\nWe ended up looking at reading and math tests, which Texas students take in grades three through 11. (Other subjects are tested only in certain grades.) We examined the scores four ways:\n\nFrom one grade and year to the next, such as third-grade reading 2003 vs. fourth-grade reading 2004.\nFrom one grade to the next for the same year, e.g., third-grade reading 2004 vs. fourth-grade reading 2004.\nThe same grade over two years, e.g., third-grade reading 2003 vs. third-grade reading 2004.\nThe same grade over two subjects, e.g., third-grade reading 2004 vs. third-grade math 2004.\n\nThe analysis went beyond scatterplots and used a statistical tool called regression, which looks for the relationship between two variables. Based on that relationship, regression makes educated predictions. So given a school's score last year, regression determines how well it should perform this year.\n\nIn Wilmer's case, it scored 231 points higher than expected in 2004, based on its 2003 score. To put that in context, most schools scored within 37 points of their predicted score.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-may-june-boat-accident-data-sheds-light-on-dangers","slug":"2005-may-june-boat-accident-data-sheds-light-on-dangers","url":"https://dwillis.github.io/uplink/article/2005-may-june-boat-accident-data-sheds-light-on-dangers.html","year":2005,"month":"May-June","monthNum":5,"headline":"Boat accident data sheds light on dangers","kicker":"SPOTLIGHT: WATER & WATERWAYS","author_name":"Natalya Shulyakovskaya","author_title":"Reporter","affiliation":"The Orange County Register","summary":"An examination of boating accidents on the Colorado River highlights an increase in alcohol-related incidents and hit-and-run accidents, with a focus on the tragic death of local teenagers.","keywords":["boating accidents","Colorado River","alcohol"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["GPS","spreadsheet"],"full_text":"Three local teenagers were killed and one was severely injured in a hit-and-run collision on a narrow stretch of the Colorado River called the Parker Strip at the end of the 2003 summer boating season.\n\nThe Register first reported the story by talking to the kids' friends, boaters, locals and law enforcement officials who patrol the waters on the Arizona and California sides of the river and its tributaries...\n\nUsing the Coast Guard data, I ran queries summarizing data by year, looking at how boating accident trends in California, Arizona and Nevada were different from the national trends, comparing various lakes and rivers, looking at sizes of boats involved in accident.\n\nWhile the number of boating accidents, fatalities and injuries nationally had declined steadily, I found a sharp two-year spike in the hit-and-run accidents and a growing number of alcohol-related accidents and fatalities.\n\nThe increase in hit-and-run accidents turned out to be a product of dirty data. After talking to boating law administrators in about a dozen states with the sharp increases, I learned there had been data entry errors when the U.S. Coast Guard switched to a Web-based reporting form.\n\nSo booze and boating became the focus of my reporting.\n\nI spent the July 4, 2004, holiday weekend on Lake Havasu and Parker Strip. The lake and the strip, although connected by water, seemed like two separate worlds. The lake was open and calm, with a few large boats and partiers congregating in coves. The strip was more crowded, its waves choppier.\n\nAs I wrote my first draft, the story became more about the river and the people who gathered there for its party atmosphere. Soon, a phone call from a relative led me to the accident that became the center of the story.\n\nThe story turned into a tale of two Orange County men who went to the river for a July 4 weekend, where their paths crossed. Both had alcohol in their blood. One of them died in an accident. Another one was arrested and later convicted of negligent homicide. The numbers and trends I got from the data became the frame that held the story together.\n\nI returned to the river with my Garmin eTrex Legend Global Positioning System unit and logged the locations of accidents, the sheriff station, and the hospital. I drove to key locations checking times and distances. I also started noting and logging drinking establishments along the river.\n\nLater, in the office, using the records obtained in my reporting, I created a timeline of the accident. I assembled a spreadsheet tracking witnesses mentioned in police reports and what they said they saw...\n\nYou can read the story, \"A Dangerous Mixture\" online at www.ocregister.com/ocr/2004/10/03/sections/news/news/article_260375.php.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-may-june-finding-overlap-in-public-private-interests","slug":"2005-may-june-finding-overlap-in-public-private-interests","url":"https://dwillis.github.io/uplink/article/2005-may-june-finding-overlap-in-public-private-interests.html","year":2005,"month":"May-June","monthNum":5,"headline":"Finding overlap in public-private interests","kicker":"OUTSOURCING","author_name":"Catherine Rentz Pernot","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists use computer-assisted reporting and database analysis to investigate government contracts with private companies, uncovering connections between major contractors and campaign contributors in Wisconsin and exposing war profiteering through private military companies in Iraq.","keywords":["contracts","database","government"],"topics":["Investigative Methods","Business & Economy","Government & Politics"],"technologies":["Excel"],"full_text":"From security in Iraq to voting machines in the United States, the private sector's work with the government is taking an increasingly visible role in the lives and finances of Americans. Below are three ways journalists have recently reported on government contracts with private business using computer-assisted reporting.\n\nTo investigate the politics behind state government transportation contracts in Wisconsin, reporter Patrick Marley of the Milwaukee Journal Sentinel did a fairly simple analysis of two databases.\n\nFor the article, \"Transportation contracts jumped 45% in 5 years,\" Marley compared five years of historical transportation contract data given to him by the Wisconsin Department of Transportation with a campaign contributions database maintained by a local nonprofit. Once he obtained the data, Marley said it took him less than an hour using Microsoft Excel and a Web-based database to make two large discoveries.\n\nFirst, from the government's contract data, Marley found out that engineering consulting contracts for transportation work leaped 45 percent, to $119.1 million from $82.1 million, over the past five years - several times the rate for all state contracts.\n\nNext, he found that employees of four of the top five recipients of engineering contracts were also some of the largest campaign contributors to state politicians. To compare contributors and contractors, Marley cross-checked state records compiled by a Wisconsin nonpartisan group advocating campaign finance reform, Wisconsin Democracy Campaign, against the contracts data.\n\nTo find change in government contracts, Marley did a simple percent-change calculation in Excel from the contract totals given to him by the state.\n\nThe relationship between the top contributors and contractors was a bit more difficult to crack. Marley began by summing amounts for the contractors. He then plugged in the top contractor names into the nonprofit's Web site, which has a searchable list of donors, to get a feel for how much the top contractors were giving to state officials.\n\nOverall, Marley said the data and the analysis were fairly straightforward. He said obtaining the data from the state also was fairly easy. It took him just two days to receive the Excel file containing the transportation contracts from the DOT with several hundred rows of contract data.\n\nView the article online at www.jsonline.com/news/state/oct04/269824.asp.\n\nWar profits\n\nReporters Joseph Neff and Jay Price of The (Raleigh, N.C.) News & Observer brought the Iraq war close to home by investigating a North Carolina company doing contract work in Iraq. At the same time, their article, \"Contractors in Iraq make costs balloon,\" exposed the private and complicated world of government contracting and subcontracting in Iraq.\n\nThe mutilation of several American contractors in Iraq propelled the investigation. Several of the contractors worked for Blackwater USA, a North Carolina company that guards the U.S. ambassador and provides other security details in Iraq.\n\nThe authors tracked a string of contracts starting with Jerry Zovko - one of Blackwater's contractors who was killed in Iraq while guarding convoys. They followed the contracts up from Zovko through three other subcontractors and eventually to Halliburton, the prime contractor.\n\nThe business of contracting in Iraq is a closed world, Neff said. Many people declined to talk until the articles started running. \"If we were reporting from Kuwait or Iraq, it would have been easier.\"\n\nSuch an environment made the database reporting challenging, Neff said. Although U.S. taxpayers ultimately paid Blackwater's bills, all of the subcontracts were hidden from the public eye either because of the confidentiality of the work or because the company was a subcontractor.\n\nThe major part of Neff's analysis used a federal contracts database. That database, which Neff obtained from the IRE and NICAR Database Library, gave him a picture of how Blackwater's business with the government grew since 1999 - before the recent wars in Afghanistan and Iraq. He analyzed the data using FoxPro relational database manager.\n\nNeff used the contractor name field to select Blackwater contracts and tally how much its business had grown with the government to $19 million in 2003 from $205,000 in 2000. He also used the database to obtain contract numbers and make Freedom of Information Act requests for related documents.\n\nHowever, the database provided only a partial picture of Blackwater's relationship with the government because it did not include any subcontracts the company had received.\n\nThe Blackwater contract that Neff and Price were looking into was a subcontract with Halliburton, so they had to seek out other sources. Getting information from Halliburton wasn't feasible. Halliburton had failed to document almost half of a $4 billion invoice it submitted to the Pentagon, according to a recent audit by the Defense Contract Audit Agency. The missing half included the Blackwater contract.\n\nSo the reporters relied on other sources and unraveled four layers of contracts above the original Zovko contract. At each layer, the reporters found tacked-on profit margins hidden from the public. They found Blackwater marked up the Zovko contract by 36 percent. However, the other markups went undiscovered. The reporters said \"whether the cost for Zovko doubled, tripled or quadrupled\" remains unclear as Halliburton has refused to disclose the details of the spending.\n\nRead the article online at www.newsobserver.com/nationworld/blackwater/story/1762376p-8044834c.html.\n\nFor more information about the Federal Procurement Data System and ordering options, visit the IRE Web site at www.ire.org/datalibrary/databases/fedcontacts.\n\nDefense dollars\n\nIn \"Outsourcing the Pentagon,\" Larry Makinson and a team of reporters and analysts at The Center for Public Integrity completed what is perhaps one of the most thorough CAR analyses of the Department of Defense's private contracts.\n\nThe investigative team of about 20 journalists used database reporting to discover the Department of Defense has outsourced about half its budget since 1998. In addition, they found about 40 percent of the department's contracts were awarded without competitive bidding during the same time period.\n\nThe basis of their analysis was the Pentagon's procurement database that they described as \"public information that had been posted for years on an obscure Defense Department Web site.\"\n\nAron Pilhofer, database editor for the Center (and former Campaign Finance Information Center director for IRE), said the Center used the procurement, Federal Election Commission campaign contribution and a lobbying databases for comparison.\n\nThe Center downloaded five years of contracting data in text format from the Defense Department Web site (www.dior.whs.mil/PEIDHOME/guide/procoper.htm), which houses data on contracts issues since 1966.\n\nThe Center then imported the files from the department's Web site into a Microsoft SQL Server database manager. The Center focused on the contractors with $100 million or more in Defense of Department contracts.\n\nHowever, the information in the database was far from perfect and required substantial cleaning, according to information posted on the Center's Web site.\n\n\"We discovered very early that the Department of Defense database was rife with errors - that literally thousands of records had incorrect or outdated IDs for the contractor's ultimate parent,\" the Center reported.\n\nTo mitigate the errors, the Center examined contracts for companies below the $100 million threshold to make sure that did not miss any company due to possible miscoding.\n\nThe Center eventually found more than 700 businesses that had $100 million contracts or more with the Pentagon.\n\nAfter identifying the major contractors, the reporters researched campaign contributions to Congress and the president. For that, they analyzed the FEC campaign contribution database and lobbying reports filed with the U.S. Senate's Office of Public Records.\n\nThey said that the lobbying data online is searchable, but hard to use, so they scraped the data from the Web site using Perl into their own database. (Web scraping is using programming languages to gather data from Web sites and assemble it into database form).\n\nIn the end, they found that many of the top contractors were also generous political contributors, especially to President Bush, who was the top recipient of the contractors' contributions, receiving $5.4 million from the top contributors since 1998.\n\nRead the Center's stories at www.publicintegrity.org. Contact Catherine Rentz Pernot by e-mail at Catherine@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-bits-and-bytes","slug":"2005-may-june-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2005-may-june-bits-and-bytes.html","year":2005,"month":"May-June","monthNum":5,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR have updated their OSHA workplace safety inspection database with records through February 2005, containing over 3 million records since 1972 that journalists can use to investigate business safety and regulatory enforcement. The organization also offers resources through its Campaign Finance Information Center for investigating national and state campaigns, including searchable federal contract and election data.","keywords":["OSHA","safety","campaigns"],"topics":["Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Fresh OSHA data\n\nThe IRE and NICAR Database Library has updated the Occupational Safety and Health Administration inspections database to include records into February 2005. The data contains more than 3 million records of workplace safety inspections since 1972.\n\nThe OSHA data consists of 10 tables detailing workplace violations, penalties and accidents. Journalists have used previous releases of the data to investigate the safety of local businesses, and the effectiveness of OSHA inspections and enforcement.\n\nMany of these stories and tipsheets about investigating workplace safety are available through the IRE Resource Center.\n\nContact the Resource Center at 573-882-3364 or rescntr@ire.org.\n\nJournalists can order the data for a state or the entire country by contacting the Database Library at 573-884-7711 or 573-884-7332.\n\nFor more information about the OSHA workplace safety database, or to download an order form, visit www.nicar.org/data/osha.\n\nCampaign finance\n\nJournalists can tap in to the resources inside IRE's Campaign Finance Information Center (www.campaignfinance.org) to investigate national and state campaigns. The CFIC Web site features a searchable database of federal contracts and campaign finance data, and links to relevant tipsheets and news stories.\n\nIn addition, the Database Library has Federal Election Commission data for the 2006 election cycle.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-seleman-joins-ire","slug":"2005-may-june-seleman-joins-ire","url":"https://dwillis.github.io/uplink/article/2005-may-june-seleman-joins-ire.html","year":2005,"month":"May-June","monthNum":5,"headline":"Seleman joins IRE","kicker":"","author_name":"","author_title":"senior reporter","affiliation":"IRE and NICAR","summary":"Pricel Issa Seleman, a senior reporter from The African in Tanzania, joins IRE and NICAR as an Alfred Friendly Press Fellow to study investigative reporting, FOIA, and computer-assisted reporting while contributing to IRE publications and the Columbia Missourian.","keywords":["journalist","fellowship","investigative"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"IRE and NICAR welcomes Pricel Issa Seleman, a 2005 Alfred Friendly Press Fellow. Seleman is a senior reporter for The (Tanzania) African, where he also assists with editing and translation from Kiswahili to English.\n\nSeleman has worked for the African since 1999, in addition to writing for the Berliner, and London-based Africa Confidential. In addition to English and Kiswahili, Seleman is fluent in Nyanja and Shona, the languages of Zambia and Zimbabwe.\n\nSeleman is spending six months at IRE and NICAR studying the U.S. Freedom of Information Act, investigative reporting and computer-assisted reporting. He also will be writing for IRE publications and reporting for the Columbia Missourian, the community newspaper operated by the Missouri School of Journalism.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-time-to-say-thanks-to-our-writers","slug":"2005-may-june-time-to-say-thanks-to-our-writers","url":"https://dwillis.github.io/uplink/article/2005-may-june-time-to-say-thanks-to-our-writers.html","year":2005,"month":"May-June","monthNum":5,"headline":"Time to say thanks to our writers","kicker":"","author_name":"David Herzog","author_title":"","affiliation":"NICAR","summary":"Uplink is a grassroots journalism newsletter featuring volunteer contributions from journalists at various news organizations who share their work and expertise for free to help others learn from their experiences.","keywords":["grassroots","journalists","contributors"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Uplink is a true grassroots publication.\n\nLook at any issue and you'll see articles written by journalists from news organizations big and small. For this issue alone, journalists from major market television, a nonprofit reporting center and a diverse mix of newspapers have pitched in.\n\nWe don't pay our contributors a dime for their work. They generously share their stories so others can learn from their successes and their missteps. And they share their contact information so if you want to ask a contributor some questions, you can simply shoot them an e-mail.\n\nOur contributors provide a great service to journalism by giving their time. Starting this issue, we're going to tell you more about their backgrounds and accomplishments in our new \"About our contributors\" feature (see below). If you're looking for their e-mail addresses, you can find those where they've always been: at the end of the articles.\n\nContact David Herzog by e-mail at dherzog@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-about-our-contributors","slug":"2005-may-june-about-our-contributors","url":"https://dwillis.github.io/uplink/article/2005-may-june-about-our-contributors.html","year":2005,"month":"May-June","monthNum":5,"headline":"About our contributors","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A profile of contributors to a journalism publication, highlighting reporters and editors from major newspapers who specialize in computer-assisted reporting and investigative journalism, many of whom are alumni of IRE and NICAR training programs.","keywords":["journalists","reporting","contributors"],"topics":["Journalism & Newsroom"],"technologies":["GIS"],"full_text":"About our contributors\n\nJeff Claassen is the CAR specialist on the Fort Worth Star-Telegram investigative team. He has used data analysis to report on pollution and sports. Claassen is a 1994 alumnus of the IRE and NICAR CAR computer-assisted reporting boot camp and the 2004 advanced statistics mini-boot camp.\n\nDani Dodge recently joined The San Diego Union-Tribune as a general assignment reporter. Before that, she reported for the Ventura County Star. Dodge is a 2005 alumna of the boot camp.\n\nRon Nixon is the CAR editor for projects team at the Star Tribune in Minneapolis, Minn. He is the former training director for IRE and NICAR. Nixon is a 1998 alumnus of the boot camp.\n\nNatalya Shulyakovskaya is an investigative reporter for The Orange County Register who specializes in data analysis. She is a graduate of the Missouri School of Journalism and a former analyst in the IRE and NICAR Database Library. Shulyakovskaya is a 2000 alumna of the boot camp.\n\nHolly Hacker is the higher education reporter for The Dallas Morning News. She is a former analyst for the IRE and NICAR Database Library and a graduate of the Missouri School of Journalism.\n\nMatthew Waite is a reporter for The St. Petersburg Times who specializes in computer-assisted reporting and the use of geographic information systems (GIS). Waite is a 1999 alumnus of the boot camp.\n\nMatt Williams is a government reporter for the News & Record in Greensboro, N.C. He is a graduate of the Missouri School of Journalism, where his studies included computer-assisted and investigative reporting. His Web log, The Inside Scoop, is available at http://blog.news-record.com/scoopblog.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-uncovering-bad-bridges-at-risk-in-mudslides","slug":"2005-may-june-uncovering-bad-bridges-at-risk-in-mudslides","url":"https://dwillis.github.io/uplink/article/2005-may-june-uncovering-bad-bridges-at-risk-in-mudslides.html","year":2005,"month":"May-June","monthNum":5,"headline":"Uncovering bad bridges at risk in mudslides","kicker":"FIRST VENTURE","author_name":"Dani Dodge","author_title":"Reporter","affiliation":"Ventura County Star","summary":"A journalist uses data analysis to investigate poorly maintained bridges in Ventura County, California after mudslides kill ten people, discovering that 99 of 485 county bridges are deficient or obsolete and four of those damaged in the disaster had poor condition ratings.","keywords":["bridges","mudslides","data"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["Access","National Bridge Inventory"],"full_text":"FIRST VENTURE\n\nUncovering bad bridges at risk in mudslides\n\nBy Dani Dodge, Ventura County Star\n\nI had just jumped onto the hotel treadmill in Columbia, Mo., for a quick morning workout before computer-assisted reporting boot camp started when a picture of my hometown fire chief flashed on the television news.\n\nRecord rains had triggered massive mudslides back home in Ventura County, Calif. Ten people were dead. And I was missing the big story. My editors wanted me to finish the week-long IRE and NICAR Boot Camp, so I did what any journalist would have done and figured out an angle that I could begin to cover from more than 1,500 miles away.\n\nMy story: Money for bridge maintenance was going down in the state and county as the bridges were aging beyond easy repair. The consequence: An extra surge of water could wash them away.\n\nIt had taken nearly a year to persuade my editors to allow me to attend the January 2005 Boot Camp. We were into the third day of training when heavy rains flooded Ventura County, causing mudslides and mayhem. So while other boot campers researched federal Small Business Administration loan records or calculated AIDS rates for women, I used Microsoft Access to delve into the data for bridges and dams. If hillsides were coming down in the rain, bridges might be also. I bought the National Bridge Inventory database from the IRE and NICAR Database Library before I returned to Ventura County.\n\nAfter I got back, I was quickly thrown into the daily coverage to relieve reporters who had been working 18-hour days. But I also continued to probe the data. The flooding had destroyed two bridges in Ventura County and damaged others beyond repair. Had those bridges been maintained well? What about the other bridges in the county? I looked in the data for answers. I ran queries for the state, neighboring counties and Ventura County. In the queries, I asked about how many bridges were deficient and obsolete. I compared the rates of disrepair. I looked at the condition ratings of bridges.\n\nThe data showed that 99 of Ventura County's 485 bridges were deficient or obsolete. Four of the bad bridges were the ones that had been washed out or severely damaged. I discovered that within Ventura County, city- and state-maintained bridges had a much lower percentage of bad bridges than the county-maintained bridges. When I called county officials, though, they said their bridges were completely safe despite the poor condition ratings and the rains: \"Just a little damage.\" So I went after the paper inspection records. At first, the county public works officials balked: Someone would have to pull them for me, and they were too stretched, they said.\n\nI threatened to file a public records request, but at the same interview I also asked to see the area where the records were stored. Once I had seen how easy they were to access, the county official realized he had no excuse for not letting me go through them myself. From there, the fun began.\n\nIn their reports, state inspectors wrote that several county bridges were \"unstable\" and needed immediate attention. The date on the inspections: 2001. But, according to the reports and interviews, no work had yet been done. County officials could no longer continue to deny problems existed. Looking at some of those bridges and several others in the county that inspectors had rated as structurally deficient, I found one bridge with rotted timbers that I could poke a stick through. I also watched heavy trucks rumble over another bridge with a visibly undermined foundation - water had washed away the river bottom under the concrete pier that was supposed to hold it up.\n\nI spent two and a half weeks on the story. Only a small portion of that was spent working with the data, but I went back to it often. Several times, I found something in the paper records, and then found a corresponding trend in the data. I realized only after looking at the reports the importance of words like scour, which means erosion.\n\n\"Scour critical\" became an important benchmark in my story. I brushed up on the language of engineers by reading Web sites such as the American Association of State Highway and Transportation Officials at www.transportation.org/aashto/home.nsf/FrontPage.\n\nThen I hit a snag: a discrepancy in my numbers. I was down to my final cross-checking when I decided to match the numbers I had from my analysis with the numbers on the Federal Highway Administration's Web site. Mine showed nearly one-third more bridges in California than the Web site. I felt that same cold rush of panic I had experienced seeing the Ventura County fire chief on the news.\n\nFirst, I called a Federal Highway Administration public relations officer. He said the numbers on the Web site were correct. He didn't know how I could have come up with more bridges. I called Jeff Porter, the Database Library director for IRE and NICAR. He said it hadn't been a problem before, and suggested looking at several things, including whether I had double-counted bridges. He said that some bridges are counted twice because the administration considers the bridge underneath, or underbridge, as a separate bridge. I didn't think that was the problem because each of the bridges I counted had a unique ID number. It didn't make sense to me that any of them would be underbridges.\n\nSo, I searched further. I e-mailed a computer analyst at the state transportation department. He didn't respond. I called the number on the fact sheet for the Federal Highway Administration. A woman told me she couldn't talk to reporters.\n\nThe world around me started to get black. So I begged. I got teary. I explained I wouldn't quote her, but please don't let me put bad information into the newspaper. She quickly told me several items to check. One was underbridges just as Porter had said. When I explained I had already tried to check that by making sure I included only bridges with unique numbers in my count, she explained that the underbridges also had their own ID numbers. Five minutes later, the problem was solved. (If I had researched Porter's suggestion further, I would have saved myself an hour of panic.)\n\nThe story, \"County's aging bridges at the breaking point,\" ran a month after I graduated from boot camp. Sidebars, graphics and a Web video presentation I had shot accompanied it. It can be seen at www.VenturaCountyStar.com.\n\nThe county official who hadn't wanted to give me the records was impressed with the report. He asked me if I expected to win a Pulitzer Prize with the story. Yeah, right after I hit the lottery. Nevertheless, I did learn some lessons:\n\n1. Practice is key. To do computer-assisted reporting, I found I have to rewire my brain. The key to that is practice. I try to do a little bit each week.\n2. Read first. Analyze later. On the bridge data from the Database Library, there is a readme file and a guide to the fields in the data. Though I thought I knew what I was looking for, I found there was much more to know and that the data had much more to give. I realize I would have saved a lot of time if I had read all the materials that came with the data before diving into the numbers. In my hurry, I did it the other way around and wasted time.\n3. Log queries. Keep records. When I started queries on the data, it seemed so simple to do and then so tedious to write. I realized later I should have heeded the advice of the boot camp instructors and written down everything. Then, when I needed to re-create the queries with different modifications, I could have done it easily. Logging queries also seems to give editors a much better sense of security. In addition, I learned it can be very difficult to re-create queries two weeks later under deadline pressure with a stressed-out editor looking over your shoulder. From now on, I'm keeping a notebook by the computer to log the queries.\n4. Check your results against other sources. If the numbers are good, they will stand up. If I hadn't checked the California Department of Transportation Web site, and then the Federal Highway Administration Web site, I never would have known how far off I was. I would have risked a massive correction. I would have had fewer chances to do computer-assisted reporting in the future.\n5. Give yourself time. The exercises learned in boot camp can be much easier to do than grabbing data in real time. I found it took me much longer to do CAR on my own. And getting the numbers is just the starting point.\n\nContact Dani Dodge by e-mail at Dani@DaniDodge.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-readmetxt","slug":"2005-may-june-readmetxt","url":"https://dwillis.github.io/uplink/article/2005-may-june-readmetxt.html","year":2005,"month":"May-June","monthNum":5,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This article highlights online resources for journalists covering hurricanes and tropical storms, including UCF's interactive hurricane forecast mapping tool and the U.S. Navy's tropical weather tracking page with downloadable forecast data.","keywords":["hurricane","journalists","forecasting"],"topics":["Public Safety","Technology","Journalism & Newsroom"],"technologies":["GIS","shapefile"],"full_text":"Additional resources for journalists looking for hurricane and tropical storm information are available on the Web.\n\nThe University of Central Florida offers an interactive mapping application at http://hurricane.methaz.org that allows users to plot the results of different hurricane forecast models on one map. This allows journalists to get an idea about the uncertainty of the storm track.\n\nThe tropical weather tracking page from the U.S. Navy (www.nlmoc.navy.mil/cgi-bin/main.pl?tropical) contains downloadable shapefiles of forecast tracks that can be used in a GIS program and basic wind models.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-911-data-shows-systems-shortcomings","slug":"2005-may-june-911-data-shows-systems-shortcomings","url":"https://dwillis.github.io/uplink/article/2005-may-june-911-data-shows-systems-shortcomings.html","year":2005,"month":"May-June","monthNum":5,"headline":"911 data shows system's shortcomings","kicker":"EMERGENCY RESPONSE","author_name":"Scott A. Zamost","author_title":"Reporter","affiliation":"NBC","summary":"After investigating a drowning where a nanny's three 911 calls went unanswered, journalists discovered a systemic failure in Broward County's 911 emergency response system by obtaining call records that revealed lengthy delays before operators answered calls.","keywords":["drowning","911","investigation"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["AccuTrak"],"full_text":"It started with an intriguing tip. And it ended with major changes in the works for the 911 emergency response system in Broward County, Fla.\n\nA source suggested we examine a drowning in which a nanny called 911 three times. Instead of getting an operator, no one answered the phone, so she rushed the child to a nearby fire station. The toddler later died.\n\nThis was something entirely new. Reporter Patricia Andreu and I knew there had been periodic stories about problems in 911 systems around the country. Most of those stories had focused on delays in getting to locations depending on where someone lived. But none had shown a systemic failure to answer calls in a timely manner using police records.\n\nWe first obtained the police file of the drowning, which confirmed the nanny had dialed 911 three times. The police reports did not specify how long the phone rang each time. To get that, we asked the Broward Sheriff's Office, which handles most of the 911 calls for Broward County for the call records the day of the drowning.\n\nPrior to our request, a source told us that every time someone calls 911, a record of the call is logged into the sheriff's computer system. This information, stored in a system called AccuTrak developed by Gemini Software Technologies, includes the time the call came in, when an operator answered the phone, when the operator disconnected the call and if the caller hung up. That call data would prove to be a gold mine for our story.\n\nAt the same time, we learned that any time there is a question or complaint about a 911 call, an official investigation is opened. We obtained the Broward Sheriff's Office investigative files about the drowning and found remarkable information: The nanny's three phone calls to 911 lasted 37 seconds, 23 seconds and 11 seconds before she hung up and took the child in her own car to the fire station. This was the kind of detail that would form the basis for a broader investigation. If this happened in a drowning where a child died, how long do other emergency phone calls ring before they are answered?\n\nAt first, the sheriff's office told us it wasn't sure such records existed. We then made a public records request for the AccuTrak records for the drowning.\n\nWe received the paper records for the drowning incident, but there was a problem. It was impossible to decipher them without a law enforcement expert from the agency we were investigating. Some of the lines were coded with information about which operator picked up. For instance, \"C08\" meant the operator in the eighth position. The public information officer gave us a basic explanation, but we needed more.\n\nWe decided to ask for a telephone interview with the head of the police communications division to explain the records and how the 911 system worked. Since they knew we were looking into answer times anyway, we figured knowing this information was critical to deciding whether we had a bigger story.\n\nOver the phone, I spoke with Maj. Paul Lauria of the Broward Sheriff's Office. He explained what the fields meant - when a call came in, when it was answered and when the operator disconnected it. The records also showed how long \"abandoned calls\" rang before the caller hung up.\n\nLauria confirmed the call answer times in the drowning incident. He told me the operators answered calls \"as quickly as possible\" and there was no standard they had to meet because it would be \"self-defeating.\"\n\nHowever, we later learned there was a standard the sheriff's office agreed to meet: 90 percent of 911 calls must be answered within 10 seconds. It's a state standard we discovered in a phone interview with Florida's 911 coordinator. It would turn out to be a key element in our story.\n\nWe looked at the AccuTrak records for the day the toddler drowned. The call answer times for that day were well below the 90 percent standard. It appeared we were onto something so we asked for six months of records. The records are kept electronically and the Florida public records law states that agencies should be able to redact non-public information. Nevertheless, the sheriff's office insisted it could not electronically redact the confidential phone numbers of the callers. We consulted someone in our own information technology department who told us this was possible. The sheriff's spokesman insisted it couldn't be done and said the office could provide the records on paper with the numbers crossed out, a process that would take weeks or even months.\n\nIn the interest of moving the story forward, we asked for six months of AccuTrak records on paper. The sheriff's office, as it had done throughout our investigation, responded in a way that kept delaying the story. It told us it would take several months to cross out the phone numbers and estimated the cost could be higher than $8,000. That was unacceptable to us. We continued to press the office about getting the records in electronic form but kept getting denials.\n\nWe ultimately decided to look at two randomly selected weeks, which would amount to nearly 29,000 emergency calls. The state 911 coordinator said that would be an excellent sampling and more than anyone had ever analyzed. The paper records, which still took nearly a month to get, cost about $600.\n\nKeying the call information into an electronic format was a monumental task. A group of dedicated interns helped input the time each call came in and the time it ended into a Microsoft Excel spreadsheet. We then wrote formulas that calculated the answer time for every call, and the daily totals. We broke down the calls into 10-second increments. We included a column for the abandoned calls and the times it took before the callers hung up.\n\nThe office's performance was much lower than the state standard. Only 65 percent of the calls were answered within 10 seconds. Hundreds of other calls took 20, 30, 40 seconds - even more than a minute. When we finally interviewed Lauria on camera, he not only acknowledged his department could not meet the standard, he said it doesn't do what we did - track call answer times.\n\nWhile we were gathering the numbers, Andreu concentrated on persuading the parents of the boy who drowned to go on camera. Still in shock, they finally agreed with the hope that the attention would ensure this wouldn't happen again. We also tracked down the nanny who made the call as well as an elderly woman who hung up after her call to 911 rang for more than one minute. We ended up with two compelling examples on camera as well as all the strong data we had assembled.\n\nReaction was swift. Shortly after the story aired in November, two cities agreed to study whether they would be better off handling 911 calls on their own.\n\nIn late January, Broward Sheriff Ken Jenne credited our investigation with major changes in how 911 calls are handled. The sheriff's office installed software that allows it to track call answer times on a daily basis. Keeping track of calls and better training have already improved answer times. In addition, the sheriff told us he would ask for additional staff to handle non-emergency calls. He hopes to set up a system in which those calls are immediately routed to special operators to free up emergency lines. Finally, the sheriff insisted his agency must work toward reaching the state standard of 90 percent.\n\nContact Scott Zamost by e-mail at Scott.Zamost@nbcuni.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-readmetxt-2","slug":"2005-may-june-readmetxt-2","url":"https://dwillis.github.io/uplink/article/2005-may-june-readmetxt-2.html","year":2005,"month":"May-June","monthNum":5,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Online resources for journalists covering hurricanes and tropical storms include UCF's interactive mapping application for comparing forecast models and the U.S. Navy's tropical weather tracking page with downloadable shapefiles and wind data.","keywords":["hurricane","tracking","forecast"],"topics":["Investigative Methods","Technology"],"technologies":["GIS"],"full_text":"Additional resources for journalists looking for hurricane and tropical storm information are available on the Web.\n\nThe University of Central Florida offers an interactive mapping application at http://hurricane.methaz.org that allows users to plot the results of different hurricane forecast models on one map. This allows journalists to get an idea about the uncertainty of the storm track.\n\nThe tropical weather tracking page from the U.S. Navy (www.nlmoc.navy.mil/cgi-bin/main.pl?tropical) contains downloadable shapefiles of forecast tracks that can be used in a GIS program and basic wind models.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-the-dividing-line","slug":"2005-may-june-the-dividing-line","url":"https://dwillis.github.io/uplink/article/2005-may-june-the-dividing-line.html","year":2005,"month":"May-June","monthNum":5,"headline":"The dividing line","kicker":"LAPPING","author_name":"Matt Williams","author_title":"Reporter","affiliation":"News & Record","summary":"A journalist uses GIS mapping and Census data to quantify racial segregation in Greensboro, North Carolina, revealing that the city is divided almost perfectly in half by race, with dramatically different retail and business presence on each side.","keywords":["segregation","mapping","inequality"],"topics":["Investigative Methods","Demographics & Census","Business & Economy"],"technologies":["ArcView","Excel","TIGER","GIS"],"full_text":"Not long after moving to Greensboro more than a year ago, I realized that to buy groceries or clothes or see a movie, I always drove in the same direction: west. This, even though I lived right in the middle of downtown.\n\nDecades after the end of legal segregation, thousands of residents still did most of their shopping on the \"white\" side of town.\n\nIt's the same problem I'd hear often from minority residents of the east side of town: There just aren't as many places to shop in our part of town as there are in yours. The places we used to shop steadily closed and have been abandoned, while all the new stores open up on your side of town.\n\nBut community leaders were never really able to quantify the sense shared by many that there was an inequity. Faced with this, I started to piece together the News & Record's attempt to do that.\n\nPutting together the numbers was the easy part. Finding out the \"why\" was the challenge.\n\nI began by creating the base for my comparison, in this case block-level data from the 2000 Census. Using the smallest geographic level of Census data allowed us to see with ESRI ArcView 8.3 geographic information system (GIS) the patterns of racial distribution. We got the demographic and map data through the Census Bureau's Maps and Cartographic Resources Web site at http://tiger.census.gov.\n\nIn ArcView, I joined the demographic data file, which includes population by race, to the Census blocks map. I then set the symbology properties to show the percentage of whites living in each block.\n\nRight away, the racial segregation was clear to the eye, with two major streets in the Census TIGER street file running diagonally through downtown delineating the two sides of Greensboro. We selected those roads as our \"dividing line.\"\n\nThe western half of the city was home to a little less than two-thirds of the city's population and was 80 percent white. The eastern half was home to a little more than one-third of the total population and was 77 percent black - a virtual mirror image. Within those two there were large areas where the dominant ethnic group made up more than 95 percent of the residents.\n\nBefore we even started, we had numbers that would surprise most of our readers. Like me, they might have assumed that the city was somewhat segregated but had no way of quantifying how much it is segregated. From there, I gathered names and addresses to place businesses of all types throughout the city. Some of the data was publicly available, such as the location of bank branches insured by the Federal Deposit Insurance Corporation. Those files can be found at http://www2.fdic.gov/idasp/main.asp. The rest were simply copied into Microsoft Excel spreadsheets one-by-one out of online phone directories. Those were then exported to comma-delimited text fields that were imported to ArcView.\n\nArcView's geocoding function takes a table containing street address data and turns it into a point map using a street file, such as a Census TIGER street file, as a reference. We displayed each type of business with a different color, and we could display only banks or just grocery stores.\n\nAfter those businesses were geocoded, we created a simple tally of the number of locations on either side of the dividing line. Once we took population into consideration, it was easy to see the disparities. We obtained the population of each side of the city by selecting the Census blocks for each and using ArcView's summarize function to calculate a total.\n\nPer capita, there were half as many banks on the \"black\" side of town and a third fewer grocery stores. In other categories there was no contest: all of the 60 movie screens, eight urgent care centers and all but one major retailer were on the \"white\" side of town. If the movie screens were distributed evenly based on the population, there should be 40 on the west and 20 on the east.\n\nFast-food restaurants and dry cleaners, which were also included in our analysis, showed roughly equal distributions. Pawn shops, check cashers and payday loans were skewed the opposite way. They were twice as common in minority neighborhoods as white ones.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-tracking-hurricanes-path-to-the-coast","slug":"2005-may-june-tracking-hurricanes-path-to-the-coast","url":"https://dwillis.github.io/uplink/article/2005-may-june-tracking-hurricanes-path-to-the-coast.html","year":2005,"month":"May-June","monthNum":5,"headline":"Tracking hurricanes path to the coast","kicker":"SPOTLIGHT: WATER AND WATERWAYS","author_name":"Matthew Waite","author_title":"Reporter","affiliation":"St. Petersburg Times","summary":"A guide for journalists on how to use GIS mapping software to track hurricane paths by extracting forecast data from the National Hurricane Center website and formatting it in Excel for visualization in ArcView.","keywords":["hurricanes","mapping","tracking"],"topics":["Technology","Public Safety"],"technologies":["Excel","ArcView","Access","ArcMap"],"full_text":"Tracking hurricanes path to the coast\n\nBy Matthew Waite, St. Petersburg Times\n\nWhen a hurricane barrels over water toward the coast, everyone wants to know one thing: Where is it heading? Everyone will talk about it, your family, your neighbors, your editors, coworkers, shoppers in line at the grocery store. Where will that storm hit?\n\nEvery six hours, the National Hurricane Center uses its computer models to predict a hurricane's path and publishes the latitude and longitude coordinates for the projected track. That's good news for journalists who use geographic information system (GIS) programs to map data.\n\nThe place to start is the National Hurricane Center's Web site at www.nhc.noaa.gov. The storms are listed by name, with links to the Public Advisory, Forecast/Advisory, Discussion, Probabilities, Maps and Charts and Archives. If you've never been to the site, each link is worth reading. For mapping purposes, go to Discussion, where the forecast points are kept, usually toward the end.\n\nBuilding the foundation\n\nCopy the forecast points and paste them into a new Microsoft Excel worksheet. You'll have to use to Data\\Text to Columns function in the menu to place the data into individual columns. Name the columns, using lat for latitude and long for longitude so ArcView can use these later.\n\nIf you've used a GIS before, you've probably plotted x/y coordinates, usually latitude and longitude. It's not difficult to do, but doing it repetitively, every six hours, would be tedious. Fortunately, with ESRI's ArcView 8 to 9, you can automate the task with little effort and create updated maps for reporters and editors.\n\nInsert a column at the beginning called forecast age. Here we're going to add data that will tell our GIS which points to display. For your newest forecast, fill in the cells in forecast_age with something that says this is what I want - I use \"current\" and then call the older points \"old.\" The use for this will come later.\n\nNext to that column, add another and call it forecast. Here, manually fill in which forecast it comes from (i.e. Friday 5 a.m. or Saturday 11 p.m.). That will help you if you want to look at the evolution of the forecast track.\n\nThen add a column called forecast_step. Fill that in, starting with 1 and ending with 8. If you want to connect the dots later using automated means, this step will save you time in having to edit the file.\n\nLast, and this is optional, convert the forecast wind speeds from knots in the last column to miles per hour. To do that, use search and replace to delete the KTs and multiply knots by 1.15077945 in the next column.\n\nNow we will get the latitude and longitude in a format that ArcView can recognize. Use the search and replace tool in Excel to delete all the Ns in the lat column and then do the same to delete the Ws in the lon column. The lon needs to be represented as a negative number, so create a new column and multiply the cells in the lon column by negative 1. (Longitudes in the Western hemisphere are negative numbers). Copy the contents of the new column and choose Paste special\\Values to put the values in the original lon column.\n\nWith properly formatted longitude and latitude points, you could go ahead and map these points. But let's add some more data.\n\nWith each new forecast, just open the spreadsheet, paste the new information in, copy the formulas down and save it. I usually use storm names for my spreadsheet names.\n\nTracking maps\n\nNow it gets much easier.\n\nOpen Microsoft Access. Create a new Access database and import your spreadsheet. Close the database.\n\nOpen ArcMap. For hurricane tracking base maps, I've found that Census TIGER maps work just as well as anything. I have statewide county boundary maps, statewide place maps, statewide major roads (interstates and freeways) and statewide water polygons in a map.\n\nWith those layers up, go to Tools/Add XY Data in the menu. Under \"Choose a table from the map or browse for another table,\" browse to your Access database file and find your table inside it. (The newest generations of ArcView, from versions 8.x on, directly read Access data). Make sure that ArcMap recognized your latitude and longitude fields in the box that opens. Click OK and hurricane track points should appear on the map.\n\nIf this is the first time you're mapping points, you'll only have one track on the map. If this is the fourth or fifth forecast you're mapping, you'll see more sets of points on the map. We can show just the current forecast track using the forecast_age field.\n\nDouble-click on your hurricane layer name in the ArcMap table of contents. Then select the Definition Query table and enter forecast_age = \"Current\" to limit the displayed data to the current points.\n\nRepeat as needed\n\nWhen you get a new forecast, all you have to do is update your database. Copy the points into Excel, repeat the steps and be sure to change the forecast_age column to reflect the new forecast as the current one, then import it into Access and ArcMap will update the points without you having to do a thing.\n\nIf you do this consistently - putting each forecast into your database - by the end of the storm you can query all of the forecast positions that start with Initial. Those are the exact point of the storm every six hours. With those points, you have the general path of the storm. One weakness is that hurricanes don't move in straight lines, which is what your map will show. They wobble, angle and turn slowly. So remember that the path you can create from initial forecast points is relative.\n\nIf you'd like to turn your points into a line, there are few options available. The best is an extension called XTools Pro (See www.xtoolspro.com). XTools costs $129 for a single-use license, but is worth the money because of the number of tools in it. One of the tools converts points to a polyline file. With three clicks, you can turn your hurricane points into a hurricane path.\n\nSince Hurricane Jeanne, I mapped the predicted track of the storms that followed, and created maps that helped editors decide where to send reporters.\n\nMapping forecast points won't result in a story itself, but it will help guide dozens of them. And it only takes a few minutes to pull off, after you get used to the steps.\n\nContact Matthew Waite by e-mail at mwaite@sptimes.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-readmetxt-3","slug":"2005-may-june-readmetxt-3","url":"https://dwillis.github.io/uplink/article/2005-may-june-readmetxt-3.html","year":2005,"month":"May-June","monthNum":5,"headline":"readme.txt","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This article highlights web-based resources for journalists covering hurricanes and tropical storms, including UCF's interactive hurricane forecast mapping tool and the U.S. Navy's tropical weather tracking page with downloadable forecast data.","keywords":["hurricane","resources","forecast"],"topics":["Technology","Journalism & Newsroom","Public Safety"],"technologies":["Internet","World Wide Web","GIS","shapefile"],"full_text":"Additional resources for journalists looking for hurricane and tropical storm information are available on the Web.\n\nThe University of Central Florida offers an interactive mapping application at http://hurricane.methaz.org that allows users to plot the results of different hurricane forecast models on one map. This allows journalists to get an idea about the uncertainty of the storm track.\n\nThe tropical weather tracking page from the U.S. Navy (www.nlmoc.navy.mil/cgi-bin/main.pl?tropical) contains downloadable shapefiles of forecast tracks that can be used in a GIS program and basic wind models.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-extra-extra-your-guide-to-the-latest-investigative-work","slug":"2005-may-june-extra-extra-your-guide-to-the-latest-investigative-work","url":"https://dwillis.github.io/uplink/article/2005-may-june-extra-extra-your-guide-to-the-latest-investigative-work.html","year":2005,"month":"May-June","monthNum":5,"headline":"EXTRA! EXTRA! YOUR GUIDE TO THE LATEST INVESTIGATIVE WORK","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"The article highlights computer-assisted reporting examples from IRE's Extra! Extra! resource, including investigations by the Salt Lake Tribune on excessive residential water consumption and the St. Petersburg Times on rapidly escalating waterfront property values in Florida.","keywords":["water","investigation","data"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":[],"full_text":"If your newsroom is parched for story ideas, Extra! Extra! on the IRE Web site (www.ire.org/extraextra), carries links to and summaries of articles about water and waterways that used computer-assisted reporting. Here are some examples:\n\nThe Salt Lake Tribune analyzed public utilities data to identify which homes used the most water in the area. One estate consumed 7.1 million gallons in one year - more than 21 times the use of an average homeowner. Former NBA star Karl Malone's home used more than 3.3 million gallons, the newspaper found.\n\nThe St. Petersburg Times conducted an analysis showing that, since 1998, the median price of waterfront homes in Pinellas County, Fla., was increasing more than twice as fast as real estate off the water. The newspaper reported that the typical waterfront property cost more than $500,000 in 2004 - more than double what it cost six years earlier.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-grappling-with-data-on-epa-mainframe","slug":"2005-may-june-grappling-with-data-on-epa-mainframe","url":"https://dwillis.github.io/uplink/article/2005-may-june-grappling-with-data-on-epa-mainframe.html","year":2005,"month":"May-June","monthNum":5,"headline":"Grappling with data on EPA mainframe","kicker":"Tech tip","author_name":"Jeff Claassen","author_title":"","affiliation":"Fort Worth Star-Telegram","summary":"Journalists investigating EPA enforcement of air pollution laws at oil refineries obtained enforcement data from the EPA's IDEA database system after struggling with initial records requests, requiring technical knowledge of the agency's custom programming language and remote access through a terminal emulator.","keywords":["EPA","enforcement","database"],"topics":["Data & Databases","Investigative Methods","Health & Environment"],"technologies":["mainframe","World Wide Web"],"full_text":"We were struggling to verify a tip: the U.S. Environmental Protection Agency had dramatically slowed its enforcement air pollution laws at oil refineries.\n\nThe clues would come from the agency's administrative records - the ones listing the dates and results of inspections, enforcement actions, court settlements and administrative fines.\n\nWe asked the Dallas regional EPA office for the records, but the requests were taking months to fulfill, and the information they eventually provided was woefully inadequate.\n\nThose EPA record-keepers were trying to pull information from the agency's enforcement-tracking databases. That piece of information gave us our first break - we needed to get copies of those databases to work with ourselves.\n\nOne database, the AFS (Air Facility Subsystem), covered enforcement of the Clean Air Act. Another, the PCS (Permit Compliance System), stored enforcement records for the Clean Water Act. A third, ICIS (Integrated Compliance Information System), held all the court and administrative settlement records.\n\nThe agency has good descriptions of these databases and others at www.epa.gov/compliance/planning/data.\n\nThe EPA's enforcement managers in Washington, D.C., were willing to help. But they thought it would be impossible to use the massive databases. Those databases contain terabytes, or thousands of gigabytes, of data.\n\nIncluded are the enforcement history and all of the pollution-output records for every emissions point in every factory, refinery and other facility in the country that has a pollution permit.\n\nNext, we went to the people who know that material the best - the agency's programmers, who run search services on the EPA's Web site. Those search engines include ECHO, an enforcement history search, which environmental reporters use often. (See www.epa.gov/echo/compliancereport.html)\n\nThe programmers told us the agency has much smaller versions of the databases we wanted, updated every month for public use. Those databases included administrative actions, such as inspections, but not the massive list of individual emissions reports.\n\nGraduate students, environmental activists, and the people who run the EPA Web site searches used that trimmed-down data the most. The agency's Web-site searches pull from these smaller databases.\n\nThe only hitch: public access was through an old, awkward Web page connection, at https://trex.rtpnc.epa.gov. The core of this Web site is a terminal emulator that wires you into a supercomputer at the agency's computer center in North Carolina. Our company firewall blocked it, so all the programming and downloading had to happen from home.\n\nIt took weeks to learn the system and download the data. The system, called IDEA (Integrated Data for Enforcement Analysis), uses a programming language created entirely by EPA computer technicians. Everything you want to do requires terminology you won't see anywhere else.\n\nThe EPA gave us several technical guidance documents, including one that is more than 200 pages long. Programmers also gave us some sample programs, and worked diligently with us to get the syntax right so it would create the data sets that we wanted.\n\nTo show how idiosyncratic the syntax is, here is one program we used. This script pulls specific fields from the AFS dataset, (\"FS.AFSID,\" etc.), copies the unique identifier field to each record (\"DEC 100\") and designates enough server processing time and hard drive space for our new file:\n\nINPUT NONLINKED SELECT FS.AFSID NE OUTPUT JOB TIME 40 REPORT AFS1 DATASET AFSNX SHR SPACE LENGTH 190 PRIMARY 2350 SECONDARY 1175 RELEASE /* DIR TITLE \"AFS PULL 1\" DETAIL AFS. AFSID DEC 100 AFS CNTY AFS. STAB AFS. PLNT AFS. APC1 AFS. REGN AFS. AQCA AFS. PNME AFS. STRT AFS. CYNM AFS. SICI AFS SIC2 AFS. INSPDAY AFS. FEDREP AFS ZIPC AFS. DTAI AFS ACSEQ AFS ANT1 AFS. PAMI AFS INFE AFS YIFE AFS. INFO AFS YIFO AFS RDTA1 AFS. RANT1 AFS RATP1 AFS. RPAM1 AFS DDTAI AFS. DANT1 AFS DATP1 AFS. DPAM1 PARM BACKYEARS_16 PARM MINHEAD PARM DELIMIT A\n\nThere's one note of caution hidden in the mostly unreadable program. If you download data from IDEA, use the shift-6 (^) as your delimiter, as specified in the last line of code. Fields contain commas, making those unusable and IDEA is unable to use tabs or other field delimiters other than alphabetic characters.\n\nDownloading also took a while because we needed to sign onto IDEA in the evenings and weekends, because of costs. During weekday hours, the EPA charges 2,100 per CPU hour (hour of computer processing time. For example, time to run queries and save data sets). Night hours cost half that, and weekend hours are one-quarter of the regular cost.\n\nOnce IDEA created the data sets for us, downloading a single file took hours, even though I was using a DSL connection at home. The EPA's FTP server in North Carolina is slow. Downloading was complicated, but worth it. We finally had three very comprehensive tables containing:\n\nmore than 2 million records for 175,000 air-permit facilities\nmore than 1.3 million records for 330,000 water-permit facilities\nabout 230,000 entries for 59,000 court and administrative settlement cases\n\nThe EPA supercomputers store the data in old-fashioned hierarchical format. That means that different types of information for a facility (address and location, inspections, violation notices etc.) are contained on different lines of data, one after the other, with no common ID number to link them together.\n\nFortunately, the EPA programmers had come up with a way to generate repeat ID numbers in the output data, so they were attached to each record. We were able to thread this information back together using the common ID. I used FoxPro, more out of habit than for any other reason. Access would have handled these files as well.\n\nAnother note for anyone who uses this data: to count inspections, enforcement actions, and other agency activities, you must use what EPA record-keepers call \"rolled-up\" data. For reasons the agency could not clearly explain, records are often duplicated in its \"detail\" data. To check that, we downloaded each type of record - \"rolled up\" and \"detail\" - and did see the repetition in the detail records.\n\nThe data went back to the 1970s and created a broad picture of enforcement patterns over the decades. The number of inspections, violation notices and other actions spiked upward under the first President Bush, after flattening under Reagan, then crested during the Clinton years, and finally dropped sharply after 2001. The data showed that our tip was right.\n\nSince 2001, inspections have gone down 52 percent for oil refineries and 4 percent for all industries, including refineries. Violation notices decreased 68 percent for refineries, and 24 percent for all industries. The number of court and administrative settlements dropped by 77 percent for refineries and 7 percent for all industries. Stack tests, examinations done by private contractors and turned over to the EPA, were down 9 percent for refineries and up 43 percent for all industries.\n\nWe also used other data - the EPA's Toxics Release Inventory and Texas' Point Source Air Emissions Inventory, among others - to flesh out our description of the pollutants that refineries have released to the air, water and land.\n\nOur reporting found that the EPA had in recent years turned increasingly to legal agreements called consent decrees, and was not inspecting refineries during the negotiations or even after them. Experts said that the lack of comprehensive inspections had a cascading effect, driving down the number of violation notices, administrative penalties and other actions.\n\nThe two-day series got responses from around the country, including neighboring Louisiana, which also has a large number of oil refineries.\n\nContact Jeff Claassen by e-mail at claassen@star-telegram.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-may-june-cleaning-your-files-with-textpipe","slug":"2005-may-june-cleaning-your-files-with-textpipe","url":"https://dwillis.github.io/uplink/article/2005-may-june-cleaning-your-files-with-textpipe.html","year":2005,"month":"May-June","monthNum":5,"headline":"Cleaning your files With TextPipe","kicker":"CAR TOOL","author_name":"Ron Nixon","author_title":"","affiliation":"Star Tribune","summary":"TextPipe Pro is a user-friendly text manipulation tool that helps journalists clean and process dirty data without requiring programming knowledge, offering 100 filters and powerful search-and-replace capabilities for data files.","keywords":["data","TextPipe","cleaning"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["TextPipe","Perl","Python","PHP","Visual Basic","Excel","mainframe"],"full_text":"Cleaning your files With TextPipe\n\nBy Ron Nixon, Star Tribune (Minneapolis)\n\nHere are three challenges familiar to journalists who have worked with dirty data:\n\n1. Your data exists in a text file, but when you try to import the file into your program, there's a problem: the data lines up incorrectly. There's a tab or comma delimiter that's out of place or a space that you can't see.\n2. You need to search and replace hundreds of times in your data, sometimes using exact searches and others with pattern searches.\n3. You need to extract information out of a text file that's unstructured or pull data out of a HTML or XML file.\n\nFor any of these challenges you could use a programming language such as Perl, Python, PHP, or even Visual Basic to craft a solution. There are several advantages of using any of these. Among them is the power and, with the exception of Visual Basic, the availability of free downloads on the Web. But there's a downside: each comes with a steep learning curve.\n\nEven Python, which can be easy to use, has a learning curve. An alternative tool is a utility called TextPipe Pro. TextPipe costs $299 and you can download a free 30-day trial from www.crystalsoftware.com.au/index.html\n\nMade by DataMystic, an Australian company formerly known as Crystal Software, the program has an easy-to-use interface. Users can have it up and running in a few minutes and it is especially well-suited for the non-programmer.\n\nDon't let the ease of use fool you - it's still a powerful text manipulation tool. TextPipe comes with 100 filters that can be applied to everything from a simple search and replace to extracting telephone numbers from a text file. What really makes it shine, though, is its ability to do hundreds, even thousands of search-and-replace routines at the same time. And that's just the beginning.\n\nAlong with the searching and replacing, you can add filters to remove white space, add a new line or add a line feed, all in one session. You can even do targeted searching and replacing by column or even by row. You can even use a Microsoft Excel spreadsheet or a text file as a lookup table for TextPipe's searching and replacing. Simply type the strings to find in one column and the replacement in another and import the data.\n\nWhat's more, TextPipe is fast and can handle files of several hundred megabytes. I've used it on files 2 gigabytes or more. TextPipe can be set up to run as a scheduled program, so you can have it running and working while you do other things. You can also run it from the command line.\n\nOther TextPipe text file capabilities include merging dozens of files, sorting information and converting text files from mainframe, PC and Unicode data formats.\n\nTextPipe keeps an audit log of all your changes so you can document your work. The software has a feature called the Scratch Pad where you can try your filter on practice text before using it on your real text file.\n\nThose who need to do even more complex text processing can use Perl-style regular expressions. For those who are intimidated by regular expressions, TextPipe offers EasyPatterns, a text processing tool that has all the power of regular expressions but is easy to understand.\n\nFor example, to describe a phone number in regular expressions you'd write: \\d\\d\\d-\\d\\d\\d-\\d\\d\\d\\d In Easy Patterns it's: [PhoneNumber].\n\nDataMystic provides an introduction to Easy Patterns at www.crystalsoftware.com.au/easypatterns.html.\n\nEven though there are programming languages available for free download, TextPipe's ease of use and flexibility make it well worth the price.\n\nContact Ron Nixon by e-mail at rnixon@startribune.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-immigrants-deadly-rides","slug":"2005-septemberoctober-immigrants-deadly-rides","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-immigrants-deadly-rides.html","year":2005,"month":"September/October","monthNum":9,"headline":"Immigrants' deadly rides","kicker":"WORKPLACE","author_name":"Mike Schneider","author_title":"Reporter","affiliation":"The Associated Press","summary":"Farmworkers in Immokalee, Florida, face hazardous transportation conditions while traveling to work, raising concerns over state regulation and safety in agricultural labor.","keywords":["farmworkers","transportation","safety"],"topics":["Investigative Methods","Public Safety","Agriculture & Land"],"technologies":["Excel"],"full_text":"The early morning fog and darkness were still two hours from lifting, but already the parking lot of the La Mexicana grocery store in the farming community of Immokalee, Fla., was packed with 200 farmworkers waiting to be picked up for a day of harvesting in southwest Florida's orange and vegetable groves. Creaky buses that looked two decades old pulled up. Some farmworkers jumped into the back of U-Haul trucks that were then pulled closed, shutting out fresh air. Others jumped into vans that had their seats pulled out to make room for more people. Most of these vehicles belonged to third-party labor contractors who were hired by growers to put together work crews. The contractors were licensed and monitored by federal and state agencies, but neither seemed to be doing a good job in the area of transportation based on a cursory look at the vehicles. Transportation is a central determinant in the lives of farmworkers because it is difficult for many to get driver's licenses because of their illegal status. Without driver's licenses or cars, the farmworkers in Immokalee are forced to live in substandard housing within blocks of La Mexicana so they can walk to the parking lot where the contractors pick them up for jobs. With this captive audience, the grocers and landlords in the area charge more for food and housing than in other surrounding communities. Transportation, I would find during my research, also is the No. 1 cause of death for Florida's farmworkers. Curious about how well the state regulated contractors, I asked the state Department of Business and Professional Regulation for its database about violations of state labor law made by Florida labor contractors since 1997. They didn't have one, even though Florida has one of the nation's largest farmworker populations. Those paper records were gathered in rubber bands in cardboard boxes at the state agency's headquarters in Tallahassee. So I created a database. I drove to Tallahassee, laptop computer in hand, with the expectation it would take me two days to enter the records into a Microsoft Excel spreadsheet where I could look for patterns, find the worst offenders and glean other information. After several weeks of data entry during spare moments, I finally had 878 entries, covering all types of violations, that spanned January 1997 to March 2004. The database revealed that more than one-third of the farm labor violations were related to transportation, such as using vehicles that had not been inspected by the state, lacking driver's insurance or employing an unlicensed driver. State regulators had not taken transportation violations as seriously as others, such as failing to pay delinquent taxes, something which often led to a contractor's license being revoked. Not one contractor had a license revoked for a driving-related violation, even though those violations could have deadly consequences. Repeat violators of farm labor transportation laws still got their contractor licenses renewed. While I was putting the database together, tragedy struck. A van jammed with 19 farmworkers rolled over on Interstate 95 near Florida's Atlantic coast, killing nine men. Almost everyone was ejected because nobody wore seatbelts, and bodies were scattered on the highway. About the same time, the Florida Legislature, at the urging of Gov. Jeb Bush, passed legislation increasing the fines for farm labor law violations and making it easier to report violations. The law was created in response to a handful of recent cases involving farmworkers made to work in slave-like conditions. The law, however, ignored transportation-related issues.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-septemberoctober-philip-meyer-journalism-award-contest-honors-stats-pioneer","slug":"2005-septemberoctober-philip-meyer-journalism-award-contest-honors-stats-pioneer","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-philip-meyer-journalism-award-contest-honors-stats-pioneer.html","year":2005,"month":"September/October","monthNum":9,"headline":"PHILIP MEYER JOURNALISM AWARD Contest honors stats pioneer","kicker":"","author_name":"Amanda Buck","author_title":"","affiliation":"IRE and NICAR","summary":"Ted Mellnik created the Philip Meyer Journalism Award to recognize journalists who use social science research methods and computer-assisted reporting, work that often goes unrecognized in prestigious journalism contests despite producing significant contributions to the field.","keywords":["journalism","awards","research"],"topics":["Journalism & Newsroom","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"The first time Ted Mellnik met Philip Meyer was at a seminar in the late 1980s. Meyer, already considered the father of precision journalism, was teaching journalists like Mellnik how to use social science research methods in their work.\n\nMellnik, database editor at The Charlotte Observer, still remembers that seminar. A few years ago, as he considered the impact of Meyer's use of social science research methods on modern journalism, Mellnik decided he should do something to help honor Meyer's contributions.\n\nFurthermore, although Mellnik knew from personal experience that journalists across the country use computer-assisted reporting and social science methods to produce \"spectacular\" stories, their work often was not recognized in prestigious contests.\n\nThat, Mellnik thought, should change.\n\nThree years later, it has. In September, the National Institute for Computer-Assisted Reporting (NICAR), IRE and the Knight Chair in Journalism at Arizona State University's Walter Cronkite School of Journalism and Mass Communication introduced an award designed to recognize the best journalism done using social science research methods. NICAR is a joint program of IRE and the Missouri School of Journalism.\n\nNamed in honor of Meyer, who holds the Knight Chair in Journalism at the University of North Carolina at Chapel Hill, the Philip Meyer Journalism Award is planned as an annual contest that will make real the kind of recognition Mellnik imagined.\n\n\"I felt we had a lot of people who did spectacular work, and that they needed to be recognized,\" Mellnik says. \"In awards, we tend to honor stories that get results - that put people in jail or result in reform legislation or something like that. But there's also a benefit to stories that establish a methodology and give reporters across the nation a tool that enables them to do good work for their readers.\"\n\nMellnik approached others with his idea, including IRE Board Member Steve Doig, the Knight Chair in Journalism at Arizona State University. Formerly associate editor for research at The Miami Herald, Doig has years of experience using social science research methods. For the past several years, he also has served as a member and chair of the IRE Awards committee.\n\nDoig's work with the committee convinced him that some award-worthy work was being overlooked.\n\n\"The problem with investigative awards (in this context) is that one of the real criteria is that the story should help uncover information that others were trying to keep secret,\" he says. \"Yet there's a great deal of good CAR, precision journalism kinds of things, for instance dealing with the census, that isn't strictly investigative. You're not uncovering wrongdoing; you're uncovering how society works, for instance.\"\n\nMeyer is known for doing that. He pioneered using survey research as a reporter for Knight Ridder newspapers to explore the causes of race riots in the 1960s.\n\n\"He inspired the profession to be more sophisticated and precise in its understanding of and reporting on social issues,\" says IRE and NICAR Executive Director Brant Houston. \"He has been a mentor to many, many journalists - including myself and he encouraged us to learn and use methods that made us and our profession much more credible.\"\n\nDoig called Meyer the \"prime mover\" in raising awareness among journalists to the need for using social science methods.\n\n\"It started with his 1973 book, 'Precision Journalism.' He wrote that really 10 years before it started becoming easier to do the things he was talking about,\" Doig says.\n\nNow in its fourth edition, \"Precision Journalism,\" is widely used as a textbook in journalism schools.\n\nSocial science research methods that would make stories eligible for the award draw on tools used across a variety of disciplines, from sociology and demographics to geography and political science.\n\nThe award \"will help recognize stories that have used social science methods to reveal patterns, trends and sometimes abuses in areas including social or economic or criminal systems,\" Houston says.\n\nBecause IRE has been the leading organization in the area of using social science methods in investigative reporting and training people to do that, it made sense for the organization to cosponsor the awards, Houston says.\n\nNominations for the award must be postmarked by Oct. 31. Appropriate work published or aired between Oct. 1, 2004, and Sept. 30, 2005, will be considered.\n\nA panel of judges including journalism educators and social scientists will choose first, second and third place entries. Judging will be complete by late December, and the awards will be presented during the 2006 CAR Conference scheduled for March 9-12 in Newark, N.J.\n\nTo see a list of judges, download an entry form and read further details about the contest, visit www.ire.org/meyeraward. Additional questions should be directed to the IRE Resource Center at 573-882-6668 or rescntr@ire.org.\n\nAll entries will be placed in IRE's Resource Center story library. Contact Amanda Buck by e-mail at amandab@ire.org.\n\nhe should do something to help honor Meyer's contributions.\n\nFurthermore, although Mellnik knew from personal experience that journalists across the country use computer-assisted reporting and social science methods to produce \"spectacular\" stories, their work often was not recognized in prestigious contests.\n\nThat, Mellnik thought, should change.\n\nThree years later, it has. In September, the National Institute for Computer-Assisted Reporting (NICAR), IRE and the Knight Chair in Journalism at Arizona State University's Walter Cronkite School of Journalism and Mass Communication introduced an award designed to recognize the best journalism done using social science research methods. NICAR is a joint program of IRE and the Missouri School of Journalism.\n\nNamed in honor of Meyer, who holds the Knight Chair in Journalism at the University of North Carolina at Chapel Hill, the Philip Meyer Journalism Award is planned as an annual contest that will make real the kind of recognition Mellnik imagined.\n\n\"I felt we had a lot of people who did spectacular work, and that they needed to be recognized,\" Mellnik says. \"In awards, we tend to honor stories that get results - that put people in jail or result in reform legislation or something like that. But there's also a benefit to stories that establish a methodology and give reporters across the nation a tool that enables them to do good work for their readers.\"\n\nMellnik approached others with his idea, including IRE Board Member Steve Doig, the Knight Chair in Journalism at Arizona State University. Formerly associate editor for research at The Miami Herald, Doig has years of experience using social science research methods. For the past several years, he also has served as a member and chair of the IRE Awards committee.\n\nDoig's work with the committee convinced him that some award-worthy work was being overlooked.\n\n\"The problem with investigative awards (in this context) is that one of the real criteria is that the story should help uncover information that others were trying to keep secret,\" he says. \"Yet there's a great deal of good CAR, precision journalism kinds of things, for instance dealing with the census, that isn't strictly investigative. You're not uncovering wrongdoing; you're uncovering how society works, for instance.\"\n\nMeyer is known for doing that. He pioneered using survey research as a reporter for Knight Ridder newspapers to explore the causes of race riots in the 1960s.\n\n\"He inspired the profession to be more sophisticated and precise in its understanding of and reporting on social issues,\" says IRE and NICAR Executive Director Brant Houston. \"He has been a mentor to many, many journalists - including myself and he encouraged us to learn and use methods that made us and our profession much more credible.\"\n\nDoig called Meyer the \"prime mover\" in raising awareness among journalists to the need for using social science methods.\n\n\"It started with his 1973 book, 'Precision Journalism.' He wrote that really 10 years before it started becoming easier to do the things he was talking about,\" Doig says.\n\nNow in its fourth edition, \"Precision Journalism,\" is widely used as a textbook in journalism schools.\n\nSocial science research methods that would make stories eligible for the award draw on tools used across a variety of disciplines, from sociology and demographics to geography and political science.\n\nThe award \"will help recognize stories that have used social science methods to reveal patterns, trends and sometimes abuses in areas including social or economic or criminal systems,\" Houston says.\n\nBecause IRE has been the leading organization in the area of using social science methods in investigative reporting and training people to do that, it made sense for the organization to cosponsor the awards, Houston says.\n\nNominations for the award must be postmarked by Oct. 31. Appropriate work published or aired between Oct. 1, 2004, and Sept. 30, 2005, will be considered.\n\nA panel of judges including journalism educators and social scientists will choose first, second and third place entries. Judging will be complete by late December, and the awards will be presented during the 2006 CAR Conference scheduled for March 9-12 in Newark, N.J.\n\nTo see a list of judges, download an entry form and read further details about the contest, visit www.ire.org/meyeraward. Additional questions should be directed to the IRE Resource Center at 573-882-6668 or rescntr@ire.org.\n\nAll entries will be placed in IRE's Resource Center story library. Contact Amanda Buck by e-mail at amandab@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-borrowers-buried-under-debt-fees","slug":"2005-septemberoctober-borrowers-buried-under-debt-fees","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-borrowers-buried-under-debt-fees.html","year":2005,"month":"September/October","monthNum":9,"headline":"Borrowers buried under debt, fees","kicker":"SPOTLIGHT: BUSINESS","author_name":"Carrie Teegardin","author_title":"Reporter","affiliation":"The Atlanta Journal-Constitution","summary":"Georgia's transformation into a creditor-friendly state has left many borrowers struggling under high debts and fees, with a focus on fast foreclosure processes and small loan companies.","keywords":["debt","foreclosure","loans"],"topics":["Data & Databases","Business & Economy","Investigative Methods"],"technologies":["Access","MapInfo","Excel"],"full_text":"James Oglethorpe founded the colony of Georgia as a haven for people who had been imprisoned for their debts. But today, Georgia has become perhaps the least forgiving state in the nation for people who owe money. A team of reporters at the Journal-Constitution - Ann Hardie, Alan Judd and myself went to work last year to understand how Georgia was transformed from a debtor's colony to a creditor's paradise. We mined lots of data to understand the lending landscape in Georgia. The series, which kicked off with four parts in January and is continuing, was filled with findings unearthed by our analysis of various databases. Of the 100 counties nationally with the highest rate of personal bankruptcy filings, 45 are in Georgia. We obtained county-level bankruptcy data compiled by the Administrative Office of the U.S. Courts and combined it with county-level population data from the Census Bureau to produce a personal bankruptcy rate for every U.S. county. We used Microsoft Access to perform the analysis. Our database editor, David Milliron, produced a national county-by-county map, using MapInfo Professional, that ran on the first day of the series. We knew the state of Georgia had a relatively high bankruptcy rate, but by presenting the data at the county level we could see compelling information. Legislators who were in session at the state Capitol when the series was published passed around copies of the map. We spent several days in bankruptcy court, interviewing people who had filed, along with bankruptcy attorneys, judges and trustees. We found that Georgia's lightning-fast foreclosure process had prompted many people to seek bankruptcy protection - they were simply trying to save their homes. We were able to establish that no state allows a faster foreclosure process than Georgia, where a house could be sold on the courthouse steps in as few as 37 days after a bank initiated the process. The number of foreclosures initiated against properties in metro Atlanta's 13 core counties in 2004 was more than double the number in 2000. We went to a private contractor to acquire five years of data for every foreclosure initiated in metro Atlanta. The vendor, Equisystems LLC, provided the data to us in a Microsoft Excel file at no cost. Georgia's 'small loan companies' old-time storefront lenders who offer loans to people with poor credit - are alive and well. Their interest rates are high and they increase the cost of the average loan by adding on with more than three questionable insurance policies. Our look at Georgia's 1,000 small loan companies was the most laborious analysis of the series. The companies are required to file detailed annual reports to the state Insurance Commissioner. But the most recent reports were on paper. We hired a data entry contractor to key in the crucial data from the most recent reports. We then crunched the numbers for several years' worth of data, using Access. We did a variety of queries to determine the characteristics of the average loan - how much did the average customer borrow? How many credit insurance policies did they buy? What was the actual cost of borrowing? The findings were eye-opening: most of the 1 million loans taken out by poor Georgians every year came at an effective annual cost of more than 100 percent and the commissioner's office had permitted excessive rates for the credit insurance policies. We estimated that in 2003 small loan customers had overpaid by $65 million for credit insurance. We arrived at that figure by calculating loss ratios on the insurance products.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-septemberoctober-visiting-journalist","slug":"2005-septemberoctober-visiting-journalist","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-visiting-journalist.html","year":2005,"month":"September/October","monthNum":9,"headline":"Visiting journalist","kicker":"Bits & Bytes","author_name":"","author_title":"editor and senior reporter","affiliation":"IRE and NICAR","summary":"Yong-Jin Kim, a veteran KBS reporter from South Korea, has joined IRE and NICAR as a visiting scholar to study U.S. investigative reporting techniques and computer-assisted reporting. The Korea Press Foundation is sponsoring his year-long visit.","keywords":["journalist","investigative","scholarship"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Yong-Jin Kim, a veteran reporter for KBS, South Korea's largest television network, has joined IRE and NICAR as a visiting scholar. At KBS, Kim served as editor and senior reporter for the Investigative and Current Affairs teams. He also is the former director of the National Union of Media Workers.\n\nKim will spend the school year learning about U.S. investigative reporting, including computer-assisted reporting, and working on a reporting project. He received his master's degree in journalism from Dong-A University in Busan, Korea, and is the recipient of numerous awards from the Journalists Association of Korea.\n\nThe Korea Press Foundation is sponsoring his visit with IRE and NICAR.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-global-gathering","slug":"2005-septemberoctober-global-gathering","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-global-gathering.html","year":2005,"month":"September/October","monthNum":9,"headline":"Global gathering","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR participated in the third Global Investigative Journalism Conference in Amsterdam from September 29 to October 2, featuring over 25 computer-assisted sessions led by international journalists on topics like data analysis and mapping.","keywords":["journalism","conference","training"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["World Wide Web"],"full_text":"IRE and NICAR participated at the third Global Investigative Journalism Conference Sept. 29-Oct. 2 in Amsterdam. Top international journalists led debates, workshops, and discussions of case studies on topics affecting global investigative journalism. David Boardman, president of IRE's board of directors, and Brant Houston, executive director of IRE and NICAR, as well as other IRE members, participated.\n\nThe VVOJ, the Dutch-Flemish Organization for Investigative Journalists, organized the event. More than 25 computer-assisted sessions were on the schedule, with topics ranging from \"Life Beyond Google\" to \"Social Network Analysis\" to three sessions covering \"Mapping the News.\"\n\nFor more information about the conference, and other IRE and NICAR training events, visit www.ire.org/training.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-change-of-seasons-in-database-library","slug":"2005-septemberoctober-change-of-seasons-in-database-library","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-change-of-seasons-in-database-library.html","year":2005,"month":"September/October","monthNum":9,"headline":"Change of seasons in Database Library","kicker":"","author_name":"David Herzog","author_title":"","affiliation":"Missouri School of Journalism, IRE and NICAR","summary":"The Missouri School of Journalism introduces its new graduate research assistants for the Database Library who serve as contributing editors to Uplink, including returning students Brian Hamman and Catherine Rentz Pernot, and newcomer Ben Welsh.","keywords":["journalism","assistants","database"],"topics":["Journalism & Newsroom","Data & Databases"],"technologies":[],"full_text":"It's the start of a new academic year here at the Missouri School of Journalism - home to IRE and NICAR - so it's time to introduce the graduate research assistants in the Database Library who also serve as contributing editors to Uplink.\n\nWe have a talented crew, as usual. You may have already talked to the staff if you have called to order data or ask a technical question.\n\nBrian Hamman is a second-year master's student who is returning to the Database Library. He has worked as a reporter, graphics reporter and online assistant city editor for the Columbia Missourian. He is also an editor for MyMissourian, a citizen journalism publication based at the Missouri School of Journalism.\n\nCatherine Rentz Pernot is a second-year master's student specializing in business reporting and is returning to the Database Library. Over the summer she had the Pulliam Fellowship for business reporting at The Indianapolis Star. She also has reported for the Columbia Missourian.\n\nBen Welsh is a second-year master's student in newspaper journalism and he is new to the Database Library. Before coming to Missouri, where he\n\n\"Social Network Analysis\" to three sessions covering \"Mapping the News.\"\n\nFor more information about the conference, and other IRE and NICAR training events, visit www.ire.org/training.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-pollution-data","slug":"2005-septemberoctober-pollution-data","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-pollution-data.html","year":2005,"month":"September/October","monthNum":9,"headline":"Pollution data","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR's Database Library has updated its Toxics Release Inventory data from the EPA through 2003, containing information on chemical releases and waste management from industrial facilities. The cleaned dataset includes enhanced fields like standardized dates and geographic coordinates to help journalists analyze toxic chemical reporting.","keywords":["toxics","database","chemicals"],"topics":["Health & Environment","Data & Databases","Investigative Methods"],"technologies":["GIS"],"full_text":"The IRE and NICAR Database Library recently updated its Toxics Release Inventory (TRI) data from the U.S. Environmental Protection Agency for 2003. The data is available starting with 1987.\n\nThe EPA now requires reporting of about 650 toxic chemicals and toxic chemical categories. The TRI consists of information about on- and off-site releases of these chemicals and other waste management activities reported annually by industries and federal facilities. The data shows what happened to the chemical after it left the facility, such as whether it was recycled, treated or released into the environment, and where the chemicals were sent. One-time accidental releases are also included in the database.\n\nThough this data is available for free through a download from the EPA Web site, IRE and NICAR's Database Library analysts have cleaned the data significantly, making it easier for journalists to use immediately. The Database Library provides additional fields containing standard date formats and latitudes and longitudes that can be mapped using geographic information system (GIS) software. For a sample of the table, samples of other tables and more information about TRI see www.ire.org/datalibrary/databases/toxic.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-looking-anew-at-dams-taxes-bridges","slug":"2005-septemberoctober-looking-anew-at-dams-taxes-bridges","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-looking-anew-at-dams-taxes-bridges.html","year":2005,"month":"September/October","monthNum":9,"headline":"Looking anew at dams, taxes, bridges","kicker":"FIRST VENTURE","author_name":"Lee Davidson","author_title":"special projects reporter","affiliation":"Deseret Morning News","summary":"A veteran journalist returning to local reporting discovers how computer-assisted reporting (CAR) dramatically streamlines investigative work, using a dam safety story updated from 20 years prior as a practical example of how databases and spreadsheets accelerate research.","keywords":["journalism","databases","reporting"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Excel","Access","Google","World Wide Web"],"full_text":"My children say I am ancient. They may be right, because I now see how much older than computer-assisted reporting I am.\n\nThe realization came as I made some first ventures into CAR by essentially updating stories that I wrote 20 years ago - then without the help of computers - about the conditions of local dams and bridges, and about those who fail to pay property taxes.\n\nIt is much easier to dig into these stories with computers, even for a CAR rookie like me who must ask my teenage and twenty-something kids how to fix computer crashes and burps. I feel like the caveman who discovered that hunting with a spear beats hunting with rocks (I also envy the true CAR experts who hunt with the equivalent of rifles and bazookas).\n\nAfter 17 years as the Washington, D.C., correspondent for the Deseret Morning News, I recently returned to Utah as the newspaper's special projects reporter. Because I had limited knowledge of CAR, my editors sent me to an IRE and NICAR Boot Camp in January.\n\nOne of the boot camp training exercises used data from the National Inventory of Dams compiled by the U.S. Army Corps of Engineers. Using a Microsoft Excel spreadsheet, I saw how quickly I could sort and count dams by inspection date, river, nearby cities - and whether they are \"high hazard,\" meaning people would likely die if they fail.\n\nIt brought back memories of 20 years ago when I sat in the state engineer's office for days poring over paper records for a time-consuming but good project about the most dangerous dams in Utah.\n\nI decided to update my old story by testing my new CAR skills. At that moment, the National Inventory of Dams was a couple of years out of date. (The IRE and NICAR Database Library has updated data that it provides to journalists for a fee; see www.nicar.org/data/dams/).\n\nBecause I wanted to see, in part, whether Utah was regularly inspecting high-hazard dams, I surfed the Web to find a more current and local version of the database. A Google search for Utah dams led me to an online database at http://des.utah.gov/nathaz/damsafety.htm It had the same information as the national inventory and is updated monthly. Plus, it has links to online inspection files.\n\nI copied and pasted data from the state Web site into my own Microsoft Excel spreadsheet for sorting and counting.\n\nI called the state dam safety engineer to tell him I was updating my old story, and wanted to interview him (as I did his predecessor long ago) about which dams he felt were the most dangerous. He told me his office maintains another database - unavailable online - that gives a numerical safety rating for each dam. He e-mailed it to me as an Excel file, and I could quickly see which dams the state considered \"unsafe\" because they did not meet current guidelines.\n\nSorting allowed me to see how many unsafe dams were upstream from major Utah cities and to make lists of the worst dams. I discovered that the engineer's office also had a database about dam failures in the state and what caused them. I obtained a copy, which helped make a nice graphic.\n\nSome lessons learned on my first CAR story:\n\n1. Every reporter could easily do a story about the condition of local dams.\n2. Look online for databases kept locally.\n3. Ask local officials for databases they maintain for internal use.\n\nContinuing the theme of updating old stories, I also decided to look online to identify local officials who fail to pay property taxes on time. Twenty years ago, I did that by sorting through paper lists at the county assessor's office and hoped I could recognize names of prominent people. Occasionally, I got lucky.\n\nI found this time that Salt Lake County lists all tax delinquents online. Again, I copied and pasted information from the online database into an Excel spreadsheet, and then imported it to Microsoft Access database manager.\n\nNext, I made another table of information about all known elected officials living in Salt Lake County by searching government Web sites. I included, when available, their addresses, birth dates, spouses and businesses. I then joined the databases by name to find possible matches. After making phone calls to weed out bad matches, I found three officials, including U.S. Sen. Bob Bennett, R-Utah, who were delinquents.\n\nBut I found a much better story once I started playing with the data. I developed lists of businesses and individuals that were the biggest delinquents, often owing hundreds of thousands of dollars. I mostly used Excel, creating pivot tables for amounts owed, and sorted to see the highest amounts.\n\nI realized that even healthy businesses, especially real estate developers, were often choosing not to pay property taxes because penalties and interest are so low in Utah. Several delinquents confirmed that on the record, including the manager of two hotels owned by Bennett that had racked up nearly $400,000 in back taxes.\n\nI also wrote a story about the condition of bridges in Utah, another story that I had done some 20 years ago with paper records.\n\nThe Federal Highway Administration provides bridge summary data by state online in Excel format at www.fhwa.dot.gov/bridge/britab.htm. After downloading and sorting the data, I asked the Utah Department of Transportation for its more detailed database of the 506 \"deficient\" bridges in the state. The data included a deficiency rating from 0 to 100 for each bridge, which allowed quick identification of the worst bridges by Excel sorting.\n\nThe big lesson is, don't be a dinosaur. Dare to do some CAR work. Start with the basics and improve your skills. Maybe update some of your old stories. If an ancient like me can do it, anyone can - just ask my children who will verify that quickly and firmly.\n\nContact Lee Davidson by e-mail at lee@desnews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-uneven-hand-in-enforcement","slug":"2005-septemberoctober-uneven-hand-in-enforcement","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-uneven-hand-in-enforcement.html","year":2005,"month":"September/October","monthNum":9,"headline":"Uneven hand in enforcement","kicker":"PARKING","author_name":"Patrick Lakamp","author_title":"Staff reporter","affiliation":"The News","summary":"A Buffalo News investigation used Freedom of Information requests and geographic information system mapping to analyze 366,500 parking tickets issued over 22 months, revealing patterns in enforcement after reporters noticed frequent parking citations in their community.","keywords":["parking","mapping","Buffalo"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Access","ArcView","CD-ROM"],"full_text":"Our idea to map Buffalo's parking tickets originated in a coffee shop where employees shout warnings to customers when they spot a parking enforcement officer. The ritual can render them practically hoarse.\n\nHaving coffee with a source for an unrelated story, we heard four warnings in an hour. Meanwhile, the source complained about her eight tickets in two years and $300 in fines.\n\nAfter that meeting, staff reporter Jay Rey and I - under the supervision of Susan Schulman, head of The News' investigative reporting team - decided to study the hundreds of thousands of parking tickets issued in the city using computer-assisted reporting. (For more details about reporting the story, see the September-October issue of The IRE Journal.)\n\nWe filed a state Freedom of Information Law request for an electronic database of all tickets written January 2003 through October 2004. The city provided, at no cost, a Microsoft Access database on CD containing 366,500 parking tickets.\n\nWe ran hundreds of queries in Access and analyzed the data using ESRI ArcView 3.2 geographic information system (GIS) software. News researcher Andrew Bailey helped with the data work.\n\nThe data was fairly clean. Parking officers issue tickets by using a handheld device capable of scanning a vehicle registration sticker and automatically recording the time and date. The officers enter the location and violation type.\n\nWe found inconsistencies in the recorded street addresses. Some officers, for example, entered \"Bailey Ave.\" while others entered \"Bailey Avenue.\" We needed consistency with addresses, because we planned to use ArcView's geocoding function to turn those addresses into points on a map. We wanted one map showing the tickets by neighborhood.\n\nBailey created a new table from a GroupBy and Count query of street name to identify streets with the most parking tickets.\n\nMapping patterns\n\nBailey had a good handle on the geocoder's preference for how street names should be identified (Northrup West, rather than W. Northrup), so that's how he standardized street names.\n\nWe found inconsistencies in the recorded street addresses.\n\nSometimes he used Access' Find and Replace function to standardize addresses, but more often he relied on update queries. When Bailey standardizes addresses, he usually creates a table that has one column containing the variations of the street name and then creates a column that contains the correct, standardized, street name. He then uses this table in the update query.\n\nBailey geocoded the cleaned-up addresses and we ended up with 290,000 matches out of our original 366,500 records. Complete success is elusive when geocoding. Some ticket addresses couldn't be mapped because officers sometimes failed to record the street number in the violation location field.\n\nWe could overcome that for violations on short streets contained in one neighborhood, but not for those on streets that cut through several neighborhoods. Ashton Place, for example, is a one-block street in Buffalo's South Park neighborhood. Tickets written on Ashton Place, but lacking a precise address, could therefore be attributed to South Park.\n\nBailey calculated the number of tickets for each neighborhood by using ArcView's spatial join. Spatial joins allow you to assign information from one map layer to another. Bailey overlayed the polygon neighborhoods map from City Hall and joined it to a map layer of parking ticket points. Then, after the join, he calculated the number of tickets in each neighborhood. That number was then displayed for each neighborhood as the percentage of all tickets mapped.\n\nWe could see downtown had the greatest percentage of all parking tickets. But several Upper West Side neighborhoods, each having 5 percent to 10 percent of the tickets, stood out, too. This indicated the Upper West Side was the most ticketed residential area of the city.\n\nZooming in\n\nAnother map created using ArcView showed the Upper West Side close up. Here, Bailey took the county property parcel map and zeroed in on the residential neighborhood. He grouped individual parcel polygons into street blocks - grouping roughly 3,500 parcels into the roughly 160 blocks in our area and assigning a name for each block. Bailey did this by manually selecting the polygon parcels and then using the Dissolve Feature tool in the Geoprocessing Extension to create the larger polygons. Bailey named these larger polygons with the street name and two cross streets. So the block on Cleveland, intersecting Elmwood and Delaware, was called ClevelandElmwoodDelaware.\n\nBailey exported the resulting table and then imported it in Access. He ran some basic queries in Access to determine the address range of each block.\n\nThen Bailey queried the database for the 35 or so streets in the heavily ticketed Upper West Side area. This resulted in about 60,000 tickets. Using the address ranges from the block table, Bailey assigned a specific block to each ticket.\n\nA query grouping by the block was used to create a table with the results, and then was imported back into ArcView for the final block map.\n\nWe verified addresses of some corner properties in question by walking along blocks in the neighborhood and recording house addresses in each block. The violation locations we could not assign to a block, and thus excluded from this map, represented less than 1 percent of the tickets.\n\nWe shaded the map by number of tickets issued in each block. But, of course, not all blocks are created equal. A typical block on Lafayette Avenue, for instance, covers the same distance as three on Ashland Avenue. Our first thought was to group blocks for the streets with smaller blocks, such as those on Ashland. We quickly abandoned this notion because we felt that it would provide an inaccurate depiction of the data by glossing over the variations.\n\nDigging deeper\n\nMost of our maps were not published, but were pivotal in our analysis.\n\nWe created maps for each type of parking violation. This showed us the obvious, that meter overtime violations were mostly downtown, but it also alerted us to ask why there were so many more alternate-side-of-street parking violations in a North Park neighborhood and so many more no-standing violations in a South Buffalo neighborhood.\n\nWe also mapped violations for certain times of the day. This allowed us to see where the parking officers started their days, and which residents had less time to get away with an alternate-side parking violation than others. We mapped morning violations in 20-minute increments.\n\nThe map revealed Ashland Avenue to be the block with the most violations right after 9 a.m., so we knew to take a closer look at that street's data.\n\nOur findings did not surprise those who start their day at the coffee shop in the city's Upper West Side. While downtown and the relatively affluent Upper West Side were prime targets for the city's parking officers, they spent a lot less time in Buffalo's crime-ridden East Side streets. The all-white ticketing force is afraid to go there, we were told.\n\nThe data prompted us to ask about deployment. With eight parking officers working the day shift, one covers South Buffalo and most of the East Side. Another works North Buffalo. The rest work downtown or the Upper West Side.\n\nSo in Buffalo, your fortune with parking tickets hinges on where you park, we reported in a special report called \"Ticketing the city.\"\n\nContact Patrick Lakamp by e-mail at plakamp@buffnews.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-gis-software-discount-for-ire-members-only","slug":"2005-septemberoctober-gis-software-discount-for-ire-members-only","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-gis-software-discount-for-ire-members-only.html","year":2005,"month":"September/October","monthNum":9,"headline":"GIS software discount for IRE members only","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"IRE members can obtain free ArcView GIS software licenses from ESRI with a three-year maintenance agreement, provided they complete qualifying GIS training through IRE, NICAR, or ESRI.","keywords":["software","training","discount"],"topics":["Technology","Journalism & Newsroom"],"technologies":["ArcView","Spatial Analyst","3D Analyst","Geostatistical Analyst"],"full_text":"Members of Investigative Reporters and Editors, Inc., qualify for discounts on geographic information system (GIS) software from ESRI, the publisher of ArcView.\n\nESRI is offering ArcView GIS single-use licenses at no charge to IRE members who agree to attend a GIS training event conducted by IRE and NICAR or ESRI. Purchasers must sign a three-year maintenance agreement with ESRI at a cost of $400 a year, with the first year's fee waived. ArcView, the GIS program most widely used by journalists, lists for $1,500. During the maintenance agreement period, purchasers will receive software upgrades and technical support.\n\nIRE members must attend a qualifying training session within one year of entering the agreement with ESRI, which is based in Redlands Calif., and has been a regular exhibitor at the annual IRE and CAR conferences. Qualifying sessions are IRE and NICAR's Mapping Data for News Stories mini-boot camp, offered two times a year with the next scheduled for Jan. 6-8, 2006; an online ESRI Virtual Campus course, and ESRI classroom training.\n\nFor more information about IRE and NICAR training see: www.ire.org/training/advbootcamps.html. For more information about ESRI training see: www.esri.com/training_events.htm\n\nIRE members also can purchase discounted extension programs, which expand the analytical capabilities of ArcView. The single-license cost for Spatial Analyst, 3D Analyst and Geostatistical Analyst is $1,500 each. That is a 40 percent discount off the list price of $2,500 each.\n\nTo obtain an order form, please contact John Green, membership services coordinator for IRE, at jgreen@ire.org or 573-882-2772.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-homes-drought-a-fiery-mix","slug":"2005-septemberoctober-homes-drought-a-fiery-mix","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-homes-drought-a-fiery-mix.html","year":2005,"month":"September/October","monthNum":9,"headline":"Homes, drought a fiery mix","kicker":"MAPPING","author_name":"Scott North","author_title":"","affiliation":"The Herald","summary":"A journalism team in Washington state used GIS data analysis to investigate wildfire risks in their rapidly growing county, discovering that 5,500 residents live in high-risk fire zones with about 100 new homes being built there annually.","keywords":["wildfire","analysis","GIS"],"topics":["Investigative Methods","Health & Environment","Technology"],"technologies":["GIS","ArcView","dBase","Access","SQL"],"full_text":"It was a moment of potentially dangerous convergence.\n\nMy editors looked at their story budget and saw a hole on the front page a few days hence. At that moment, I didn't have anything pressing on my schedule.\n\nTag. I was it.\n\nThe assignment - find out how an unusual winter drought had affected this normally soggy place - could have been just another weather yarn. Instead, some quick data analysis using geographic information system (GIS) software highlighted a developing hazard that had been taking shape for years in the forestland outside area cities and towns.\n\nThe Herald serves a community of more than 630,000 people in rapidly growing Snohomish County. The county, roughly a half-hour drive north of Seattle, is about the size of Connecticut. Most of the population is concentrated not far from the tideland shores of Puget Sound. To the east are rugged foothills, evergreen forests and thousands of acres of Cascade Mountain Range wilderness, including a slumbering volcano.\n\nLots of rain and snow usually keep this place green. Not this year. The driest winter in 110 years led the governor to declare a drought emergency. Officials forecast what could be the worst wildfire season in years.\n\nIf the fires got going, where would they likely appear, and who would be at greatest risk?\n\nUsing mostly free data arrayed in a GIS, we were able to report that 5,500 people in our county appear to be living in the areas most at risk for wildfires. We found planners have been approving about 100 new homes in those fire-prone areas each year, many outside fire-protection districts.\n\nMy partner on the story, Diana Hefley, knew the state Department of Natural Resources had identified portions of the county as particularly at risk for wildfires. The determinations were based on a number of factors: fire history, presence of brushy fuel, terrain, ease of access, a low level of organized fire protection.\n\nA quick check showed the department had created mapping shapefiles of these fire zones and had posted them on the Web. The state also had 30 years' of wildfire incidents, their locations mapped using X,Y coordinates.\n\nAfter consulting the metadata for these records, we opened the files in ESRI ArcView 9. It is at moments like this that I love Washington's strong public records laws. In addition to the free data on wildfire threats, past and future, my GIS data cache already contained shapefiles of property parcel lines, a decade of building and construction permits already geocoded into points for use in another project, fire district and city boundaries, major roads and rivers, population estimates at the Census block level, plus a fat database containing property tax assessment records.\n\nAfter making sure all the GIS data was in the same projection, I set ArcView to the task of clipping information that would form the core of our analysis.\n\nClipping has become one of my most-used GIS tools. The process is simple. First, decide what data you want to clip, and place that into an ArcView map. Next, overlay the shapefile that contains the area you want to use as a cookie cutter to punch out the relevant records. If you have your GIS ducks in a row, clipping works equally well to extract a section of points (say wildfire locations, or building permits) or to snip out relevant polygons, such as census blocks, or property parcels.\n\nWe clipped data using the state's shapefiles of fire hazard zones. ArcView automatically creates a new shapefile of the data it clips, complete with its own underlying database table in dBASE format.\n\nArcView has SQL query capability, but I still like to export data tables to Microsoft Access database manager for analysis. In short order, I knew how many people lived in the fire zones (by analyzing clipped Census block data), how many new homes on average had received construction permits each year (by analyzing clipped building permits) and the parcel numbers of each piece of property within the hazard areas.\n\nIt was then time to help the data start talking.\n\nParcel numbers are the key to making joins to property data. Using Access, I joined the data table of clipped parcels to property assessment records. Because assessors record the year of construction for each home they record, I was able to track the development history in the clipped area, year by year.\n\nAfter tabulating the home construction data, I dropped the results into a Microsoft Excel spreadsheet and prepared a chart. The resulting trend line, and some quick math, showed that more than three-fourths of the homes in the fire-prone areas had been built in the past 40 years.\n\nBy increasing the transparency of the fire-zone shapefiles, we were able to see clearly the building permit locations, spread across the land like so many freckles. Adding fire-protection district boundaries to the map demonstrated that significant construction had occurred in places not only at risk of burning, but also outside fire protection districts, and rapid assistance from fire crews.\n\nI knew many of the fire-prone areas were in forested mountain foothills. Some quick Web browsing led to digital images that I could open in my ArcView mapping project to accurately demonstrate topography. These images brought some on-the-ground reality to our analysis.\n\nI then turned to the 30 years of fire data at my fingertips. It showed that people had sparked most of the fires, primarily by burning trash and other debris around their homes. I used a land survey shapefile of townships, with its big, uniform-sized blocks, to section off the county. I calculated the number of fires in each section and prepared a graduated color map that showed how many fires had occurred in each part of the community over the years. The map showed the state had been accurate in its concerns about the fire-prone areas.\n\nHefley and I reached out to the experts, sharing Adobe PDF copies of our data maps. We found fire officials and county planners not only aware of the problem, but in some cases, frustrated with their inability to spark attention. There is no law in Washington against building a home in a place that is prone to fire. One former fire marshal told us she feared that Washington - damp, green and growing in population - is just an ill-fated spark away from the devastation and loss of life that has hit other places in the west, especially California.\n\nHer fears seem founded. One of the people we talked with is a state expert on wildfire. The land shows scars from devastating fires that have swept through the forests in these parts every 150 to 300 years, he said. People now call those areas home. We are in the midst of one of the worst dry spells in more than a century. Do the math. You may find a similar story developing in your neck of the woods.\n\nContact Scott North by e-mail at snorth@heraldnet.com\n\nin the past 40 years.\n\nBy increasing the transparency of the fire-zone shapefiles, we were able to see clearly the building permit locations, spread across the land like so many freckles. Adding fire-protection district boundaries to the map demonstrated that significant construction had occurred in places not only at risk of burning, but also outside fire protection districts, and rapid assistance from fire crews.\n\nI knew many of the fire-prone areas were in forested mountain foothills. Some quick Web browsing led to digital images that I could open in my ArcView mapping project to accurately demonstrate topography. These images brought some on-the-ground reality to our analysis.\n\nI then turned to the 30 years of fire data at my fingertips. It showed that people had sparked most of the fires, primarily by burning trash and other debris around their homes. I used a land survey shapefile of townships, with its big, uniform-sized blocks, to section off the county. I calculated the number of fires in each section and prepared a graduated color map that showed how many fires had occurred in each part of the community over the years. The map showed the state had been accurate in its concerns about the fire-prone areas.\n\nHefley and I reached out to the experts, sharing Adobe PDF copies of our data maps. We found fire officials and county planners not only aware of the problem, but in some cases, frustrated with their inability to spark attention. There is no law in Washington against building a home in a place that is prone to fire. One former fire marshal told us she feared that Washington - damp, green and growing in population - is just an ill-fated spark away from the devastation and loss of life that has hit other places in the west, especially California.\n\nHer fears seem founded. One of the people we talked with is a state expert on wildfire. The land shows scars from devastating fires that have swept through the forests in these parts every 150 to 300 years, he said. People now call those areas home. We are in the midst of one of the worst dry spells in more than a century. Do the math. You may find a similar story developing in your neck of the woods.\n\nContact Scott North by e-mail at snorth@heraldnet.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-database-answers-bankruptcy-questions","slug":"2005-septemberoctober-database-answers-bankruptcy-questions","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-database-answers-bankruptcy-questions.html","year":2005,"month":"September/October","monthNum":9,"headline":"Database answers bankruptcy questions","kicker":"SPOTLIGHT: BUSINESS","author_name":"Rosemary Winters","author_title":"Reporter","affiliation":"The Salt Lake Tribune","summary":"The Salt Lake Tribune conducted a computer-assisted investigation into Utah's high personal bankruptcy rate by analyzing a random sample of 1,053 cases filed in 2003-2004, collecting detailed data from court records to identify underlying causes.","keywords":["bankruptcy","database","investigation"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Access","Excel","PACER","PDF"],"full_text":"Since 2002, Utah has held the dubious distinction as the personal bankruptcy capital of the nation, with one of every 37 households filing for bankruptcy each year.\n\nYet the reasons for the state's high rate were largely unknown before The Salt Lake Tribune launched a computer-assisted project last year.\n\nThe Tribune's business desk first studied Utah's rising bankruptcy rate in 1999 as the state was soaring to record levels, but before it hit No. 1. This time we decided to take a closer look at the cases and record more information from each filing.\n\nBusiness editor Lisa Carricaburu led discussions about what kind of data needed to be collected for the project and what questions we wanted to answer. Beat reporters had theories they wanted to test. For example, real estate reporter Lesley Mitchell wondered whether Utah's high rate of home ownership and high mortgage foreclosure rates play into bankruptcy filings. Health care reporter Linda Fantin wanted to test a common perception that aggressive collection tactics by a Salt Lake City-based health care provider led to many bankruptcy filings.\n\nBecause more than 20,000 personal bankruptcy cases are filed in Utah each year, we decided to study a representative sample. We consulted statisticians at the University of Utah and Brigham Young University to determine a reliable sample size. Selecting every 20th case, we obtained a random sample of 1,053 cases filed between July 1, 2003, and June 30, 2004. At a 95 percent confidence level, the sample had a margin of error of plus or minus 3 percentage points.\n\nElectronic copies of bankruptcy case documents are kept as Adobe Portable Document Files by the U.S. Bankruptcy Court for Utah in Salt Lake City. Using PACER, Utah bankruptcy cases can be accessed through the court's Web site at www.utb.uscourts.gov, but they cost 8 cents per page and are about 30 pages or longer. Downloading the 1,053 cases we studied would have cost more than $2,000.\n\nInstead, we obtained permission to use public computer terminals at the bankruptcy clerk's office to access the records for free. Tribune reporters Julie DeHerrera and I compiled the database, using laptops and the court's Internet access to open case files on the court's computer and then transcribe information into a Web form, which submitted the data into a Tribune database. The Web form and underlying Microsoft Access database were created by the Tribune's new media editor, Tony Semerad.\n\nWe used sequential case numbers to open and record every 20th case, assuring the randomness of our sample. We recorded up to 170 items from each case, including income, property, number of dependents and types of debt. We spent about 15 minutes on each case.\n\nThe data entry took nearly four months to complete and was finished in August 2004.\n\nOnce the database was completed, Semerad chose Microsoft Excel to analyze the data because of its easy sorting and statistical functions. He primarily used means, medians and ratios.\n\nDeHerrera and I also flagged potential outlier cases based on data that looked unusual when it was being entered. Semerad distilled general trends from the data, including credit card use, housing expenses, average income, family size, charitable giving, medical debt and whether incomes increased or decreased in the years prior to filing. DeHerrera, I and business reporters Steven Oberbeck, Linda Fantin and Bob Mims helped flesh out the findings. Data from the story was compared to state and national averages.\n\nReporters added interviews with people who filed for bankruptcy, attorneys and financial experts. Using names from the database, they were able to speak with several people who had gone through the bankruptcy filing process. This was perhaps the most challenging piece of the project, since most people didn't want to have their financial troubles publicized.\n\nWe found that Utah's economy, with wages below the national average but living costs at or above average, combines with a culture that emphasizes large families and material success to push people to the brink of financial ruin more often than in other states. Many Utahns live paycheck to paycheck, juggling mortgage payments and other debt with living expenses, so a catastrophe such as a medical problem or job loss makes them more vulnerable to bankruptcy.\n\nAnd, belying the myth that bankruptcy filers are irresponsible spenders, we found evidence that filers chose bankruptcy as a last resort, trying alternatives such as debt counseling, borrowing money from relatives or even selling blood plasma to pay off their debts.\n\nAlthough the Federal Housing Administration recommends that no more than 30 percent of net monthly income pay for housing, we found that bankruptcy filers, on average, spend 47 percent. We found that 85 percent of people who declare bankruptcy are gainfully employed and nearly 40 percent of filers experienced a significant decrease in their annual incomes during the three years prior to filing. Average take-home pay was $1,577 a month, but average monthly living expense was $2,126.\n\nThe cost of health care was also a significant factor. Few filers reported health insurance as a monthly expense but 60 percent reported having unpaid medical bills, a percentage double the national average. Intermountain Health Care's collection tactics were not deemed a significant factor in the bankruptcy rate.\n\nReligious culture also appeared to be a factor. In a state where about 70 percent of the population are members of The Church of Jesus Christ of Latter-day Saints, social norms often dictate that residents marry young and have large families. Many Utahns, in turn, get married, buy homes and have children at a time when they are at the lowest point of their earning power. Often, bankruptcy was filed in an effort to avoid a home foreclosure.\n\nAlmost two-thirds (63.6 percent) of people who filed for bankruptcy had one or more dependent children, making them twice as likely to be supporting children than the average household nationally. Three cases even had seven, eight or 10 kids.\n\nIn addition, members of the LDS Church typically donate 10 percent of their income to the church. Tithing, which was reported by 12 percent of filers, may play an ancillary role in some cases by reducing disposable incomes and allowing people to obtain more credit than they can afford when loans are based on their pre-tithe income.\n\nThrough our research, we were able to show that Utah is No. 1 for bankruptcy for a web of related reasons. The state's socioeconomic forces place more people on the edge, making it easier for people to slip into financial disaster when pushed by unexpected debt.\n\nContact Rosemary Winters by e-mail at rwinters@sltrib.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-extra-extra-your-guide-to-the-latest-investigative-work","slug":"2005-septemberoctober-extra-extra-your-guide-to-the-latest-investigative-work","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-extra-extra-your-guide-to-the-latest-investigative-work.html","year":2005,"month":"September/October","monthNum":9,"headline":"EXTRA! EXTRA! YOUR GUIDE TO THE LATEST INVESTIGATIVE WORK","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"The article provides examples of computer-assisted reporting investigations on business topics, demonstrating how journalists use data analysis to uncover stories about lending discrimination, debt patterns, agricultural decline, food safety, mining violations, workplace safety, and government accountability.","keywords":["data","investigation","business"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":[],"full_text":"Looking for ideas of business stories you can do with computer-assisted reporting? Extra! Extra! on the IRE Web site (www.ire.org/extraextra) carries links to and summaries of stories that used data analysis. Here are some recent examples from the business beat:\n\nThe Charlotte Observer analyzed 2.2 million mortgage applications from the top 25 U.S. lenders and found that blacks were four times as likely as whites to get high interest rates for loans. The disparities existed even when blacks had much higher incomes.\n\nThe Dallas Morning News used Census and state economic numbers to show that the wealthy residents of Collin County, Texas, carry larger debts than residents in other high-income counties across the country.\n\nCensus data helped the San Jose Mercury News tell the story about California's decline in young farmers. The number of farmers younger than 35 fell 44 percent in California and 18 percent nationwide from 1997 to 2002.\n\nThe (Allentown, Pa.) Morning Call used restaurant inspection data to scrutinize food safety in the Lehigh Valley and the rest of Pennsylvania. School cafeterias faired well but the report concluded that the \"patchwork of food safety laws and public health agencies often fails to provide even the minimal monitoring of restaurants and food retailers.\"\n\nThe Charleston Gazette compared color overlays of old mine maps to newer digital maps in their analysis of a coal company building silos outside its permit area and within 300 feet of a school.\n\nThe (Staunton, Va.) Daily News Leader analyzed five years of state Department of Labor and Industry safety inspections and found that many Virginia businesses are not inspected each year. They also reported \"the labor department performed too few safety inspections between 1999 and 2003 to reach even a quarter of the construction businesses in more than a third of the state.\"\n\nThe San Diego Union-Tribune analyzed county grant receipts and discovered \"records for 54 grants totaling nearly $1 million are missing.\"","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-detecting-bloat-in-job-claims","slug":"2005-septemberoctober-detecting-bloat-in-job-claims","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-detecting-bloat-in-job-claims.html","year":2005,"month":"September/October","monthNum":9,"headline":"Detecting bloat in job claims","kicker":"SPOTLIGHT: BUSINESS","author_name":"Mike McAndrew and Michelle Breidenbach","author_title":"","affiliation":"The Syracuse Post-Standard","summary":"Journalists at The Syracuse Post-Standard used Freedom of Information requests and Excel spreadsheets to investigate New York state economic development grants, discovering that 47 percent of companies failed to create the jobs they promised, despite receiving $195 million in taxpayer funding. The investigation revealed that the Empire State Development Corp. rarely imposed penalties on underperforming grant recipients and sometimes provided additional aid instead.","keywords":["accountability","investigation","grants"],"topics":["Investigative Methods","Business & Economy","Government & Politics"],"technologies":["Excel"],"full_text":"Detecting bloat in job claims\n\nBy Mike McAndrew and Michelle Breidenbach, The (Syracuse, N.Y.) Post-Standard\n\nPoliticians sometimes sweep into town to announce they will create or save jobs by giving an economic development grant to a local business.\n\nIt's usually a surefire way for elected officials to get good publicity.\n\nThat's because journalists often report those rosy predictions without going back to see if they come true.\n\nIn New York, 47 percent of the companies awarded $195 million in state grants or loans by a state public authority failed to employ as many workers in 2004 as they promised they would, The Post-Standard discovered.\n\nCompanies had to promise to create or retain specific numbers of jobs to get the taxpayer-funded grants.\n\nGov. George Pataki's top economic development arm, the Empire State Development Corp. (ESD), can impose penalties on grant recipients that miss their job targets. But more often than not, ESD waived penalties. Sometimes the authority offered even more aid to businesses that had not delivered the required jobs.\n\nTo get this story, we used a Microsoft Excel spreadsheet to bolster some old-fashioned digging.\n\nAs a public authority, ESD operates under a governor-appointed board of directors in New York City, outside the normal scrutiny of the state legislature and comptroller.\n\nWe heard that several years ago a state senator had filed a Freedom of Information Law request, just as an ordinary citizen would, for a copy of the authority's annual report. She discovered that, unlike many other state authorities, ESD does not produce one. Instead, ESD provided the senator with a jobs report - one that showed in clear terms that many companies receiving economic development grants were not creating jobs.\n\nWe filed our own Freedom of Information request for the ESD's 2004 jobs report in electronic format.\n\nThe Excel file the ESD sent by e-mail identified nearly 500 economic development grants or loans it made across the state. It included the date the grant was approved, each recipient's starting employment total, the number of workers each company promised to hire or retain, and the businesses' total jobs in the spring of 2003 and 2004.\n\nAn accompanying summary prepared by the ESD staff for the authority's directors noted that ESD had required some companies that missed job targets to repay some of the grants. But the summary didn't identify which companies were penalized or which got waivers. The database also included no addresses for the businesses.\n\nSo we filed a second FOI request with ESD, asking for fields that provided the location of each grant recipient and details about penalties imposed on the nearly 200 companies that had failed to comply.\n\nWe received an Excel spreadsheet with addresses quickly. But when it came to providing details about penalties and waivers, ESD dragged its feet. It took months of telephone calls and a letter hand-delivered to ESD's chairman before we received that information.\n\nOnce we had it, we asked our Excel spreadsheet some questions. Which companies missed their job target by the largest number? What businesses had fewer workers in 2004 than they did when they got their grant? Who eliminated the most jobs, but escaped without a penalty? What companies got ESD to reduce their jobs obligation for 2005?\n\nWe mined the list for potential anecdotes. For instance, why was an aerospace museum in Niagara Falls given $750,000 to create only two jobs?\n\nWe wrote formulas in Excel subtracting current employment from employment at the time of the grant to show layoffs, and subtracting current employment from target employment to show businesses failing to live up to promises. Then we sorted the data.\n\nComputer-assisted reporting enabled us to identify faster the grant recipients who were likely to become the best examples to shine a spotlight on in our story.\n\nThe same calculations pointed out success stories, too. Then we read published reports, corporate Web sites and grant documents, and conducted interviews with officials at companies, to check the accuracy of some job totals that ESD provided. We discovered the state was overstating some of its successes in its report.\n\nFor instance, ESD claimed General Motors had hired 2,000 new workers since it received a $5 million grant. In reality, we learned that GM's Buffalo area engine plant was employing about 1,000 fewer workers than it did before it got the state aid. When we asked ESD about that, the authority admitted it made an error.\n\nWe found the state also was taking credit in its jobs report for helping to retain jobs that had been around for more than a century at one firm. The owner of another business admitted he would have expanded and hired more workers with or without the state's money.\n\nESD declined to allow us to interview anyone but its spokesman. But, in minutes of meetings of the ESD board of directors - a document we had obtained in electronic format via a FOI request - we discovered one nugget. A director had questioned in 2002 whether ESD would be criticized by the press because 27 percent of the economic development grant recipients had missed their job targets that year. Two years later, ESD could only dream of the days when it had a 27 percent failure rate.\n\nBy using CAR tools to take a deeper look at the numbers, The Post-Standard discovered the results of ESD's economic development program were sometimes gloomy - certainly in sharp contrast to the happy talk of politicians announcing they are going to save jobs by giving away a grant.\n\nContact Mike McAndrew by e-mail at mmcandrew@syracuse.com.\n\nContact Michelle Breidenbach by e-mail at mbreidenbach@syracuse.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-dissecting-biggest-public-companies","slug":"2005-septemberoctober-dissecting-biggest-public-companies","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-dissecting-biggest-public-companies.html","year":2005,"month":"September/October","monthNum":9,"headline":"Dissecting biggest public companies","kicker":"SPOTLIGHT: BUSINESS","author_name":"Helen Huntley","author_title":"","affiliation":"St. Petersburg Times","summary":"The St. Petersburg Times describes how it compiles an annual report ranking the 10 largest public companies in the Tampa Bay area by gathering financial data from SEC filings and calculating key metrics using spreadsheets and online tools.","keywords":["companies","financial","rankings"],"topics":["Business & Economy","Data & Databases","Investigative Methods"],"technologies":["Excel","Bloomberg terminal","Yahoo Finance"],"full_text":"Does your area have a billionaires' club? That's what the St. Petersburg Times calls its list of the 10 largest public companies in the Tampa Bay area. You can tally up the wealth of public companies in your area fairly easily by using the Internet to access U.S. Securities and Exchange Commission filings and a spreadsheet to crunch the numbers.\n\nThe Times has been publishing an annual report about the top public companies in our area for 10 years. Initially, we hired an outside vendor to collect and analyze the statistics for us, but we quickly figured out we could do it ourselves. Over the years we have made numerous changes in how the report is prepared. We now do a streamlined report, focused on the 10 largest companies (down from 50), called the Times 10.\n\nOur starting point is the running list we keep of all public companies in our area traded on the New York or American stock exchanges or the NASDAQ Stock Market (bulletin board stocks are excluded). We publish closing prices on these stocks daily. I narrow the list to the ones that have large enough revenues to have a chance of making the top 10 cutoff before I get down to serious data collection. (After you've done this one year, you'll have a good idea of the contenders the following year.)\n\nWe make our comparisons based on the most recently completed fiscal year. Since most companies have a Dec. 31 year-end, I start compiling the report in April and we publish it in May. An alternative approach is to use the four most recently completed quarters. However, that becomes more complicated, so I do it only for one company that is always late getting its results out.\n\nThe primary source for my information is the Form 10-K annual report that companies file with the SEC at the end of the fiscal year. You can find these reports on the SEC Web site (www.sec.gov) or via links on the company's Web site. I use the commercial 10k Wizard (www.10kwizard.com), which provides easy searches of the filings.\n\nAlthough financials can be downloaded, I prefer to write down the information I want and print the pages to check my numbers against later. I collect these numbers for the most recent year and the previous year: Revenues, net income, net income per share and shareholder equity. I average the two years of equity to get the figure used in calculating return on equity.\n\nI use a Bloomberg terminal to get the closing stock price on my chosen date (typically the last Friday in April) and the date two years earlier, to calculate the total return for that period and to get the dividend and market capitalization. If you don't have access to a Bloomberg terminal, you can use Yahoo Finance (http://finance.yahoo.com) to get closing stock prices. If you multiply shares outstanding (from the 10-K) by the share price, you have market cap. I type all this information into a Microsoft Excel spreadsheet.\n\nThe top 10 rankings are based solely on total revenues. Once I have the top 10, I use the spreadsheet to calculate rankings for those companies that made the list. First I type in the formulas to get the information I want, such as the year-over-year change in revenue (2004 revenue 2003 revenue / 2003 revenue) and the return on equity (2004 net income / average shareholder equity for 2003 and 2004). Then I use the Sort function to rank the companies by fastest growing, most profitable, most valuable (based on market cap) and best and worst investments.\n\nWe publish a short profile about each company in the Times 10, including basic information such as address, telephone number, type of business, top executives and number of employees. Normally all this can be found in the 10-K. If you don't follow the company closely, call to verify that the executives haven't changed since the 10-K was published.\n\nOne caution: when companies lose money, it will do funky things to the calculations of year-to-year changes. When I calculate the fastest-growing profits list, I deal with this issue by only considering companies that were profitable in both years. For the individual profiles, I calculate a change in total profits and use positive or negative values based on what makes sense even if it is not mathematically correct. I show a positive number for a company that went from a loss to a profit and a negative number for a company that went from a profit to a loss. If a company lost money both years, I show a positive number if the loss got smaller and a negative number if it got larger. I note this in a footnote and have never had anybody complain about this approach.\n\nContact Helen Huntley by e-mail at huntley@sptimes.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-resources-for-covering-business","slug":"2005-septemberoctober-resources-for-covering-business","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-resources-for-covering-business.html","year":2005,"month":"September/October","monthNum":9,"headline":"Resources for covering business","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR provide computer-assisted reporting resources and databases for business journalists, including tools for investigating labor violations, SBA loans, and federal contracts with real-world examples of impactful investigations.","keywords":["databases","reporting","investigation"],"topics":["Business & Economy","Investigative Methods","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"Resources for covering business\n\nIRE and NICAR offer resources for journalists interested in using computer-assisted reporting to cover business topics.\n\nJournalists can download the \"CAR Can Drive the Business Beat\" presentation developed by IRE from the Donald W. Reynolds National Center for Business Journalism at the American Press Institute Web site. (www.businessjournalism.org/content/6864.cfm). The presentation lists useful Web sites for the business beat, and shows how journalists can integrate simple data analysis into their coverage.\n\nLabor: Find out which local companies have bucked minimum wage, family leave and other federal labor laws with the Wage and Hour Enforcement database. It includes employee and employer information, number of violations and the amount of penalties. In 1998, USA Today did a computer-assisted analysis that showed working children are at risk and that the federal government does little to find or punish those employers who break child labor laws. (Story 15123)\n\nBusiness loans: Which local businesses have had loans guaranteed by the federal government? The SBA7a Business Loans database includes information about loans guaranteed by the U.S. Small Business Administration. It shows which businesses have repaid their loans and under which program the businesses received the loans. A computer-assisted investigation by The Kansas City Star revealed that the SBA gave most of its government-guaranteed loans to firms located in middle- and high-income areas. The paper discovered that minority-owned businesses received fewer than 15 percent of the SBA loans, despite efforts by the administration to target minority owners. (Story No. 16986)\n\nFederal contracts: The Federal Procurement Data System was created by the Government Services Agency. Contracts included in the 2004 data show services being performed or items being produced in all U.S. states, as well as U.S. territories and some foreign countries. The list of services or products being contracted includes telecommunications, maintenance, office furniture, food products, nursing home care contracts, consulting services, military equipment, computer equipment and software, janitorial services, removal and cleanup of hazardous materials, hotel/motel lodging, construction of troop housing, textile fabrics and fuel products. The Corpus Christi (Texas) Caller-Times did a series of computer-assisted stories in 2002 on the House Armed Services Committee third-ranking Democrat, Rep. Solomon Ortiz, and his influence on and personal gain from defense federal contracts in his district. (Story No. 19602)\n\nFor more details and a comprehensive list of what the Database Library has to offer for reporting on business go to www.ire.org/datalibrary/databases. To order databases, call 573-884-7711 or 573-884-7332.\n\nTo order story reprints contact the IRE Resource Center at 573-882-3364 or rescntr@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-tech-tip-making-xml-useful","slug":"2005-septemberoctober-tech-tip-making-xml-useful","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-tech-tip-making-xml-useful.html","year":2005,"month":"September/October","monthNum":9,"headline":"Tech tip Making XML useful","kicker":"","author_name":"Jeff Porter","author_title":"","affiliation":"IRE and NICAR","summary":"This article explains XML (Extensible Markup Language) and its practical challenges for database users, particularly in government agencies, demonstrating how XML's flexible structure differs from traditional flat files and can be imported into tools like Excel and Access.","keywords":["XML","database","government"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["XML","Excel","Access"],"full_text":"If you have not had to import XML data into a database manager, you're lucky so far - more and more government agencies use XML to store and transmit information. If the data is relatively small, that's good. But XML files can easily become bloated monsters, frustrating a user of off-the-shelf database software.\n\nWhat is XML? Several definitions are floating around the Web.\n\nExtensible Markup Language is a simple, very flexible text format Originally designed to meet the challenges of large-scale electronic publishing, XML is also playing an increasingly important role in the exchange of a wide variety of data on the Web and elsewhere. (www.advancinginsights.com/mybiz/)\n\nA widely used system for defining data formats. XML provides a very rich system to define complex documents and data structures such as invoices, molecular data, news feeds, glossaries, inventory descriptions, real estate properties, etc. As long as a programmer has the XML definition for a collection of data (often called a \"schema\") then they can create a program to reliably process any data formatted according to those rules. (www.unitedyellowpages.com/internet/terminology.html)\n\nCapable of describing many different kinds of data. Its primary purpose is to facilitate the sharing of structured text and information across the Internet. Languages based on XML (for example, RDF, RSS, MathML, XSIL and SVG) are themselves described in a formal way, allowing programs to modify and validate documents in these languages (http://n.wikipedia.org/wiki/XML)\n\nIf those definitions leave your head spinning (as they did mine), we will focus on the practicality of dissecting XML files - making them useful for data analysis in your software of choice. An XML data file does not have the usual feel of a flat file, such as ASCII and the like. There are no headers, no fixed fields, no columns. Instead, each element is defined line by line, as in this example from the Microsoft support Web site (http://support.microsoft.com).\n\n<ROOT> <Customers> <CustomerId>1111<CustomerId> <CompanyName>Sean Chai</CompanyName> <City>NY</City> </Customers> <Customers> <CustomerId>1112</CustomerId> <CompanyName>Tom Johnston</CompanyName> <City>LA</City> </Customers> <Customers> <CustomerId>1113</CustomerId> <CompanyName>Institute of Art</CompanyName> </Customers> </ROOT>\n\nA bit of translation is in order here.\n\nThe tag <Customers> (and its counterpart, </Customers>) identifies each record that pertains to a given customer. It's repeated for each new record. You can trace the same syntax for CustomerId, CompanyName and City fields. The ROOT tags surround it all. The design can be simple, but in some ways, it's reminiscent of hierarchical data - bringing back the term \"variable length\" - in that each field does not necessarily have to show up in each record.\n\nMicrosoft Excel or Access can import and use many of the XML online files. For practice, download U.S. House roll call voting data that you can get from http://clerk.house.gov/evs/2005/index.asp. For this example, we chose to save this page: http://clerk.house.gov/evs/2005/roll464.xml.\n\nExcel 2003's File > Open menu item has an option to choose XML as file type. After you choose this, it will say \"The file you are opening contains stylesheet(s). What would you like to do:\" Your options are to use an existing stylesheet, ending with the suffix \"xsl\" - in essence, a technical record layout - or opening the file without applying a stylesheet - in other words, relying on the general structure of tags in the file.\n\nWhere does the stylesheet come from? In this instance, it's on the House Web site. Online, the XML file looks nice and pretty. But viewing it with a text editor or using your browser's View Source function, you can easily find a reference for the stylesheet:\n\n<?xml version=\"1.0\" encoding=\"UTF-8\"?> <!DOCTYPE rollcall-vote PUBLIC \"-//US Congress//DTDs/vote v1.0 20031119 EN\" \"http://clerk.house.gov/evs/vote.dtd\">\n\n<?xml-stylesheet type=\"text/xsl\" href=\"http://clerk.house.gov/evs/vote.xsl\"?>\n\nIf you go to http://clerk.house.gov/evs/vote.xsl, you can view the entire document - a mixture of record layout, code sheet, and tips about the internal data the House uses to create the XML pages.\n\nThat brings us back to the question about whether to use the stylesheet or the file's tags. Which is better? It varies from case to case, so it is a good idea to try both during your first attempt at importing the data.\n\nIn this particular instance, using the stylesheet generates an Excel file that looks suspiciously like the Web page. Go with the other option, pick as an XML list or a read-only workbook and you get what I believe is a better-looking spreadsheet. You actually gain a bit more information, too: party affiliation. Using the stylesheet, parties are identified only by font: Roman type is Republican, italic, Democrat. Independents are underlined. Without stylesheet, the parties are identified with a column containing D for Democrat, R for Republican and I for Independent. And as an added bonus, you get the unique House ID number for each legislator - helpful in joining tables later.\n\nThe loser in this test was Access 2003. You can import the XML using the Import Wizard. Just open or create a database, go to File > Get External Data > Import and choose XML as file type.\n\nBut in this case, the results are disappointing. Like many dataset examples, the XML is structured into several separate \"tables,\" so our results include five tables. The most vital one lists names of legislators and individual votes. Party affiliation and the House IDs are missing.\n\nOther applications are available for working with XML:\n\nXMLSpy from Altova. This program was suggested by several people in and out of journalism. You can download a 30-day evaluation copy from the company Web site at www.altova.com.\n\nNavicat from PremiumSoft. Navicat is a popular graphical interface for MySQL relational database manager that provides a high level of control during importing. A 30-day evaluation copy is available at www.navicat.com.\n\nSAS. This powerhouse stats and data management program (www.sas.com) includes the SAS XML Mapper tool that, in a point-and-click interface, lets the user create a text file of instructions for field names, format and location. You can also edit the file for more control.\n\nPerl. Users of this programming language can download modules to translate XML files.\n\nXML was never designed (in my humble opinion) as a format to store mammoth amounts of data, but increasingly it is being used for large and complex sets of information. Chances are that journalists doing computer-assisted reporting will probably run into a database in this format. Whether it's a huge file, or a small one, there are plenty of tools for CAR practitioners to try.\n\nContact Jeff Porter by e-mail at jeff@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-septemberoctober-home-mortgage-resources","slug":"2005-septemberoctober-home-mortgage-resources","url":"https://dwillis.github.io/uplink/article/2005-septemberoctober-home-mortgage-resources.html","year":2005,"month":"September/October","monthNum":9,"headline":"Home mortgage resources","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The Charlotte Observer's analysis of Home Mortgage Disclosure Act data revealed that black borrowers receive high-interest loans at four times the rate of white borrowers, prompting journalism resources like IRE and NICAR to provide guidance for reporting on mortgage lending disparities.","keywords":["mortgages","disparities","lending"],"topics":["Data & Databases","Investigative Methods","Demographics & Census"],"technologies":[],"full_text":"The Charlotte Observer, working with new Home Mortgage Disclosure Act data, reported in August that black borrowers from the largest U.S. lenders were four times more likely than whites to receive high-interest loans.\n\nThe Observer analyzed more than 2 million records of home purchase loans from 25 of the largest lenders in the United States. The Observer was able to do the story because federal banking regulators, starting in 2004, began requiring lenders to report whether a loan carries a high interest rate.\n\nIRE and NICAR offer resources for journalists who are interested in using the HMDA data to detect racial disparities or report on housing trends.\n\nA good start for beginning an examination of mortgages is the IRE Beat Book, \"Home Mortgage Lending: How to Detect Disparities,\" by Jo Craven McGinty. The beat book explains how to access and analyze home mortgage data, what questions to ask, and what other resources are available. The Beat Book is available for purchase from the IRE Store at www.ire.org/store or the Resource Center at 573-882-3364.\n\nThe Home Mortgage Disclosure Act database, available from the IRE and NICAR Database Library, features more than 41 million records of loan applications for 2003, with archival data reaching back to 1992. The data includes demographic information about the loan applicants, as well as the reason for the loan and whether it was accepted or rejected. Journalists can purchase data for the country or a state by calling 573-884-7711. The Database Library hopes to make updated data available to journalists in October.\n\nIn addition to these two key resources, the IRE Resource Center offers stories and tipsheets:\n\n\"Burrowing into HMDA,\" by John Byczkowski of The Cincinnati Enquirer provides guidance for journalists looking to dig into the home mortgage data. This tipsheet offers cautions about interpreting the data. (Tipsheet No. 1774)\n\nIn \"Seeing Red: How to Expose Redlining Practices,\" Penny Loeb of U.S. News & World Report provides useful background information for journalists working with the HMDA data. (No. 410)\n\nThe (Fremont, Calif.) Argus, analyzed home mortgage data and found stark disparities in rejection rates between minority and nonminority applicants. They found that Hispanics and blacks were twice as likely to be turned down in the various counties in California. The article also concentrated on possible ways of solving the problem. (Story No. 21008)\n\nThe (Nashville) Tennessean focused on how race played into home mortgage lending in the Nashville area, showing that black home owners were denied for mortgage loans \"up to three times more often than whites even when incomes were similar.\" (No. 21136)\n\nCall 573-882-3364 or e-mail rescntr@ire.org to order copies of these tipsheets or stories.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-novemberdecember-planning-for-the-worst","slug":"2005-novemberdecember-planning-for-the-worst","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-planning-for-the-worst.html","year":2005,"month":"November/December","monthNum":11,"headline":"Planning for the worst","kicker":"DISASTERS","author_name":"Theresa Wieberg","author_title":"","affiliation":"IRE and NICAR","summary":"Journalism veterans suggest maintaining a library of essential databases like Census data and property tax rolls to prepare for reporting on disaster aftermaths.","keywords":["Disaster Preparedness","Databases","Census"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["CD-ROM"],"full_text":"Preparing for disaster might sound like an impossible task – after all, events such as this season's hurricanes do not wait until those in their paths are ready for their fury. But there are steps journalists can take to get ready for computer-assisted reporting after the big one hits. Journalism veterans say there are some essential databases for newsrooms to use in reporting on the aftermath of a disaster. So what should you keep on hand and updated? \"Any databases that give you a comprehensive picture both of the people and the structures of your area, such as detailed Census data, property tax roll, flood plain maps in database form ready for mapping, and information about the transportation system of the area,\" says Steve Doig, a journalism professor at Arizona State University and a member of Investigative Reporters and Editors' board of directors. Property tax rolls, available in electronic form, are at the heart of any investigation about who and what is affected by a disaster. These databases provide information about the characteristics of each property, such as buildings and land use, and who owns it. Property records are also useful in tracking activity as communities rebuild, says Margot Williams, database research editor at The New York Times. \"They may give us a way to figure out down the road where people went, to see where they were, what new ownership will be and where people used to live,\" Williams says. \"It allows us to take a look back when we see what places like Louisiana look like a year or two from now.\" Data from the 2000 Census – the most recent – contains detailed population information, down to areas as small as a city block. Journalists can use this data to generate estimates of the number of people stricken by disaster and describe the affected areas. Williams says the newspaper's strategy includes backing up databases on servers that are accessible off-site. It also has a newsroom already set up elsewhere if journalists need to relocate from the newspaper's main newsroom. Doig stressed the importance of off-site backup as part of a plan to get a newsroom back in operation. \"Individual reporters can dump files onto a CD, but the IT department should also have a plan that puts [the files] in another location.\" Doig recalled the 2001 anthrax contamination at the American Media Inc. building in Boca Raton, Fla., which shut down the building. AMI, which publishes supermarket tabloids, abandoned the building. \"With a little bit of thought, you can anticipate certain disasters, and with a relatively small amount of money, you can make sure it won't be a crippling one.\"","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-novemberdecember-probing-femas-chronic-illness","slug":"2005-novemberdecember-probing-femas-chronic-illness","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-probing-femas-chronic-illness.html","year":2005,"month":"November/December","monthNum":11,"headline":"Probing FEMA's chronic illness","kicker":"AFTER A HURRICANE","author_name":"John Maines","author_title":"","affiliation":"South Florida Sun-Sentinel","summary":"The South Florida Sun-Sentinel's year-long investigation into FEMA revealed systemic waste, fraud, and a lack of accountability in disaster relief payments.","keywords":["FEMA","Fraud","Disaster Relief"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["ArcGIS"],"full_text":"The meltdown of the Federal Emergency Management Agency's response to Hurricane Katrina stunned America and the rest of the world. At the South Florida Sun-Sentinel in Fort Lauderdale, there was a lot less surprise. By the time Katrina hit New Orleans, we had spent the better part of a year documenting troubles with the FEMA relief efforts that followed Florida's four hurricanes in 2004. We found one problem after another, ranging from widespread fraud and waste in the agency's disaster assistance program to the discovery that many FEMA housing inspectors had criminal histories. Our work led to a U.S. Senate investigation and a demand by senators for sweeping changes at FEMA to shore up sloppiness and improve accountability. FEMA responded to 313 declared disasters across the United States and in U.S. territories between 1999 and 2004. We mapped out the money by ZIP code using ArcGIS 9 and compared our data maps to any document we could find – public records of damage in these areas, newspaper stories, police or fire calls, ambulance runs, public utility flooding reports, and the National Climatic Data Center's storm events database. During the investigation Sun-Sentinel reporters visited the inner cities of Baton Rouge, Cleveland, Detroit, and Los Angeles. In each, residents told us almost mirror versions of what we had heard in Miami: FEMA was an easy mark. As a result of the Sun-Sentinel's articles, in May hearings were held before the U.S. Senate Committee on Homeland Security and Governmental Affairs. Two months later the committee sent then-FEMA director Brown a letter saying \"serious shortcomings\" existed in the nation's disaster relief program that \"allowed taxpayer dollars to be wasted.\" FEMA announced a series of reforms in its Building on Success report, outlining plans for greater scrutiny or work by inspectors, who now are required to take photographs of damaged property.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-novemberdecember-spotlight-for-more-about-hurricane-destruction-see","slug":"2005-novemberdecember-spotlight-for-more-about-hurricane-destruction-see","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-spotlight-for-more-about-hurricane-destruction-see.html","year":2005,"month":"November/December","monthNum":11,"headline":"SPOTLIGHT: For more about hurricane destruction see:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This spotlight section directs readers to multiple resources about hurricane destruction, including articles on mapping storm damage with GIS technology, identifying vulnerable populations, and reporting on coastal communities.","keywords":["hurricane","damage","reporting"],"topics":["Public Safety","Technology"],"technologies":["GIS","GPS"],"full_text":"SPOTLIGHT: For more about hurricane destruction see:\n- GIS and GPS plot storm damage, p. 6\n- Identifying at-risk populations, p. 8\n- Covering coastline stories, p. 10\n- The November-December 2005 IRE Journal","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-novemberdecember-assault-data-riddled-with-holes","slug":"2005-novemberdecember-assault-data-riddled-with-holes","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-assault-data-riddled-with-holes.html","year":2005,"month":"November/December","monthNum":11,"headline":"Assault data riddled with holes","kicker":"TROUBLE IN SCHOOLS","author_name":"Jeff Roberts and David Olinger","author_title":"","affiliation":"The Denver Post","summary":"An investigation found that Colorado school districts use narrow legal definitions to exclude serious violent incidents from their public accountability report cards.","keywords":["School Violence","Accountability","Colorado Education"],"topics":["Crime & Justice","Education","Investigative Methods"],"technologies":[],"full_text":"When is an assault not an assault? When public schools in Colorado say so, as these cases illustrate: A child is beaten on his way home from middle school by five older boys in ski masks. Five girls ambush a 10th-grader with a blow to the head. They hold her down, pull up her sweater and burn her with a cigarette. A 13-year-old girl suffers bruised kidneys and blood in her urine after a girl jumps her from behind. None of these were assaults. Not according to the yearly \"report cards\" – or School Accountability Reports (SARs) – the schools delivered to parents last year along with their children's performance on the state assessment tests. Parents get a nice pamphlet with all sorts of information about their kid's school such as student-teacher ratios, academic ratings, teacher experience and average daily attendance. While Jefferson County, the state's largest school district, listed nearly 650 fights and assaults on its report cards to parents, Denver Public Schools listed only 103. Suburban Cherry Creek schools reported just one fight for the entire school year, as did the Aurora school district. Police reports, detailing each time officers were called to a school because of violence, were the key. From the reports we learned Denver police had recorded at least 345 assault arrests at high schools last year, while the SARs listed only 38 assaults. State education officials told us they had limited the \"assaults/fights\" category on accountability reports to first- and second-degree felony assaults to avoid listing every shoving match, but this definition effectively hid serious violence from parents.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-novemberdecember-entries-in-philip-meyer-journalism-awards","slug":"2005-novemberdecember-entries-in-philip-meyer-journalism-awards","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-entries-in-philip-meyer-journalism-awards.html","year":2005,"month":"November/December","monthNum":11,"headline":"Entries in [Philip Meyer Journalism Awards]","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"The Philip Meyer Journalism Awards, a new annual award recognizing journalism that employs social science research methods, is currently in its inaugural judging phase with 28 entries competing for prizes ranging from $200 to $500. The awards will be presented at the 2006 CAR Conference in March and are named after Philip Meyer, a pioneer in precision journalism.","keywords":["journalism","awards","research"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"Judging for the Philip Meyer Journalism Awards, created to recognize journalism that uses social science research methods, is under way.\n\nThe awards, new this year, will honor the best use of methods such as surveys, content analyses, field experiments, social network analyses and statistical analyses in news reporting.\n\nFor the inaugural awards, judges are choosing from 28 entries postmarked before the Oct. 31 deadline. The judging is scheduled to be completed on Dec. 20 and the awards will be presented at the 2006 CAR Conference, which is scheduled for March 9-12 in Newark, N.J.\n\nFirst, second, and third prizes will be awarded.\n\nThe awards were announced by IRE, NICAR and the Knight Chair in Journalism at Arizona State University this year. NICAR is a joint program of IRE and the Missouri School of Journalism.\n\nThe annual award is named in honor of Meyer, author of \"Precision Journalism\" and a pioneer in using social science methods in news reporting. Meyer is the Knight Chair in Journalism at the University of North Carolina at Chapel Hill.\n\nThe awards include cash prizes of $500 for first place, $300 for second place, and $200 for third place.\n\nFor more information, visit www.ire.org/meyeraward.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-novemberdecember-boating-resources","slug":"2005-novemberdecember-boating-resources","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-boating-resources.html","year":2005,"month":"November/December","monthNum":11,"headline":"Boating resources","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE provides comprehensive resources for journalists covering boating accidents, including databases, tipsheets, and archived articles accessible through their dedicated webpage.","keywords":["boating","journalism","databases"],"topics":["Public Safety","Investigative Methods"],"technologies":["World Wide Web"],"full_text":"IRE offers resources to journalists covering boating accidents, such as the Lake George, N.Y., tour boat accident in October.\n\nThe boating accidents resources Web page includes information about databases useful in covering boating accidents that are available from the IRE and NICAR database library. In addition, there are synopses of related tipsheets and stories available from the IRE Resource Center, and references to relevant past articles in Uplink and The IRE Journal.\n\nThe site can be found at www.ire.org/inthenews_archive/tourboat.html.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-novemberdecember-hurricane-help","slug":"2005-novemberdecember-hurricane-help","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-hurricane-help.html","year":2005,"month":"November/December","monthNum":11,"headline":"Hurricane help","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE provides journalists with resources for hurricane coverage including stories, tipsheets, databases, and links to useful reporting tools available on their website and through their Database Library.","keywords":["hurricanes","databases","journalism"],"topics":["Journalism & Newsroom","Data & Databases"],"technologies":["Internet","World Wide Web"],"full_text":"Visit www.ire.org/inthenews_archive/katrina.html on the IRE Web site for stories, databases and tipsheets available for journalists covering the aftermath of hurricanes.\n\nThe Web site includes detailed descriptions of stories about hurricanes and tipsheets for reporting on hurricanes that are in the IRE Resource Center. In addition, the Web site includes links to other sites that journalists will find useful in their reporting.\n\nJournalists also can get more information about databases available from the Database Library that can be used in reporting about hurricanes and their economic effects.\n\nFor more information about the databases visit www.ire.org/datalibrary or call the Database Library at 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-novemberdecember-covering-the-big-story","slug":"2005-novemberdecember-covering-the-big-story","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-covering-the-big-story.html","year":2005,"month":"November/December","monthNum":11,"headline":"Covering the big story","kicker":"","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"David Herzog provides a roadmap for journalists to use computer-assisted reporting to investigate the long-term impacts of Hurricane Katrina on local communities.","keywords":["Hurricane Katrina","Data Journalism","FEMA"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":[],"full_text":"The aftermath of Hurricane Katrina and its lesser siblings is the big U.S. news story of the year. As journalists document how evacuees are settling into nearly every corner of the United States, it is easy to see how the ripple effects of this devastating storm will generate local news stories in the years to come. Here are five key areas that journalists using computer-assisted reporting can focus on: 1. The resettlement. Journalists will want to examine how the resettlement affects schools, social services, housing, employment and the economy in their towns. 2. Racial inequity. New Orleans residents left behind were disproportionately African-American; data can show if minorities in other areas are similarly at risk. 3. FEMA's failures. Journalists can use FEMA's data to show if aid was wasted on people who suffered no losses locally. 4. Coastal safety. Use data to report on population growth versus the erosion of natural safeguards like wetlands. 5. The reconstruction. Federal spending, loan programs, and property databases can track the flow of billions of dollars.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-novemberdecember-about-our-contributors","slug":"2005-novemberdecember-about-our-contributors","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-about-our-contributors.html","year":2005,"month":"November/December","monthNum":11,"headline":"About our contributors ...","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This section lists the professional backgrounds and expertise of journalism contributors, many of whom specialize in computer-assisted reporting (CAR) and data analysis across various major news organizations.","keywords":["journalists","reporting","data"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["GIS"],"full_text":"About our contributors ...\n\nAdam Bell specializes in computer-assisted reporting at The Charlotte Observer.\n\nChase Davis is a senior at the Missouri School of Journalism. Through December, he is working as a reporting intern at The Boston Globe. His Web site focusing on computer-assisted reporting is at www.car-chase.net.\n\nJohn Keefe is news director for WNYC-New York Public Radio. He is an alumnus of the 2005 Editor mini-boot camp.\n\nJohn Kelly is space editor at Florida Today and is one of the newspaper's computer-assisted reporting specialists.\n\nJohn Maines is assistant research manager/computer-assisted reporting specialist at the South Florida Sun-Sentinel in Fort Lauderdale. He is a 2000 alumnus of the Advanced Statistics mini-boot camp.\n\nDavid Olinger has been a reporter at The Denver Post since 1997. He has written many stories with the assistance of public databases or in-house statistical analyses: on unregulated gun trades after the Columbine High massacre, predatory foreclosure investors, child abuse deaths, neighborhood pollution and Census trends.\n\nJeff Roberts is The Denver Post's computer-assisted reporting editor. He has worked at The Post, as a reporter or editor, since 1984. He attended the IRE and NICAR Computer-Assisted Reporting Boot Camp in 1997 and has been a presenter and instructor at the annual IRE and CAR conferences.\n\nMatthew Waite is a reporter for the St. Petersburg Times who specializes in computer-assisted reporting and the use of geographic information systems (GIS). Waite is a 1999 boot camp alumnus.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-novemberdecember-suspensions-underreported","slug":"2005-novemberdecember-suspensions-underreported","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-suspensions-underreported.html","year":2005,"month":"November/December","monthNum":11,"headline":"Suspensions underreported","kicker":"TROUBLE IN SCHOOLS","author_name":"Adam Bell","author_title":"","affiliation":"The Charlotte Observer","summary":"The Charlotte Observer used a detailed suspension database to uncover significant racial disparities and a high frequency of repeat suspensions in local schools.","keywords":["School Suspensions","Racial Disparities","Charlotte-Mecklenburg"],"topics":["Education","Demographics & Census","Data & Databases"],"technologies":["Access"],"full_text":"This year, when we examined the Charlotte-Mecklenburg Schools' suspension rate, we found that more than 450 students had been suspended at least 10 times. We started our analysis with a Microsoft Access database that listed 52,600 suspensions in a district of 114,000 students. For a follow-up story, we requested from the district more complete data. This revised database allowed us to give interested parents and others a way to look at schools and compare them to others. We felt the more accurate picture on race was provided by using the total individuals suspended rather than total suspensions. We made clear the distinction. Overall, CMS suspended 8 percent of its white students and 30 percent of its black students, disparities that exist nationally, but are not easily explained. This complex issue touches on several factors, including racism, cultural misunderstanding by teachers and poverty. In Charlotte's home county, black school-age children were four times more likely to live in poverty than white children. After the series ran, the CMS board called for a review of discipline practices to stop the surging suspensions.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-novemberdecember-destination-failing-schools","slug":"2005-novemberdecember-destination-failing-schools","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-destination-failing-schools.html","year":2005,"month":"November/December","monthNum":11,"headline":"Destination: Failing schools","kicker":"TROUBLE IN SCHOOLS","author_name":"John Keefe","author_title":"","affiliation":"WNYC-New York Public Radio","summary":"WNYC's analysis of enrollment data found that special education students in New York City are disproportionately concentrated in the city's worst-performing high schools.","keywords":["Special Education","Enrollment Data","Failing Schools"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["spreadsheet"],"full_text":"For years, there had been rumblings that kids with special needs were being \"dumped\" into New York City's worst high schools. Reporter Beth Fertig decided to see whether enrollment numbers confirmed what the parents of these children had long suspected. While our analysis could not detect evidence of intentional dumping, it did show that special education students and English language learners were disproportionately overrepresented in the city's failing and most violent public high schools. At the same time, those kids – as a group – were losing out on the new, small schools created to improve education. Fertig started by filing a request under New York's freedom of information law for enrollment data about each of the city's 337 public high schools. After we compiled the data, we created a column to calculate the percent of special needs students in each school and then sorted the data by that percentage in descending order. At a glance it was easy to see the list.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-novemberdecember-gis-gps-after-the-storm","slug":"2005-novemberdecember-gis-gps-after-the-storm","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-gis-gps-after-the-storm.html","year":2005,"month":"November/December","monthNum":11,"headline":"GIS, GPS after the storm","kicker":"MAPPING IT OUT","author_name":"Matthew Waite","author_title":"Reporter","affiliation":"St. Petersburg Times","summary":"Matthew Waite explains how to use GPS and GIS tools to rapidly map hurricane destruction, providing better visual context and reporting for readers.","keywords":["GIS","GPS","Hurricane Mapping"],"topics":["Investigative Methods","Technology","Public Safety"],"technologies":["GPS","GIS"],"full_text":"After a hurricane strikes, one of the hardest things to do early is get a good grip on the scope of the damage. One of the best ways to do it is with mapping. You will need a geographic information system (GIS) program and a global positioning system (GPS) receiver. Mark damage points when you see them. If you have more than one GPS unit, you can send them out with other reporters and compile the data later. I did that for Hurricane Ivan in 2004 and used GPS to map damage. Once airborne in a helicopter, power up your GPS. Every time you spot newsworthy damage, create a waypoint. Note the waypoint numbers in your notebook and describe what you see for each. You can download the points to your GIS after you land. The next day, we published dramatic pictures, details of damage in areas closed off to other reporters and a damage map, both in print and interactively online.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-novemberdecember-maps-identify-at-risk-areas","slug":"2005-novemberdecember-maps-identify-at-risk-areas","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-maps-identify-at-risk-areas.html","year":2005,"month":"November/December","monthNum":11,"headline":"Maps identify at-risk areas","kicker":"AFTER A HURRICANE","author_name":"John Kelly","author_title":"Reporter","affiliation":"Florida Today","summary":"Florida Today used GIS to identify thousands of residents in hurricane-prone areas who were eligible for evacuation assistance but had not signed up for the program.","keywords":["Evacuation","Vulnerable Populations","GIS"],"topics":["Public Safety","Demographics & Census","Investigative Methods"],"technologies":[],"full_text":"After Katrina, we decided to test our community's readiness to move people to safety in event that a really monstrous storm ever roared our way. Are there people living in hurricane-threatened areas of our county who might not have the means to escape? We combined data from local, state and federal sources to look at the number of people, homes and businesses located in the most vulnerable areas of the county. The answer was \"Yes.\" Not as many as New Orleans, but the numbers show thousands of seniors, poor people and families without cars living in risky places. Brevard County has a program for free rides to people who are elderly or disabled, but you have to sign up in advance. About 2,000 people had signed up; our numbers revealed at least 20,000 might qualify. We used Census block groups to identify these neighborhoods. The resulting story and map prompted good discussions among emergency management officials, and at least a couple of hundred people contacted the county to get on the pick-up list.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-novemberdecember-covering-coastlines-with-ire-resources","slug":"2005-novemberdecember-covering-coastlines-with-ire-resources","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-covering-coastlines-with-ire-resources.html","year":2005,"month":"November/December","monthNum":11,"headline":"Covering coastlines with IRE Resources","kicker":"AFTER A HURRICANE","author_name":"Amanda Buck and Rhitu Chatterjee","author_title":"","affiliation":"IRE and NICAR","summary":"A guide to IRE resources and past investigative stories that can help journalists cover issues related to coastal development and environmental vulnerability.","keywords":["Coastal Change","IRE Resources","Investigative Reporting"],"topics":["Investigative Methods","Health & Environment","Journalism & Newsroom"],"technologies":[],"full_text":"IRE and NICAR offer numerous resources that can help journalists report on coastline change using computer-assisted reporting. Highlights include the series \"Boom on the Beach\" by USA Today, which examined economic growth along coasts, and \"Shoreline in Peril\" by Newsday, which found property owners building at public expense. Tipsheets such as \"Land Scandals by the Sea\" focus on how to look at court documents and environmental records for coastal development stories.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2005-novemberdecember-google-maps-api-mapping-hacks","slug":"2005-novemberdecember-google-maps-api-mapping-hacks","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-google-maps-api-mapping-hacks.html","year":2005,"month":"November/December","monthNum":11,"headline":"[Google Maps API / mapping hacks]","kicker":"","author_name":"Chase Davis","author_title":"","affiliation":"","summary":"Google Maps API offers journalists a powerful tool for visualizing spatial data and creating interactive maps, though reporters must be aware of its limitations including precision issues, performance constraints with large datasets, and geocoding challenges. The technology can enhance news coverage by revealing story ideas and making data more accessible, despite requiring creative problem-solving for some technical obstacles.","keywords":["mapping","journalism","API"],"topics":["Technology","Journalism & Newsroom","Investigative Methods"],"technologies":["Google Maps API","Internet Explorer","TIGER","Perl"],"full_text":"this meaning on their own. But as with any other source, journalists must be skeptical.\n\nReporters should beware never to take points plotted on a Google Map as precise locations, especially in cases where a measurement of a few yards is likely to matter. The maps are great for eyeballing clusters and showing trends in areas, but more precise measurements should be left to GIS packages.\n\nThe API also does not handle multiple points plotted at exactly the same coordinates well. No matter how many points are plotted there, only one will appear. In those cases, I usually alter the coordinates by an infinitesimal amount so the points appear offset at higher zoom levels.\n\nFinally, Google does not want corporations using their maps exclusively on local intranets. Under the API's terms of use, all maps have to be displayed on the open Internet and must be universally accessible. I got around this by placing mine on my personal Web site, and it seems Google is willing to compromise on other solutions.\n\nThere are technical issues as well. Microsoft Internet Explorer sometimes chokes trying to map more than a few hundred points. Although performance is quickly improving, Google Maps was not designed to handle thousands of events at a time.\n\nBecause no function exists for it in the API, geocoding – assigning latitude and longitude coordinates to addresses – requires some creative programming. Many programmers scrape the coordinates from www.geocoder.us, which condones the practice, or set up a large Census TIGER database on their servers and draw from it using Perl.\n\nSolutions for these problems and others are often discussed by the extensive community that has coalesced around the API since Adrian Holovaty, a 2001 graduate of the Missouri School of Journalism recently hired by The Washington Post, helped popularize maps hacking with chicagocrime.org earlier this year.\n\nGoogle's message boards are loaded with information, and Google representatives often log in to answer questions. It seems that Google continues to supports the innovation born from tinkering with their technology, and the potential is clear.\n\nAlthough it is not without hiccups, the Google Maps API has already been used to create gorgeous online graphics. Within the newsroom, it can also be turned into a useful reporting tool that could help relieve some of the deadline dependence reporters have on CAR gurus.\n\nBut more importantly, it can make spatial data display a more prominent and accessible component of news coverage, revealing new story ideas and helping create a better-informed newsroom.\n\nContact Chase Davis by e-mail at chase.davis@gmail.com.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2005-novemberdecember-car-workshops","slug":"2005-novemberdecember-car-workshops","url":"https://dwillis.github.io/uplink/article/2005-novemberdecember-car-workshops.html","year":2005,"month":"November/December","monthNum":11,"headline":"CAR Workshops","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer intensive boot camps and specialized seminars to train journalists in computer-assisted reporting, data analysis, and using databases and mapping software to uncover news stories, with follow-up support provided after training.","keywords":["training","data","journalism"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet","GIS"],"full_text":"These unique seminars, taught by IRE and NICAR's experts, train journalists to acquire electronic information, use spreadsheets and databases to analyze the information and to translate that information into high-impact stories. In addition, IRE and NICAR provide follow-up help when participants return to their news organizations.\n\n2006 Boot Camps:\nColumbia, Mo.\n\n• Jan. 8-13, 2006\n• March 26-31, 2006\n• May 21-26, 2006\n• Aug. 6-11, 2006\n\nMapping Data for News Stories:\nColumbia, Mo.\nLearn how to uncover news stories by mapping data with geographic information system software.\n\n• Jan. 6-8, 2006\n\nEditor Boot Camp\nColumbia, Mo.\nThe boot camp will teach editors the things they need to know to make CAR successful in their newsrooms.\n\n• April 7-9, 2006\n\nAdvanced CAR Statistics\nTempe, Ariz.\nAimed at strengthening the skills of reporters who want to move beyond basic CAR and use statistical analysis in their work.\n\n• Feb. 17-19, 2006\n\nWhat previous Boot Campers have said about the experience:\n\n\"It was intense, but totally inspiring. I feel like I came away with some solid skills.\"\n\n– Abigail Sterling Vazquez,\nKRON-San Francisco\n\n\"This was one of the best training workshops I've ever attended. You've taken a very intimidating topic and made it easy to understand and apply in my own newsroom.\"\n\n– Patrick Powers,\nBelleville News-Democrat\n\nFellowships\nA limited number of minority and small-news organization fellowships are available for IRE and NICAR workshops. Visit www.ire.org/training/fellowships.html for more information and an application.\n\nwww.ire.org/training/bootcamps.php","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-january-february-giving-proper-credit","slug":"2006-january-february-giving-proper-credit","url":"https://dwillis.github.io/uplink/article/2006-january-february-giving-proper-credit.html","year":2006,"month":"January February","monthNum":1,"headline":"Giving proper credit","kicker":"CAR ISSUES","author_name":"Danielle Cervantes","author_title":"research analyst","affiliation":"The San Diego Union-Tribune","summary":"Computer-assisted reporting (CAR) experts who conduct database analysis and investigative work often go uncredited in published stories, despite their crucial contributions to major journalistic discoveries.","keywords":["credit","reporting","database"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"A disaster strikes a community. A jury announces a big verdict. A team of reporters scrambles to gather information and reactions. Credit is spread generously among all contributors to the story.\n\nA computer-assisted reporting expert uses a database manager to match a list of deadbeat parents with a list of prisoners and finds the most-wanted deadbeat sitting in a cell just down the hall from the sheriff's office that is trying to locate him. A reporter airs the revelation in a big story with no credit to the CAR expert.\n\nFor many journalists doing CAR, unfortunately, the second scenario has become common.\n\nUplink first raised the question of proper crediting in 1992, with an article by Dan Woods, former database editor with The (Raleigh, N.C.) News & Observer. Woods found that CAR contributors were often left out of the credit loop. And that sentiment is certainly credit-worthy.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-january-february-sept-11-loans-steered-to-the-unscathed","slug":"2006-january-february-sept-11-loans-steered-to-the-unscathed","url":"https://dwillis.github.io/uplink/article/2006-january-february-sept-11-loans-steered-to-the-unscathed.html","year":2006,"month":"January February","monthNum":1,"headline":"Sept. 11 loans steered to the unscathed","kicker":"SPOTLIGHT: EXAMINING HOMELAND SECURITY","author_name":"Frank Bass","author_title":"Investigative Reporter","affiliation":"The Associated Press","summary":"The article investigates the distribution of post-9/11 disaster loans, revealing that many funds were allocated away from directly impacted areas, raising questions about financial oversight.","keywords":["Loans","9/11","SBA"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["dBase","Access","spreadsheet"],"full_text":"When Dirk Lammers, an AP reporter in Sioux Falls, S.D., called me in New York one afternoon, I didn't know he had a really great story idea. I knew just enough, however, to know I wouldn't mind a small piece of it. \"What's a South Dakota radio station doing with a Sept. 11 Small Business Administration loan?\" he wanted to know. Lammers had found out about the radio station, \"Hit Kickin' Country KBFS-AM\" from a database of U.S. Small Business Administration disaster loans obtained from the IRE and NICAR Database Li- brary. He had been researching drought issues. I downloaded the dBASE file myself from the Database Library's subscription service Web site, unzipped it and imported it into Microsoft Access. Sure enough, the radio station's owner, Ultimate Caps Inc. of Belle Fourche, S.D., was on the list, along with 10,000 other borrowers. Lammers had done some preliminary research; it turned out that all borrow- ers had to file the standard financial paperwork, plus an explanation of how the Sept. 11 attacks had affected their business. The SBA told me I would have to file federal Freedom of Information Act requests with four different regional of- fices, and it could take a while before getting the results back. Lammers and I reconsidered. Instead of filing for all 10,000 explanations, we decided to pick out two- or three-dozen loans to businesses that sounded inter- esting in different parts of the country. We found a pet boutique in Utah; a perfume bar in the Virgin Islands; a gun dealer in Guam; a car wash in Phoenix; and quite a few bars and restaurants. We filed the FOIAs in mid-March; docu- ments began trickling back in mid-April, in various stages of redaction. Some just had the company financials covered, per the FOIA exemption that protects busi- ness trade secrets and financial infor- mation; others were completely blacked out. We appealed the redaction of one batch that was completely unusable. The data, slightly better this time, arrived in mid-May. The documents offered a glimpse into the rationales granting the loans. An Oregon winery suffered because, as its owner later joked, \"people started hit- ting the hard stuff.\" A Maryland skydiv- ing school almost started laying off staff because small planes were grounded for more than a month after Sept. 11, and besides, who felt like jumping out of an airplane? The Utah dog boutique was hit hard because no one was going on vacation, and dog boarding was a big part of their business. The story, I thought, might focus on the integration of the economy, the idea that the attacks in New York had enormous ramifications for a restaurant owner in San Diego. But we decided we needed to do more reporting. Since the story in- volved a federal agency, we contacted our newly created multimedia investi- gations team in Washington, D.C., to help us coordinate all the elements that would be needed-audio, video, online and photos. We decided to send a copy of the spreadsheet to AP bureaus in all 50 states. We also began running more queries on the data to answer some questions: Which cities had gotten the most? Which states? What industries needed the most help? How many happened inside and outside the immediate disaster area? One of the first things we discovered was that barely half of all loans had gone to small businesses in New York City or Fairfax County, Va., where the Pentagon is located. We decided to focus upon the companies outside the immediate areas of the attacks and split the list of companies whose documents we obtained under the FOIA request to see what else was there. Lammers was also talking to experts. He called me one day with alarming news. \"Have you heard of STAR loans?\" \"Huh?\" \"Apparently, there's a whole lot more money out there.\" \"More than $1.2 billion?\" I was scheduled to interview Hector Bar- reto, the SBA administrator, in another week. I called the public affairs office. \"So there's $1.2 billion in the economic injury disaster loan (EIDL) program?\" I asked. \"Right,\" public affairs officer confirmed. \"And is the STAR program part of that \"No!\" she said. \"That's another $3 billion or $4 billion.\" It took about two weeks but I obtained from the SBA a Microsoft Excel spread- sheet with about 8,000 rows of data. Sure enough, there were about $3.7 billion in STAR loans. We imported the new data into Ac- cess, and put together an abbreviated spreadsheet so it would be easier for journalists in AP bureaus to interpret. We included the business name, type, ad- dress, city, state, ZIP code, date of loan and amount for each borrower. We now had records of close to 19,000 loans, totaling $4.9 billion. We began looking for patterns again in Access and found some beauties, such as $6 million in loans to more than 50 Dunkin' Donuts franchises. Were Dunkin' Donuts really hurt by terror attacks? We did some more analysis, and found that the amount of SBA loans going to New York and Washington had dropped, to barely 11 cents on the dollar. I thought that was a very strong element of the story that could be combined with the overall \"ripple effect\" theme. We decided to send a copy of the spreadsheet to AP bureaus in all 50 states. In addition, we would create an online database for our members to lo- calize their own stories. John Solomon, the head of our multimedia team, wanted me to go to Ground Zero and find out how New Yorkers felt about the distribu- tion of funds. I spent an afternoon around Ground Zero with a cameraman, listening to New Yorkers fume about trying to re- build their business in what now is - for Manhattan, anyway - a ghost town. We headed across the Brooklyn Bridge and interviewed Rep. Nydia Velasquez, the ranking Democrat on the House Small Business Committee. \"It's outrageous,\" she said. And, she offered, it would get worse: There had been a two-year deferral period on the loans. So while our earlier analysis might have shown a miniscule default rate, she predicted it would start to be- come huge. But there was something else disturbing that was beginning to happen: Soloman, Lammers and I were getting calls from AP bureaus who said borrowers denied receiving SBA loans. We groaned. Dirty data is an everyday occurrence in com- puter-assisted investigations. A pattern emerged: People within the original loan recipient group did not have a problem discussing how Sept. 11 had harmed their business. But STAR loan recipients generally were outraged and denied that they had received any Sept. 11 loans. Fortunately, some were curi- ous and mad enough to call their lenders and confirm that they received Sept. 11 assistance. Lammers and I regrouped; he was on vacation in Florida, but was lugging around a cardboard box with all his story notes and papers, just in case. We were about a week away from the projected run date, and kept getting calls from bureaus. Finally, an SBA official gave us a bit of guidance: The agency had left it up to the banks to determine whether the Sept. 11 attacks had economically harmed borrowers, making them eligible for the STAR loans. Some banks may have applied a loose standard and decided every business applying for an SBA loan had been hurt by the attacks. Dirk called the National Association of Guaranteed Lenders to confirm our information. The president of the as- sociation shrugged it off. Everyone had been affected. But there was one thing we ought to know, he added: The SBA had been worried that it would run out of money in its traditional loan program, so it had encouraged banks to put as many people in the Sept. 11 program as possible. The anecdotes started flowing into our e-mail from around the country. \"That's scary,\" said one businessman who provided clowns for special events. No one had told him he had gotten a STAR loan. We decided to focus upon the companies outside the immediate areas of the attacks. The story landed on front pages across the country, accompanied by more than three dozen state AP sidebars. Three U.S. senators called for an investiga- tion. Nearly a month after the story ran, Lam- mers and I looked at other angles. We have reported that the SBA has been slow to approve loans for the victims of Hurricanes Katrina and Rita. And we reported on how recipients of Sept. 11 loans have been defaulting in large numbers. We learned (or, I should say, relearned) quite a few things from the story: 1. The computer analysis is the first step. It gives you the questions, not the answers. 2. Don't assume there is ever just one government program that handles all needs. We are quite sure there are more Sept. 11-related programs around, and all of them bear examina- tion. 3. \"Follow the money\" has become such a cliché that I hate to mention it. So it is probably more relevant to remember there are two times when big money is made: When empires are being built, and when they are being destroyed. In this case, we had both elements. 4. Question your assumptions. Bounce your ideas off other people. Also re- member Dumbledore's maxim: We may be rather cleverer than most wizards, but as clever wizards, our mistakes are correspondingly larger. Don't be carried away by your own cleverness. Not all computer investigations need to involve a scripted VB module that invokes a batch file to download 50 gigabytes of text files and runs them through a series of nested SAS macros. Sometimes, simple is better. Contact Frank Bass by e-mail at fbass@ap.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-january-february-analysis-finds-bias-in-jury-picks","slug":"2006-january-february-analysis-finds-bias-in-jury-picks","url":"https://dwillis.github.io/uplink/article/2006-january-february-analysis-finds-bias-in-jury-picks.html","year":2006,"month":"January February","monthNum":1,"headline":"Analysis finds bias in jury picks","kicker":"STATISTICS","author_name":"Jennifer LaFleur","author_title":"CAR editor","affiliation":"The Dallas Morning News","summary":"A statistical analysis of jury selection in Dallas County reveals continued racial bias in juror exclusions, highlighting slow progress since a 1986 exposé.","keywords":["Bias","Jury","Statistics"],"topics":["Crime & Justice","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"Nearly 20 years ago, The Dallas Morning News reported that Dallas County prosecutors were excluding nine out of 10 blacks from jury service. So when the U.S. Supreme Court agreed to review a Texas death-penalty case that alleged jury discrimination, we thought it was time to see whether anything had changed.\n\nAfter two years of research, multiple court orders and regression models, we had our answer: race still played a key role in who serves on a jury.\n\nBut because the trend was not as blatant as it was in 1986 - when reporters Steve McGonigle and Ed Timms found that four out of five juries were all white - the story required a more sophisticated statistical tool.\n\nWe used logistic regression, which computes the relationship between variables, when the dependent (or outcome) variable is dichotomous - meaning it has two possibilities for the value.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-january-february-bits-and-bytes","slug":"2006-january-february-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2006-january-february-bits-and-bytes.html","year":2006,"month":"January February","monthNum":1,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Data Library assisted multiple broadcast news stations in analyzing various datasets, including crime rates on college campuses, felon hunting licenses, poverty rates, and sex offender locations, resulting in significant policy changes and public safety improvements.","keywords":["analysis","crime","mapping"],"topics":["Data & Databases","Investigative Methods","Crime & Justice"],"technologies":["SPSS","ArcView"],"full_text":"Bits & Bytes\n\nData Analysis\n\nThe IRE and NICAR Data Library recently analyzed data for several high-impact broadcast news reports.\n\nFor ABC News' Primetime, data analysts used SPSS to calculate violent crime rates on college campuses across the nation. The analysis found smaller institutions often had higher crime rates than larger institutions.\n\nFor KUTV-Salt Lake City, analysts compared felon databases against hunter licenses databases and found 7,000 felons had hunting licenses. After the report, the state confiscated 76 tags and changed the hunting license applications.\n\nFor WSB-Atlanta, analysts used U.S. Census data and ArcView geographic information system (GIS) software to map poverty rates and found that, in one area, three of every four residents lived below the poverty line.\n\nAnalysts mapped sex offenders' residences and public school bus stops, finding that at least 140 sex offenders lived within 500 feet of a school bus stop for WLEX-Lexington. At one bus stop, there were six offenders within 500 feet. After the program aired, the school moved the bus stop.\n\nAnalysts mapped sex offenders for WOOD-Grand Rapids by location and helped the station identify clusters in places, such as homeless shelters.\n\nFor information about how the Database Library can assist your reporting project, contact Jeff Porter at 573-882-1982 or jeff@ire.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-january-february-meth-stories-win-first-meyer-award","slug":"2006-january-february-meth-stories-win-first-meyer-award","url":"https://dwillis.github.io/uplink/article/2006-january-february-meth-stories-win-first-meyer-award.html","year":2006,"month":"January February","monthNum":1,"headline":"Meth stories win first Meyer Award","kicker":"INSIDE NICAR","author_name":"","author_title":"","affiliation":"","summary":"Three major journalism investigations were recognized with the inaugural Philip Meyer Awards for their use of social science methods, including exposés on methamphetamine regulation failures, Veterans Affairs bureaucratic delays, and Florida wetland destruction.","keywords":["investigation","journalism","data"],"topics":["Investigative Methods","Data & Databases"],"technologies":["GIS"],"full_text":"Investigations into the methamphetamine epidemic, entrenched failures at the U.S. Department of Veterans Affairs and the loss of Florida's wetlands were recently named winners of the first Philip Meyer Awards.\n\nThe awards recognize the best uses of social science methods in journalism and are administered by the National Institute for Computer-Assisted Reporting (a joint program of Investigative Reporters and Editors and the Missouri School of Journalism), and the Knight Chair in Journalism at the Walter Cronkite School of Journalism and Mass Communication at Arizona State University.\n\nThe winners are:\n\nFirst Place: The Oregonian for \"Unnecessary Epidemic,\" a series of articles showing how Congress and the Drug Enforcement Administration missed opportunities to stop the growth of meth abuse by aggressively regulating the import of chemical ingredients. Lead reporter Steve Suo used statistical analysis of treatment-admission, drug purity, price and arrest data.\n\nSecond Place: The Knight-Ridder Washington Bureau for \"Discharged and Dishonored,\" a yearlong series that revealed how bureaucratic delays by the U.S. Department of Veterans Affairs harmed disabled veterans. Reporters Chris Adams and Alison Young analyzed survey data and the VA's database of 3.4 million claims to discover that more than 13,700 vets died while waiting for their claims to be resolved.\n\nThird Place: The St. Petersburg Times for \"Vanishing Wetlands,\" which found that 84,000 acres of Florida wetlands have been destroyed since 1990, when President George H.W. Bush declared a national policy of no net loss of wetlands. Reporters Matthew Waite and Craig Pittman calculated the loss using before-and-after satellite imagery and geographic information system software.\n\nThe awards were presented March 10 at the IRE and NICAR Computer-Assisted Reporting conference in Newark, N.J.\n\nA panel of five contest judges picked the winners from more than two dozen entries. The awards are given in honor of Philip Meyer, the Knight Chair in Journalism at the University of North Carolina at Chapel Hill and a pioneer in using social science methods in news reporting. For more information see www.ire.org/meyeraward/05winners.html","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-january-february-about-our-contributors","slug":"2006-january-february-about-our-contributors","url":"https://dwillis.github.io/uplink/article/2006-january-february-about-our-contributors.html","year":2006,"month":"January February","monthNum":1,"headline":"About our contributors","kicker":"","author_name":"","author_title":"","affiliation":"Associated Press","summary":"A profile of seven investigative journalists who specialize in computer-assisted reporting (CAR), detailing their professional backgrounds, current positions at major news organizations, and training in data journalism techniques.","keywords":["journalists","reporting","databases"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Frank Bass is an investigative reporter for The Associated Press in Washington, D.C. He has worked for the AP since 1997 as director of computer investigations. Bass has worked for the Wall Street Journal's Texas Journal, the Houston Post and the Alabama Journal, where he shared the 1988 Pulitzer Prize for General News Reporting. He is the author of the AP Guide to Internet Research and Reporting. He is an alumnus of the 2004 Advanced CAR Statistics Boot Camp and the 1998 SQL Server Boot Camp.\n\nDanielle Cervantes is a research analyst for The San Diego Union-Tribune and specializes in CAR and demographics. Before that she was a research librarian for five years. She is a 2003 alumna of the CAR Boot Camp.\n\nDoug Haddix is projects editor for The Columbus Dispatch. He is a 2001 alumnus of the CAR Boot Camp.\n\nJennifer LaFleur is the CAR editor for The Dallas Morning News. She was IRE and NICAR's first training director, serving 1994-95, and is co-author of \"Mapping for Stories,\" an IRE Beat Book. She is a 1997 alumna of the SQL Server Advanced Boot Camp and the 1990 CAR Boot Camp.\n\nAron Pilhofer is a database editor for The New York Times and has written extensively for Uplink about open-source software. He is the former director of IRE and NICAR's Campaign Finance Information Center and co-author of \"Unstacking the Deck: A Reporter's Guide to Campaign Finance,\" a beat book from IRE. Pilhofer is an alumnus of the 2004 Mapping Boot Camp and 1999 CAR Boot Camp.\n\nDave Smith is weekend editor for the Sentinel, in Fairmont, Minn., where he has worked for six years. He covers city government, business, health and medicine, and politics. Smith also writes a weekly humor column. He is a 2005 alumnus of the CAR Boot Camp.\n\nMaurice Tamman is a writer on the Sarasota Herald-Tribune special projects team. Before that, he was database reporter on the The Atlanta Journal-Constitution's special projects team.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-january-february-farm-story-gets-car-started","slug":"2006-january-february-farm-story-gets-car-started","url":"https://dwillis.github.io/uplink/article/2006-january-february-farm-story-gets-car-started.html","year":2006,"month":"January February","monthNum":1,"headline":"Farm story gets CAR started","kicker":"FIRST VENTURE","author_name":"Dave Smith","author_title":"Weekend editor","affiliation":"Sentinel (Fairmont, Minn.)","summary":"A series of investigations into the methamphetamine crisis and other issues received the first Philip Meyer Awards, highlighting significant contributions of social science methods in journalism.","keywords":["Meth","Awards","Journalism"],"topics":["Investigative Methods","Agriculture & Land","Data & Databases"],"technologies":["Excel","pdftotext","Internet"],"full_text":"After attending the IRE and NICAR Computer-Assisted Reporting Boot Camp last year my head was swimming with all the stories I planned to work on with my newfound expertise. I returned to the newsroom of the small southern Minnesota daily where I work walking a little taller, buoyed by my sudden trans- formation into a CAR expert. I carried a list of no fewer than 20 CAR stories I planned to work on, but with a special section looming, I decided to use my new powers for that assignment. The section-Spring - Ag Update- one we do every year. I usually struggle to come up with good agricultural stories because it is not my beat. I consulted my list from the boot camp to see if I thought of anything while the iron was hot down in Columbia, Mo. And there it was: farm accidents. That would be just the topic to call on my new CAR skills as well as provide our readers in this farming region with an important and interesting topic. I decided to keep it simple in my first attempt at a full-fledged CAR story. I started with a Google search of farm accidents in Minnesota. I am no stranger to using the Internet for searching for data, but I quickly dis- covered I would need to employ some of the Boolean search techniques picked up at the boot camp. The infor- mation I found initially was not in a form I could download and analyze. While I was able to use some of the informa- tion from the initial searches later for background and expert opinion, it was not what I needed to get the start I wanted: accident statistics. Those Boolean search tactics I learned at the boot camp are what eventually got me the statistics, because even the most logical places I searched, such as the Farm Bureau and Minne- sota Extension Web sites, had articles using the data I wanted but only as it pertained to their narrower, specific purposes. The deeper searching took me to the University of Minnesota Agricultural and Health Safety Program where I was able to find a link to the accident and fatality data I needed. I was excited to have finally unearthed the gold I was mining for so I could get it into a Micro- soft Excel spreadsheet and work it over. There was a catch, however: the data was all in Adobe Portable Document Format. Normally this would require a lot of manual entry and a great deal of time, but now I had new tools. I remembered pdftotext, a program we used in the lab that converts tables in a PDF into a fixed-width text file that can be imported to Excel. I downloaded the PDF accident files, fired up my boot camp CD with pdftotext on it and con- verted them all to text. I opened the text in Excel and, using my boot camp skills once again, cleaned it up into a nice piece of data that I could easily sort, filter and compare. (For a step-by-step tutorial on installing and using pdftotext, visit www.ire.org/training/nettour/pdf/ PDFTOTEXT.pdf) Once I had the data in Excel I used techniques picked up at the boot camp to compute average ages, median ages and to count the total number of ac- cidents. I was able to separate the ac- cident victims by sex, age, county and type of accident. I could have sorted by month the ac- cident happened, but it is well known in this farming community that harvest season is when most accidents hap- pen, so I left that out. I also was able to access expert analy- sis on the Internet to verify what I found in my research as well as extension educators and farm bureau sources to help provide more information. One thing my CAR work on this story pointed out is that farm safety education efforts are paying off. The number of accidents and deaths went down in the past couple of years. This is especially true for children. I also discovered that our county, Martin, has one of the highest accident rates in the state. However, more than anything else, this is because we have many more farms than other counties. This was a small and simple story but it accomplished several things. First of all, it built up my confidence in using what learned at the boot camp. I found I could come up with a more complete, useful story in a relatively short amount of time simply using the tools and techniques gathered there. It really increased the credibility of IRE and NICAR and its staff when the things that seem to work so easily in their controlled lab also worked for me when I got back to the little news- room in rural Minnesota. Second, this story got my managing editor on board. He liked what I did and how I was able to use the boot camp experience to make the paper better. Following this story he allowed me to share some things with the rest of the newsroom during a staff meeting. Just as Brant Houston, the executive director of IRE and NICAR, said would happen, I am now the CAR expert at the Sentinel. I have helped other reporters use CAR methods in their stories and have taught them to use them them- selves. One reporter covering a school administrator contract issue was able to access documents online and find resources that broke the real story for her. Since this story, I have done a three- part series on bridge safety in our town, county and a neighboring county (I told you my editor loved this stuff). I have gathered five years' of county and city budgets and city salaries and am in the process of converting them into useful formats. This is another instance where the things learned at the boot camp, as well as the IRE and NICAR staff's willingness to help has continued to pay off. My editor asked me to put together a binder of sources that could be used by the newsroom to do basic CAR work. That project has turned into a CAR library with books we have started ordering from IRE, as well as our own sources. These will be available for any- one, allowing other writers' experience sto be accessed by everyone. I have talked with our publisher and he is strongly considering getting another computer to use strictly for setting up permanent databases for reference and also getting more CAR training for the newsroom staff. We are a smaller daily, like so many others with little or no training budget. Because I attended the boot camp on a small-news organization fellowship, at least one newspaper and several report- ers are better for it. And at least a couple of CAR evangelists have been born. Contact Dave Smith by e-mail at dave@fairmontsentinel.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-january-february-disaster-loans","slug":"2006-january-february-disaster-loans","url":"https://dwillis.github.io/uplink/article/2006-january-february-disaster-loans.html","year":2006,"month":"January February","monthNum":1,"headline":"Disaster loans","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library has updated its Small Business Administration Disaster Loan database, now current through September 2005 and containing over 867,000 records dating back to 1980, which journalists can use to analyze government disaster response and loan repayment rates.","keywords":["loans","disasters","databases"],"topics":["Data & Databases","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"Disaster loans\n\nThe IRE and NICAR Database Library has just updated its Small Business Administration Disaster Loan database. Current as of September 2005, the data includes more than 300 loan records for the month following Hurricane Katrina.\n\nThe database details loans made to businesses and individuals as disaster assistance. The data identifies the borrower, the disaster, the amount and, for business borrowers, whether the loan was paid in full or deemed uncollectible. The data includes a total of 867,631 records dating back to 1980.\n\nJournalists can use this data to analyze the government's response to natural disasters by finding who received the loans. Also, journalists can calculate loan repayment rates.\n\nFor a sample of the SBA data and more information about SBA, visit www.nicar.org/data/sba/sbadis.htr or call the Database Library at 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-january-february-brazilian-gathering","slug":"2006-january-february-brazilian-gathering","url":"https://dwillis.github.io/uplink/article/2006-january-february-brazilian-gathering.html","year":2006,"month":"January February","monthNum":1,"headline":"Brazilian gathering","kicker":"","author_name":"","author_title":"","affiliation":"IRE","summary":"IRE participated in Brazil's first International Congress on Investigative Journalism in Rio de Janeiro, where the Brazilian Association for Investigative Journalism organized the event for over 300 attendees with support from IRE and the Knight Center.","keywords":["Brazil","journalism","congress"],"topics":["Journalism & Newsroom","Investigative Methods"],"technologies":[],"full_text":"IRE participated in Brazil's first International Congress on Investigative Journalism in Rio de Janeiro, Oct. 27-29. The Brazilian Association for Investigative Journalism (ABRAJI) conducted the conference with more than 300 journalists, students, and teachers. IRE provided training and support for the event, which was supported by the Knight Center for Journalism in the Americas.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-january-february-come-to-newark-for-tips-inspiration","slug":"2006-january-february-come-to-newark-for-tips-inspiration","url":"https://dwillis.github.io/uplink/article/2006-january-february-come-to-newark-for-tips-inspiration.html","year":2006,"month":"January February","monthNum":1,"headline":"Come to Newark for tips, inspiration","kicker":"CAR CONFERENCE","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"After attending a CAR Boot Camp, the author utilizes CAR techniques to analyze farm accident data, showcasing the potential of data journalism in local reporting.","keywords":["Farming","Accidents","Data"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Internet","spreadsheet"],"full_text":"The 2006 Computer-Assisted Report- ing Conference in Newark, N.J., is just around the corner - March 9-12 - and will offer more than 50 sessions and two-and-a-half days of intense hands- on training. While neither Bruce Springsteen nor the Sopranos will be in attendance (our host The Star-Ledger is, however, the paper of record for the Sopranos) this confer- ence will spin out plenty of hot tracks on CAR methods and stories, a variety of hands-on classes from beginners to advanced, and special presentations of the latest techniques. (See www.ire.org/ training/newark06) As in recent years, this year's pro- gram.gets the CAR engines ignited on Thursday, focusing on the needs of the advanced CAR practitioner. Sessions will range from better intranets to Web scraping to the latest in mapping data and social network analysis. The day's speakers are top-of-the-chart profes- sionals who are generous in sharing their knowledge. The main conference will be packed with panels on criminal justice, money and politics, bulletproofing stories, social re- search methods and many other topics. The panelists will show how computer- assisted reporting helps in every beat and on a daily and weekly basis. We will have sessions that deal with every medium. A highlight will be the presentation of the first Philip Meyer Awards for excellent work in using social science methods in journalism. Meyer, a pioneer in the field, will join us at the ceremony and reception. Our hands-on classes will include us- ing the Internet effectively, working with spreadsheets and database managers, mapping, programming, statistics and social network analysis. In addition, there will be a special mini- boot camp: a series of panel sessions each morning, then classes throughout the afternoon. Designed for CAR begin- ners, the classes are similar to sessions conducted during lengthier boot camps at the IRE and NICAR headquarters. The classes begin with basic spread- sheet calculations and work through using CAR techniques for investigative projects or deadline stories. It's strongly recommended that confer- ence registrants sign up for classes before the conference begins, available through IRE's online registration pro- cess. You can start here at www.ire.org/ training/newark06 and follow the link for registration. Attendees also can register for classes on site by visiting staff mem- bers in the Demo Room area. The Demo Room, too, will be a hub of activity and will feature special sessions on uses of new software and techniques for news stories. The sessions have an informal approach and often turn into freewheeling discussions among par- ticipants. You also can road test IRE and NICAR services and data from our ever growing government database collection. Staffers will be on hand to an- swer questions and make suggestions about how to better use CAR. This conference also will help all at- tendees at every level to become bet- ter journalists and leaders in the use of new technology and reporting methods in their newsrooms. Be sure to take advantage of this great network of col- leagues, whether in the panel rooms, the demo room or in the informal meetings in the hallways, coffee shop or bars.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-january-february-tracking-home-foreclosures","slug":"2006-january-february-tracking-home-foreclosures","url":"https://dwillis.github.io/uplink/article/2006-january-february-tracking-home-foreclosures.html","year":2006,"month":"January February","monthNum":1,"headline":"Tracking home foreclosures","kicker":"MAPPING","author_name":"Doug Haddix","author_title":"Projects editor","affiliation":"The Columbus Dispatch","summary":"Columbus Dispatch reporters used GIS mapping software to visualize 12,000 foreclosed properties across Franklin County, revealing that the foreclosure crisis extended far beyond poor inner-city neighborhoods and surprising county officials with its widespread scope.","keywords":["foreclosure","mapping","journalism"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["ArcView"],"full_text":"It's one thing to point out in a news story that 12,000 houses in the Columbus area were listed for sheriff's auction during the past five years. It's quite another to publish an eye-popping map that instantly shows the toll of foreclosures in communities across Franklin County - far beyond the inner city and poor neighborhoods.\n\nProjects reporters Jill Riepenhoff, Geoff Dutton and I called it our \"holy crap map.\"\n\nNo one - lenders, Franklin County government officials or homeowners - could diminish the scope of the problem. The map provided the foundation for \"Brokered Dreams,\" a four-day series about foreclosures and mortgage fraud published Sept. 18-21, 2005. (The series is posted online at www.dispatch.com/reports/reports.php) A little less than two months later, the Chicago Tribune reported how widespread mortgage fraud was hurting poor communities in the city.\n\nWe used ESRI ArcView geographic information system (GIS) software repeatedly during the reporting, writing and editing of the series. Mapping software would strengthen just about any story involving property parcels.\n\nBy far, the countywide map of properties sent to sheriff's sales from 2000-04 posed the biggest challenge. The time gathering and cleaning data paid off because the map guided much of the reporting that followed.\n\nThe sheriff's department provided a text file with properties sent to auction. That data set included the parcel identification number, which proved invaluable in linking the information to the more-detailed auditor data. Joining the data on the street address field would have been a data nightmare of epic proportions, given the inconsistencies, typos and other data-entry problems.\n\nOnce we produced a master table of properties sent to sheriff's sale, we spent hours cleaning it up and standardizing information gathered by two separate county offices. The auditor's data included fields that helped us produce a master table that included only single-family properties. We excluded commercial buildings, apartments and other properties that went on the sheriff's auction block to keep our focus on single-family homes.\n\nWhen we were confident that our data showed what we wanted, we joined the table using the parcel ID field to the auditor's layer file. Presto: our \"holy crap map.\"\n\nThe map breathed life and meaning into our table of nearly 12,000 single-family homes sent to sheriff's sale. We expected to see dense concentrations of sheriff's sales in poor, central-city neighborhoods but never dreamed there would be so many foreclosures outside Interstate 270, the outer belt ringing Columbus.\n\nA printout of the map left Franklin County Treasurer Richard Cordray nearly speechless. \"Wow,\" he said. \"Wow.\"\n\nFor a time, we considered using the map as the lead art on Page 1 with the kick-off story. Ultimately, we chose a photo showing workers removing furniture from a foreclosed house, a visceral image that best represented loss of a home. The map also would have eaten up too much space on the cover. The fact that we even considered a large map as lead art shows how far our newsroom has come in its use of digital mapping.\n\nWith our countywide ArcView map, we were able to take a closer look at subdivisions all around the outer belt. Many of those neighborhoods have fairly new houses - homes built within the past five or 10 years. We created a theme map showing foreclosed properties based on the home builder. In some subdivisions, several builders had put up houses. In others, a single developer had built the homes. ArcView enabled us to see quickly whether clusters of foreclosures involved a single builder.\n\nMapping played a pivotal role in driving home the key findings in the series. Other key maps produced for \"Brokered Dreams\" included:\n\nA detailed look at one Columbus subdivision, Galloway Ridge, which had been riddled with sheriff's sales and personal bankruptcies. The map vividly showed the devastating toll on a neighborhood where nearly one in six households had faced foreclosure, bankruptcy or both. We produced the map by combining sheriff's sale and bankruptcy data and then linking it to a parcel layer. Riepenhoff took a printout of the map with her as she knocked on doors and talked with Galloway Ridge residents about their deteriorating neighborhood. Many knew that foreclosures and bankruptcies were harming their eight-year-old subdivision, but few realized the scope of the problem until they saw the map.\n\nAn overview showing how 390 houses financed by a New York City hedge fund were clustered in poor Columbus neighborhoods. Those ramshackle houses had been flipped (sold quickly at huge markups) and left vacant, adding more eyesores to struggling areas. A tipster alerted Dutton to irregularities involving houses financed by the Stillwater Asset Backed Fund.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-january-february-privacy-security-block-access-to-data","slug":"2006-january-february-privacy-security-block-access-to-data","url":"https://dwillis.github.io/uplink/article/2006-january-february-privacy-security-block-access-to-data.html","year":2006,"month":"January February","monthNum":1,"headline":"Privacy, security block access to data","kicker":"SPOTLIGHT: EXAMINING HOMELAND SECURITY","author_name":"Ben Welsh","author_title":"","affiliation":"","summary":"Journalists across the United States face increasing obstacles in obtaining federal data through Freedom of Information Act requests, with government agencies citing privacy exemptions to deny disclosure at record rates, particularly evident in Florida newspapers' battles with FEMA over hurricane relief records.","keywords":["FOIA","privacy","data"],"topics":["Government & Politics","Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Although they usually lack the intrigue and visibility of some of the latest clashes between the press and gov- ernment, journalists' ongoing battles for federal data are felt in newsrooms across the United States. The increased concern about terror- ism sparked by the 9/11 attacks, and the more recent devastation of the Gulf Coast by a series of hurricanes have prompted legislators to pour billions of dollars into programs designed to speed recovery and better protect the country. At the same time, officials at all levels of government are attempting to rein in the efforts of journalists to scrutinize these programs. Trends suggest that Freedom of Information Act officers have been given greater latitude to deny requests, particularly when they position them behind the spacious pro- tections of the right to privacy. Two battles over the use of the privacy exemption are happening in Florida, where Gannett Newspapers and the South Florida Sun-Sentinel are seeking data to help shed light on the Federal Emergency Management Agency's response to four hurricanes that struck the state in 2004. The Gannett newspapers and the Sun-Sentinel independently have sued FEMA and asked judges to force the agency to release records that detail the names and addresses of aid recipients. FEMA is withholding them by citing privacy laws. Consid- ering the agency's documented poor performance, and the spending of even more aid money after hurricanes Katrina and Rita, how FEMA responds after a disaster is a newsworthy ques- tion. Privacy outbreak At the federal level, use of the privacy exemption to deny FOIA requests for documents and data has skyrocketed. A study of the annual reports filed by government agencies by Jennifer LaFleur, CAR editor for The Dallas Morning News and a former fellow at The Reporters Committee for Freedom of the Press, found the use of privacy exemptions to deny FOIA requests increased more than 600 percent between 1998 and 2002, rising from 55,000 to 380,000. After the passage of the USA Patriot Act and formation of the massive Homeland Security Department, a number of com- mentators predicted a crackdown on public disclosure through the national security exemption. But a report is- sued by The Coalition of Journalists for Open Government in November found otherwise. The group's analysis of re- quests handled by 22 federal agencies between 2000 and 2004 discovered the number of privacy denials has shot upwards as the number of national se- curity denials decreased. (Investigative Reporters and Editors is a member of the coalition.) \"The national security exemption is just one way that they can withhold in- formation,\" says LaFleur, former train- ing director for IRE and NICAR. \"The rulemaking is very complex. There are hundreds of ways in which things can be withheld.\" Overall, the coalition report concluded the use of exemptions to deny FOIA requests increased 22 percent, despite a 13 percent drop in the total number of requests answered. However, the study omitted requests made to the Depart- ment of Veteran Affairs, the Social Se- curity Administration and the Depart- ment of Health and Human Services on the grounds that they process millions of routine requests which the coalition said would cloud the results. Now closed What sparked the spike in privacy ex- emptions? A study into homeland secu- rity policy conducted by The Reporters Committee for Freedom of the Press points to a policy statement issued from the highest ranks of the Justice Depart- ment on Oct. 12, 2001, now known as \"The Ashcroft Memorandum.\" Then-Attorney General John Ashcroft sent a notice to government agencies altering legal policy on FOIA requests to support withholding information on security or privacy grounds if \"sound legal basis\" could be found. FOIA of- ficials were \"assured\" that government lawyers were ready to go to court. The order revoked the standard established by former Attorney General Janet Reno that encouraged \"maximum responsi- ble disclosure.\" The Justice Department instructed FOIA officers to withhold access to any information that might expose the government's vulnerabilities or that could be construed as possibly aiding terrorists in attacking American interests. \"What is happening is that the govern- ment has moved from a presumption of openness - unless they can claim a privacy interest worth protecting to one of presumptive closure,\" says Charles Davis, executive director of the Freedom of Information Coalition at the University of Missouri. From the start, pundits, columnists and press advocates have criticized the memo. The American Society of Newspaper Editors has even provided two ready-made \"stump speeches\" on their Web sites for editors to use. The Ashcroft language was repudiated in a subsequent editing of the FOIA manual by Rep. Henry Waxman, D-Ca- lif., and Rep. Dan Burton, R-Ind., but its influence can still be felt. Evidence can be found in a September 2003 audit by the Government Accountability Of- fice. While its survey of federal FOIA officers found that 48 percent said they had not noticed any changes in how their agency deals with requests, about one-third said they were less likely to disclose information. And 75 percent of that one-third cited the memo as encouraging their change in behavior. The inconsistency captured by those results is often a source of frustration for journalists working with different departments. In response to a FOIA request made by the IRE and NICAR Database Library, the Department of Defense declined to release a master list of the unique identifiers attached to businesses in the federal contracting database. This is despite the fact that those numbers are readily released piecemeal by other government agencies, including another arm of the defense department. The DOD uses business numbers pro- vided by the private company Dun and Bradstreet, which are now considered a private trade secret despite their use as a primary identifier in public records. \"It makes no sense to me,\" says Jeff Porter, Database Library director. Porter has run into similar problems with other agencies. The Department of Transportation will release the HAZMAT database containing a list of accidents involving hazardous materials, including plane crashes. But the Federal Aviation Administration, an organization under the umbrella of the DOT, omits all en- forcement actions involving hazardous materials from its database. The result is that an accident like the 1996 crash of Valujet Flight 592, which is believed to have been caused by oxygen canisters that burst into flame, will be included in the DOT's HAZMAT database but not the FAA's database. The U.S. Citizenship and Immigration Services, formerly known as Immigra- tion and Naturalization Services, is balking at releasing \"public use files\" of legal residency. The database, col- lected by the Database Library, has no names and, in its current version, goes to the metropolitan statistical area at the smallest. It includes country of origin, age, marital status, sex, OC- cupation and other somewhat benign information. The agency cites its new \"confidentiality\" policy. President George W. Bush took action on FOIA on Dec. 14 when he signed an executive order instructing each federal agency to hire a FOIA chief and increase efficiency by streamlin- ing requests. It did not address the Ashcroft memo. \"The executive order is a nice political move in that it gets the administration in front of the Cornyn-Leahy bill,\" Davis says. The legislation, working its way through Congress, would enact penalties on in- compliant FOIA officers. \"And it essen- tially adds little or nothing to FOIA.\" Fighting back So what is a journalist to do when con- fronted by a FOIA exemption he or she feels is unjustified? How can the press strike back? Some journalists, like The Des Moines Register's Bert Dalmer, have found ways to cover homeland security spending from the bottom-up by fer- reting out the local numbers. \"It's last resort to go through the feds for anything,\" Dalmer says. \"The feds almost always channel their money through the states and the locals. It's the capture there that should give re- porters a way of entry. In lowa they often don't mind sharing that stuff if they're a pass-through. It's the path of least resistance and most opportunity.\" But the big solution, LaFleur, Davis and others experts say, will only come through congressional action or orga- nized legal opposition. \"I don't think the media is aggressive enough,\" LaFleur said. \"Some of the big pushes for things [like the names of Guantanamo Bay captives] have come from other groups.\" It took more than a year of legal wran- gling, but the group of Gannett papers in Florida fighting with FEMA recently won many of the documents they sought, including agency memos and other internal communications. But their effort to procure a database with the names and addresses of aid recipients was denied by U.S. District Court Judge John Steele for exposing citizens to \"predictable and obtrusive invasions of privacy.\" The case brought by the Sun-Sentinel was pending, but FEMA has agreed to release the names and wages of disaster assistance inspectors. Still in question is the database containing the names and addresses of aid recipients as well as the comments about each claim recorded by FEMA inspectors. The Gannett papers plan to challenge Steele's ruling with an appeal. And the line drawn between privacy rights and the public's right to know, particularly when it comes to detailed information like names and addresses, is sure to be the central issue. Contact Ben Welsh by e-mail at ben.welsh@gmail.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-january-february-creating-data-points-with-google-maps","slug":"2006-january-february-creating-data-points-with-google-maps","url":"https://dwillis.github.io/uplink/article/2006-january-february-creating-data-points-with-google-maps.html","year":2006,"month":"January February","monthNum":1,"headline":"Creating data points with Google Maps","kicker":"Tech tip","author_name":"Maurice Tamman","author_title":"Writer","affiliation":"Sarasota Herald-Tribune","summary":"A Florida newspaper used Google Maps APIs to create an interactive property tax database displaying 400,000 parcels across five counties, combining parcel coordinates with tax roll data instead of relying on expensive traditional mapping servers.","keywords":["mapping","database","interactive"],"topics":["Technology","Data & Databases","Investigative Methods"],"technologies":["Google Maps","ArcGIS","AJAX","XML","JavaScript"],"full_text":"In recent years, newspapers have been using maps online to add inter- active Web components to projects and stories. Typically, they have ported geographic data to a Web-based map server or used plug-ins for ArcGIS or similar prod- ucts to create HTML-based maps. Over the summer, we looked for a way to add an interactive component to a property tax project we planned to publish in the fall. We explored the traditional options, but we wanted it to be truly dynamic and to display almost every parcel (about 400,000) in five Florida counties. We considered renting space on a mapping server but it was expensive: $5,000 for four to six months. It was also slow, clunky and particularly difficult to use for the uninitiated Web user. At the time, Google had just re- leased the beta version of the Google Maps APIs (application programming interfaces). Google Maps is clearly the best online mapping service (Yahoo! has just started a similar mapping service) and its underlying maps can be built into anyone's Web-based ap- plications. It quickly became clear to us that we could use Google Maps to deliver a far more satisfying online experi- ence. Here is how we did it: Using ArcGIS 9.1 we calculated the center point of every parcel in the five counties: Google Maps uses latitude and longitude coordinates to plot points and lines. From there we extracted market, assessed and taxable values from the counties' 2004 and 2005 tax roll data and created a table in databases with the parcel coordinates, property values and calculated property taxes for each of the years. (Example 1) GEvent.addListener( map, \"click\", function(a,p) { //here if == null && map.getZoomLevel > { map.centerAndZoom p, map.getZoomLevel } else if ( o == null && map.getZoomLevel()>0) map.centerAndZoom p, map.getZoomLevel( -1);1 (Example 2) function parcel() { // Generate a new XMLHTTP Object xmlhttp = false; try { xmlhttp = new ActiveXObject (\"Msxml2.XMLHTTP\"); } catch (e) { try { xmlhttp = new ActiveXObject (\"Microsoft.XMLhTtp\"); } catch (E) { xmlhttp = false; } if (!xmlhttp && typeof XMLHttpRequest!='undefined') { xmlhttp = new XMLHttpRequest // Get the current map bounds var b = map.getBoundsLatLng // Send an XMLHTTP Request, asking for the bubble data xmlhttp.open(\"*GET\",\"SH/cgibin/geo2.cgi?x=\"+b.minX+\"&y=\"+b.m inY+\"&x2=\"+b.maxX+\"&y2=\"+b.maxY,f true) xmlhttp.onreadystatechange = process; xmlhttp. send (null) (Example 3) # The table columns we want to output my @col = (id lon lat name address soh just assd txbl home tax cat sohtax rate cnty brk name_05 soh_05 just_05 assd 05 txbl_05 home 05 brk_05 rate_05 sohtax 05 tax_05 cat_05 justg sohg assg txblg taxg sohtaxg sohbrk sohbrk_05); my $ret = \"Content-type: text/xml\\n\\n<items>\"; # Only do 2/3 of the available area my $x=(p(x2')- p(x')) (1/6); my by=(p(y2')- p(y'))*(1/6); = my (p('x')+p('x2')) / 2 my (p('y') + p('y2')) # Request a slice of parcels my $q = $db->prepare ( \"SELECT *, SQRT (POW (parcel_lon- $cx,2) + POW (parcel_lat - $cy,2)) as C FROM parcels WHERE parcel_lon >= ? AND parcel_lat >= ? AND parcel_lon <= AND parcel_lat <= and par- cel_just_05 > 0 ORDER BY C LIMIT 40:f is $q->execute p('x') $x, p('y') $y, p('x2') $y The project examined Florida's de- cade-old Save Our Homes amend- ment, which caps annual increases in assessed values (and so taxable values) for resident homeowners. We wanted the Web application to show how the amendment had created wild discrepancies in property taxes for neighbors, even those living in virtually identical houses. Google's map does its magic through client-side scripting and Asynchro- nous JavaScript and XML (AJAX). AJAX allows a programmen to grab data from a server without reposting a page. It is the essence of how Google Maps works. In our case, we hired John Resig, a Rochester Institute of Technology senior, to build the skeleton of the application and we adapted his code. We found him by doing a Google search and came across his Web blog (http://ejohn.org) describing techniques using the Google Maps API. (For more about the APIs see www.google.com/apis/map.) Anyone planning to work with Google Maps (or the similar Yahoo! Maps) must be a proficient or know a profi- cient programmer in avaScript and a server-side language. In our case we used PERL on the server, but could have picked from other languages, including the various NET flavors. The Google APIs are pretty com- plete, but because of differences in browsers, the programming can be tricky. Microsoft's Internet Explorer 6, for example, renders icons on a map very slowly. Mozilla Firefox, however, renders icons almost instantly. We also wanted to change some of the behavior of the map from the standard Google Maps. The most important being that a click on the map drills down two levels and cen- ters on the click. (We wanted to get users down to the neighborhood as quickly as possible.) (Example 4) Print out the resulting XML $ret .= \"<item join map{ $row->{\\\"parcel_$_\\\" = s/&/&amp;/g; $row->{\\\"par cel_$_\\\"} = s/'/&apos;/g; $row->{\\\"parcel_$_*)\\\" nso } @col ) \"/>\"; (Example 5) // from the XML, gets the cat of property var n = it [i] .getAttribute (\"cat_05\") 0; var 35 ( * 5 ) ; // and figure out if is a homesteaded property var ic = new GIcon icon ic.image = \"img/bubble\" + [i] getAttribute (\"home 05\") == \"1\" ? \"check\" na ) n + \" . png\" ; // and creates an icon ic.iconSize = new GSize j, j ic.iconAnchor = new GPoint j - 20, j - 10 ) ; c.infoWindowAnchor = new GPoint 10, 10 // and places the icon on the map using the latitude and longitude of the parcel contained in the XML var III = new GMarker ( new GPoint ( it [i] getAttribute (\"lon\"), .getAttribute lat\") ),ic); Example 1 shows the vaScript code to add click and zoom func- tionality. This snippet is part of the windows.onload function(). Effec- tively, it grabs the point of a click and zooms in two levels, centered on the point of the click. At the lowest extent of the map, a client-side function fires that initiates a PERL script on the server that in turn queries a database and returns XML-formatted data. The client-side script then picks back up and places the icons on the map. Example 2 shows the vaScript that fires the PERL script on the server. The first half of the code allows the browser to accept XML-formatted data. The second half grabs the coordinates of two opposite corners of the map and sends them to the PERL script. See Example 3 for PERL code to grab the data from a MYSQL server using the map boundaries to gather the appropriate parcel data for the map. After doing some server-side calcu- lations to assign a value that would determine the size and type of icon to be displayed on the map, we then returned the data to the client using the code in Example 4.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-march-april-mine-safety-stories","slug":"2006-march-april-mine-safety-stories","url":"https://dwillis.github.io/uplink/article/2006-march-april-mine-safety-stories.html","year":2006,"month":"March-April","monthNum":3,"headline":"Mine safety stories","kicker":"IN THE NEWS","author_name":"Brian Hamman","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists utilized computer-assisted reporting techniques to investigate mine safety in the wake of the Sago Mine disaster, leveraging data from the U.S. Mine Safety and Health Administration.","keywords":["mine","safety","data"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["Excel","World Wide Web"],"full_text":"In the days and weeks after the Sago Mine disaster in West Virginia, journalists used their computer-assisted reporting skills to dig more deeply into stories about mine safety. The U.S. Mine Safety and Health Administration (MSHA) offers more than 20 years of accident, inspection, and violation data on its Web site that journalists can use to add context to a quick-hit story or as the basis of an investigation. When they set out to study the violations at Colorado's mines, Katy Human and Jeff Roberts of The Denver Post found they had to begin with a more basic question: \"Our first question was, do we have underground coal mines in Colorado?\" Human says. To get an overview of mining in Colorado, Human and Roberts turned to the data retrieval system provided by MSHA, available at www.msha.gov/drs/drshome.htm. an employment table with details about the number of employees, the amount of production, mine type and other information, and an incident table with details about each injury and illness, including a narrative of the incident and information about the experience, shift and job title of the person involved. Data is available for 1983-2004. By joining the employment table with the injury table, it is possible to calculate an injury rate for each mine, as well as a national average. Once you have identified mines of interest, however, you will need to turn to the data retrieval system to view citations and fines. Much of the attention following the Sago Mine focused on the time it took the first rescue team to begin work. Ward reported in The Gazette that a rescue team did not enter the mine until almost 12 hours after the explosion. To add depth to the story, Ward used a nationwide list of mine rescue teams available from MSHA from the mine rescue link off the home page, or www.msha.gov/MineRescue/MAP/ASP/minerescuehome.asp. The data comes as an Excel file with a row for each rescue team nationwide. The columns include number of members, the type of activities, the number of monthly training hours, and whether the team participates in mine rescue competitions. By combining the rescue team data with other MSHA data, Ward was able to report there was only one rescue team for every four mines nationwide. Human and Roberts caution journalists to be careful to note what dates are included in their analysis. Since MSHA updates its data retrieval system frequently, summary information such as violations and accident totals can change throughout the reporting of the story. To get acquainted with the terminology and regulations associated with the mining industry, Ward recommends several sources, including the Pennsylvania Department of Environmental Protection mining glossary (www.dep.state.pa.us/dep/deputate/minres/dms/website training/glossary.html) and the U.S. Code of Federal Regulations, which is available from the MSHA home page. Ward also recommends obtaining underground mining textbooks through interlibrary loan at your local library. In addition to the textbooks, some colleges offer mining programs. The faculty there can provide another resource. Tomich drew on the experience of a local mining professor to help him explain the differences between the amounts mines were fined. Contact Brian Hamman by e-mail at brianh@ire.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-march-april-data-lifts-lid-on-delay-money","slug":"2006-march-april-data-lifts-lid-on-delay-money","url":"https://dwillis.github.io/uplink/article/2006-march-april-data-lifts-lid-on-delay-money.html","year":2006,"month":"March-April","monthNum":3,"headline":"Data lifts lid on DeLay money","kicker":"SPOTLIGHT: MONEY & POLITICS","author_name":"Jonathan D. Salant","author_title":"","affiliation":"Bloomberg News","summary":"Analysis of financial contributions and ethics violations reveals insights into the influence and power held by Rep. Tom DeLay in Congress.","keywords":["politics","finance","ethics"],"topics":["Government & Politics","Investigative Methods"],"technologies":[],"full_text":"Until he stepped down as U.S. House majority leader in September and then announced in January that he would not attempt to reclaim his position, Rep. Tom DeLay was arguably the most powerful member of the House of Representatives.\n\nEven after he was rebuked three times in 2004 by the House ethics committee, DeLay had no trouble winning re-election to the No. 2 position in the House Republican leadership.\n\nOnly one member of the conference, Christopher Shays of Connecticut, called on DeLay to step down in the face of continuing ethical questions, including whether lobbyist Jack Abramoff","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-march-april-advanced-gis-analysis-targets-lost-wetlands","slug":"2006-march-april-advanced-gis-analysis-targets-lost-wetlands","url":"https://dwillis.github.io/uplink/article/2006-march-april-advanced-gis-analysis-targets-lost-wetlands.html","year":2006,"month":"March-April","monthNum":3,"headline":"Advanced GIS analysis targets lost wetlands","kicker":"MEYER AWARD WINNER","author_name":"Matthew Waite","author_title":"Reporter","affiliation":"St. Petersburg Times","summary":"A GIS-based investigation discovers large-scale loss of wetlands in Florida, underscoring the failure of federal commitments to protect these critical areas.","keywords":["GIS","wetlands","analysis"],"topics":["Health & Environment","Investigative Methods","Technology"],"technologies":["Landsat","SPOT","satellite imagery"],"full_text":"I hadn't been working in the St. Petersburg newsroom for more than a month when Craig Pittman, the environment reporter at the Times, said he had an idea he wanted to run past me. He wanted to measure the wetlands that had been destroyed in the past 15 years, but had no data and no idea how to do it. I told him it would probably require an analysis of satellite imagery. \"Probably,\" I said, lying through my teeth. I had no idea if I could do it. \"Can you do it?\" he asked. Eventually, I figured it out, and the answer to Pittman's question was not so simple. Between 1990 and 2003 - a time when the federal government pledged to stem the destruction of wetlands - about 84,000 acres of Florida's wetlands were replaced by urban uses; houses, schools, gas stations, malls, etc. That figure formed the backbone of an investigation that revealed how poorly the U.S. Army Corps of Engineers has enforced the Clean Water Act. What did it take to do it? Two college courses, $5,000 in software, 220 GB of hard drive space, editors with an abundance of patience and faith and about 10 months of free time. If I haven't scared you off yet, and you think you might try your hand at satellite imagery analysis, here's how we handled the problem of detecting change with imagery. The first step in any satellite imagery analysis is to remember Philip Meyer, the father of precision journalism and a professor at the University of North Carolina. Meyer has been telling us that we should use the scientific method for journalism. With satellite imagery analysis, it is a survival method. You need to clearly understand your problem and thoroughly understand your data before starting your analysis. Why? Lots of reasons. If you don't understand what you are after, you could spend months and money analyzing data, only to come up with nothing. If you misunderstand your question, you might buy the wrong data. That's right: More often than not, you are going to be buying data. Some of it is expensive. You do not want to spend hundreds of dollars on Landsat imagery only to realize you needed to spend thousands on SPOT data. To help you understand your question, you need to understand what imagery can show you. For instance, Landsat data has a resolution of 30 meters- one pixel in the image equals 30 meters on the ground. If you're looking for individual houses, it's the wrong data. If you are looking at how whole subdivisions have changed the landscape, you're in the right place. You also need to know what data you need in your imagery. Imagery can contain just visible light - three bands, red, green and blue - or it could contain slices of the infrared spectrum. You need to know what the bands can tell you and which bands you need for your analysis. It's harder than it sounds, and could fill an Uplink article by itself. It might seem esoteric to talk about being sure you understand your own question, but it's not. It's critical. I learned the hard way. We originally set out to find how many acres of wetlands had disappeared between 1990 and 2003. After months of work, I had an answer, but it was tens of thousands of acres beyond what any of our reporting had found. So I looked at what we had - a map showing where wetlands had increased and decreased between 1990 and 2003. In a few minutes, I found the problem - tides. In one map, the tide was out, in another, the tide was in. So thousands of acres of saltwater marshes were gone, but not really. Other changes were time differences - one image was taken during a wetter month than another. All over Florida, natural changes - rains, tides, etc. - affect the size of wetlands. Our question was too simplistic. What we were really interested in was how many acres of wetlands man had destroyed permanently. More on that in a minute. Obtaining data is both easier and more difficult in satellite imagery analysis. It is easier because there are only so many places you can get it. It is harder because the ways in which imagery is stored are a little arcane and, most of the time, costs money. Before you buy data, though, ask around. You might be surprised who has some and who might be willing to share. I found a complete 2003 dataset through a state agency, which made it free to me under Florida's public records law. I found a complete Landsat dataset for the late 1980s through the University of Maryland's Global Land Cover Facility. They make imagery from all over the country freely available to anyone for download. The upside? It's free. The downside? It may not be the time period you want, or they may not have your area. If you need something more detailed than Landsat, it gets more difficult. For instance, Space Imaging, the people who run the Ikonos satellite, will sell you 1-meter high-resolution imagery, but it's expensive. Many states and the U.S. Geological Survey do aerial photography at 1 meter - USGS will sell it to you or your state might give it away. You might be able to avoid paying for imagery, but you won't duck the software price tag. And it's a doozy. Leica ERDAS IMAGINE , the industry stan- dard software tool, is $3,500 last time I checked. Leica makes an extension for ArcGIS called ArcView Image Analysis. It is $2,500, and it is the one we went with, though I regret that now. Comparing IMAGINE to Image Analysis is like comparing SAS to Excel. The power difference is huge, and you will outgrow Image Analysis before you're done. There are other software packages, but you don't even want to know how much they cost. With software and data, we set out. The analysis I devised was based on a half a dozen academic papers I read that attempted something similar to the same thing. It's intentionally simple, and it's called unsupervised classifications. (The detailed methodology with bibliographic information is available at www.sptimes.com/2005/05/22/State/In_depth_methodology.shtml.) In unsupervised classifications, the software takes pixels and groups them together into assesses-clusters of pixels of close to the same values. You can set the number of pixel classes you want. We went with 100 to give us maximum flexibility. The trouble with unsupervised classifications is that the computer creates the classes but you have to identify what the class is. How do you do that? You click. A lot. Using maps of ancillary data - land- use maps, road maps, anything you think will help - you determine that all the Class 10s fit inside ponds, so you identify all the Class 10s as ponds. We kept it simple, identifying things as wetlands or not wetlands. After you get some practice, one Landsat scene might only take a day's work. Until then, it can take days. But with 14 Landsat scenes covering Florida, it was extremely tedious. After classifying all the imagery twice - once for each year - you can then compare them. Image Analysis has a change detection algorithm built into it. The algorithm will compare two raw images, but the work that goes into comparing raw images is extremely complex and far beyond this article. With classified images, the algorithm will detect where it was once one thing and is now another. From there, the analysis gets more complicated. To answer my question - to see where wetlands had been replaced by urban uses - I had to rely on another dataset, a satellite-derived map of urbanized areas. It was based on the same data, so I just totaled up the acres of area that had been wetlands but were now urbanized areas. To do that, I used another extension, Spatial Analyst. Spatial Analyst, a $2,500 extension to ArcGIS, works with raster data, which is what satellite data is. Spatial Analyst has powerful tools that are like Excel for raster data. There are functions and formulas you can write to get the information you need. The syntax is a little tricky, but the ESRI support reference is very good. Our analysis was, in the end, easily the most complicated work I've ever done. If you decide to try this, I strongly urge you to first take college courses in this. It is well worth the effort and a tremendous addition to your CAR toolbox. But it is complex data and not something you can learn in a tipsheet or an afternoon class. I took two remote sensing classes, which are offered by many large universities. Many of these introductory classes are often offered for those who are getting geography degrees, and the advanced courses are often graduate level. You can read Our Vanishing Wetlands online at www.sptimes.com/wetlands. Contact Matthew Waite by e-mail at waite@sptimes.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-march-april-new-staff-member","slug":"2006-march-april-new-staff-member","url":"https://dwillis.github.io/uplink/article/2006-march-april-new-staff-member.html","year":2006,"month":"March-April","monthNum":3,"headline":"New staff member","kicker":"","author_name":"","author_title":"assistant database library administrator","affiliation":"IRE and NICAR","summary":"An analysis of campus crime statistics highlights the University of Massachusetts as having the highest rate of violent crime, prompting discussions on safety measures.","keywords":["campus","crime","safety"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":[],"full_text":"Megan Means (formerly Christensen) has rejoined the staff at IRE and NICAR as an assistant database library administrator. After working as a graduate research assistant in the Database Library from 2001 to 2003, Means spent a year at the Detroit Free Press through the Knight Ridder specialty development internship program, as a metro reporter and CAR specialist on the investigative team. Major projects included a mapping analysis of childhood lead poisoning in Detroit and a review of the Ford Crown Victoria's accident history. Most recently, she covered the K-12 education beat for the Columbia (Mo.) Daily Tribune. She is working toward completing her master's degree at the Missouri School of Journalism.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-march-april-campus-crime","slug":"2006-march-april-campus-crime","url":"https://dwillis.github.io/uplink/article/2006-march-april-campus-crime.html","year":2006,"month":"March-April","monthNum":3,"headline":"Campus crime","kicker":"INSIDE NICAR","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Introduction of OpenInfo, a tool for intranet-based full-text search across newsroom databases, offering a cost-effective and efficient solution for data access.","keywords":["database","search","intranet"],"topics":["Crime & Justice","Data & Databases","Education"],"technologies":[],"full_text":"ABC News Primetime reported in November that the University of Massachusetts had the highest rate of violent on-campus crime among the largest universities in the United States, based on an analysis of data by the IRE and NICAR Database Library. The analysis also found that forcible sexual assault was reported more frequently than any other type of violent crime. IRE and NICAR analyzed 2002-03 Campus Crime Statistics data for the report. The analysis separated universities and colleges into four groups based on enrollment. The federal Jeanne Clery Disclosure of Campus Security Policy and Campus Crime Statistics Act requires colleges and universities to release campus crime and security procedures each year. The Database Library has the 2001-2003 data, which is available to journalists for a processing fee. The dataset includes incidents of liquor and drug violations and possession of illegal weapons. A separate table tracks hate crimes, in which victims were picked out because of their race, gender, religion, sexual orientation, ethnicity or disability. For more information about how the Database Library can perform analysis for your news organization, contact Jeff Porter, director of the Database Library, at 573-882-1982 or jeff@ire.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-march-april-yes-we-still-say-computer-assisted-reporting","slug":"2006-march-april-yes-we-still-say-computer-assisted-reporting","url":"https://dwillis.github.io/uplink/article/2006-march-april-yes-we-still-say-computer-assisted-reporting.html","year":2006,"month":"March-April","monthNum":3,"headline":"Yes, we still say \"computer-assisted reporting\"","kicker":"","author_name":"David Herzog","author_title":"NICAR representative","affiliation":"NICAR and Missouri School of Journalism","summary":"Computer-assisted reporting remains a vital journalistic skill that goes beyond simple web searching, involving data analysis through databases and spreadsheets to uncover stories that wouldn't be apparent from surface-level web browsing.","keywords":["journalism","data","analysis"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet","GIS"],"full_text":"A contributor to Poynteronline recently asked \"Does Anyone Still Say \"Computer- Assisted Reporting'?\" in a brief essay that suggested CAR has gotten as mundane as using the telephone to gather news. Well, it hasn't and plenty of journalists still utter the words \"computer-assisted reporting\" when they are talking about analyzing data using spreadsheets, database managers and mapping programs for their news stories. It's true that many journalists are tapping into Web tools to enhance their reporting, making good use of sites such as Google, reverse-lookup directories and the local assessor's online real estate tax rolls. But many journalists know CAR means more than effectively scouring the Web for information. A journalist visiting the U.S. Mine Safety and Health Administration Web site would not learn that the agency lets many of the major fines assessed against mine operators go unpaid. But journalists who have obtained MSHA's enforcement data and analyzed it using a database manager discovered that newsworthy piece of information, among others. That's what we still call \"computer-assisted reporting.\" Contact David Herzog by e-mail at dherzog@nicar.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-march-april-truck-inspections","slug":"2006-march-april-truck-inspections","url":"https://dwillis.github.io/uplink/article/2006-march-april-truck-inspections.html","year":2006,"month":"March-April","monthNum":3,"headline":"Truck inspections","kicker":"","author_name":"","author_title":"","affiliation":"Database Library","summary":"The Database Library has updated the U.S. Department of Transportation truck inspection dataset through 2004, allowing journalists to investigate truck safety violations by company, state, and cargo type. The resource enables reporting on enforcement gaps, unsafe trucking companies, and industry lobbying efforts.","keywords":["trucks","safety","violations"],"topics":["Public Safety","Data & Databases"],"technologies":[],"full_text":"The condition of trucks, crossing our nation's highways by the millions, is a critical matter for protecting the public's safety. The Database Library has updated the U.S. Department of Transportation truck inspection dataset through 2004. Journalists can review national or state truck inspection violations, by company or material carried to identify dangers to the public. Previous stories have delved into careless enforcement of regulations, poor safety records of trucking companies and lobbying by trucking companies to gain support for bigger, heavier trucks.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-march-april-finding-surprises-in-drug-arrest-files","slug":"2006-march-april-finding-surprises-in-drug-arrest-files","url":"https://dwillis.github.io/uplink/article/2006-march-april-finding-surprises-in-drug-arrest-files.html","year":2006,"month":"March-April","monthNum":3,"headline":"Finding surprises in drug arrest files","kicker":"FIRST VENTURE","author_name":"Liz Carey","author_title":"Reporter","affiliation":"Community Press (Cincinnati)","summary":"A journalist describes how computer-assisted reporting training helped uncover a drug trend in a Kentucky town by analyzing police data, emphasizing that successful data journalism requires both technical skills and strong source relationships.","keywords":["data","reporting","investigation"],"topics":["Investigative Methods","Data & Databases","Crime & Justice"],"technologies":["Excel","CD-ROM"],"full_text":"Two of hardest things for me to wrap my mind around at the IRE and NICAR Computer-Assisted Reporting Boot Camp were how to get the data I wanted, and what to do with it. Data analysis is more than taking apart a spreadsheet, or pawing through a database. Data analysis, we learned, is the middle step of a story. You cannot base a story on the data alone, and you cannot get the data unless you have the contacts to get it. All of this became clear to me when I returned from the March 2005 Boot Camp and was chomping at the bit to find a story to use my newfound skills. I bought the National Inventory of Dams database for my state from IRE and NICAR Database Library and searched through the IRE Web site for anything I could make a story out of, certain I could blow the lid off something in my Northern Kentucky beat, despite being at a weekly newspaper. Little did I know, the story was right beneath my feet. As part of my regular beat, I regularly wrote up police reports in the small towns covered, transcribing them from paper to my laptop. I began to notice an increasing number of arrests on drug-related charges. Cold Spring, Ky., was an example of this. Cold Spring, a city of just more than 4,000, was named the fastest growing city in Kentucky in 2004. The four-lane highway U.S. 27 runs through the middle of the city, connecting the eastern half of Northern Kentucky to I-275, the bypass that circles Greater Cincinnati. The city is hardly an urban center, boasting a generally white, upper-middle class population, with home values in the upper-$250,000 range. I was noticing more frequently that regular traffic stops in Cold Spring were turning into drug arrests. Additionally, shoplifting charges in the four major grocery stores were increasing and, more often than not, the things being stolen could be used in methadone production. I decided to take closer look. I didn't find a smoking gun, but an existing trend that was being investigated by the police. It is evidence of the importance of source relationships. We started by requesting all of the police reports since January 2000. At first, the request was met with a modicum of disbelief. In short, the department clerk told me I was free to go through all 5,000-plus handwritten reports and take from them whatever information I wanted. But the police chief told us that, since 2003, those records were kept in a database, and he would be willing to give them to us on CD. That data dump took more than an hour, not in processing time, but in negotiation over each data category and whether or not it should be divulged. The process meant sitting with the chief and discussing each category in depth, sometimes giving in, sometimes citing public record law. There was no charge for the data. Some categories of data were easily determined such as age, offense, date of offense and location of offense. Information about name, address, arresting officer and Social Security number had to be thoroughly discussed to determine how essential the information was to our story, and whether or not it violated anyone's privacy. In the end, we were able to get everything, except Social Security numbers, in a comma-delimited text file which we imported into a Microsoft Excel spreadsheet. I work in a Mac-based newsroom and Excel was readily available. In addition, I worked with Excel in the corporate world before becoming a reporter and it was easy for me to work with the data that way and turn over a story quickly, instead of relearning the Structured Query Language also covered in Boot Camp. The result was a detailed report of every arrest or citation in the previous three years that was easily sorted and manipulated for analysis. That analysis started with breaking down information by year. Each year's arrests and citations were then categorized by type of citation. All those not falling into a drug category were eliminated from the table. Then, data was sorted per citation. Each citation was sorted by name of offender. Offenders were checked against one another to make sure that they were not multiple charges on the same citation. Additionally, citations were sorted according to ZIP code to determine where the biggest number of offenders was coming from. The data proved that most offenders lived in the county, contrary to law enforcement professionals' belief that the majority of the offenders were coming from outside the county. Data also revealed offenders came more from rural areas of the county, than from the more urban areas. Those arrested were more likely to be mid-20s to mid-30s, instead of college-aged children coming from Cincinnati or nearby Northern Kentucky University as many law enforcement officials previously thought. Because of space considerations, much of that information was not used in the final story. After culling the data down to the essential information, saving each spreadsheet along the way, we were able to track the increase, however slight, over the three years covered in numbers and types of drug arrests and citations. That is where the story started, not ended. At the same time, our law enforcement sources were telling us that area police officers were looking into the same things we were. As it turned out, police officers in Campbell County, Ky., were working to clamp down on drug use along the same stretch of highway where we had noticed problems. A six-week sweep netted 70 busts for drug possession and use. It was the anecdotal evidence that helped make the story come alive. Highland Heights Police Chief Carl Mullen, who organized the sweep, said the problem stemmed from users traveling south from Cincinnati, as well as from residents from every municipality in the county. \"I was surprised at the number arrested; we all were actually,\" Mullen said. \"We knew it was a problem. But we didn't know how big of a problem it was.\" Mullen said arrests included pregnant women, social workers, students and even mothers with their children in the car with them. \"It was all walks of life. It's not isolated to any class,\" he said. \"We had one where a girl had told us that she at one point had a $400-a-day habit. When she was approached, she didn't have any drugs on her, but she admitted to us that she had an addiction problem.\" One officer talked about arresting a woman for shooting up heroin in her car while driving her children to day care. Our examination of the trends showed that while there was a 17 percent increase in the number of arrests made for drug-related offenses in Cold Spring alone, there was a staggering 146 percent increase in arrests made for possession of a controlled substance. Campbell County police continue to work together, nearly nine months later, on controlling the flow of drugs through the area. In the words of Mullen, what happened over those six weeks is just the tip of the iceberg. \"We got 70 people,\" he said. \"How many others do you think are out there?\" Contact Liz Carey by e-mail at lcarey@communitypress.com","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-march-april-covering-congress-states-getting-trickier","slug":"2006-march-april-covering-congress-states-getting-trickier","url":"https://dwillis.github.io/uplink/article/2006-march-april-covering-congress-states-getting-trickier.html","year":2006,"month":"March-April","monthNum":3,"headline":"Covering Congress, states getting trickier","kicker":"SPOTLIGHT: MONEY & POLITICS","author_name":"Derek Willis","author_title":"Reporter","affiliation":"The Washington Post","summary":"Computer-assisted reporting on elections has become more complex due to campaign finance law changes, requiring journalists to track data from multiple sources including 527 committees and state PACs beyond just the FEC database. Reporters covering 2006 elections must learn to gather and integrate federal, state, and IRS data to avoid missing important stories about campaign funding.","keywords":["elections","campaign","data"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["FTP","Internet"],"full_text":"Using computer-assisted reporting to cover congressional elections used to be so much simpler. The Federal Election Commission's FTP site was the first stop, where pretty much all of the campaign finance data you would ever need resided. Now, thanks to 527 political committees, state PACs and the 2002 McCain-Feingold campaign finance law, things are more complicated. It is no longer enough to limit your data gathering and analysis to the two or more candidates in a given congressional or statewide race. Do that in 2006 and you could miss the big story, or at least trail the rest of the pack on it. While the FEC is still the starting point for federal races, it is not the finish line. Those using CAR for elections should assess the contests they will be covering to see whether the old boundaries between federal and state races still apply. If your state has a hot race, then you will need to know how to gather data from multiple sources and integrate it into a single system. Consider the election for Virginia's attorney general in 2005 (Virginia is one of a few states that have significant off-year elections). While Republican and Democratic candidates raised millions for this statewide post, the GOP candidate got more than a third of his money from a single source: the Republican State Leadership Committee. Because the RSLC is a 527 organization reporting to the Internal Revenue Service, Virginia regulators and journalists knew only that it had given a bunch of money to a state candidate. They did not know where that money came from and, thanks to the IRS reporting schedule, they would not find out until after the election. Elections in 2006 will see more frequent disclosure from 527s, which follow a reporting schedule roughly equivalent to the FEC in even-numbered years. And like the FEC, the IRS releases its data in bulk - meaning that CAR folks will have to download the entire set of 527 electronic filings and then find the committees they are after. If you have never done this before, a little practice wouldn't hurt. Unlike the FEC, dealing with the IRS data doesn't require any reformatting of fields (something you can avoid by purchasing FEC data from the IRE and NICAR Database Library), but it does lump original and amended filings in the same compressed file. Updates are available every Sunday at http://eforms.irs.gov, and The Center for Public Integrity also has a regularly updated site that provides downloadable contributions and expenditures for 527 committees (www.publicintegrity.org/527). In federal races, some advertising activity by 527 committees must be reported to the FEC, thanks to the McCain-Feingold law. These expenditures, known as \"electioneering communications,\" occur when an organization not registered with the FEC airs a television or radio ad that mentions or identifies a federal candidate within 30 days of a primary election or 60 days of the general election. Such ads fall under the scope of the law only if they are targeted to a candidate's constituency; if the Sierra Club runs an ad about a New Jersey candidate in Arizona, it would not meet the criteria. The good news about electioneering communication reports is that they include very specific information about ad buys, including the stations and amounts involved, and reports are due within 24 hours of the ad's initial appearance. The bad news is that those reports are filed by fax only. For journalists outside Washington, D.C., that means waiting on the FEC to scan and post images of those faxes, which sometimes are difficult to read. It also means that there is no database of such activity, unless you decide to build one (The Center for Public Integrity did so for the 2004 elections). bulk form but are available on a filing-by-filing basis. Expenditures are easily the least-covered aspect of campaign finance, and sometimes the most important. They can show the tactics used by campaigns (direct-mail fundraising or big, flashy events?) or provide home addresses for campaign workers. Particularly interesting are the expenditures of candidates who face little or no opposition - what do they spend their money on? Such reporting can yield great stories, but if you go this route, remember that you'll need to keep track of the filings, because amendments appear often. I've found that it's useful to track filings themselves, and you'll probably want to do the same if you plan to maintain the latest and greatest version of a committee's data. The electronic filings end with the file extension \".fec\", but they are delimited text files that can be imported into Microsoft Excel or other programs. But you'll want to be familiar with the layouts and keep in mind that each filing contains all of the transactions associated with that report: contributions, expenditures and summary figures, all stacked on top of each other. A campaign finance junkie would do well to keep checking the FEC's FTP site regularly, however; the agency often will place custom collections of data there that might prove useful. One last tip, which has been a personal interest of mine: don't forget the meta-data. In campaign finance, the metadata means things like when a committee reports, how many amendments it files or even the length of the reports. Such details are worth tracking because they can provide insight into the operations of the campaign. Multiple amendments filed on the same day are a sure sign that either the campaign's accountant found a mistake or the FEC's auditors did. Either circumstance is worth a look, and may yield a good story. Contact Derek Willis by e-mail at willisd@washpost.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-march-april-simplifying-database-full-text-searches","slug":"2006-march-april-simplifying-database-full-text-searches","url":"https://dwillis.github.io/uplink/article/2006-march-april-simplifying-database-full-text-searches.html","year":2006,"month":"March-April","monthNum":3,"headline":"Simplifying database full-text searches","kicker":"OPEN SOURCE","author_name":"Brad Heath","author_title":"Database Reporter","affiliation":"The Detroit News","summary":"OpenInfo is a free, open-source intranet solution that enables newsrooms to conduct simultaneous full-text searches across multiple databases without expensive tools or extensive coding, featuring customizable search forms and cross-referencing capabilities built on Apache, MySQL, and PHP.","keywords":["database","search","newsroom"],"topics":["Data & Databases","Investigative Methods","Technology"],"technologies":["Apache","MySQL","PHP","Excel","Google Maps"],"full_text":"One of the most important things you can do in your newsroom is spread the wealth of information. But finding a way to let other reporters and editors get the data they need, whenever they need it, is a challenge. The easiest way to do it is on your company's intranet-but most of the tools are cumbersome or expensive. Here is my solution. OpenInfo is a collection of intranet scripts you can use to run simple, full-text searches across dozens of newsroom databases simultaneously. You can add data to the site with a few minutes of work, instead of spending hours writing new search forms. You can customize it to help people in your newsroom find information most relevant to them, with easy searches and automatic cross-referencing of records. Users who want to go deeper will find forms they can use to run advanced searches or write data queries (without having to know Structured Query Language). Best of all, it runs entirely on open-source platforms (an Apache Web server, a MySQL database server and the PHP scripting language). If you already have the hardware, you can get the whole package set up in an afternoon, at no cost. And because it relies on MySQL to do the heavy lifting, it can easily accommodate searches of millions of records. (I have used it on one server with almost 70 million records spread across 60 tables.) Download the OpenInfo file from www.nicar.org/downloads Once you provide OpenInfo with basic information about what kinds of data are stored in your table, it does the rest: building search forms, a query wizard and integrating the records into a global full-text search. Once you have the scripts set up (more on that in a minute), searching is very straightforward. The home page offers a single form that will search across every database that you incorporate into OpenInfo, using syntax similar to what you are accustomed to using for searching the Web or your electronic archives. (Figure 1) Because the search runs against a full-text index, you don't need to know the order in which your search words appear, or whether they're in the same field in the database. In addition to simple searches, OpenInfo gives you an almost unlimited ability to drill into and cross-reference your data. Take, for example, my own voter registration information. To find it, I'd start by searching for Heath, Brad* (yes, OpenInfo supports wildcards), then click on the matching results to see the full record. (Figure 2). OpenInfo shows all the fields incorporated into the database (unless you have chosen to hide a few), and makes the contents of each field clickable. Clicking any of the links will take you to a pop-up guide that will help you cross-reference that information against other Openinfo records, Google, or any other online database with an accessible URL syntax (Yahoo! News, your archives, your photo assignments, etc.). If you want, OpenInfo can also do some of this work for you. For example, you can teach Openinfo how to assemble the fields in your database to make a complete address. If you do, the script displays an embedded map for each record (as well as linking to Google Maps), and gives you tools for cross-referencing the address against other records. It will also automatically check for other records in that table with a matching address and show you a list. That means when you look at my voter registration record, OpenInfo automatically finds my wife's registration, too. (Figure 3). You also can have OpenInfo mash together other fields from the record that you think might be helpful for users looking to find related information. For example, it might be helpful to have one link that will search based on a person's first name, last name, city and state. For users who want more power, OpenInfo has more advanced search functions. You can search individual fields in a database, or use a built-in query grid to search through or summarize your data and export the results to Microsoft Excel (users who want to do even more can download full data tables as dBASE files). Users can create custom groups of sources, or search based on the type of information associated with a field, such as names and addresses. If you want, you can allow users to create their own databases on the fly, for their own use or to add to the public collection of sources. OpenInfo tracks searches and record views so you can find out quickly what is most popular with your newsroom. Setup is easy and inexpensive. To start, you need a MySQL database server, an Apache Web server and the PHP scripting language, all of which are open source. (If you don't already have these, you can add them fairly easily by downloading XAMPP, which installs all three at once, from http://sourceforge.net/projects/xampp.) After that, unpack the script files into one of the folders on your server. Then sign on to your database server as a root user and run the setup.sql file included with the scripts. (You might need to make a few other adjustments to your server, depending on how it's configured. A complete readme file is included with the scripts.) www.mysqlfront.de.) After you have data on your server, sign in to OpenInfo as an administrator, go to the main page and follow the \"Grab an existing database\" link. You'll be prompted to choose a database and table, then to enter a name and description of what information is stored in the table. Next, OpenInfo reads the structure of your data and asks you to fill in basic information about the table, such as the field that contains a unique identifier for each record and the field you want to use as the main display line for a record. (Figure 4) You also need to decide which fields you want included in the full-text search. (Note that this applies only to text fields, and because of limitations in MySQL, you can choose, at most, 16 fields.) You will be able to add additional information later, including adding detailed annotation to fields, specifying sort orders and setting up rules to create addresses and quick links. Openinfo is yours to play with and improve. If you don't like the way it works, change it. If you have a suggestion for a feature you like, let me know and I'll work to incorporate it. Among the improvements I'm already polishing up (they'll be ready soon) are a system that picks out relevant records based on previous searches and the ability to search based on how a word sounds, not just its literal spelling. Contact Brad Heath by e-mail at bheath@detnews.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-may-june-making-the-switch-to-acs","slug":"2006-may-june-making-the-switch-to-acs","url":"https://dwillis.github.io/uplink/article/2006-may-june-making-the-switch-to-acs.html","year":2006,"month":"May-June","monthNum":5,"headline":"Making the switch to ACS","kicker":"CENSUS","author_name":"Paul Overberg","author_title":"","affiliation":"USA Today","summary":"The Census Bureau's American Community Survey faced funding challenges but is likely to replace the long form of the 2010 Census.","keywords":["Census","ACS","data"],"topics":["Demographics & Census","Data & Databases"],"technologies":[],"full_text":"Once again, the Census Bureau's American Community Survey faced a budget challenge that could have killed it. It wasn't until a congressional conference committee hashed out details in mid-November that it was funded at sustainable levels for this year. Nothing is certain in Washington, but this means it's increasingly likely that after years in pilot, ACS finally will take wing and replace the long form of the 2010 Census. Each year, 3 million households will get the ACS survey. Each summer, data from the preceding year will be released for big places (over 65,000 people). By summer 2008, data from 2005-07 will be released as three-year averages for places of at least 20,000 people. Finally, in summer 2010, data from 2005-09 will be released as five-year averages for places as small as census tracts (3,000 people) and in some cases block groups (600 people). Every year after that, new annual data...","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-may-june-rolling-hazards-in-rail-cargo","slug":"2006-may-june-rolling-hazards-in-rail-cargo","url":"https://dwillis.github.io/uplink/article/2006-may-june-rolling-hazards-in-rail-cargo.html","year":2006,"month":"May-June","monthNum":5,"headline":"Rolling hazards in rail cargo","kicker":"SPOTLIGHT: TOXIC LIVING","author_name":"David Danelski","author_title":"The (Riverside, Calif.) Press-Enterprise","affiliation":"The Riverside Press-Enterprise","summary":"Investigation into freight train derailments revealed ineffective track inspections and other issues that endangered residents.","keywords":["rail cargo","hazards","derailments"],"topics":["Investigative Methods","Public Safety","Health & Environment"],"technologies":[],"full_text":"A freight train derailment last year left several tank cars containing deadly chemicals twisted and dented just yards from homes in densely populated San Bernardino. The accident raised obvious questions about the transport of hazardous materials by railroad through our region. Our investigation ultimately found ineffective track inspections, misidentified tank cars, and communications problems between railroad and police officials that allowed hundreds of residents to return home before it was safe to do so. The Press-Enterprise covers Riverside and San Bernardino counties, an area in Southern California that serves as a railroad conduit for growing volumes of goods that have arrived from Asia at the ports of Los Angeles and Long Beach. Much of our railroad coverage has focused on increasing rail traffic, toxic diesel exhaust from locomotives and long waits at rail crossings. The threat of potentially deadly chlorine gas escaping from a ruptured tank car became a new target for our reporting after the April 4, 2005 derailment. The San Bernardino wreck, we reported, was a symptom of a national hazard that claimed more lives in 2005 than in the previous 20 years combined. San Bernardino had escaped a disaster...","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-may-june-er-gridlock-exposed","slug":"2006-may-june-er-gridlock-exposed","url":"https://dwillis.github.io/uplink/article/2006-may-june-er-gridlock-exposed.html","year":2006,"month":"May-June","monthNum":5,"headline":"ER gridlock exposed","kicker":"HEALTH CARE","author_name":"Augusta Brennan Jones","author_title":"investigative reporter","affiliation":"WBAL-Baltimore","summary":"Investigation into Maryland ERs revealed significant delays and highlighted a crisis in emergency healthcare.","keywords":["healthcare","ER","Maryland"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"If you have visited a hospital emergency room, you've probably experienced a wait. But I'll bet it wasn't as long as the time I rushed my husband, John, to the hospital. It was a Monday night last July. John is a cardiac patient, and he was having neck and jaw pain. That visit, which stretched to more than eight hours, inspired our investigative unit at WBAL to examine ER waiting times using freely available data from a Maryland state agency. Shortly after I got John to Northwest Hospital outside Baltimore, he was called to the triage unit where they checked his vital signs and determined he could wait. So they sent him back to the waiting room with dozens of other sick-looking patients. After about an hour, he still had not seen a doctor; that's when a nurse announced there would be a six-hour delay. I said, 'I'll take him to Sinai Hospital.' She said, 'Sinai is on an eleven-hour delay.' I couldn't believe it. There didn't seem to be any option but to wait, and it gave me lots of time to wonder what's going on inside Baltimore ERs...","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-may-june-storm-season","slug":"2006-may-june-storm-season","url":"https://dwillis.github.io/uplink/article/2006-may-june-storm-season.html","year":2006,"month":"May-June","monthNum":5,"headline":"Storm season","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library has updated its Storm Events database through December 2005, containing over 600,000 records of U.S. storms including Hurricane Katrina with details on fatalities, injuries, property damage, and geographic coordinates for mapping purposes.","keywords":["storms","database","hurricanes"],"topics":["Public Safety","Data & Databases","Technology"],"technologies":["GIS"],"full_text":"Just in time for storm season, the IRE and NICAR Database Library has updated its Storm Events database through Dec. 31, 2005. The data contains information about last season's hurricanes, including Hurricane Katrina.\n\nThe database includes more than 600,000 records of U.S. storm events ranging from hail and flooding to hurricanes and tornadoes. Each record contains details about fatalities and injuries, damage to crops and property, and remarks about the storm. Many of the storm records include latitude and longitude data, which can be used to map the locations using geographic information system (GIS) software.\n\nIn addition to information on recent storms, the database has archival information as far back as 1950.\n\nTo order the storm events data, contact the database library at 573-884-7711 or download an order form at www.ire.org/datalibrary/orderform.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-may-june-deadly-roads","slug":"2006-may-june-deadly-roads","url":"https://dwillis.github.io/uplink/article/2006-may-june-deadly-roads.html","year":2006,"month":"May-June","monthNum":5,"headline":"Deadly roads","kicker":"Bits & Bytes","author_name":"","author_title":"System administrator","affiliation":"WCBS-New York","summary":"A database library analysis of the Fatality Analysis Reporting System examined fatal vehicle accidents in New York and New Jersey for WCBS-New York, revealing that most accidents occurred during daylight hours in fair weather conditions.","keywords":["accidents","highways","fatalities"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["FARS"],"full_text":"In February, the database library analyzed fatal vehicle accidents data for a WCBS-New York story.\n\nAssistant administrator Megan Means analyzed the Fatality Analysis Reporting System (FARS) Data, available from the database library, to count fatalities on highways in New York and New Jersey. She found the majority of accidents occurred in daylight during fair weather conditions.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-may-june-car-its-a-starting-point","slug":"2006-may-june-car-its-a-starting-point","url":"https://dwillis.github.io/uplink/article/2006-may-june-car-its-a-starting-point.html","year":2006,"month":"May-June","monthNum":5,"headline":"CAR: It's a starting point","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"Uplink","summary":"The article discusses computer-assisted reporting (CAR) training at the IRE conference in Fort Worth, where journalists learn to use databases and data analysis tools to enhance their reporting. Despite initial complexity, CAR provides a foundation for producing better news stories.","keywords":["journalism","databases","reporting"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["GIS","SQL"],"full_text":"Hundreds of journalists were on hand for the computer-assisted reporting day at the annual IRE conference in June in Fort Worth, Texas. The one-day focus on CAR is an annual rite for many journalists and, for others, an introduction to the use of databases, spreadsheets and Internet resources for reporting.\n\nFor many newcomers, it's easy to get overwhelmed by the possibilities of CAR and all of the jargon (think GIS, SQL, regression analysis, intranets). But those journalists who persist can develop new reporting tools that can help them produce better news stories. That's what it's all about, right?\n\nIn each issue of Uplink, you read about these news stories. And while you might notice differences in the software, techniques and obstacles overcome, you'll see one common thread: CAR provided the starting point for even more reporting.\n\nSpeaking of Uplink, we're working on publishing the newsletter electronically. If you have any suggestions for use, please let me know. I'd love to hear them.\n\nThe March-April Uplink \"Mapping it Out\" feature incorrectly identified the location of a village near Peoria, Ill., that was struck by a tornado. It is South Pekin.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-may-june-about-our-contributors","slug":"2006-may-june-about-our-contributors","url":"https://dwillis.github.io/uplink/article/2006-may-june-about-our-contributors.html","year":2006,"month":"May-June","monthNum":5,"headline":"About our contributors","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Profile of six journalists working across various U.S. newsrooms, highlighting their specializations in investigative reporting, data journalism, and beat coverage in areas including environmental issues, education, and crime.","keywords":["journalists","reporting","newsrooms"],"topics":["Journalism & Newsroom"],"technologies":[],"full_text":"Augusta Brennan Jones is executive producer-special projects at WBAL-Baltimore, where she manages the station's I-Team. She began her broadcast career as a producer at WRTV-Indianapolis.\n\nDavid Danelski covers air pollution and does investigative projects for The Press-Enterprise in Riverside, Calif. He is an alumnus of the 2004 Advanced CAR Statistics Boot Camp.\n\nPaul Overberg, a database editor at USA Today, helps coordinate its demographic coverage. He began using data from the American Community Survey nine years ago and has been invited to participate in several user workshops on development of its products.\n\nEric Sagara has reported the Tucson Citizen for five years and has covered the crime, higher education, business and wildfire beats. He currently covers primary and secondary education. He is an alumnus of the 2005 Advanced CAR Statistics Boot Camp.\n\nDuane Schrag is special projects reporter at the Salina (Kan.) Journal. He has worked for the Harris Enterprises newspaper group, based in Hutchinson, Kan., since 1984. Previously, he was editor and publisher of the Chanute (Kan.) Tribune.\n\nKen Ward Jr. is a reporter for The Charleston (W.Va.) Gazette and has received national reporting awards for stories about environmental topics. He is chairman of the Society of Environmental Journalists' First Amendment Task Force.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-may-june-stats-help-deflate-big-crowd-count","slug":"2006-may-june-stats-help-deflate-big-crowd-count","url":"https://dwillis.github.io/uplink/article/2006-may-june-stats-help-deflate-big-crowd-count.html","year":2006,"month":"May-June","monthNum":5,"headline":"Stats help deflate big crowd count","kicker":"FIRST VENTURE","author_name":"Eric Sagara","author_title":"","affiliation":"Tucson Citizen","summary":"A journalist learned data analysis techniques to accurately count crowd attendance at the La Fiesta de los Vaqueros Rodeo Parade in Tucson, finding the actual attendance was roughly one-third of the organizers' official estimates of 175,000-180,000 people.","keywords":["crowd","counting","parade"],"topics":["Investigative Methods","Data & Databases","Journalism & Newsroom"],"technologies":["Excel"],"full_text":"If you have ever strained to count a crowd of people milling around a concert, parade or festival, you know how hard it can be to get an accurate head count.\n\nThe difficulty of crowd counting came up briefly at an IRE and NICAR Advanced CAR Statistics Boot Camp at Arizona State University, and I learned there is a way to go beyond official estimates and find a more accurate estimate for readers.\n\nWithin days of my completing the Boot Camp, editors wanted to find a way to accurately count the number of people that attended La Fiesta de los Vaqueros Rodeo Parade.\n\nTouted as the world's longest, non-motorized parade at a length of 2.1 miles, it marks the beginning of the Tucson Rodeo, the first event of the season on the U.S. professional rodeo circuit.\n\nOrganizers say the parade and the events over the ensuing days are a major economic boon to the community with an estimated 200,000 people attending the parade alone every year.\n\nAs a native Tucsonan who had never been to the rodeo or the parade, I found that figure hard to believe. It meant that as many as one in every four people living in the county attended the opening day event.\n\nLast year, our count put attendance at roughly a third of the parade committee's official estimates, which ranged from 175,000 to 180,000.\n\nStats camp instructor Steve Doig, a professor at Arizona State University, talked about several different methods of approaching the problem, including using of aerial photography and software designed specifically for the task.\n\nWe decided the parade did not warrant the expense of chartering an airplane or purchasing software. Instead, I opted for a notebook, an Excel spreadsheet and my mountain bike.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-may-june-endangered-data-helps-find-deadly-mercury","slug":"2006-may-june-endangered-data-helps-find-deadly-mercury","url":"https://dwillis.github.io/uplink/article/2006-may-june-endangered-data-helps-find-deadly-mercury.html","year":2006,"month":"May-June","monthNum":5,"headline":"Endangered data helps find deadly mercury","kicker":"SPOTLIGHT: TOXIC LIVING","author_name":"Ken Ward Jr.","author_title":"Reporter","affiliation":"The Charleston (W.Va.) Gazette","summary":"TRI data was used to investigate mercury pollution from chlor-alkali plants, revealing significant environmental concerns.","keywords":["TRI","mercury","pollution"],"topics":["Health & Environment","Investigative Methods","Data & Databases"],"technologies":[],"full_text":"At the annual computer-assisted reporting conferences and on the NICAR listserv, I always hear about how bad government data is. When the conversation steers toward the U.S. Environmental Protection Agency's Toxic Release Inventory, the criticisms seem especially harsh. And they're probably mostly true. There are lots of problems with the TRI. But right now, the biggest problem is that we might lose it. The EPA is proposing major cutbacks, including a plan to make the now-annual data collection an every-other-year deal for the chemical industry and other manufacturers. I also know that some of the best stories I do are based in some way on TRI, or at least use some nugget from the TRI for that crucial 'computer-assisted paragraph' that makes the story solid. TRI is still the best basic set of pollution numbers we have. With that in mind, here's how TRI helped me turn what could have been a boring daily story from a news release into a more meaningful Sunday and Monday package of stories...","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-may-june-excel-and-vb-help-crack-school-finances","slug":"2006-may-june-excel-and-vb-help-crack-school-finances","url":"https://dwillis.github.io/uplink/article/2006-may-june-excel-and-vb-help-crack-school-finances.html","year":2006,"month":"May-June","monthNum":5,"headline":"Excel & VB help crack school finances","kicker":"the Detables Tech tip","author_name":"Duane Schrag","author_title":"","affiliation":"The Salina (Kan.) Journal","summary":"A Kansas journalist investigated the true cost of implementing school finance reforms by analyzing state enrollment data and formulas using Excel and programming, challenging conflicting estimates ranging from $236 million to $1 billion. The story demonstrates how data analysis and mathematical formulas can verify complex policy claims that state education officials initially dismissed.","keywords":["financing","education","analysis"],"topics":["Investigative Methods","Education","Data & Databases"],"technologies":["Excel","Access","Visual Basic"],"full_text":"All across the United States lawsuits have been filed in recent years accusing state governments of failing to provide a suitable education for their citizens. Many - perhaps most - of the cases turn on the question of adequate financing: Are schools being given enough money to educate students most at risk?\n\nKansas is one of those states. More than two years ago, a county district judge ruled that too little money was being spent to educate low-income, bilingual and special education students. During the litigation the State Board of Education - the principal defendant - commissioned a study to find out whether the Kansas school finance formula was appropriate and, if not, how much it would cost to implement one that was.\n\nThe study by consultants Augenblick & Myers proposed a new formula for school spending. The study's findings were presented as evidence in the case, along with the state's estimate of the financial impact of the formula. In ruling for the plaintiffs, the judge repeatedly said the cost of fixing school finance in Kansas would be at least $1 billion. However, the state had submitted a figure of $853 million. And the study estimated it at $236 million.\n\nMy story set out to answer the question: How much would it really cost to implement the findings? And how would it affect the 75 school districts in our readership area? I used Microsoft Excel, Access and Visual Basic programming to answer the questions.\n\nEarlier, when I had asked the gurus at the state education department about that, they were dismissive. \"Augenblick & Myers got it wrong,\" they said. I have worked with these people for years, and I took their word for it. If I had listened more closely to my instincts, we would have had this story earlier.\n\nThe Kansas Supreme Court later upheld the lower court's ruling, so the funding was in the news again. I requested figures on how the ruling would affect the 302 school districts in Kansas. The Kansas Department of Education sent me the information, with the stipulation that I not attribute it. \"Even the school districts haven't seen this,\" I was told. That was a red flag.\n\nInitially, the thought of calculating the cost on my own was too intimidating. But after looking closely at the formulas, I realized it was not out of the question. It required algebra, not rocket science. Calculating the cost involved:\n\nformulas to compute weighting factors for district size, low-income, bilingual, and special education. These factors assign more \"weight\" to low-income students, for instance, which increases state aid for them.\nenrollment figures for each school district, and the subgroups within each district.\n\nThe enrollment information was relatively easy to get. The state does a fairly good job of posting it and many other key tables online. Most importantly, the data is usually available as Microsoft Excel spreadsheets, so rekeying is not necessary. If possible, get this data in electronic, not paper, format. Transcription errors are too easy to make.\n\nIn my calculations, I had to make sure I was using full-time equivalents when those were required, headcounts when headcounts were required. I had to re-run the data after learning the special education figures posted online were incomplete - they excluded children in gifted programs.\n\nThe formulas themselves are straightforward - a series of If/Then statements, with assorted multiplication and division operations. Excel has a built-in If() function that I found too limited for my purposes. Instead, I wrote custom functions in Visual Basic.\n\nExample 1 examines the adjustment for school size. This is how the consultant presented the formula for a base budget per pupil of $4,550.\n\nExample 2 is the equivalent in Visual Basic. I named the function size_weight(), which takes as arguments the base budget per pupil (b) and the district's enrollment (e).\n\nYou may notice the base budget per pupil is a variable, not a constant - this made it possible to compute the cost for any base figure. For our story, we used $5,000, which was the study's original figure adjusted for inflation.\n\nWith the formulas in hand, and the raw numbers in spreadsheet format, it was a breeze to compute the weightings, and thereby create the exact budget figure for each school district. At this point, I knew how much the proposal would cost, but that was only half the problem.\n\nThe other half of the problem was comparing the proposal to the status quo. Again, finding the raw data was not difficult - every school district's budget is posted online, in sufficient detail to extract the information needed for the story.\n\nExtracting the correct figures required some work. In order to compare apples and apples, I needed to deduct expenses for transportation, capital outlay, food service and adult education. The budget is a massive spreadsheet - it consists of 14 separate sheets, each with 302 rows (one for each school district) and as many as 20 columns of figures. Spending for the services in question were scattered across those sheets.\n\nFirst, I sought help from the local school district. I sat down with the business manager and made certain I knew which columns had to be deducted to account for the necessary change. There were dozens.\n\nOnce again, I wrote a Visual Basic function to systematically extract the necessary information. While it is possible to do this by hand, the number of operations (about 10,000) makes it highly likely a mistake will be made.\n\nNow I could make the proper comparison. I had the true cost of applying the proposal, district by district. And I had the actual comparable cost under the existing formula. I used Microsoft's Excel and Access to manage the data, although relying solely on Excel would be possible.\n\nA&M's model was never adopted, so it is impossible to say what it would have ended up costing. But in January 2006, an exhaustive study by the Kansas Legislative Division of Post Audit on the \"true\" cost of providing an education said an additional $400 million is required; this comes on top of an extra $143 million added last year. That put the additional spending right where we said A&M would have put it.\n\nPerhaps the most important lesson this story reinforced for me is that journalists must do a tricky balancing act - know when to ask for information, know when to do the work yourself.\n\nSources for this story gave wildly different interpretations of what the A&M study said. Some who should have known it backwards and forwards were dead wrong. Only by reading it, re-reading it, discussing it, and re-reading it further was I able to offer the story's conclusions with confidence.\n\nContact Duane Schrag by e-mail at dschrag@saljournal.com","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-july-august-a-bumpy-ride","slug":"2006-july-august-a-bumpy-ride","url":"https://dwillis.github.io/uplink/article/2006-july-august-a-bumpy-ride.html","year":2006,"month":"July-August","monthNum":7,"headline":"A bumpy ride","kicker":"LATIN AMERICAN CAR","author_name":"Alejandra Fernández Morera","author_title":"","affiliation":"Missouri School of Journalism","summary":"Investigative journalists in Latin America are increasingly adopting computer-assisted reporting (CAR) to expose corruption, although challenges like limited technology and varying open-records laws persist.","keywords":["Latin America","CAR","investigative reporting"],"topics":["Investigative Methods","Business & Economy","Journalism & Newsroom"],"technologies":["Excel"],"full_text":"Cars travel on highways, but in Latin America, CAR travels on unpaved roads. While computer-assisted reporting has gained popularity in Mexico and Brazil, it is still terra incognita in Central America, and few can point out more than one journalist who excels in the genre's many techniques in South American countries such as Chile, Venezuela and Peru.\n\n\"Few practice and master [CAR],\" said Pedro Armendares, executive director of the Mexican Investigative Reporting Center, or CPI, a partner organization of Investigative Reporters and Editors, Inc., based at the Missouri School of Journalism. \"There are countries where it isn't practiced at all.\"\n\nIn the countries where CAR is practiced, the success has been clear. In one case, a Mexican reporter discovered that taxpayers' money was used to pay for $400 towels for the president's residence. The reporter used data from a government asked me 'How do you practice CAR without Excel or a computer?' Well,","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-july-august-program-for-disabled-misses-the-mark","slug":"2006-july-august-program-for-disabled-misses-the-mark","url":"https://dwillis.github.io/uplink/article/2006-july-august-program-for-disabled-misses-the-mark.html","year":2006,"month":"July-August","monthNum":7,"headline":"Program for disabled misses the mark","kicker":"SPOTLIGHT: ON THE JOB","author_name":"Tom Detzel","author_title":"Investigations Editor","affiliation":"The Oregonian","summary":"The Oregonian investigated the Javits-Wagner-O'Day program, uncovering that charities were redefining disability and overpaying executives while mismanaging federal contracts.","keywords":["disability","federal contracts","nonprofits"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["Excel","Access","Nexis","Dun & Bradstreet","Guidestar Premium"],"full_text":"Our stories on nonprofits that employ severely disabled workers sparked an FBI investigation and calls for reform of a little-known federal contracting program that spent $2.25 billion last year, much of it on military goods and services. For these stories, simple databases produced profound results. The Javits-Wagner-O'Day program sets aside government business for charities that mainly employ workers who are blind or severely disabled. But our reporting found that nonprofits were increasingly using workers with mild or questionable disabilities. The biggest contractor, the National Center for the Employment of the Disabled in El Paso, Texas, repeatedly failed to document that it was using enough disabled labor. And it had paid the management firm run by its president a stunning $8.7 million over two years. Although much of our information came from sources inside the charities themselves, reporters Jeff Kosseff, Bryan Denson and Les Zaitz used the federal Freedom of Information Act and Internal Revenue Service charity disclosure forms (Form 990) to get a grasp of overall trends in the program and target specific charities for investigation. Kosseff used FOIA to obtain five years of annual reports from the 627 charities. The reports, kept by a tiny federal agency (Committee for Purchase from People Who are Blind or Severely Disabled) that oversees the program, contained data on annual sales, wages paid to the disabled, and the hours worked on federal contracts. When he received the data in a Microsoft Excel spreadsheet, it was immediately clear that the bulk of the contracts went to the 50 largest charities. We decided to focus our reporting on this group, which the data showed had tremendous revenue growth since 9/11. Denson wanted to know the impact of the wars in Afghanistan and Iraq. He looked up the Dun & Bradstreet D-U-N-S nine-digit identification numbers of each of the 50 contractors, punched them into a spreadsheet and sent them to the Defense Department, asking for a list of their military contracts. The Pentagon can find contracting records quickly if you provide a contractor's D-U-N-S. You can find these numbers online on Nexis, Hoover's and other sites. The Pentagon turned the request around in two days. We then had a list of every defense contract for each charity over five years, detailing contract amounts and the type of goods or services provided. The bulk of the growth had indeed come from a burst in military spending. Defense business was only part of the picture. What about other sources of income – contracts from local governments and private firms – for the charities? And how could we measure whether workers or charities managers were benefiting? To get aggregate numbers, Denson turned to the charities' tax forms. Gail Hulden, the newsroom's head researcher, used Guidestar Premium to acquire 990s from the top 50 charities for 2000 and 2003. We built a data entry form in Microsoft Access, and Denson and two copy aides began the painstaking process of building and proofing a database. The information they punched into the database included the 9-digit employee identification number (EIN); the tax return year; the name of the nonprofit; the total amount of government contributions; total revenue; total expenses; net assets; officer compensation; total salaries and wages; other wages; related income; and four-year income. This data allowed us to measure growth in all categories. In a access form, they focused on payments to charity officials, entering total compensation for up to five officers, five employees and five contractors paid $50,000 or more. In all, they entered and verified the names, compensation totals and deferred compensation totals for 1,165 officers, employees and subcontractors. Denson also flagged any insider or related-party transactions noted on the forms for follow-up reporting. In the case of the National Center for the Employment of the Disabled, they were numerous enough to lead us to add a reporter to the project and look hard at the charity's president. We later added 2004 data when it became available. It allowed us to report that average compensation for top executives at the 50 biggest charities grew three times faster than hourly wages for their disabled workers. At least a dozen execs pulled in $350,000 a year or more in pay and benefits, while many workers were paid less than minimum wage. The management firm run by Bob Jones, who ran the El Paso nonprofit, had received $14 million in charity payments over five years and loaned millions more to other businesses in which Jones had an interest. Records we requested under FOIA, plus documents from internal charity sources, showed that the charity had repeatedly been unable to document its workers' disabilities. Many were counted as disabled because they spoke little or no English. Jones resigned the day our story about him appeared under the headline: \"Texas charity a launchpad for entrepreneur's empire.\" He is now under investigation by the FBI, the IRS and the Texas attorney general's office. Our first-day story about weak oversight of the contracting program, \"Charity leaders prosper as 'disabled' is redefined,\" prompted calls on Capitol Hill for an overhaul of the Javits-Wagner-O'Day program. It also sparked an intense and ongoing debate within the community of nonprofits that serve the disabled. The stories and accompanying slide shows can be found online at www.oregonlive.com/special/disability.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-july-august-poor-conditions-plague-medical-helicopters","slug":"2006-july-august-poor-conditions-plague-medical-helicopters","url":"https://dwillis.github.io/uplink/article/2006-july-august-poor-conditions-plague-medical-helicopters.html","year":2006,"month":"July-August","monthNum":7,"headline":"Poor conditions plague medical helicopters","kicker":"AIR SAFETY","author_name":"Alan Levin","author_title":"Aviation safety reporter","affiliation":"USA Today","summary":"A USA Today investigation using a custom database discovered that lax industry safeguards and poor visibility lead to a high rate of fatal air ambulance helicopter crashes.","keywords":["aviation safety","air ambulance","NTSB"],"topics":["Public Safety","Investigative Methods","Data & Databases"],"technologies":["Access","spreadsheet"],"full_text":"There's no substitute for precision on a big project. USA Today set out more than a year ago to learn more about why air ambulance flights were crashing at an alarming rate. A quick review of the clips revealed several fatal crashes that had killed patients. The numbers of crashes were climbing. Our guts told us we had a good story. But you can't attribute major findings to gut instinct. That started several months of searching for hard data that eventually led us to create our own database of crashes from scratch. Along the way, we occasionally cursed the sometimes archaic ways of aviation regulation and recordkeeping, which can make data retrieval very awkward. But, in the end, we had a roadmap laying out the worst crashes and the major safety shortfalls. Our data was more complete than anything the government had compiled. Since the Vietnam War, air ambulances have gradually become commonplace. Every major metro area has a steady stream of helicopters landing on the roofs of major trauma centers, bringing the most severely injured and gravely ill patients to the best facilities. In recent years, air ambulances have helped rural areas cope with major changes in the health care system. As more and more small rural hospitals shut down, helicopters have become increasingly important. For millions of Americans, the only quick way to an emergency room is by air. During the 1980s, the number of air ambulance flights surged. But in the race to provide more service, safety standards suffered. In the years 1985 through 1988, there were 15 fatal air ambulance helicopter crashes, compared to only seven in the previous four-year period. The National Transportation Safety Board (NTSB) attributed the spike in crashes and deaths to carelessness that had crept into overstressed operations and to inadequate federal oversight. The crashes fell rapidly under the glare of attention from investigators and federal aviation regulators. As we began our reporting last November, we suspected that many of the same dynamics had crept into the industry over the past few years, but the NTSB was just beginning its own review of the crashes and had not issued any findings. So we started our own data collection. As a general rule, the NTSB keeps one of the cleanest databases in government. Because the agency has no regulatory power and relies solely on the bully pulpit, it is also unusually generous with its public records. But when it comes to niches such as the air ambulance world, the database can be difficult to work with. In order to be included in the NTSB's database, an accident must meet a test that requires factors such as fatalities, serious injuries or significant damage to the aircraft. All air ambulance accidents that met the NTSB's statutory requirements were in its database, but there is nothing in the data to flag these flights. They are simply listed under a broad category of charter flights. We did our best to collect lists of the accidents from a variety of sources. NTSB provided us with a spreadsheet of accidents after 1995 and we combed reports and other sources. Paul Overberg, a USA Today database editor, helped design the database and created a data entry form in Microsoft Access. We set our able intern, Marie Skelton, loose on the NTSB's database. The web version of the database includes a fairly robust search function that allows broad text searches. She began plugging in words like \"nurse,\" \"ambulance,\" and \"hospital\" to look for any cases we might have missed. To be as thorough as possible, we also entered into our database the accidents involving medical flights on planes, not just helicopters. Working one or two days a week, Marie finished in about two months. I don't recall the exact number, but by using this crude-but-effective manual search, she found a handful of accidents that others had missed. At our conclusion, we had 279 accidents from 1978 to the early months of 2005. It turned out that finding the crashes was the relatively easy part of the job. Next, I had to audit Marie's work (which was extremely clean) and to categorize all of the accidents. The NTSB provides detailed information about the cause of each accident. But that information comes in the form of a fairly lengthy string of text, and over the years, the agency has used different formats. We needed to boil this down to key components, namely the pilot's error, in order to know what patterns were behind the crashes. How important were errors by pilots versus mechanical failures? Were crashes occurring in bad weather? At night? The only way to identify whether these broad types of issues were leading to crashes was to read over every single NTSB cause finding. I then added a bunch of additional fields to our database for every conceivable type of accident, from pilot error and mechanical failure to factors such as the time of day. I went through each record twice to ensure consistency and accuracy. Later on, I met with a statistician at the Federal Aviation Administration to compare our database with one they had compiled to make certain we had not missed any accidents or included cases that didn't belong. I also went through the data and made sure we had accurate counts of the dead. In particular, we needed to know how many patients had died on flights. It wasn't pretty, but I hand-entered crash totals and various pieces of information into a Microsoft Excel spreadsheet. That allowed us to help visualize some of the trends. For example, it became obvious very quickly that poor visibility from bad weather and flying at night was frequently a contributing factor to crashes. We were far from the first news organization to notice that air ambulance flights were crashing at alarming rates. But the painstaking collection of data (and my editors' patience while I completed the work) allowed us to write about the problem with authority. Even though the NTSB had not yet issued its own findings about the growing number of air ambulance accidents, we were able to reach strong findings on our own. Here are two of our main conclusions: \"Industry safeguards are so lax that pilots have repeatedly caused accidents by knowingly flying into bad weather, failing to check weather conditions or otherwise violating federal or company regulations.\" \"About two-thirds of fatal air ambulance crashes occur in poor visibility, the newspaper's analysis shows. Even so, pilots are not required to have special training about what to do when they encounter fog, snow or darkness.\" The information on these crashes was available for anyone to see, but no one had bothered to compile it in one place. The raw number of crashes was the highest in history, but the industry had grown considerably in the past decade. Of course, the data was only one of the elements we relied on to report the story. But without our computer analysis, the story would not have been half as good.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-july-august-bits-and-bytes","slug":"2006-july-august-bits-and-bytes","url":"https://dwillis.github.io/uplink/article/2006-july-august-bits-and-bytes.html","year":2006,"month":"July-August","monthNum":7,"headline":"Bits & Bytes","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Over 800 journalism professionals attended IRE's 30th annual conference in Fort Worth featuring 130+ panels on reporting and data analysis, including specialized computer-assisted reporting workshops. The IRE and NICAR Database Library also updated its Federal Contracts database to provide more timely access to information about contracts between federal agencies and contractors.","keywords":["conference","journalism","data"],"topics":["Journalism & Newsroom","Data & Databases","Investigative Methods"],"technologies":["Excel","Access","CrimeStat"],"full_text":"More than 800 journalists, educators and students attended the 30th Annual IRE Conference, June 15-18 in Fort Worth, Texas.\n\nThe conference offered more than 130 panels and workshops on covering beats, writing, editing, producing and hands-on classes in data analysis.\n\nAbout 300 people showed up for Thursday's optional CAR day, which included panels on issues as far-ranging as introducing newsroom intranets and diagnosing medical and care data.\n\nThe conference also featured two rooms exclusively geared towards CAR: a hands-on room teaching the basics of Microsoft Excel spreadsheets and Access database manager, and a demo room where journalists showed off some of the newest and most advanced technology with presentations on CrimeStat, satellite imagery, Web scraping, social network analysis and more.\n\nFederal Contracts Update\n\nThe IRE and NICAR Database Library recently updated its Federal Contracts database, making the latest additions available to journalists faster.\n\nThe database contains data about contracts between about 70 federal agencies and thousands of contractors. Those contractors include large corporations, local businesses, universities, nonprofits, and local, state and foreign governments. The database lists – among other things – the agency and office, the amount of money, date, contractor, purpose of the contract and where the work is performed.\n\nJournalists can use the data to examine the local economic impact of a federal project. Combine it with campaign finance information to look into the relationships between the government awarding the contract and the contractor who earns the business.\n\nPreviously, the data was updated annually, but for the first time, the Database Library is making new data available as soon as the agency's contractor releases updates.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-july-august-tune-in-to-car-tips","slug":"2006-july-august-tune-in-to-car-tips","url":"https://dwillis.github.io/uplink/article/2006-july-august-tune-in-to-car-tips.html","year":2006,"month":"July-August","monthNum":7,"headline":"Tune in to CAR tips","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"NICAR instructor","affiliation":"NICAR and Missouri School of Journalism","summary":"IRE and NICAR are now offering digital audio recordings of sessions from the 2006 IRE Conference, providing members with accessible training on computer-assisted reporting.","keywords":["training","audio recordings","IRE conference"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["MPEG-3"],"full_text":"Check out this incredible resource for helping you learn more about computer-assisted reporting from the top practitioners in the business. For the first time, IRE and NICAR offer direct access to digital audio recordings from panels and speeches at the 2006 IRE Conference in Fort Worth, Texas. In the past, journalists who wanted recordings had to order tapes or CDs from a vendor. IRE members who attended the 2006 Conference can simply visit the conference Web page at www.ire.org/training/dallasfortworth06 and follow the links to find .mp3 (MPEG-3) files. Listen to them online or download for playback on a computer or portable digital audio player. Conference attendees also may order a CD containing audio from all of the panels, paying only for shipping. IRE members who did not attend the conference can purchase the CD for $20 plus shipping. Order online from the conference Web page or contact the Resource Center at 573-882-3364. So, if you want to learn more about mapping data, social network analysis, using CAR to report on campaign finance – you name it – you can revisit Fort Worth and get an earful of what you missed.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-july-august-mapping-finds-holes-in-siren-coverage","slug":"2006-july-august-mapping-finds-holes-in-siren-coverage","url":"https://dwillis.github.io/uplink/article/2006-july-august-mapping-finds-holes-in-siren-coverage.html","year":2006,"month":"July-August","monthNum":7,"headline":"Mapping finds holes in siren coverage","kicker":"FIRST VENTURE","author_name":"Gerry Lanosga","author_title":"Investigative producer","affiliation":"WTHR-Indianapolis","summary":"A WTHR-Indianapolis team used GIS mapping software to identify significant coverage gaps in central Indiana's tornado siren network, helping residents find dead zones near their homes.","keywords":["GIS mapping","tornado sirens","ArcView"],"topics":["Public Safety","Technology","Investigative Methods"],"technologies":["ArcView","ArcMap","Access","GPS","Google Maps"],"full_text":"Taking my first crack at ArcView 9.1 reminded me of the first time my dad let me use a chainsaw – a little bit scary (probably more for him than me) – but a hell of a lot of fun. I had tinkered around with ESRI's geographic information system (GIS) program for a couple of small maps after attending IRE and NICAR's Mapping Boot Camp last summer, but in March we decided to get serious with mapping by taking a look at tornado sirens in central Indiana. It's certainly ground that has been tread before, but not in any in-depth way in our viewing area. With tornado season approaching, we figured it would be a high-interest story for our audience that would also give us a good initiation with ArcMap. From start to finish, the project took just about every bit of three months for reporter Bob Segall, photojournalist Bill Ditton and me. First we made calls to get siren locations in Marion and the eight surrounding counties in the Indianapolis metropolitan area. Most of the counties either lacked GIS files on the sirens or had files that didn't fit our needs. So we decided to begin from the beginning, building our own database of sirens from which to create maps. Getting the lists of locations was an adventure in itself. Some counties were extremely well-organized; others not so much. In Hancock County, just east of Indianapolis, it took several phone calls and interviews just to get to the person who reportedly knew where all the sirens were. When we found her at the sheriff's department, she jotted out a list of locations from memory. And, when we went to verify the locations, we found some of them didn't exist. The problems in Hancock, characteristic of the smaller counties in our area, would end up being central to one story in our series. With the locations in hand, I built a database in Microsoft Access and then pulled the data into ArcView for geocoding to create a point map of the sirens. I was able to match only about half of the 350 sirens. The rest we plotted the hard way. Bill, Bob and I fanned out with handheld GPS units and gathered coordinates (in UTM, or Universal Transverse Mercator, projection) at each of the remaining siren locations. This proved to be more painful than geocoding. In fact, it was the most time-consuming aspect of the project. We logged hundreds of miles searching for sirens, and the locations we were given weren't always as accurate as we'd hoped. Once we had the data, I made the first of many novice mistakes on this project by hand-keying the data captured in the GPS units into the database. In any event, after the database was complete, it was a snap to add the data to the map, as X,Y points. After adding geographic features such as townships, populated places, and census tract data, I created buffers around each siren. We had our maps – the first comprehensive picture of tornado siren coverage in central Indiana. I should note that we didn't get to this point without some epic battles with ArcMap. I struggled with everything from projections to buffers. At one point, the program simply crashed repeatedly, sometimes without warning and other times with cryptic error messages. After scouring technical papers, talking to technical support and re-installing the software, the problem persisted on a couple of county files. Our initial stories looked at the overall picture of tornado sirens in the area, from how well they work to how much area they cover. Using census block data, we were able to report roughly how many people live in dead zones where there are no tornado sirens. On the WTHR Web site, we posted our own maps of the siren coverage for each county and hired a consultant who created an interactive Google Maps application that allowed users to look up sirens near their homes. I learned a few lessons and would do things differently if starting over. Initially, we had decided to use a generous 1.5-mile siren coverage radius, even though only the best sirens under ideal conditions operate at that range. In retrospect, I wished I had handled those buffers differently, creating multiple rings rather than one radius. In fact, for a second take on our stories, we did that, showing graduated half-mile distances from the sirens and explaining the variance in siren range in a legend. For this second go-around, we were able to report how many schools and parks are outside the practical range of the sirens. Another thing I'd change relates to the display. If I had to do it again, I would create one master data table that includes a county field. While ArcMap is exhilarating to use in terms of its analytical power, I also discovered that it's just plain fun to create these visuals that can add so much to our storytelling. (And I didn't even saw off my foot.)","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-july-august-making-sense-of-osha-data","slug":"2006-july-august-making-sense-of-osha-data","url":"https://dwillis.github.io/uplink/article/2006-july-august-making-sense-of-osha-data.html","year":2006,"month":"July-August","monthNum":7,"headline":"Making sense of OSHA data","kicker":"SPOTLIGHT: ON THE JOB","author_name":"Mike Casey","author_title":"Projects reporter","affiliation":"The Kansas City Star","summary":"A reporter from The Kansas City Star describes how to use federal OSHA inspection data to investigate workplace safety, identify high-violation industries, and track workplace fatalities.","keywords":["OSHA","workplace safety","database analysis"],"topics":["Public Safety","Data & Databases","Investigative Methods"],"technologies":["Access"],"full_text":"How much is a worker's life worth? What are the most deadly workplaces in your town? Are there factories in your town that are exposing workers to toxic chemicals? The Occupational Safety and Health Administration database has the answers. OSHA is the federal agency responsible for worker safety, and its database details millions of inspections of factories, steel mills and construction sites since 1972. Don't let the size of the database or its complexity intimidate you. For a recent project about worker safety around Kansas City, The Star purchased the data from the IRE and NICAR Database Library. To tailor the analysis for our circulation area, I extracted data for a dozen area counties. I used Microsoft Access database manager for my analysis of nearly 30,000 records. One way to approach the database is to think of it as a news story. The main table is like the lead of a story that gives you a little about the most important aspects of each inspection. The accompanying tables are like the rest of story; they provide more detail and context. The main table has more than 50 fields that give the name of the company, its address, the reason for the inspection, the number of OSHA violations, penalties the company paid and a number of other items. Other tables describe the inspection in more detail. If the inspection concerns a fatality, the database tells you the name of the accident victims as well as the accident's causes. Another table explains the violations in more detail and references the specific regulations that the company violated. Yet another table provides details about hazardous chemicals found in the workplace. What connects all of these tables is the unique inspection number, which OSHA calls the activity number. It's just a matter of using your database manager to connect the tables using that field. I was interested in looking at fatalities. In OSHA's codes, a fatality is classified as an 'A' in the inspection-type field. I selected all 'A' inspection types from the main table and joined that to the others using the inspection number. Here are some of the things I learned: Low fines for workplace deaths or injuries are common even when OSHA cites employers for a serious violation. In 80 such fatal and injury accidents, half of the fines Kansas City area employers paid were $3,000 or less. A table detailing the citations showed that the agency sometimes downgraded its most serious willful violations to unclassified and then substantially cut its penalties in these fatal accidents. To find the change, look in the section that details the types of violations. 'W' stands for willful and 'U' stands for unclassified. The table that details fatalities gave the names of workers and I noticed that more workers with Hispanic names were dying on the job. After checking death records, I learned that indeed more Hispanics – particularly immigrants – were killed at local worksites. The data also showed that falls and electrocutions often caused workplace fatalities in the Kansas City area. Like all databases, OSHA's has strengths and weaknesses. I've found the database information nearly always matched the information in the written reports. However, some of the OSHA fields contain dirty data. For example, Ford Motor Co. can be called Ford Motor, Ford or Ford Motor Co – no period. That creates a challenge in finding the companies with the highest number of violations. Start by selecting and grouping the company names. Once you get the top five or 10, check their addresses to see if the name of the facility is connected to the same address. Using the address, city and ZIP code fields can help you come up totals for each worksite. For construction inspections, the worksite is listed as the location, so a construction company may have a number of different sites listed. That makes coming up with a top violator among construction companies very difficult, if not impossible. Still, if the data shows a company with a high number of violations you can probably say the company was among the top violators. Another area to look at is the industries with the most violations. You can do that by using the Standard Industrial Classification (SIC) codes in the inspections table. Picking the industries with the most violations and then looking for companies in those industries with a number of violations will give your story an immediate focus. The OSHA database goes back to the days of Richard Nixon's presidency. I focused on a 10-year span from 1994 to 2004. These people are eager to tell their stories, which help make for a dramatic project.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-july-august-extra-extra-your-guide-to-the-latest-investigative-work","slug":"2006-july-august-extra-extra-your-guide-to-the-latest-investigative-work","url":"https://dwillis.github.io/uplink/article/2006-july-august-extra-extra-your-guide-to-the-latest-investigative-work.html","year":2006,"month":"July-August","monthNum":7,"headline":"EXTRA! EXTRA! YOUR GUIDE TO THE LATEST INVESTIGATIVE WORK","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"Journalists use computer-assisted reporting and government databases to investigate workplace issues including employee compensation, occupational safety violations, and retirement benefits. Resources like OSHA inspection records and wage enforcement databases enable reporters to identify dangerous working conditions and uncover trends affecting workers.","keywords":["workplace","databases","safety"],"topics":["Investigative Methods","Business & Economy","Health & Environment"],"technologies":[],"full_text":"Check out the stories below for examples of journalists using computer-assisted reporting to report about the workplace. These stories and others can be found at Extra! Extra! on the IRE Web site (www.ire.org/extraextra).\n\nKeith Rogers of the Las Vegas Review-Journal used Department of Labor statistics to review compensation applications made by sick employees who once worked at six radioactive and toxic materials sites. Rogers found that the Nevada Test Site had a compensation approval rate of 6 percent, the lowest of the sites. Yet only at the Nevada site did employees work in areas that had been detonated by nuclear devices. Nearly 900 cases were denied because former employees couldn't prove beyond a 49 percent likelihood that radioactive materials contributed to their illnesses.\n\nRandy Ludlow of The Columbus Dispatch reports on respiratory illnesses contracted by more than a quarter of the employees working in a TRW brake plant in Mt. Vernon, Ohio. He also reviews the lack of Occupational Safety and Health Administration standards to regulate practices that caused these respiratory problems. As he describes in the July-August issue of The IRE Journal, Ludlow found illness trends by using a database of TRW employee complaints from the Ohio Bureau of Workers' Compensation. He incorporated that data with OSHA's inspection data and anecdotes gathered from dozens of interviews.\n\nIn Time magazine, Donald L. Bartlett and James B. Steele review how employees lost their retirement benefits when 401(k) plans tanked and pension plans were pulled out from under them. Company-sponsored pension plans are under-funded by $450 billion in the private sector and are quick to vanish when companies hit financial troubles. While 2004 reports claimed the average 401(k) is worth $61,000, the authors say that the median value was only $18,000.\n\nResources for workplace reporting\n\nIRE and NICAR offer resources for journalists who are interested in using computer-assisted reporting to cover workplace issues.\n\nThe Occupational Safety and Health Administration inspections database, available from the IRE and NICAR database library, contains more than 3 million records of OSHA inspections in all states and U.S. territories from 1972 to February 2006. The OSHA database contains a variety of information on workplace conditions and accidents, including hazardous substance accidents. Journalists can use the database to identify companies that have dangerous working conditions for employees and to examine the safety record of companies involved in breaking news.\n\nThe Database Library also offers the Wage and Hour Enforcement database kept by the U.S. Department of Labor. It shows violations and penalties related to labor laws, including the Fair Labor Standards Act, Family and Medical Leave Act, and the Migrant and Seasonal Agricultural Worker Protection Act. It contains records regarding more than 700,000 cases that have been closed between the 1980s and 2004. Journalists can investigate the fairness of compensation paid by employers, identify industries with frequent offenders, and check adherence to child labor laws.\n\nIn addition, IRE resources can provide guidance in using the databases to for stories. Chapter 12 of The Investigative Reporter's Handbook has a section specifically about safety and health in workplace.\n\nIn the March-April 2002 issue of Uplink, Ron Nixon explains pitfalls of OSHA data and offers some measures to overcome them. In the same issue, Stephanie Armour lists workplace data resources, along with some specific story ideas.\n\nTipsheets contributed by journalists are also good resources to refer to in covering workplace issues.\n\nBarbara Clements and Daniel Wickert of The (Tacoma, Wash.) News Tribune offer advice about analyzing OSHA data and suggestions for making the data more manageable. They also provide a list on web resources regarding an OSHA project. (Tipsheet No. 2458)\n\nJustin Pritchard of The Associated Press explains where to get information about death in the workplace and how to handle that information. (Tipsheet No. 2062)\n\nJournalists have written a number of stories about the workplace and available from the IRE Resource Center:\n\nDavid Barstow of The New York Times revealed that OSHA declined to seek prosecution in 93 percent of the cases in which it determined an employer's \"willful safety violations\" caused a death. The report also finds that \"OSHA's reluctance to seek prosecution persisted\" even when the violations caused multiple deaths, or when the victims were teenagers. (Story No. 20850)\n\nAn investigation by The (Fredericksburg, Va.) Free Lance-Star disclosed that state government agencies are among the most common violators of worker safety standards, but they never get fined. The story identifies the area's top violators in the past 25 years, including the City of Fredericksburg and the County of Stafford. (Story No. 18656)\n\nContact the Database Library at 573-884-7711 to order workplace data. More information about the databases is available online at www.ire.org/datalibrary/databases.\n\nContact the Resource Center at rescntr@ire.org or 573-882-3364 to order copies of the stories or tipsheets.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-july-august-wrestling-useful-data-from-pdfs","slug":"2006-july-august-wrestling-useful-data-from-pdfs","url":"https://dwillis.github.io/uplink/article/2006-july-august-wrestling-useful-data-from-pdfs.html","year":2006,"month":"July-August","monthNum":7,"headline":"Wrestling useful data from PDFs","kicker":"CAR TOOL","author_name":"Neil Reisner","author_title":"Associate professor of journalism","affiliation":"Florida International University","summary":"This article reviews software tools like XPDF, Foxit, and PDF2XL that allow journalists to extract structured data from PDF files for analysis in spreadsheets and databases.","keywords":["PDF conversion","data extraction","software reviews"],"topics":["Technology","Investigative Methods","Data & Databases"],"technologies":["PDF","XPDF","PDFTOTEXT","Foxit Software","PDF Text Viewer","Foxit PDF to Text Converter","Cogniview PDF2XL","Excel","Investintech Able2Extract Pro","OCR","Word","HTML"],"full_text":"Let's start with the obvious: PDFs are devils masquerading in angelic garb. They emerged in the early '90s as a purportedly benevolent effort by developer Adobe Systems Inc. to create 'a universal way to communicate documents across a wide variety of machine configurations, operating systems, and communication networks.' Translation: Let's create software that, with a mouse click, can change any file – including data tables – into an image that cannot be imported into a spreadsheet or database manager. But PDFs have been driving geeks crazy ever since. Fortunately, there are a number of free or inexpensive programs that extract data from PDFs and turn it into text that imports easily into CAR software. The format resolved an important dilemma because they can be created from the output of any software and can be viewed on any computer. Moreover, because PDFs are hard to edit, there was a reasonable assurance that a PDF version was the same as the original. Today all it takes is any of the widely available PDF print drivers distributed for free. Which means it's even easier for records custodians to make our lives harder. So, how do we deal with them? The key is to know the public records law for your state. In Florida, for example, the law requires agencies to provide copies in the medium requested if they maintain it that way. If a PDF is all you can get, it's time to turn to several free or low-cost programs. For the last few years, XPDF/PDFTOTEXT has been the way to go. It's command-line driven and available at no cost. Foxit Software offers PDF Text Viewer ($39) with a simple Windows interface and Foxit PDF to Text Converter ($59) which includes batch conversion tools. Cogniview PDF2XL ($95) converts PDF tables directly into Excel and handles scanned PDFs with some success. Investintech Able2Extract Pro ($119.95) also extracts text and imports into Excel, Word, or HTML using OCR. These programs add more bullets to our gun belts and maybe a wooden stake and some wolf's bane potion as well.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-july-august-finding-diamonds-in-the-rough","slug":"2006-july-august-finding-diamonds-in-the-rough","url":"https://dwillis.github.io/uplink/article/2006-july-august-finding-diamonds-in-the-rough.html","year":2006,"month":"July-August","monthNum":7,"headline":"Finding diamonds in the rough","kicker":"SCHOOL DATA","author_name":"MaryJo Sylwester","author_title":"Database editor","affiliation":"St. Paul Pioneer Press","summary":"The St. Paul Pioneer Press used multiple regression analysis to identify schools in low-income areas that were outperforming their predicted standardized test scores.","keywords":["education data","regression analysis","test scores"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["SPSS"],"full_text":"The results from Minnesota's standardized test – used to measure progress for No Child Left Behind – yields a predictable list when sorted best to worst. At the top are schools from the wealthy neighborhoods and suburbs. At the bottom are schools from poor neighborhoods. This didn't seem like a fair way to assess how schools are performing. In June, the Pioneer Press published a series called 'Schools that Work,' stemming from a statistical analysis of test scores that revealed which elementary schools were doing better than expected, considering demographic factors. We found 13 schools serving predominantly low-income students that had consistently done better than expected. Then we visited those schools and came up with five things they were doing that experts said were keys to success. The bulk of the work was traditional street reporting, but data analysis pointed us in the right direction. Test score data is available in Minnesota from the state department of education. Numerous studies have shown that family income is the most significant predictor of test performance. The statistical procedure we used – regression – estimates the impact of income and uses that to 'predict' what each school's average should be. We used multiple regression with free/reduced-price lunch, English proficiency, and special education as independent variables. We used SPSS statistical software to run separate regressions on each set of scores and checked the 'residual,' which shows the difference between the predicted score and the actual score. We averaged the residuals to ensure schools labeled 'above predicted' were consistently better than expected. We decided to focus on schools with 60 percent or more low-income students to create a roadmap for success. We tapped other data sources like student demographics and teacher experience. Instead of publishing residuals, we identified schools as above, within, or below predicted to avoid confusion. I convinced my editors to put a good story in the paper; the analysis was just the path to get there.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-september-october-monitoring-airport-activity","slug":"2006-september-october-monitoring-airport-activity","url":"https://dwillis.github.io/uplink/article/2006-september-october-monitoring-airport-activity.html","year":2006,"month":"September-October","monthNum":9,"headline":"Monitoring airport activity","kicker":"TRANSPORTATION","author_name":"Trebor Banstetter","author_title":"airline and travel reporter","affiliation":"Fort Worth Star-Telegram","summary":"The article discusses how the Star-Telegram used U.S. Department of Transportation databases to analyze the impact of low-fare carrier AirTran Airways on Dallas-Fort Worth Airport competition.","keywords":["transportation","airlines","databases"],"topics":["Business & Economy","Investigative Methods","Technology"],"technologies":[],"full_text":"Dallas-Fort Worth Airport is one of the largest airports in the world, with more than 2,000 daily flights to more than 170 cities worldwide. For travelers who live in North Texas, however, the mammoth airport has long been a mixed blessing. It's home to a giant American Airlines hub, which means nonstop flights and high frequency service to countless cities. But American aggressively protects its turf and has a history of running off competitors. That's part of the reason why fares at D/FW tend to be higher than the national average. So when low-fare carrier AirTran Airways launched a bid to operate a mini-hub at D/FW, travelers in Fort Worth and Dallas were watching closely. At the Star-Telegram, where I cover airlines and travel for the business desk, we explored U.S. Department of Transportation databases routinely used by transportation analysts and researchers to gauge the impact AirTran was having on fares, traffic demand and competition with American.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-september-october-databases-show-vets-get-uneven-treatment","slug":"2006-september-october-databases-show-vets-get-uneven-treatment","url":"https://dwillis.github.io/uplink/article/2006-september-october-databases-show-vets-get-uneven-treatment.html","year":2006,"month":"September-October","monthNum":9,"headline":"Databases show vets get uneven treatment","kicker":"SPOTLIGHT: MILITARY MANEUVERS","author_name":"Chris Adams","author_title":"","affiliation":"McClatchy Washington Bureau","summary":"This investigation explores how U.S. military veterans receive inconsistent benefits and treatment depending on their geographic location despite uniform federal regulations.","keywords":["veterans","military","disability"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"All things being equal, a military veteran on the West Coast should get the same treatment as his colleague from the East, right?\n\nYou would think so. After all, the U.S. Department of Veterans Affairs has but one set of regulations, which applies to veterans whether they live in Sacramento, Calif., or Syracuse, N.Y..\n\nBut in fact, a veteran from one state is unlikely to get the same service, treatment or monthly disability check as his","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-september-october-tired-truckers-endanger-tenn-drivers","slug":"2006-september-october-tired-truckers-endanger-tenn-drivers","url":"https://dwillis.github.io/uplink/article/2006-september-october-tired-truckers-endanger-tenn-drivers.html","year":2006,"month":"September-October","monthNum":9,"headline":"Tired truckers endanger Tenn. drivers","kicker":"ROAD SAFETY","author_name":"Nancy Amons","author_title":"I-Team Investigator","affiliation":"WSMV-Nashville","summary":"An investigation by WSMV-Nashville analyzed five years of Tennessee truck accident reports to determine the prevalence of driver fatigue in fatal and injury crashes.","keywords":["trucking","accidents","road safety"],"topics":["Investigative Methods","Public Safety","Data & Databases"],"technologies":["Access","CD-ROM"],"full_text":"Sammy Pruitt stopped at 6:30 every morning for bacon and biscuits at C & E market on a rural highway south of Nashville. Last summer he was pumping gas into his pick-up when a tractor-trailer came sliding toward the gas station on its side. Pruitt died instantly. The truck driver, Robert Peyton, told emergency workers he had fallen asleep behind the wheel. Peyton had been driving all night to deliver a load of air conditioners. The trucker was never prosecuted or ticketed and still has a valid Kentucky commercial driver's license – which shows no sign of the fatal accident. He could be hired to haul hazardous material tomorrow. How big a problem are sleepy truckers in Tennessee? WSMV's I-Team analyzed truck accident reports statewide over the last five years and found 64 crashes where a trucker who was listed as \"apparently asleep\" injured or killed someone. In 70 percent of those cases, the trucker never got a ticket. We found another 22 cases where \"driver fatigue\" was listed as a factor in a crash where someone died or was injured. We used half a dozen databases to find out how often Tennessee truckers fall asleep at the wheel and kill or injure someone, and what happens when they do. Tennessee, like all states, uses a standardized form to report accidents. It then forwards these reports – called Uniform Traffic Crash Report in Tennessee – to the federal government. If you remember taking those standardized tests when you were in school – the ones where you use the number-two pencil to fill in the bubble – that's what these crash reports look like. I had reviewed accident reports in the past and knew there was a collection of fields that describe \"driver factors.\" Police officers can check up to three boxes that describe the \"driver factor conditions.\" One of the driver conditions is \"apparently asleep,\" another is \"fatigue.\" (If you do a similar story, request a blank crash report so you will know all the fields.) I asked the Tennessee Department of Safety to extract reports that involved crashes that involved a large truck and injuries or fatalities. We looked at five years' worth of data. If your state law does not require officials to write programs to extract specific data, you might have to work with a larger set of data. The Tennessee Department of Safety produced the data within a couple of weeks of my request. It came on CD, already in a database format. It contained two tables: the main table, called \"crash parent\" and a second, named \"driver vehicle.\" The latter table contained details about the truck driver. Each table held 8,700 records and 25 or so fields. I used Microsoft Access database manager to join them and run queries. The analysis was straightforward: we isolated cases where the driver condition field met the criteria we were interested in – \"apparently asleep.\" We had to be careful because \"apparently asleep\" could appear in any of the three fields that describe the driver condition. We were able to very quickly calculate how many sleepy drivers were involved in fatal or injury accidents.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-september-october-2007-car-conference","slug":"2006-september-october-2007-car-conference","url":"https://dwillis.github.io/uplink/article/2006-september-october-2007-car-conference.html","year":2006,"month":"September-October","monthNum":9,"headline":"2007 CAR conference","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The 2007 CAR Conference will be held in Cleveland on March 8-11, offering best-practices panels and hands-on classes on computer-assisted reporting techniques for journalists covering topics like census, crime, and government. The event is hosted by The Plain Dealer at the Renaissance Cleveland Hotel with hotel discounts available through February 9.","keywords":["conference","journalism","technology"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["spreadsheet"],"full_text":"The 2007 CAR Conference will be held in Cleveland on March 8-11, featuring panels by the best in the business as well as hands-on classes. The Plain Dealer hosts the conference at the Renaissance Cleveland Hotel. The conference is sponsored by the Gannett Foundation.\n\nThe best-practices panels will discuss using CAR to cover the census, crime, education, local and state government, Freedom of Information issues, and transportation, among others. In the hands-on classes, participants can learn to use spreadsheets, database managers, mapping, statistics and how to apply new technology and programming techniques in your work. IRE and NICAR welcome suggestions for panel topics, speakers or class offerings, too. Submit ideas to confideas@ire.org and reference the 2007 CAR conference.\n\nThe online registration form is available at www.ire.org/cgi-bin/secure-form.cgi?form=clevelandReg. Registrants who book their hotel rooms by Friday, February 9, can get the discounted rate of $119 plus tax per night.\n\nIf you have hotel or general conference questions, please contact Ev Ruch-Graham, senior conference coordinator at ev@ire.org or 573-882-8969. If you have registration questions, please contact John Green, membership coordinator, at jgreen@ire.org or 573-882-2772.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-september-october-in-the-news","slug":"2006-september-october-in-the-news","url":"https://dwillis.github.io/uplink/article/2006-september-october-in-the-news.html","year":2006,"month":"September-October","monthNum":9,"headline":"In the News","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR provide \"In the News\" resources to help journalists cover breaking news events by offering databases, articles, and tipsheets on relevant topics like aviation safety, terrorism, and weather disasters.","keywords":["journalism","databases","breaking"],"topics":["Journalism & Newsroom","Data & Databases","Investigative Methods"],"technologies":[],"full_text":"IRE and NICAR offer \"In the News\" updates to help journalists pursue in-depth stories while covering breaking news. These include databases, news articles and tipsheets related to the latest events.\n\nWhen British authorities disrupted a terrorism plot to blow up transatlantic aircraft in August, \"In the News\" offered resources to journalists for covering aviation safety and terrorism.\n\nAnother \"In the News\" package featured storm events and federal assistance data. The storm events database has helped in the reporting of enterprise stories, such as determining the most costly weather events to hit a certain area.\n\nSee these \"In the News\" reports and more by visiting www.ire.org/inthenews_archive.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-september-october-time-for-meyer-awards","slug":"2006-september-october-time-for-meyer-awards","url":"https://dwillis.github.io/uplink/article/2006-september-october-time-for-meyer-awards.html","year":2006,"month":"September-October","monthNum":9,"headline":"Time for Meyer awards","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"David Herzog announces the call for entries for the 2006 Philip Meyer Journalism Award, which honors journalism using social science and CAR techniques.","keywords":["Philip Meyer Award","precision journalism","awards"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"There's no doubt that journalists using social science techniques – including computer-assisted reporting – have produced some great stories. Each issue of Uplink is a testament to that, as journalists recount how these skills helped them uncover or advance their stories. Last year NICAR, a joint program of IRE and the Missouri School of Journalism; the Knight Chair in Journalism at Arizona State University; and IRE introduced the Philip Meyer Journalism Award to honor the best examples. The inaugural first-place prize went to The Oregonian for \"Unnecessary Epidemic,\" a series that used statistical analysis to show how the federal government missed opportunities to stop the growth of methamphetamine abuse. \"Discharged and Dishonored,\" which you can read about in this issue of Uplink, won second place for reporters in the former Knight Ridder Washington Bureau. The awards are given annually to recognize the legacy of Philip Meyer, Knight Chair in Journalism at the University of North Carolina at Chapel Hill and father of \"precision journalism.\" If you have a story or stories that you feel are worth consideration for the 2006 awards, see www.ire.org/meyer-award for more information. Works in print, television, books or online are eligible. Don't wait, because the postmark deadline for entry in the 2006 awards is Oct. 31. Contact David Herzog by email at dherzog@nicar.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-september-october-staying-afloat-with-boating-data","slug":"2006-september-october-staying-afloat-with-boating-data","url":"https://dwillis.github.io/uplink/article/2006-september-october-staying-afloat-with-boating-data.html","year":2006,"month":"September-October","monthNum":9,"headline":"Staying afloat with boating data","kicker":"FIRST VENTURE","author_name":"John Ferro","author_title":"City Editor","affiliation":"Poughkeepsie Journal","summary":"The projects editor at the Poughkeepsie Journal recounts his first venture into computer-assisted reporting using U.S. Coast Guard boat accident data to find local trends.","keywords":["boating accidents","CAR Boot Camp","data analysis"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Access","Excel","Nexis","FTP","IRE and NICAR Database Library"],"full_text":"A couple of years ago, when I was city editor, we sent our political reporter to the IRE and NICAR Computer-Assisted Reporting Boot Camp at the Missouri School of Journalism. When he returned, I asked him how it went. \"I can't wait to use this stuff,\" he said. I recalled that comment when I jetted off to Columbia in late March. I was sure I would be coming back flushed with a \"CAR high.\" Instead, I returned CAR-petrified. I had skills, but what to do with them? And the warning from IRE Executive Director Brant Houston's was ringing in my head – \"Use it, or lose it.\" So I started casting about. We purchased the Environmental Protection Agency's Comprehensive Environmental Response, Compensation and Liability Information System (CERCLIS) from the IRE and NICAR Database Library. Too many tables. Too many joins. No clear trends. I was in over my head. I sent a Freedom of Information request to the state education department for special-education data, but it took several months until the department and I worked out our differences over the request. The clock was ticking. My bosses were tapping their feet, waiting for results. I could feel my little bitty CAR muscles atrophying each day. I went back to the Database Library's Web site at www.ire.org/datalibrary. And there it was, after a few clicks: The Recreational Boat Accident database allows reporters to find information about the vessels, people and conditions involved in the accident. Because of the simplicity of this database, it is a good one for beginning CAR reporters to use. Perfect. A database on training wheels. I purchased the U.S. Coast Guard files, downloaded them from NICAR's FTP server and imported them into Microsoft Access database manager. Before I started running queries, I searched Nexis for stories that reported not only individual accidents, but also trends and recent legislation. I limited my searches to New York. (I kept hearing that line from the Jodie Foster film \"Contact\" – \"Small steps, Ellie. Small, steps.\") In 2004, New York had completed its phase-in of a law that requires operators of personal watercraft to take an eight-hour safety course and pass a test before they revved up their JetSkis. My first queries examined accident trends involving personal watercraft. The database has just four tables, and they are easily joined. I used Access to do counts and filter. Then I pasted the results into Microsoft Excel spreadsheets to create pivot tables and do percent-change calculations. I am the kind of guy who spends a little too much time making sure his stack of Poughkeepsie Journal copies is piled neatly on his desk and reference books are organized by size. So, as the number of my Access queries increased, my efforts to name them became more, shall we say, compulsively obsessive. I found it useful to name each query starting with its type: Filter, Count, MakeTable or Sum. So my list of queries looked something like this: CountAllPWCByState, CountBoatTypesForHudsonRiver, CountEducationForPWCForNY, FilterForDutchessUlsterInjuries, FilterForHudsonRiverDeaths, MakeTableForDutchessUlsterAccidents. This saved time when I went back to my query list. All the counts were in one place, all the filters in another. (I got a little obsessive-compulsive rush every time I opened up the query window.) My first analysis counted the number of boating accidents for each boat type – pontoon, personal watercraft, powerboat, etc. I did totals by year for New York and for the United States. I took each of those counts and pasted them into Excel. I used a pivot table to line up the totals for each boat type and each year. This allowed me to calculate the percentage of accidents for each boat type for any year. I noticed the percentage of accidents involving personal watercraft in New York had dropped significantly during the 1995-2004 period I was examining. The rate of decline was faster than any of the more commonly used vessels.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-september-october-federal-contracts","slug":"2006-september-october-federal-contracts","url":"https://dwillis.github.io/uplink/article/2006-september-october-federal-contracts.html","year":2006,"month":"September-October","monthNum":9,"headline":"Federal contracts","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library has completed its federal contracts database for fiscal year 2005 with over 1 million records, providing journalists access to detailed information about government contracting to investigate business-government connections and federal spending in their communities.","keywords":["contracts","database","journalism"],"topics":["Data & Databases","Government & Politics","Business & Economy"],"technologies":[],"full_text":"The IRE and NICAR Database Library's federal contracts database now includes the complete set of entries for fiscal year 2005. All together, the data includes some 1 million records for the fiscal year 2005, and the Database Library continues to update fiscal year 2006 contracts monthly.\n\nThe Database Library's federal contracts collection contains more than 2.7 million records. The database includes the contracting agency, name and location of the business, the worksite, the amount of the transaction, type of work, and much more.\n\nJournalists can use the data to report on connections between businesses and government officials, cover the impact of federal dollars in their communities, and monitor federal agency contract expenditures. For more information about the federal contracts database, visit www.ire.org/datalibrary/databases or call the Database Library at 573-884-7711.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-september-october-tracking-problems-with-us-sky-spies","slug":"2006-september-october-tracking-problems-with-us-sky-spies","url":"https://dwillis.github.io/uplink/article/2006-september-october-tracking-problems-with-us-sky-spies.html","year":2006,"month":"September-October","monthNum":9,"headline":"Tracking problems with U.S. sky spies","kicker":"SPOTLIGHT: MILITARY MANEUVERS","author_name":"Mike Fabey","author_title":"Reporter","affiliation":"Aviation Week/Aerospace Daily","summary":"Reporter Mike Fabey used federal contract databases and technical research to investigate budget overruns and mismanagement in the Pentagon's classified satellite programs.","keywords":["satellites","Pentagon","federal contracts"],"topics":["Investigative Methods","Data & Databases","Public Safety"],"technologies":["Federal Contracts Database","Internet"],"full_text":"Space proved to be – if not the final frontier, certainly a new one – for the military trade publication Defense News. The newspaper, where I recently worked as a reporter, had blanketed other military programs, but never those in space. Our editors decided to change that and assigned me to dig up what I could on Pentagon-related space programs. It promised to be a fruitful fishing expedition. The Defense Department had mismanaged nearly every space program it had – most of the satellite systems were billions of dollars over budget and months or years behind schedule. The question was this: what could I find that was different and had not been reported before? I simply could not serve up a reheated story. But I also knew that for all of the coverage, there had been relatively little analysis, especially financial, to put it into context. Database analysis of program spending played a key role in reporting the story. I started by contacting longtime source Loren Thompson, a space expert at the Lexington Institute in Arlington, Virginia, a think tank advocating free-market policies and limited federal governance. Thompson told me the one topic that had not been looked into for years was the development problems of the nation's eavesdropping satellites and provided some starting information. But I needed to learn a lot more to report on classified projects that few people would want to talk about on the record. Most of the material was highly technical, and I needed to find expert guides. I studied books on basic communications and the science of radio waves. I read more than a dozen technical books on satellites, the history of U.S. eavesdropping, space communications, and so on, for a basic understanding of the technology and the jargon of the field. I also read nearly 30 articles from periodicals and after-action reports from Marine and Army divisions involved in the invasion of Iraq. Because the focus of my stories was on the money issues, I read and reviewed a score of budgets, military planning documents, selected acquisition reports, special intelligence studies and other documents. I rounded this off with a file drawer full of company reports, studies and brochures on the capabilities they were promising for commercial and military use. I focused on the companies that are the main satellite builders or developers: Boeing, Lockheed Martin and Northrop Grumman. Most of these sources are available online. A good Pentagon budget summary for key programs is in the Selected Acquisition Reports at www.acq.osd.mil/ara/am/sar. As I read all of the reports, the outlook seemed gloomy for the U.S. spy satellite programs. American eavesdropping and eavesdropping satellites, many built years ago to keep tab on the former Soviet Union, are proving ineffective in the war on terror, thanks to prevalence of the Internet and cell phones. The U.S. military and intelligence communities are trying to fix the problem with costs at $60 billion and climbing. The military's main satellite-builder – the National Reconnaissance Office – is struggling to maintain its relevance as critics charge the NRO is obsolete and out of touch with its \"customers.\" Defense Week purchased an updated federal contracts database from the IRE and NICAR database library. (For more information see www.ire.org/datalibrary). The database includes detailed information about federal contracts, including those with the Defense Department. But I needed a way to find the satellite contracts I wanted. In some cases, the word \"satellite\" would not even appear in any of the contract fields, so I needed to find a way to capture any of the possible contracts for that work. Most would be funded by Air Force, the military's financial clearing house for most satellite work, even for intelligence spacecraft. I called up www.defenselink.mil and searched its Pentagon contracts, plugging in \"satellite,\" \"space\" or other terms. I learned the different possible contract numbers and types of services. I then used these contract numbers to find contracts and modifications in the Federal Contracts database from NICAR. Armed with that information, I used Microsoft FoxPro 2.6 database manager to hunt through each year's data to find groupings and amounts of contracts (space, satellites, classified, etc.) Then I used SPSS statistical software to do some of the statistical work on those groupings. It's easier and much more versatile when working with millions of records. I used SPSS to look at the percentage differences for classified and unclassified contracts. While I could have done this with FoxPro, it would have taken longer. Also with SPSS, I broke out other percentages for different types of contracts – by military service, contractor, and so forth. Most of the contracts I was trying to examine dealt with classified programs and could not be identified easily. In some cases, sources helped guide me. In others, I tailored my queries to find contracts dealing with \"space services\" or \"telecommunications.\" During the reporting I also put online contract information in Microsoft Excel spreadsheets and Access database manager for further analysis. When the analysis was done, I learned that most of the satellite contracts were classified. Classified satellite programs had grown by one-third between 2001 and 2004, as had military contracts overall. Brad Peniston, my managing editor, and I decided to start publishing stories so we could develop sources and build credibility on the subject as we published. Our first story focused on the growth in satellite classified contracts, and the difficulties federal agencies such as the Congressional Research Service and Government Accountability Office faced in analyzing program spending. The stories that followed zeroed in on the satellite program difficulties. I had a good foundation, but I needed sources to put it in perspective so I reached out to experts via ProfNet (www.profnet.com), an expert-finding service run by PR Newswire. I joined organizations like the Association of Former Intelligence Officers, which is a great resource. I also talked to executives in companies doing commercial satellite work. While officials doing government work could not discuss classified programs, they could talk about their commercial work in similar areas. As they had to overcome many of the same technical hurdles in the commercial arena, their perspective was pertinent. Also by writing stories about the commercial side of the business, I developed contacts, expertise and the trust of the satellite executives. We had decided to run the hardest-hitting piece – detailing the NRO's woes in trying to build the next generation of eavesdropping satellites – in April, the same week of the Air Force's key space-systems conference. The NRO said nothing about the piece. When dealing with classified systems like this, the only time when you hear from the folks you're writing about, is when you screw something up so we knew we were right on target.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-september-october-diagnosing-troop-mental-health-ills","slug":"2006-september-october-diagnosing-troop-mental-health-ills","url":"https://dwillis.github.io/uplink/article/2006-september-october-diagnosing-troop-mental-health-ills.html","year":2006,"month":"September-October","monthNum":9,"headline":"Diagnosing troop mental-health ills","kicker":"SPOTLIGHT: MILITARY MANEUVERS","author_name":"Matthew Kauffman","author_title":"reporter","affiliation":"The Hartford Courant","summary":"The Hartford Courant investigated systemic failures in the military's mental health screening process by analyzing data from pre-deployment health questionnaires.","keywords":["mental health","military","data analysis"],"topics":["Investigative Methods","Data & Databases","Health & Environment"],"technologies":["SQL","Defense Medical Surveillance System"],"full_text":"When reporter Lisa Chedekel and I set out to examine the adequacy of mental health screening and treatment for troops in Iraq and Afghanistan, we found powerful stories of soldiers and Marines who killed themselves after their mental problems were ignored by commanders. We could have built a series on just those tales, but we were determined to supplement our anecdotal reporting with a statistical analysis in an effort to determine whether there were systemic failures in the military's treatment of troubled soldiers. We found a possible window in Defense Department Form 2795, the two-page pre-deployment health questionnaire that is supposed to be filled out by every deploying service member. The form asks whether service members have sought counseling in the past year, and that single question is the foundation of the military's mental health screening process. The form also indicates whether a referral to a mental health professional was made, and whether the service member was ultimately deemed deployable. Three million of those questionnaires have been filled out since 1993, most – unfortunately – on paper. But we discovered that every one of those pre-deployment forms gets sent to an office in suburban Washington and entered into a database. It is all stored in something called the Defense Medical Surveillance System, a massive database – 300 million rows – chock full of data protected under the federal Health Insurance Portability and Accountability Act. But, for researchers, the military maintains a separate database that has been stripped of identifying information. It's all available to registered users at http://amsa.army.mil/DMED_Items/DMEDOverview.htm. It was a gold mine. But getting access to it wasn't going to be that easy. My application for online access was rejected. And when I contacted an official at the U.S. Army Center for Health Promotion and Preventive Medicine, which oversees the database, I was told the data is available only to Defense Department personnel. Citing the online user's manual for the database, I noted that it was supposed to be \"available to all military and civilian researchers, policy makers and others with a need to evaluate the health of active duty service members.\" Surely that applied to me, I argued. But my definition of \"need\" apparently differed from the military's. The answer was no. But they also made an unexpected offer: While they were still refusing to give me unfettered access, they agreed to query the data and give me the results. I still believed I was entitled to the raw database, and I wasn't thrilled about tipping my hand, fearing that making it obvious I was focusing on mental health might lead to the sudden withdrawal of their offer. But I was quickly seduced by the prospect of getting some numbers without an endless court battle that undoubtedly would have dragged on well past our publication date. So I took them up on the offer, asking for data, aggregated by month, which would provide counts of the various permutations of three yes-or-no questions: had service members marked that they had sought counseling, were they referred to a mental-health professional, and were they ultimately deployed. I asked for data going back to March 2003 – the launch of Operation Iraqi Freedom – though I later came to wish I had gone back farther. There were other data I would have liked, which would have allowed us to analyze any statistical differences among the branches of service, or among different military ranks. But in the end, I decided to stick with what amounted to a single, short line of SQL code that would yield a table with fewer than 400 records, hopeful that that simple request would be impossible for them to refuse. I made the request in September 2005, two months after I began negotiating for access to the data. Had I been able to query the data online, I would have had it in seconds. I figured it would take the military a couple days to get around to running the query. Six weeks (and a lot of prodding phone calls later), I was still waiting, and began to assume I would never see the numbers. But I kept pushing, and one day in November, a modest Microsoft Excel spreadsheet arrived by e-mail, offering a first-ever window into the military's pre-deployment mental health screening. It took four months of wrangling – not too bad when negotiating with the military – and it was definitely worth the wait. Although the file was small, I imported it into Microsoft FoxPro database manager so I could work with the data more easily. Every way I viewed the data, the results were astounding. We found that since the start of the war, fewer than 1 in 300 service members was referred to mental health professional, despite a 1997 law requiring that the military assess the mental health of all deploying troops. Even among troops who said they had sought mental health care in the past year, just 6.5 percent were sent for an evaluation before being deemed mentally fit for war. In addition, despite the military's promises to pay closer attention to the mental health of service members following a spate of suicides in 2003, we found that soldiers who reported psychological issues were more likely to be deployed in 2005 than at the start of the war. The numbers ratcheted up our comfort level that we had found systemic problems and not simply a cache of compelling anecdotes. And the statistics were widely cited in post-publication reporting on the series and were picked up by legislators pressing for reforms. \"The law is not being followed as it was intended,\" Senator Joe Lieberman testified in support of a bill that would expand pre-deployment screening. \"Alarmingly, the Hartford Courant's investigation found that only 6.5 percent of those indicating mental health problems were referred for mental health evaluations from March 2003 to October 2005. This is unacceptable.\" The legislation, added to the Defense Authorization Bill, requires the military to go beyond a single screening question and also assess a service member's mental health history and any current symptoms, treatment or psychotropic drug use. The bill also mandates that service members who show signs of a mental health disorder must be referred to a qualified health professional before they are deemed deployable. Getting these numbers did not require any heavy-duty analysis or sophisticated programming; it only took simple queries to evaluate the relationships (and the relationships over time) between three fields. But there are basic lessons worth repeating for those who do computer-assisted reporting. Routinely look for data – the government's got loads of it, as do other institutions – to add a broader statistical element to your stories. And don't listen to the voice that says an agency will never, ever cough up the data you're seeking. Pursue it – creatively, doggedly. If you get it one time in 10, it's worth it.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-september-october-more-tips-for-reporting-on-trucking-accidents","slug":"2006-september-october-more-tips-for-reporting-on-trucking-accidents","url":"https://dwillis.github.io/uplink/article/2006-september-october-more-tips-for-reporting-on-trucking-accidents.html","year":2006,"month":"September-October","monthNum":9,"headline":"More tips for reporting on trucking accidents:","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"This article provides journalists with key web resources and databases for reporting on trucking accidents, including the Federal Motor Carrier Safety Administration's crash data, the National Transportation Safety Board's investigation reports, and industry sources.","keywords":["trucking","accidents","resources"],"topics":["Public Safety","Investigative Methods","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"More tips for reporting on trucking accidents:\n\nThere are great Web sites for trucking stories. Start with the U.S. Department of Transportation's Federal Motor Carrier Safety Administration at www.fmcsa.dot.gov. One useful feature is the crash profiles online, where you can query Fatality Analysis Reporting System (FARS) data to get detailed information about fatal traffic accidents.\n\nIn March, the administration released a detailed report about big-truck crashes to Congress and found legal drugs were a big factor. See www.fmcsa.dot.gov/facts-research/research-technology/report/ltccs-2006.htm.\n\nThe administration's Web site also has state accident-comparison tables at http://ai.fmcsa.dot.gov/CrashProfile/NationalCrashProfileMain.asp#statereports. State-by-state summary data on enforcement activities can be found at: http://ai.fmcsa.dot.gov/ProgramMeasures/Intro/ProgramMeasuresMain.asp.\n\nThe National Transportation Safety Board Web site has accident investigation reports and other documents online at www.ntsb.gov.\n\nThe American Trucking Association is the largest trade association representing the trucking industry. Its Web site (http://truckline.com) has industry news and legislative and regulatory information.\n\nFor more detailed guidance, see \"Investigating Trucking\" (Tipsheet No. 2) by David Barnes, director of public affairs for the U.S. DOT's Office of Inspector General. IRE members","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-november-december-tallying-city-spending-on-trinkets","slug":"2006-november-december-tallying-city-spending-on-trinkets","url":"https://dwillis.github.io/uplink/article/2006-november-december-tallying-city-spending-on-trinkets.html","year":2006,"month":"November • December","monthNum":11,"headline":"Tallying city spending on trinkets","kicker":"FIRST VENTURE","author_name":"Eugene Mulero","author_title":"","affiliation":"The Arizona Republic","summary":"Eugene Mulero explores how cities spend taxpayer money on promotional items like stress-relief toys and pens.","keywords":["city spending","promotional items","public records"],"topics":["Government & Politics","Investigative Methods","Business & Economy"],"technologies":[],"full_text":"My interest in city promotional items, or trinkets, started at a local history festival last fall in Chandler, Ariz., and led to my first CAR story. The event took place at a park near the ruins of the old Southern Pacific Railroad line. To commemorate the event, city officials handed out small foam choochoo trains. The trains looked like squeezable stress-relief toys. More importantly, they had the city's logo and Web site on their tops. I took one back to the newsroom. A few weeks later, I talked to a former colleague in New Jersey. I told him I picked up the trinket, and he mentioned that cities in Jersey wouldn't spend money on those items. I realized he was right. I never saw a city in New Jersey give away pens or notepads with their logos. If they did, residents would criticize officials for overspending.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-november-december-records-reveal-ties-behind-charity-scam","slug":"2006-november-december-records-reveal-ties-behind-charity-scam","url":"https://dwillis.github.io/uplink/article/2006-november-december-records-reveal-ties-behind-charity-scam.html","year":2006,"month":"November • December","monthNum":11,"headline":"Records reveal ties behind charity scam","kicker":"SPOTLIGHT: PEOPLE AND PROFILES","author_name":"Ronald Campbell","author_title":"","affiliation":"The Orange County Register","summary":"Ronald Campbell investigates how solicitors use sound-alike charity names to collect millions of dollars from unsuspecting donors.","keywords":["charity scam","phone solicitors","fundraising"],"topics":["Investigative Methods","Business & Economy","Data & Databases"],"technologies":[],"full_text":"It's 7 p.m. and you're just sitting down to dinner when the phone rings.\n\nAnother phone solicitor.\n\nHe's pitching for the American Deputy Sheriff's Association or the Disabled Firefighters Foundation or the American Veterans Relief Foundation or one of dozens of sound-alike charities.\n\nThese groups use similar names, similar pitches, similar techniques and many of the same organizers to reap tens of millions of dollars annually from American donors. In fact, they are all connected in ways most donors never suspect.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-november-december-hard-lessons-in-first-big-data-wave","slug":"2006-november-december-hard-lessons-in-first-big-data-wave","url":"https://dwillis.github.io/uplink/article/2006-november-december-hard-lessons-in-first-big-data-wave.html","year":2006,"month":"November • December","monthNum":11,"headline":"Hard lessons in first big data wave","kicker":"AMERICAN COMMUNITY SURVEY","author_name":"Paul Overberg","author_title":"","affiliation":"USA Today","summary":"Paul Overberg analyzes the challenges and evolution of the American Community Survey's national data release.","keywords":["American Community Survey","census data","data analysis"],"topics":["Data & Databases","Journalism & Newsroom","Demographics & Census"],"technologies":[],"full_text":"As summer rolled into fall and the data from the 2005 American Community Survey just kept coming, one thing became clear: Its maturation will be slower and more difficult than even the Census Bureau anticipated. This year's release, the first of the full national sample of three million households per year, is already the sixth in the series stretching back to the Census 2000 Supplementary Survey. ACS won't emerge in full form until 2011. In journalism, that's almost geological time. ACS has been changing all the time. This year, the data changed significantly from last year. The same thing will happen in each of the next two years. For example: The 2005 data was produced from a nationwide sample that included all states and congressional districts. PLUS: This allowed comparisons from place to place that could never be made before. CAVEAT: comparisons to earlier years were often suspect.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-november-december-boating-data","slug":"2006-november-december-boating-data","url":"https://dwillis.github.io/uplink/article/2006-november-december-boating-data.html","year":2006,"month":"November • December","monthNum":11,"headline":"Boating data","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library has updated its U.S. Recreational Boating Accident Database through 2005, containing information on boating incidents resulting in deaths or significant injuries across four data tables, though California data is excluded due to state restrictions on record release.","keywords":["boating","database","accidents"],"topics":["Data & Databases","Public Safety"],"technologies":[],"full_text":"The IRE and NICAR Database Library recently updated its U.S. Recreational Boating Accident Database.\n\nCurrent through 2005, the database contains information on recreational boating incidents that resulted in a death or an injury that required more than just first aid. It consists of four tables (Prime, Dead, Injury and Vessel) and also lists vessel damage that costs more than $2,000 to repair. For the first time, the database does not include data for California because the state declined to allow the Coast Guard to release the records.\n\nFor more information, visit www.ire.org/datalibrary/databases or call the library at 573-884-7711","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-november-december-ire-conference","slug":"2006-november-december-ire-conference","url":"https://dwillis.github.io/uplink/article/2006-november-december-ire-conference.html","year":2006,"month":"November • December","monthNum":11,"headline":"IRE Conference","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The 2007 IRE Conference will be held June 7-10 in Phoenix at the Arizona Biltmore Resort and Spa, featuring over 100 panels and workshops on investigative journalism topics. The event will include optional CAR training sessions on June 7 with hands-on classes in Excel and Access databases.","keywords":["conference","journalism","Phoenix"],"topics":["Journalism & Newsroom"],"technologies":["Excel","Access"],"full_text":"Make plans now to attend the 2007 IRE Conference June 7-10 in Phoenix. The Arizona Republic is hosting the event, which will be held at the Arizona Biltmore Resort and Spa.\n\nThe annual gathering will feature more than 100 panels, workshops and presentations on covering courts, public safety, the military and business just to name a few. Register for the optional day, June 7, to attend special CAR panels or take additional hands-on classes in Excel spreadsheets and Access database manager.\n\nIRE and NICAR welcome suggestions for panel topics, speakers and classes. Send your ideas to confideas@ire.org. Please make reference to the 2007 IRE Conference.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-november-december-on-the-foia-front","slug":"2006-november-december-on-the-foia-front","url":"https://dwillis.github.io/uplink/article/2006-november-december-on-the-foia-front.html","year":2006,"month":"November • December","monthNum":11,"headline":"On the FOIA front","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"David Herzog discusses how journalists successfully use FOIA to uncover critical data from federal agencies, despite increased secrecy.","keywords":["FOIA","transparency","federal government"],"topics":["Investigative Methods","Government & Politics","Journalism & Newsroom"],"technologies":[],"full_text":"In the previous issue of Uplink, journalists wrote about how they obtained data from the federal government and analyzed it for their stories about the U.S. military. A common thread that ran through the articles was the authors' use of the Freedom of Information Act to get data from federal agencies. The requests yielded data about civilian complaints against soldiers in Iraq, mental health referrals for service members and disparities in veterans' disability payments. In the latter case, the former Knight Ridder Washington bureau had to sue the U.S. Department of Veterans Affairs. But the shared theme here is that journalists can use FOIA successfully. That should be heartening news for journalists doing computer-assisted reporting. Despite the federal government's clampdown on information since 9/11, some journalists are persistent enough to get the data for stories that inform the public about those sent to war. Let's hope these FOIA experiences encourage even more of us to use this essential, but underused, tool. Contact David Herzog by e-mail at dherzog@nicar.org.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-november-december-these-sources-speak-when-subjects-wont","slug":"2006-november-december-these-sources-speak-when-subjects-wont","url":"https://dwillis.github.io/uplink/article/2006-november-december-these-sources-speak-when-subjects-wont.html","year":2006,"month":"November • December","monthNum":11,"headline":"These sources speak when subjects won't","kicker":"SPOTLIGHT: PEOPLE AND PROFILES","author_name":"Duff Wilson","author_title":"","affiliation":")}\n\nSome of these online tools are included in ","summary":"Duff Wilson provides a guide to essential online tools and public records for backgrounding individuals for investigative profiles.","keywords":["backgrounding","online records","investigative tools"],"topics":["Investigative Methods","Data & Databases"],"technologies":["Google","Nexis","Accurint","AutoTrack","VitalChek","PACER","Internet","World Wide Web"],"full_text":"There is a wealth of records, many of them available at your fingertips for profiling people who don't want to be profiled.\n\nSome of these online tools are included in ","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":false}},{"id":"2006-november-december-training-in-korea","slug":"2006-november-december-training-in-korea","url":"https://dwillis.github.io/uplink/article/2006-november-december-training-in-korea.html","year":2006,"month":"November • December","monthNum":11,"headline":"Training in Korea","kicker":"Bits & Bytes","author_name":"","author_title":"","affiliation":"","summary":"IRE conducted investigative and computer-assisted reporting training sessions at two major South Korean news organizations: the Korean Broadcasting System and the Chosun Ilbo newspaper, helping journalists develop skills in databases, mapping, and investigative techniques.","keywords":["training","journalism","Korea"],"topics":["Investigative Methods","Journalism & Newsroom","Technology"],"technologies":["Excel","Access"],"full_text":"IRE recently conducted investigative and computer-assisted reporting training for two of South Korea's leading news organizations.\n\nIn late October, Database Library Director Jeff Porter led sessions for the Korean Broadcasting System, the country's largest television network. In Suwon, 22 journalists attended classes in Excel and Access and learned about investigative reporting techniques. At the KBS headquarters in Seoul, reporters and producers heard presentations on using databases, mapping and social network analysis for stories.\n\nThe Chosun Ilbo, South Korea's largest daily newspaper with a circulation of more than 2 million, hosted David Donald, IRE training director, for four days in early August.\n\nChosun is starting an investigative reporting program and IRE was tapped to help get it started. Donald trained journalists in investigative and computer-assisted reporting. He also worked with other Chosun editors to put together an investigative team.\n\nAccording to Donald, South Korea does not have a long tradition of investigative reporting. CAR has gained popularity only during the past few years.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2006-november-december-capitol-aides-two-jobs-leave-money-trail","slug":"2006-november-december-capitol-aides-two-jobs-leave-money-trail","url":"https://dwillis.github.io/uplink/article/2006-november-december-capitol-aides-two-jobs-leave-money-trail.html","year":2006,"month":"November • December","monthNum":11,"headline":"Capitol aide's two jobs leave money trail","kicker":"SPOTLIGHT: PEOPLE AND PROFILES","author_name":"Thomas Peele","author_title":"Reporter","affiliation":"Contra Costa Times","summary":"Thomas Peele used congressional spending reports and financial disclosures to reveal how a top aide exploited travel expenses and exceeded income limits.","keywords":["congressional spending","financial disclosure","public records"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["Excel"],"full_text":"I have never met Steven J. Ding, chief of staff to the U.S. House Resources Committee and a top political adviser to its chairman, Rep. Richard Pombo, R-Calif., who recently lost his re-election bid. I've never been in the same room with Ding, and we have spoken on the telephone only a few times. Yet after digging through congressional spending documents, personal financial disclosures, campaign finance reports and other records, and then using them to build databases, I was able to paint a picture of Ding for our readers. Ding preferred venti-sized Starbucks coffee, parked often at the Sacramento airport, usually on Mondays, and left his car for four days. He spent a lot of time on planes and at Capitol Hill hotels. Between 2000 and 2004, he received at least $114,240 in political consulting fees from California Republican politicians, some of it in violation of congressional rules. Last year, two colleagues and I began to comb through records related to the land holdings of Pombo and his family in California's Central Valley. We heard that Pombo had allowed his top aide unusual job freedom, so I began gathering documents and eventually broke away to focus on Ding. I learned that Ding traveled almost weekly between his home in Stockton, Calif., and Washington, where he stayed in a Capitol Hill hotel at taxpayer expense when the committee was in session. To examine the travel, I needed copies of Pombo's \"representational account\" reports for five years. These documents, available only in the House Clerk's Office, show all salaries and expenses paid from a House member's office budget. Two colleagues passing through Washington obtained copies for me. These documents, in tiny type, contained dozens of references to \"Ding airfare Sac-DC\" or \"Ding airfare DC-SF.\" There were other references to \"Ding lodging\" or \"Ding sustenance.\" The hotel and food entries didn't specify where the money was spent, but the dates on the travel records told me that Ding had to have incurred them in Washington. I entered all of the costs into a Microsoft Excel spreadsheet. They showed that during the three-year period in which Ding worked for the committee but lived in Stockton, he charged more than $87,000 in travel expenses to Pombo's district office. Pombo appointed Ding to run the House Resources Committee when he became its chairman in 2002; Ding had worked for the congressman in California since 1992. I also learned from the Pombo office spending documents that Ding was getting paid $100 a month as Pombo's chief of staff. This was in addition to his committee salary of more than $150,000 a year. That item in the office budget, Ding later said, permitted him to commute from California to Washington at taxpayer expense. By still doing some work on Pombo's district staff, he considered his congressional \"duty station\" to be California. That, he said, entitled him to government travel when he was needed back in Washington for committee business. If he had attempted to bill this travel to the committee, it would have been rejected. Committee rules state: \"living and commuting expenses are not reimbursable.\" In California, Ding frequently worked for Republicans with close ties to Pombo, an analysis of state and congressional records showed. Top House staffers must file annual financial disclosure statements. Ding's showed income from several political candidates and consultants, including a Sacramento firm that ran Pombo's campaigns. Congressional rules limit the amount of money that staffers can receive each year on top of their government salaries. For one year, Ding reported that he received an amount that exceeded the limit. I started digging more deeply. The California Secretary of State maintains a campaign finance Web site (http://cal-access.ss.ca.gov/Campaign) with advanced search functions. When I entered Ding's name as a vendor and searched all records, I got numerous hits. One showed that he received $30,000 in one month from a candidate in the 2003 gubernatorial recall election. I used Excel and Microsoft Access database manager to show how much Ding was paid compared to what he had reported to Congress. In all, the state finance records showed he received $35,000 in payments for that year. In contrast, Ding reported $20,000 in outside income on his House financial disclosure statement. Congressional rules limited outside income that year to about $22,000. The state records also showed that Ding received a $1,000 payment in 2000 that he failed to report to Congress. Additionally, in 2002 he reported income that exceeded the congressional limit by $5,000.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-november-december-prep-for-election-mapping","slug":"2006-november-december-prep-for-election-mapping","url":"https://dwillis.github.io/uplink/article/2006-november-december-prep-for-election-mapping.html","year":2006,"month":"November • December","monthNum":11,"headline":"Prep for election mapping","kicker":"MAPPING IT OUT","author_name":"Jacob Jost","author_title":"Journalist","affiliation":"WBIR-Knoxville, Tenn.","summary":"Jacob Jost outlines how journalists can use GIS software and precinct data to create analytical maps for election night coverage.","keywords":["GIS mapping","election data","precincts"],"topics":["Government & Politics","Technology","Investigative Methods"],"technologies":["ArcView","Access","XPDF","Excel"],"full_text":"With the November general election over, now is the perfect time to lay the groundwork for mapping data in future elections. Journalists reporting for print can follow races throughout election night and then analyze the data using geographic information system (GIS) software when the final results are released. Those of us in television and online media have another challenge: generating worthwhile analysis of the election as it develops. Preparation can help your news organization provide thoughtful coverage of the election in advance and even as it happens. There's plenty you can do right now. If an incumbent is running for office, you can analyze earlier results to see where that candidate's support was strongest. If there's a challenger, examine election results from the primary election or any earlier races that included the challenger. You also can obtain information on the number of registered voters and voter turnout in previous elections. This can give you a sense of the \"power precincts\" within your voting area. With these maps in hand, you can either present an election preview story or simply make sure your anchors are well-briefed to analyze the election as the results roll in. To election mapping you'll need a GIS program, such as ESRI's ArcView, data on voter registration and results, and mapping shapefiles that show voting precincts. Many journalists have been able to obtain registration and results in common data formats (such as ASCII text) that can be easily imported into a database manager, such as Microsoft Access. Sometimes this information is only readily available in Adobe Acrobat PDF files. If that's the case, you will need to convert these to text tables using the no-cost XPDF or a similar tool from a commercial vendor. You also will need to get election map shapefiles required to display the data in ArcView. Check first with the elections division in your state's secretary of state's office. The cost, if any, will vary. In Tennessee, a voting precinct shapefile for a county costs $10. Tennessee has 95 counties, so the cost for the entire state would run more than $1,000 after sales tax. If you're only seeking maps for your viewing or circulation area, the cost would be lower. Another option is to check with the commissioner of elections in each county, you might be able to do better on the cost. If you're unable to obtain the precinct shapefiles, you may need to shift gears and examine results by county. County election commissioners usually total the results from all their precincts, and then send them along to the state. To complement the data, you can then use county shapefiles, which are available from a number of sources. A few maps can go a long way in your reporting. Perhaps the simplest is a map showing which candidate won each county or precinct. If you're examining a two-candidate race you can easily calculate the winner in a Microsoft Excel spreadsheet. If there are more than two candidates, you may find it easier to move your data into a database manager, such as Access, and run update queries to indicate whether a candidate is the winner. In addition to showing wins, you can show the strength of victory using a color scheme or other map graphics. I recently used a 10 percent margin as my benchmark for strong support. You can adjust the benchmark, based upon voting history in your area. The victory-margin map helped for a WBIR report that showed how a powerful Republican sheriff nearly lost to a lightly funded challenger who made inroads into a Republican stronghold. The strength-of-win map showed that, although the incumbent held onto the majority of the votes in that area, it was a close race. Another informative map shows turnout. You can create this by getting data on registered voters and voter turnout records for past elections. That will show you the high-activity areas and help you see turnout trends. These are just three examples of the maps you can make for election night. The simplicity of the numbers available limits the ways in which you can work with the data. On the upside, the maps are easier for everyone to understand. Contact Jacob Jost by e-mail at jjost@wbir.gannett.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-november-december-tame-unruly-notes-with-a-wiki","slug":"2006-november-december-tame-unruly-notes-with-a-wiki","url":"https://dwillis.github.io/uplink/article/2006-november-december-tame-unruly-notes-with-a-wiki.html","year":2006,"month":"November • December","monthNum":11,"headline":"Tame unruly notes with a wiki","kicker":"CAR TOOL","author_name":"Matthew Waite","author_title":"reporter","affiliation":"St. Petersburg Times","summary":"Matthew Waite explains how using wiki software can help journalists manage and search vast amounts of interview notes and documents for long-term projects.","keywords":["wiki","data management","investigative reporting"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["MediaWiki","PHP","MySQL","Apache"],"full_text":"After 18 months of reporting, interviewing and studying, Craig Pittman and I sat down to write a series on Florida's vanishing wetlands. We had interviewed hundreds of people, reviewed tens of thousands of pages of documents and analyzed satellite imagery for the whole state. In short, we had a lot of stuff to go through. And we had no real way to manage it all, aside from boxes, so we muddled through the old-fashioned way – with a lot of paper shuffling. But it didn't have to be like that. A year earlier, Derek Willis of The Washington Post showed me how journalists there were investigating lobbyist Jack Abramoff and storing their information in a wiki. A wiki is a mouthful of an idea, but what it boils down to is this: it's a Web-based text database. Users can edit any page and add text, which the wiki software makes browsable and searchable. Wikis can run on a Web server, or they can run on your desktop. To edit the wiki, all you or your users need is a Web browser. As for who the users are, that's up to you. You could set up a wiki for yourself on your computer. You can set one up on a server where you or a wiki administrator can control who gets a password and login. You can set wikis up to use your existing newsroom security accounts, or just throw the whole thing open to anyone. Now, more than a year after the first two stories in the wetlands series ran, we store nearly all of the information we collect for our continuing series on a wiki. We have more than 200 interviews transcribed on the wiki, thousands of pages of documents stored, literally millions of words comprising our research, all organized and searchable. Instead of pawing through boxes or notebooks, I can type a term into the search area and find every instance of that term in all of our notes at once. For a follow-up story, I remembered that a source had talked about a specific issue in an interview we'd done a year before. I typed in a few keywords, and the interview popped up in seconds. No quote, fact or idea we've had on the project is out of reach anymore. Our original wiki, which I hosted on a personal server, worked so well, and so many reporters in the newsroom wanted to use it, that we set one up for the St. Petersburg (Fla.) Times newsroom. Unveiled in late October, it's in its infancy. Wikis work extremely well managing projects, but it's an open question to see how they do in a daily environment. The Reasons are simple: if the wiki is just for you, there are plenty of options for wikis that run on your computer. If it's for the whole newsroom, then your needs ramp up. I started our wiki on a personal server because I had no access to any servers at the Times, and none of our servers had the right software to run one. We now have two new servers running our internal wiki. Our IT staff bought into the idea right away, and we got what we needed. A bright side: there's no need to spend a dime on software. There are several open-source wiki programs that cost nothing. Our internal wiki runs on MediaWiki, the software that runs Wikipedia, is open-source and free of charge. MediaWiki requires PHP, a free scripting language, and MySQL, a free database program that stores all your information. If you already have an Apache-based Web server in your newsroom, you can set up a wiki without spending anything. With the back end set up, making it work is a snap, and it's an open-ended question. The wiki is a series of Web pages that are all linked together. How you create them and how you organize them is up to you. On our internal wiki, I've created areas by project, beat, bureau and down to individual reporters. Creating pages in MediaWiki is simple. You open up an edit screen (by clicking the cleverly named \"edit\" button) and enclose the name of your page in double brackets. Let's say I wanted to create a page with this article in it. In my edit screen, I would type [[wiki article]]. When I hit \"save,\" the wiki software creates a new page in the database – an empty place for you to store your text – and turns the words \"wiki article\" into a link. I could then click on the link, and it would automatically take me to an edit screen to add the text of this article. As soon as I hit \"save,\" I can search for any word in the article. Having searched for years for a better way to manage and organize the projects I'm working on, I've never seen anything better than a wiki. It's simple, sophisticated and extremely flexible. Every journalist I've shown the wiki has immediately grasped what it could do for them. It's definitely worth looking at for your newsroom. Contact Matthew Waite by e-mail at waite@sptimes.com.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2006-november-december-wiki-resources","slug":"2006-november-december-wiki-resources","url":"https://dwillis.github.io/uplink/article/2006-november-december-wiki-resources.html","year":2006,"month":"November • December","monthNum":11,"headline":"Wiki Resources","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"Matthew Waite presented on wikis at the 2006 CAR Conference, with his presentation available online, while the Special Libraries Association also featured a panel discussion on wikis with audio and materials available.","keywords":["wikis","conferences","presentations"],"topics":["Journalism & Newsroom","Technology"],"technologies":["PowerPoint","MP3"],"full_text":"Matthew Waite spoke about wikis at the 2006 Annual CAR Conference in Newark, N.J. You can view his PowerPoint presentation online at www.mattwaite.com/conference-materials.\n\nPanelists at the Special Libraries Association News Division 2006 Annual Conference also discussed wikis. An MP3 audio file of the panel and supporting materials are available at www.ibiblio.org/slanews/conferences/sla2006.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-januaryfebruary-plug-into-data-to-investigate-utilities","slug":"2007-januaryfebruary-plug-into-data-to-investigate-utilities","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-plug-into-data-to-investigate-utilities.html","year":2007,"month":"January/February","monthNum":1,"headline":"PLUG INTO DATA TO INVESTIGATE UTILITIES","kicker":"ENERGY AND UTILITIES","author_name":"Tom McGinty","author_title":"Staff Writer","affiliation":"Newsday","summary":"A journalist describes how analyzing utility data from LIPA revealed that new power plants weren't necessary, and discusses how deregulation of the electricity industry created complexity and opportunities for fraud like Enron's.","keywords":["utilities","deregulation","data"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["ASCII"],"full_text":"In July 2001, just a month after I was hired by Newsday and assigned to the utilities beat, the Long Island Power Authority, or LIPA, announced that it had to build new power plants across the island because increasing demand by residents and businesses would soon surpass the available supply.\n\nTalking to experts about the issue, I learned that utilities in the Northeast typically have plenty of kilowatts to go around on all but the hottest days of the year when air conditioners cause huge spikes in demand. I was curious about changes in demand on LIPA's system over time, so I asked the utility for three years' worth of hourly readings in electronic format.\n\nThe chief spokesman looked at me like I was from Mars (the authority had never been asked for electronic data before), but he ultimately gave me a disk with 26,000 records in fixed-width ASCII format along with an utterly pointless 500-page printout of the same data.\n\nMy resulting story documented that demand on LIPA's system, which then had a maximum capacity of 4,600 megawatts, had exceeded 4,000 megawatts during just 16 hours in the previous year. The data allowed me to explain to readers that new power plants, always unpopular with residents, wouldn't be needed if the utility and its customers could find ways to cut electric usage during those few peak hours.\n\nThroughout the ensuing years on the beat, I would turn to databases and public documents from countless sources to fact-check industry officials and investigate a wide range of issues, from fuel costs and pollution to utility salaries and the impact of deregulation on the energy industry.\n\nIn fact, few beats have more data and public documents readily available to the reporter who knows where to find them and – this is the hard part – how to decipher the incredibly arcane terminology of the industry.\n\nDeregulation tangle\n\nUntangling the inner workings of electric utilities got that much harder after the Federal Energy Regulatory Commission prodded many states to deregulate their utilities. Deregulation involved breaking up the electric monopolies that historically owned every piece of the systems – the power plants, the high-voltage transmission lines that move the juice over long distances and the low-voltage distribution circuits that feed kilowatts to users.\n\nIn states where deregulation took hold, the utilities sold their generating plants to independent power producers while keeping the wires. Those producers then competed to sell electricity to the utilities and large industrial customers. Middlemen – energy marketers and traders – began buying and selling electricity like a commodity in what were essentially spot markets.\n\nOf course, Enron became the most infamous player in this new world and its 2001 collapse sent shockwaves through the industry. In California, energy deregulation led to rolling blackouts and billions in costs to ratepayers. Many states that had considered deregulation backed off.\n\nIn New York, where LIPA operates, deregulation led to what I considered one of the best stories of my tenure on the beat. The state's largest utility, Consolidated Edison, had sold its power plants as part of deregulation. Its customers in New York City and Westchester County were now dependent on those independent power producers for their electricity.\n\nI learned through regulatory filings that Con Ed was paying a fortune to keep some of those independent producers on standby – plants that would only run during emergencies. These were called ","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":false}},{"id":"2007-januaryfebruary-data-reveals-city-workers-bonus-pay","slug":"2007-januaryfebruary-data-reveals-city-workers-bonus-pay","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-data-reveals-city-workers-bonus-pay.html","year":2007,"month":"January/February","monthNum":1,"headline":"DATA REVEALS CITY WORKERS' BONUS PAY","kicker":"FIRST VENTURE","author_name":"Matt Stiles","author_title":"","affiliation":"Houston Chronicle","summary":"A journalist uses computer-assisted reporting techniques to investigate a city payroll scandal, uncovering that bonus payments were far more widespread than officials claimed, with one in six city employees receiving extra compensation totaling $4 million since 2004.","keywords":["payroll","investigation","bonuses"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":[],"full_text":"I returned from last January's computer-assisted reporting Boot Camp in Columbia, Mo., with renewed vigor and eager to apply the new skills to my city hall beat.\n\nI didn't have to wait long. The following month, a payroll-padding scandal broke, and the story dominated our local media for weeks.\n\nThe tale was perfect for a CAR rookie. Most of the details couldn't be told well without data on city salaries, budgets and bonuses, and typically talkative sources had clammed up because of an administrative investigation.\n\nIn short, I had little choice but to turn to data.\n\nThe payroll scandal involved four employees in the city council's administrative office who were accused of taking $143,000 in improper, incremental bonus payments over approximately one year. The employees were quickly suspended and later fired. Mayor Bill White and other city officials told reporters that bonus payments – especially in such large quantities – were highly unusual for city employees.\n\nThat wasn't exactly true, but it took CAR to show it. We compiled an exclusive story showing that one in six city employees was paid extra money – totaling about $4 million – since Mayor White took office in 2004. The","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-januaryfebruary-indian-grants-gaming-revenue-both-on-the-rise","slug":"2007-januaryfebruary-indian-grants-gaming-revenue-both-on-the-rise","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-indian-grants-gaming-revenue-both-on-the-rise.html","year":2007,"month":"January/February","monthNum":1,"headline":"INDIAN GRANTS, GAMING REVENUE BOTH ON THE RISE","kicker":"TRACKING FEDERAL FUNDS","author_name":"Ziva Branstetter","author_title":"","affiliation":"Tulsa (Okla.) World","summary":"Despite significant growth in Indian gaming revenues, federal funding to tribes has increased rather than decreased, suggesting that gaming revenue and federal assistance are not inversely related. The author used federal data to show that between 2002 and 2004, tribal gaming revenues grew 30 percent while federal grants grew 44 percent.","keywords":["gaming","federal","funding"],"topics":["Data & Databases","Government & Politics","Investigative Methods"],"technologies":["FAADS"],"full_text":"Indian gaming has grown explosively during recent years. While giving few specific examples, tribes in Oklahoma claim they are putting that money to good use for programs to benefit their members. Before gaming, tribes relied almost solely on the federal government to pay for housing, medical care, education and other needs.\n\nThis led me to wonder about the relationship between tribes' gaming revenues and the funds they receive from the federal government. As tribes make more from gaming and become more self-sufficient, is there any reduction in federal funds for housing, medical care, government and the like? The answer is no, at least during the three years of data I examined.\n\nWhile revenues from Indian gaming grew 30 percent nationally during fiscal years 2002 through 2004, federal grants to tribes grew 44 percent. By 2004, federal assistance to tribal organizations totaled about $7 billion; Indian gaming revenues topped $19 billion nationally. The pattern was the same in Oklahoma.\n\nI found the answer using a database called the Federal Assistance Award Data System, or FAADS, maintained by the U.S. Census Bureau and available from the IRE and NICAR Database Library, www.ire.org/datalibrary/databases/viewdatabase.php?dbaseindex=38. Covering federal financial assistance for about 600 federal programs, FAADS includes","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-januaryfebruary-uplink-goes-electronic","slug":"2007-januaryfebruary-uplink-goes-electronic","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-uplink-goes-electronic.html","year":2007,"month":"January/February","monthNum":1,"headline":"UPLINK GOES ELECTRONIC","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"NICAR and Missouri School of Journalism","summary":"Uplink introduces its new interactive electronic version and website updates to better serve its readers with full-color graphics and searchable pages.","keywords":["Uplink","digital journalism","NICAR"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Adobe Acrobat","PDF","World Wide Web"],"full_text":"This issue of Uplink, as you've no doubt noticed, ushers in a new look that we hope will be more appealing to our readers, who are a loyal, demanding and intelligent bunch. While we're still mailing our print edition, subscribers now can download an electronic version of Uplink and take advantage of its interactive features. Just visit the NICAR Web site (www.nicar.org) and download the Adobe Acrobat PDF file for this issue. Inside, you'll find full-color graphics, hyperlinks that will take you to online resources and searchable pages. We're also sprucing up the NICAR Web site to include even more sample data files and multimedia features relating to what's on the pages of Uplink. Features will change and develop as we make the transition to this new format. We'd love to get your feedback on these changes. Please send me an e-mail and let me know what you think. And, don't forget to send in your registration for the annual CAR Conference March 8-11 in Cleveland. It will be a great opportunity to learn the tricks of the trade or recharge your CAR battery.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2007-januaryfebruary-car-conference-deadlines","slug":"2007-januaryfebruary-car-conference-deadlines","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-car-conference-deadlines.html","year":2007,"month":"January/February","monthNum":1,"headline":"CAR conference deadlines","kicker":"Bits and Bytes","author_name":"","author_title":"","affiliation":"IRE","summary":"The 2007 CAR Conference will take place March 8-11 in Cleveland, featuring panels on various journalism beats and hands-on technology training classes. Early registration and hotel reservations must be completed by mid-February to receive discounted rates.","keywords":["conference","registration","journalism"],"topics":["Journalism & Newsroom","Technology","Investigative Methods"],"technologies":["spreadsheet"],"full_text":"Make plans now for the 2007 CAR Conference to be held March 8-11 at the Renaissance Cleveland Hotel in downtown Cleveland.\n\nAttendees must reserve a room by Friday, Feb. 9, to receive the discounted rate of $119 plus tax per night. The last day to register for the conference at the early-bird rate of $150 is Feb. 19. After that date, you must register on site for $175.\n\nThe conference will feature panels by the best in the business on every beat and topic including: Census, crime, education, local and state government, freedom of information, transportation and more. Hands-on classes will be offered featuring spreadsheets, database managers, mapping, statistics and other technology.\n\nRegistration, hotel information and preliminary program details can be found at www.ire.org/training/cleveland07.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-januaryfebruary-truck-data-updated","slug":"2007-januaryfebruary-truck-data-updated","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-truck-data-updated.html","year":2007,"month":"January/February","monthNum":1,"headline":"Truck data updated","kicker":"","author_name":"","author_title":"","affiliation":"NICAR","summary":"NICAR has updated three databases tracking commercial truck accidents, inspections, and company safety information, providing journalists with detailed data on crashes, violation histories, and carrier compliance records.","keywords":["trucks","accidents","safety"],"topics":["Data & Databases","Public Safety"],"technologies":[],"full_text":"Keep an eye on jack-knifing semis and other trucks on your local roads using three databases recently updated by NICAR. The Truck Accidents database (www.ire.org/datalibrary/databases/viewdatabase.php?dbaseindex=51) shows commercial-carrier accidents as reported to local and state police, including details such as location, fatalities and vehicle identification numbers for trucks, buses or semis involved in wrecks. The Truck Inspections database (www.ire.org/datalibrary/databases/viewdatabase.php?dbaseindex=37) contains information on the number of inspection violations for each truck; date, time and place of the inspection; whether the inspection occurred before or after a crash, as well as alcohol and drug detection information. Finally, the Truck Census database (www.ire.org/datalibrary/databases/viewdatabase.php?dbaseindex=50) contains the shipper's DOT census number, company address, number and type of vehicles in the company's fleet, cargo types and safety ratings.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-januaryfebruary-new-archives-for-nicar-l","slug":"2007-januaryfebruary-new-archives-for-nicar-l","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-new-archives-for-nicar-l.html","year":2007,"month":"January/February","monthNum":1,"headline":"New archives for NICAR-L","kicker":"","author_name":"","author_title":"","affiliation":"","summary":"A new archive interface has been launched for the NICAR-L Listserv, offering improved search capabilities and organization by thread, month, and author, with advanced filtering options including boolean searches and regular expressions.","keywords":["archive","search","listserv"],"topics":["Journalism & Newsroom","Technology"],"technologies":["Excel"],"full_text":"A new archive interface makes it easier to search for tips and solutions posted on the NICAR-L Listserv, whether you're searching for a string function, a URL, author's name, or a phrase, such as \"Excel date conversion.\"\n\nThe updated system organizes entries by thread, month, and author. The advanced search feature offers helpful filtering options, ranging from familiar boolean terms to wildcard searches and pattern-matching with regular expressions. IRE members can access the archives at www.ire.org/membersonly/lists/NICAR-L.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-januaryfebruary-ny-power-agency-gives-dollar1-million-to-leaders-pet-projects-charities","slug":"2007-januaryfebruary-ny-power-agency-gives-dollar1-million-to-leaders-pet-projects-charities","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-ny-power-agency-gives-dollar1-million-to-leaders-pet-projects-charities.html","year":2007,"month":"January/February","monthNum":1,"headline":"NY POWER AGENCY GIVES $1 MILLION TO LEADERS' PET PROJECTS, CHARITIES","kicker":"ENERGY AND UTILITIES","author_name":"Michelle Breidenbach","author_title":"state government reporter","affiliation":"Post-Standard","summary":"An investigation reveals that New York Power Authority executives received higher salaries than the governor and used public funds for personal charities and political perks.","keywords":["NYPA","accountability","public records"],"topics":["Government & Politics","Investigative Methods","Data & Databases"],"technologies":["Excel","World Wide Web"],"full_text":"New Yorkers are familiar with the clean hydropower the state has been producing from the waterfalls on the St. Lawrence River for the past 50 years. But, is anyone paying attention to the agency that runs these dams? The New York Power Authority is managed by political appointees who work in White Plains, a two-hour drive from Albany. The staff treats the agency like a private business, and employees call it \"the company.\" The NYPA is owned, however, by New York state taxpayers. Its meetings are public. Its records are subject to New York's Freedom of Information Law. Its board of directors is held to the same ethical standards as the governor or anyone who works for the highway or education departments. Readers were surprised to learn from our two-part series that the NYPA's top executives make more money than the governor, fly even short distances on state-owned airplanes and donate about $1 million a year to their pet charities. They used public money to buy chandeliers for a theater in the governor's hometown and to throw a ski weekend for an Albany congressman. NYPA executives refused to be interviewed in person about their day-to-day business. So, I relied on records. To get started, I learned how New York's public power system fits into the national scene. The American Public Power Association (www.appanet.org) ranks public power utilities and put NYPA at No. 4 in generation. I researched a state-owned power company of similar size in South Carolina – Santee Cooper – for comparison. I filed a public records request for Santee Cooper's salaries. They refused to send electronic records but provided a paper list with last names, positions and annual salaries for employees who make more than $50,000. The records showed that 4.4 percent of Santee Cooper's public power workers made more than $100,000 per year. I also reviewed the salaries at the New York agencies whose budgets are controlled by the legislature and the governor. The electronic records are on file at the state comptroller's office. At these agencies, 3.8 percent of employees earn more than $100,000 per year. The governor makes $179,000 per year. Getting electronic records from the NYPA proved to be more difficult. The first response was: No, it is not state policy to distribute electronic records. So, I made an argument using opinions from the State Committee on Open Government, a branch of the Department of State. Some states, such as New York, have assigned someone within the government the task of interpreting open records and meetings laws. It is easy to search past opinions on the committee's Web site (www.dos.state.ny.us/coog/coogwww.html). New York courts ruled more than 20 years ago that government agencies could not restrict access to data simply because it was held in computers and not on paper. Therefore, NYPA staff agreed to release electronic records. Payroll records came in a simple Microsoft Excel spreadsheet with names, titles and salaries for about 1,600 people. It showed 12.5 percent of employees make more than $100,000 per year. Fifteen NYPA employees make more than the governor. Using the Find option in the Edit menu, easy word searches showed how many titles contained certain terms: there are 21 with \"vice president,\" 53 with \"director,\" 88 with \"manager\" and 15 with \"public\" or \"government affairs.\" We used this information in a graphic. A simple ranking of the list also teased out telling details: A janitor makes $85,488. That compares to less than $30,000 for janitors on the ordinary state payroll. I asked the state comptroller's office and the Assembly Energy Committee what kind of documents they had received or requested in their audits and investigations of the NYPA. The comptroller's office provided procurement reports, which detailed the authority's contracts and their bidding processes. The Assembly Energy Committee had asked for airplane records, which it said it had not received. NYPA filled my request for airplane records, which existed only on paper. I had received a tip that the governor had been overusing NYPA's airplanes, even though he had access to the state police fleet (the only other planes in state government). I split the records into piles to separate trips that included anyone who was not on my roster of NYPA employees. I made a spreadsheet of trips taken by Gov. George Pataki and his staff. In the end, this exercise proved only that the tip did not appear to be true. The records did show, however, that NYPA executives were using the planes to travel from Albany to New York City and White Plains – trips other state employees make by train. I wanted to know about the other ways the NYPA had strayed from its original mission. They provided an electronic copy of their vehicle report, which accounted for 798 electric-drive cars, vans and lawn mowers they had distributed in the name of green energy. The vehicles went to places such as Yankee Stadium. The authority's list of green energy projects showed new heating, air conditioning and lights at Rochester City Hall but not as many projects for Syracuse, a city that had not been represented on the NYPA's board of directors in a dozen years. I also asked for an accounting of what the NYPA called \"community support projects.\" I called it philanthropy. It turned out that the agency set aside $1 million per year for executives and staff to hand out to the charities of their choice. This included buying tables at fundraising galas and donating to construction-fund drives for groups in which NYPA executives are active. The agency's Excel spreadsheet showed the names of the recipients and the amount the NYPA donated. I looked up the grant recipients' IRS 990 forms at Guidestar (www.guidestar.org) and compared their board members to the names of the NYPA employees I had in my spreadsheet. There were a lot of hits. I also looked up the home addresses of NYPA board members and executives and looked for connections between their resumes and the groups on the philanthropy list. I typed these connections into a new spreadsheet and started making phone calls. Several grant recipients said they got their money because someone from the NYPA served on their board. Online county property records revealed that the NYPA's former president Eugene Zeltmann owned a home in Lake Placid in the Adirondack Mountains. The NYPA had contributed $55,000 to the Olympic Regional Development Authority, another state public authority. ORDA manages Whiteface Mountain in Lake Placid and the other venues used in the 1980 Winter Olympics. Like the NYPA, the authority is headquartered a few hours' drive from Albany and beyond the eyes of the legislature and the capitol press corps. ORDA's spokesman said they used some of the power authority's money for an annual event called the Congressional Winter Challenge. It is a fantasy camp intended for Rep. John Sweeney to bring other members of Congress and their staff members to ski, play hockey and socialize. The hope is that they will return to Washington and send federal money to the mountains. A records request at the development authority resulted in the kind of details that make a story about a winter fantasy camp come to life. They provided a copy of the invitation, the list of passengers who boarded a Washington airplane for Albany, an itinerary and team rosters. One participant was called simply \"Mary S's best friend.\" Mary S. is the congressman's daughter. The U.S. Olympic Committee, a nonprofit that rents space at ORDA's office, provided copies of opinions they had circulated from the House and Senate Ethics committees. Sweeney sought advice from the committees, which said that it is permissible for a member to take a privately sponsored trip in connection with his or her duties as an officeholder if the trip is for fact-finding and if the recreational activities were \"merely incidental to the trip.\" Only one other member of Congress attended the official fact-finding trip to Lake Placid. Other guests included at least a dozen Washington lobbyists, Sweeney family members, former Sweeney staffers and staff and relatives of NYPA executives. Sweeney lost re-election in November. It turns out no one was paying attention ORDA either. After that story, tips poured in from the North Country, located way outside our circulation area. One tipster said the agency had steered contracts for the winter weekend to its own board members who own posh resorts in Lake Placid. Another round of records requests proved that to be true. For the past two years, ORDA paid $96,378 to a resort owned by one board member and $112,040 to a resort owned by another board member.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2007-januaryfebruary-drill-for-oil-data-on-opec-site","slug":"2007-januaryfebruary-drill-for-oil-data-on-opec-site","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-drill-for-oil-data-on-opec-site.html","year":2007,"month":"January/February","monthNum":1,"headline":"DRILL FOR OIL DATA ON OPEC SITE","kicker":"ENERGY AND UTILITIES","author_name":"Jeff Porter","author_title":"Database Library Director","affiliation":"NICAR","summary":"Jeff Porter highlights the wealth of data available on the OPEC website, including downloadable tables on oil production, imports, and global pricing.","keywords":["OPEC","oil statistics","petroleum"],"topics":["Data & Databases","Investigative Methods","Business & Economy"],"technologies":["Excel","Internet","World Wide Web"],"full_text":"In \"The Wizard of Oz,\" Scarecrow asked the critical question just after the Tin Woodsman was discovered by the Yellow Brick Road: Dorothy: \"Did you say something?\" Tin Woodsman: \"Oil can.\" Dorothy: \"He said, 'Oil can.'\" Scarecrow: \"Oil can what?\" We know the answer to the Scarecrow's question: Oil can make or break politicians. Oil can drive the world's economy. Oil can make news time after time. Oil can play into data, too – data that can cover the entire world. The influential organization of countries that produce oil, the Organization of the Petroleum Exporting Countries, or OPEC, has its own Web site and numerous tables, which are easily downloadable and already in Microsoft Excel format, that can provide numbers and perspective to stories about oil imports and exports, the transportation of petroleum products and, of course, pricing. OPEC is made up of 11 oil producing and exporting countries: Algeria, Indonesia, Iran, Iraq, Kuwait, Libya, Nigeria, Qatar, Saudi Arabia, United Arab Emirates and Venezuela. For those countries, oil is the main commodity and key to their economies. The Web site, www.opec.org, offers charts, publications and background on OPEC itself and, under its Publications link, the Annual Statistical Bulletin. See www.opec.org/library/Annual%20Statistical%20Bulletin/ASB2005.htm. From there, you can check the table of contents and download tables on a variety of topics: basic indicators, oil and gas production, export and import, transportation by tanker or pipeline, oil prices, major oil companies and global flows of crude and refined oil. For example, click on the link for Oil and Gas Data, then choose Table 56: World Imports of Crude Oil by Country, 1980-2005: With a single click on the Excel icon, you can download the table and make additional comparisons or calculations. This table is just one example of the rich amount of year-by-year data available from the OPEC site. A quick comparison by region with a calculation of percentage change can show big increases in oil consumption from 1960 to 2005 in Asia and the Mideast, which helps drive the price of oil. See the category called Summary Tables and Basic Indicators. If you want to use this material in a serious way, you should read the General Notes section of the Annual Statistical Bulletin. There you can find definitions of terms such as \"Consumption of refined products,\" a list of countries included in each region and explanations of abbreviations (for example, \"b/d\" means barrels per day). Also, you can compare some of these statistics with other data sources: World Trade Organization – www.wto.int International Trade Administration – www.trade.gov Trade Stats Express – http://tse.export.gov","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2007-januaryfebruary-gas-pump-inspections-make-speedy-car-story","slug":"2007-januaryfebruary-gas-pump-inspections-make-speedy-car-story","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-gas-pump-inspections-make-speedy-car-story.html","year":2007,"month":"January/February","monthNum":1,"headline":"GAS PUMP INSPECTIONS MAKE SPEEDY CAR STORY","kicker":"ENERGY AND UTILITIES","author_name":"Lee Davidson","author_title":"special projects reporter","affiliation":"Deseret Morning News","summary":"Lee Davidson describes a quick investigation into gas pump accuracy in Utah using state inspection databases.","keywords":["gas pumps","Utah","consumer protection"],"topics":["Investigative Methods","Data & Databases","Business & Economy"],"technologies":["Access","CD-ROM","World Wide Web"],"full_text":"If you're looking for a quick, fairly easy investigation that will grab the interest of readers, then checking out the accuracy of gasoline pumps is the one for you. The project, which took me only about a week to do (and I spent most of that negotiating and waiting for a copy of a state database) came about after a string of stories about how gasoline prices had dropped significantly nearly everywhere except Utah. While the governor launched another investigation of the high prices, we decided to look at whether customers really receive what they pay for at the pump. We began our investigation with a quick search on the state's Web site, www.utah.gov, which showed us that the Division of Weights and Measures in the Department of Agriculture and Food had the data we needed. The Web site mentioned that the division tests \"all gasoline pumps for accurate measurement.\" (I also could have found that by simply looking more closely on the inspection stickers that the division places on each gasoline pump in the state.) The state provided me a full copy of its Microsoft Access database on a CD, and it already had some state-designed queries saved on it, such as \"total pumps failed with location and count,\" which made my life far too easy. A major drawback with the Utah database is that it did not include whether a \"volume failure\" occurred because a pump gave too much gasoline or too little. However, the inspection supervisor and other inspectors agreed that nearly half of the time the failed pumps gave too much, and half the time they gave too little. I had most of the numbers and information that I needed from the database in about an hour. Some simple division of the number of pumps that failed (1,898) by the total pumps inspected (44,796) gave the overall failure rate (4.2 percent). Division of the number of stations with at least one failed pump (92) by the number of total stations (1,140) gave the failure rate for stations (8 percent). I did some sorting by station and inspection date to discover that many stations had all their pumps fail repeatedly in up to five inspections over two years. Sorting by city, I also found that some of the state's biggest cities did not have any failing pumps, while some smaller cities had numerous failures. Finally, sorting and counting by date showed that virtually all known stations had been inspected in 2005, but inspections appeared to be way down in 2006. Some of our findings: Customers in Utah have a 1-in-25 chance of buying gasoline at a pump that fails to give amounts within legal ranges. About half of the pumps that fail give too little gasoline, and half give too much. So, motorists have about a 1-in-50 chance of getting more than they paid for, and 1-in-50 odds of getting less. About 8 percent of gas stations in Utah had at least one pump that failed from January 2005 through August 2006. They were scattered among 92 stations in 43 communities. Some stations had all of their pumps fail repeatedly in multiple inspections. However, the state issued no fines and closed no pumps or stations at that time. The state inspected all of the state's gasoline stations in 2005 but now has decided to inspect only about one-third of them each year. It has only eight inspectors, and they do a variety of other work, including checking scales and price scanners at stores statewide. The state said that other work had been suffering. Once I had my information from the database, I interviewed state officials and industry representatives. A photographer and I went with a state inspector to a local gas station to see and describe how inspections occur. We also found some interesting tidbits along the way. The inspector told us that pumps always give a little more gasoline when it is pumped quickly instead of slowly, so now I am sure that all of our readers pump gas as quickly as possible. State officials also say pumps are designed so that as they wear out they should err in favor of consumers. I also suspect that readers started looking for older pumps. We put a list online of all stations that had at least one pump fail. It showed the dates of inspections and how many pumps failed each time. We arranged that by city to make it easier for readers to identify nearby stations.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2007-januaryfebruary-resources-from-ire-and-nicar-for-reporting-on-energy-and-utilities","slug":"2007-januaryfebruary-resources-from-ire-and-nicar-for-reporting-on-energy-and-utilities","url":"https://dwillis.github.io/uplink/article/2007-januaryfebruary-resources-from-ire-and-nicar-for-reporting-on-energy-and-utilities.html","year":2007,"month":"January/February","monthNum":1,"headline":"RESOURCES FROM IRE AND NICAR FOR REPORTING ON ENERGY AND UTILITIES","kicker":"ENERGY AND UTILITIES","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE and NICAR offer comprehensive databases and resources for journalists covering energy, utilities, and power infrastructure, including the National Inventory of Dams, federal funding data, and federal contracts information. The article provides links to databases, tipsheets, and guidance materials for investigating energy-related issues and blackouts.","keywords":["energy","databases","utilities"],"topics":["Investigative Methods","Data & Databases","Technology"],"technologies":["Internet","World Wide Web"],"full_text":"You can find more tips for covering energy and utility in databases, stories, tips and publications available from IRE and NICAR.\n\nDon't forget that IRE maintains a Web page with resources for covering blackouts and updates the information when power outages make headlines. The listings include online resources from a variety of sources. See www.ire.org/inthenews_archive/blackout.html.\n\nDatabase Library\n\nThe National Inventory of Dams (www.ire.org/datalibrary/databases/viewdatabase.php?dbaseindex=44) includes dam maintenance records, structure, composition and owner, including power-generating dams. This update includes records for almost 80,000 dams. This update includes data through 2002. (In the past, the U.S. Army Corps of Engineers updated the database biannually, but the two-year reporting system is currently delayed.)\n\nFederal Assistance Award Data System\n\nFollow federal funding for state and local energy and water programs with FAADS (www.ire.org/datalibrary/databases/viewdatabase.php?dbaseindex=38), compiled by the General Service Administration. It shows major categories of federal spending, including grants, loans and direct payments. Entries include references to program codes used in the Catalog of Federal Domestic Assistance (http://12.46.245.173/cfda/cfda.html). The data is not a perfect picture of the government's payouts, but it does provide an overview of spending programs and money flowing to the state, county or city level.\n\nCheck the Database Library's federal contracts collection (http://www.ire.org/datalibrary/databases/viewdatabase.php?dbaseindex=17) to find information on companies involved in energy-related projects or provision of services to the government. IRE and NICAR's data covers 1979-2006, plus portions of the current fiscal year 2007. It includes the company, contact information, the agency, type of work performed and where the work was performed.\n\nResource Center\n\n• \"The Investigative Reporter's Handbook\" covers \"Investigating Energy and Communications Utilities\" in Chapter 17.\n\n• The IRE Journal (July/Aug. 2004) covered energy issues, including how to respond to power blackouts.\n\n• Nigel Jaquiss of Portland's Willamette Week provides a list of tips for journalists who are covering utility companies and explains how to find sources and appropriate documents. He also gives information on potential conflicts between consumers and shareholders. Tipsheet No. 2706\n\n• Ken Ward, The Charleston (W. Va.) Gazette, lists and discusses 10 sources of data for journalists investigating water and air quality, pollution, energy usage and other environmental issues. Tipsheet No. 1369\n\n• Ron Nixon, IRE and NICAR, and Tom McGinty, Newsday, compiled a list of Web sites helpful in covering the electric utility industry for the annual computer-assisted reporting conference in 2002. Tipsheet No. 1505\n\n• Arthur O'Donnell, California Energy Markets, provides a list of Web sites about energy information and electric and gas business news. This is from the 2002 annual IRE conference. Tipsheet No. 1620\n\n• Mike Taugher, Contra Costa Times, compiled a list of Web sites about energy, environment related issues and companies having energy for a presentation at the 2002 annual IRE conference. Tipsheet No. 1631\n\nTo order databases, contact the Database Library at 573-884-7332 or 573-884-7711. For a complete listing of the databases available from NICAR, visit www.ire.org/datalibrary/databases/databases.php.\n\nFor copies of stories or tipsheets, contact the Resource Center at 573-882-3364 or rescntr@ire.org. IRE members can download free PDF copies of most tipsheets at www.ire.org/resourcecenter.\n\nRead more about Energy and Utilities in The IRE Journal (Jan./Feb. 2006) IRE Members can view archived issues at www.ire.org/membersonly/resourcecenter/irejournal","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-marchapril-subsidies-spread-cash-far-from-struggling-farms","slug":"2007-marchapril-subsidies-spread-cash-far-from-struggling-farms","url":"https://dwillis.github.io/uplink/article/2007-marchapril-subsidies-spread-cash-far-from-struggling-farms.html","year":2007,"month":"March/April","monthNum":3,"headline":"SUBSIDIES SPREAD CASH FAR FROM STRUGGLING FARMS","kicker":"DOWN ON THE FARM","author_name":"Sarah Cohen","author_title":"Reporter","affiliation":"The Washington Post","summary":"A 2006 Washington Post investigation revealed that federal farm subsidies, often justified as supporting small family farms, were instead distributed to non-farmers, duplicate payments, and operations exceeding legal limits, with investigators analyzing USDA data to uncover billions in questionable payments.","keywords":["subsidies","investigation","farming"],"topics":["Business & Economy","Investigative Methods","Agriculture & Land"],"technologies":[],"full_text":"Skeptical of the claims that farm subsidies are needed to save the small family farm, Dan Morgan, Gilbert M. Gaul, and Sarah Cohen spent 2006 answering a simple question about federal subsidy programs: Who reaped the benefits from the taxpayer dollars, and did it help struggling small family farmers? The investigation revealed that subsidies included $1.3 billion meant for working farmers but paid to those who grow nothing; $9 billion in disaster aid that went often to farms already covered by a $20-billion crop insurance program; and payments to farms that exceeded legal payment limits. Unlocking USDA data was crucial, and The Washington Post published nine stories as part of the 'Harvesting Cash' series.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2007-marchapril-fluff-rules-in-puerto-rican-legislature","slug":"2007-marchapril-fluff-rules-in-puerto-rican-legislature","url":"https://dwillis.github.io/uplink/article/2007-marchapril-fluff-rules-in-puerto-rican-legislature.html","year":2007,"month":"March/April","monthNum":3,"headline":"FLUFF RULES IN PUERTO RICAN LEGISLATURE","kicker":"FIRST VENTURE","author_name":"Omaya Sosa Pascual","author_title":"","affiliation":"El Nuevo Día","summary":"A computer-assisted investigation revealed that the Puerto Rican legislature focuses more on trivial matters than addressing significant economic and social issues, highlighting the issue of quantity over quality in legislative measures.","keywords":["Puerto Rico","legislation","politics"],"topics":["Government & Politics","Investigative Methods"],"technologies":[],"full_text":"Politics in Puerto Rico are complicated and intense. The current legislature could also be described as frivolous and wasteful, according to a recent computer-assisted investigation by El Nuevo Dia. The San Juan newspaper used electronic legislative filings to show that Puerto Rican legislators choose quantity over quality when proposing new measures. They devote their energy to saluting special occasions or launching trivial investigations, while neglecting serious economic and social problems. In the middle of one of Puerto Rico's worst economic crises, more than half of the legislative efforts were dedicated to congratulating people, often for no extraordinary reason. Newcomer Nelson Del Valle Colón presented the most proposals overall, but almost 92 percent were congratulatory measures. Resolutions to investigate accounted for 16 percent of the total, yet most of the 690 investigations focused on inconsequential issues, and only 6 percent were completed. Joint resolutions to allocate funds accounted for 9 percent, with 340 resolutions to allocate a total of $120 million from the deficit-ridden general fund for all sorts of agencies, organizations, and individuals. This occurred within a government that had to shut down several agencies, including public schools, in June 2006 due to insufficient funds. Legislation on core issues like public health and education received little attention.","provenance":{"extracted_by":"pre-v2","extracted_at":"","verified":true}},{"id":"2007-marchapril-census-shows-blacks-driven-from-their-homes","slug":"2007-marchapril-census-shows-blacks-driven-from-their-homes","url":"https://dwillis.github.io/uplink/article/2007-marchapril-census-shows-blacks-driven-from-their-homes.html","year":2007,"month":"March/April","monthNum":3,"headline":"CENSUS SHOWS BLACKS DRIVEN FROM THEIR HOMES","kicker":"HIDDEN HISTORY","author_name":"Elliot Jaspin","author_title":"","affiliation":"Cox News Service","summary":"A journalist discovers evidence of mass racial cleansings in America through census records showing black populations mysteriously disappearing from counties between 1890-1930, uncovering a largely forgotten chapter of American history.","keywords":["cleansings","census","disappearance"],"topics":["Demographics & Census","Investigative Methods","Government & Politics"],"technologies":[],"full_text":"Old census records – I mean really old census records – are a pretty unlikely place to look for a news story. Even current census numbers aren't that exciting and, unlike a fine cognac, sitting around for 100 years doesn't help.\n\nBut as I began exploring a database of population figures that stretched from 1890 to 1930, I stumbled across something perplexing. In county after county throughout the United States, I found that the black population would suddenly and mysteriously disappear.\n\nIn places like Forsyth County in Georgia and Missouri's Lawrence County, one census would show blacks living in the county, and then a decade later, they were gone. Why had whole communities that had existed since slavery suddenly left? And, once gone, why had blacks never moved back?\n\nAt the time, I didn't know that I was looking at the unique signature left by one of the uglier events in American history: racial cleansings.\n\nLike most people, I had never heard of America's racial cleansings. In 1998, I found myself on assignment in Berryville, Ark. With a little time to kill, I wandered into a small history museum and marveled at a display of antique embalming equipment in one","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-marchapril-meyer-winners-examine-stocks-hospitals-schools","slug":"2007-marchapril-meyer-winners-examine-stocks-hospitals-schools","url":"https://dwillis.github.io/uplink/article/2007-marchapril-meyer-winners-examine-stocks-hospitals-schools.html","year":2007,"month":"March/April","monthNum":3,"headline":"MEYER WINNERS EXAMINE STOCKS, HOSPITALS, SCHOOLS","kicker":"INSIDE NICAR","author_name":"David Herzog","author_title":"","affiliation":"NICAR","summary":"Three newspapers won the 2006 Philip Meyer Award for using social science methods in investigative journalism, with the Wall Street Journal taking first place for uncovering stock option backdating, Gannett News Service second for analyzing hospital care disparities, and the Philadelphia Inquirer third for exposing irregularities in Camden school test scores.","keywords":["journalism","investigation","analysis"],"topics":["Investigative Methods","Journalism & Newsroom"],"technologies":[],"full_text":"Three newspapers recently won the 2006 Philip Meyer Award for their use of social science methods in reporting on stock options, hospital care and public schools.\n\nThe Wall Street Journal took first place for \"Perfect Payday,\" a series of stories that uncovered the practice of secretly backdating stock options to benefit corporate insiders. Reporters Charles Forelle and James Bandler used a statistical model to show how the backdating timing probably didn't happen by mere chance.\n\nGannett News Service won second place for \"Special Report: Rating Hospital Health Care.\" Database editor Robert Benincasa and reporter Jennifer Brooks analyzed national treatment data to show how patients in poor and rural areas weren't getting recommended care.\n\nThe Philadelphia Inquirer received third place for its investigation of Camden, N.J., schools. Melanie Burney, Frank Kummer and Dwight Ott found some test scores at some of the schools in the troubled district were higher than would be expected. The revelations led to the resignation of the superintendent and close oversight from the state.\n\nCongratulations to the winners. It's great to see journalists using the tools of social science to report on matters that are important to us all. More information about the winners and the Meyer Award is available online at www.ire.org/meyeraward/06winners.html. Contact David Herzog by e-mail at dherzog@nicar.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-marchapril-campus-crime-update","slug":"2007-marchapril-campus-crime-update","url":"https://dwillis.github.io/uplink/article/2007-marchapril-campus-crime-update.html","year":2007,"month":"March/April","monthNum":3,"headline":"Campus Crime Update","kicker":"Bits and Bytes","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"The IRE and NICAR Database Library has updated its Campus Crime database, allowing public access to crime statistics reported by colleges and universities from 2001-2005, including data on on-campus and off-campus crimes, arrests, discipline, and hate crimes.","keywords":["crime","campus","database"],"topics":["Education","Data & Databases","Investigative Methods"],"technologies":["World Wide Web"],"full_text":"How safe are the campuses of your local colleges or universities? The IRE and NICAR Database Library recently updated the Campus Crime database so you can find out.\n\nOn October 1st of each year, schools are required to report crime statistics for the three most recent calendar years. The data cover 2001 through 2005 and contain 8,771 records for the most recent year. On campus and off-campus crime, arrests, discipline and hate crimes are reported in separate tables that can be linked. More limited information is available for 2000. For more information and sample data, visit the Database Library's Web site www.ire.org/datalibrary/databases/viewdatabase.php?dbaseindex=54.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-marchapril-ire-spring-training","slug":"2007-marchapril-ire-spring-training","url":"https://dwillis.github.io/uplink/article/2007-marchapril-ire-spring-training.html","year":2007,"month":"March/April","monthNum":3,"headline":"IRE spring training","kicker":"","author_name":"","author_title":"","affiliation":"IRE and NICAR","summary":"IRE is offering multiple computer-assisted reporting (CAR) training events across the country, including Watchdog Workshops in Syracuse and Oklahoma City, a Money in Politics workshop in Los Angeles, and an intensive CAR Boot Camp in Columbia, Missouri.","keywords":["training","journalism","reporting"],"topics":["Journalism & Newsroom","Investigative Methods","Technology"],"technologies":["Excel"],"full_text":"A variety of IRE's upcoming on-the-road training events will feature CAR sessions.\n\nThe Watchdog Journalism series (www.ire.org/training/watchdogjournalism.html) designed for reporters and editors at small or mid-sized papers or bureaus of larger papers, will feature CAR training at several events. You must register for the Watchdog Workshop to attend the CAR sessions; space is limited.\n\nBetter Watchdog Workshop, April 14-15, in Syracuse, N.Y. (www.ire.org/training/betterwatchdog/Syracuse07.html) Beginning Excel.\n\nBetter Watchdog Workshop, April 28-29, in Oklahoma City. (www.ire.org/training/betterwatchdog/Oklahoma07.html) Beginning Excel.\n\nIRE will conduct a Money in Politics workshop (www.knightnewmediacenter.org/seminars/archives/election_08_covering_politics_in_cyberspace) April 20-21 in Los Angeles in conjunction with the Knight New Media Center seminar on politics in cyberspace.\n\nCAR Boot Camp, May 13-18, in Columbia, Mo. (www.ire.org/training/bootcamps/may2007.html) offers intensive training in spreadsheets, databases and how to use CAR in your news organization.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-marchapril-tracking-cotton-subsidies-and-learning-sql-server-on-the-fly","slug":"2007-marchapril-tracking-cotton-subsidies-and-learning-sql-server-on-the-fly","url":"https://dwillis.github.io/uplink/article/2007-marchapril-tracking-cotton-subsidies-and-learning-sql-server-on-the-fly.html","year":2007,"month":"March/April","monthNum":3,"headline":"TRACKING COTTON SUBSIDIES AND LEARNING SQL SERVER ON THE FLY","kicker":"DOWN ON THE FARM","author_name":"Megan Clarke","author_title":"Reporter","affiliation":"The Atlanta Journal-Constitution","summary":"An Atlanta Journal-Constitution reporter describes how she used SQL Server to analyze 180 million records of U.S. farm subsidy data spanning over a decade to investigate cotton subsidies in Georgia, which received $317 million in 2005.","keywords":["subsidies","database","agriculture"],"topics":["Business & Economy","Data & Databases","Investigative Methods"],"technologies":["Access","FoxPro","SQL Server","SQL"],"full_text":"Cotton is still king in Georgia. In 2005, the federal government pumped $317 million in subsidies into the state for cotton alone, making it Georgia's top subsidized crop.\n\nGeorgia's take was a piece of $23 billion in subsidies paid nationwide that year. American farm subsidies are criticized around the world for artificially depressing crop prices, and The Atlanta Journal-Constitution decided to see how those payments help or harm farmers in the United States.\n\nTo tell the story, we chose cotton subsidies because of the amount of money involved and the crop's historical importance in Georgia. My collaborators were reporters Ken Foskett and Dan Chapman, and projects editor Jim Walls.\n\nThe U.S. Department of Agriculture's Farm Service Agency is responsible for distributing subsidy money. I requested payment data for the entire country dating back to 1994. FSA initially assured me the data included all subsidy payments. But after some research and checking with a fellow CAR person, I discovered I needed databases for three more programs with nearly $4 billion a year in subsidies: tobacco transition payments, commodity certificate payments and cotton cooperative certificate payments.\n\nBy requesting a fee waiver because the information was of \"public interest,\" I avoided paying for any of the data. The payment database took about a month to receive, but the others, which are not requested as often, took a little longer. I would recommend requesting expedited action in your FOIA request. Most federal agencies do not recognize such a request, but FSA did.\n\nOnce I received 70 GB of data – 180 million records – it was clear that my usual software, Microsoft Access and FoxPro, were not going to cut it.\n\nWith trouble-shooting help from our then-CAR editor David Milliron and the book \"Sams Teach Yourself Microsoft SQL Server 2000 in 21 Days\" by Richard Waymire and Rick Sawtell, I took a crash course in SQL Server.\n\nThe capabilities of SQL Server were overwhelming at first. It can handle 1 million terabytes of data, versus 2 gigabytes in Access. (The drawback to SQL Server's power is that it comes with a hefty price tag.)\n\nI learned the organization and the differences in SQL statements on deadline. SQL Server syntax slightly differs from SQL commands in FoxPro (wildcards, for example and excluding the semi-colon after each line in a query). One handy feature with SQL Server is the color-coded command text. SELECT, DELETE, INSERT all turn blue when used properly and values turn red when your syntax is correct.\n\nIf you're transitioning from Access or FoxPro, stick with what you know (e.g. familiar queries) in the beginning. Once you get used to the software, it's easier to transition to advanced features. I continue to use Access and FoxPro for quick-hit analyses or daily CAR, but SQL Server has offered more advanced options for more organized, efficient analysis.\n\nSQL Server also provides more advanced security and a great feature called transaction logs, which documents almost every change made to the database. Like most database management software, SQL Server provides advanced processing tools.\n\nFor the subsidy story, I used summarizing queries to produce aggregates by several different categories. My biggest interest was finding oddities and outliers. (When you see a person with a Beverly Hills address collecting cotton subsidies in Georgia, it piques your interest.) Through geographic and time-ordered slices of the data, I was able to spot trends. Besides calculating the typical","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-marchapril-cotton","slug":"2007-marchapril-cotton","url":"https://dwillis.github.io/uplink/article/2007-marchapril-cotton.html","year":2007,"month":"March/April","monthNum":3,"headline":"COTTON","kicker":"DOWN ON THE FARM","author_name":"Megan Clarke","author_title":"Reporter","affiliation":"AJC","summary":"A journalist details their five-month investigation into U.S. farm subsidy data, revealing that wealthy individuals, corporations, and institutions far removed from farming—including celebrities and foreign entities—collect hundreds of millions in subsidies while just 5% of farmers receive half of all payments. The analysis uncovers how the subsidy system has been exploited by non-traditional recipients and shows how newly released FSA data can help news organizations better track subsidy recipients.","keywords":["subsidies","database","investigation"],"topics":["Agriculture & Land","Data & Databases","Investigative Methods"],"technologies":["SQL Server"],"full_text":"Combining the processing, mapping, analysis and memos, I worked on it for about five months.\n\nThe payment database includes a separate name table, which includes farm operator names and addresses. The tables are linked by a customer number, which is a surrogate for the tax ID that FSA normally uses. All of the different subsidy datasets can be linked using the customer number. The payment data also includes state codes, county codes, program names, types of crops, date and amount. It does not include the location of the farms, other than the county and state. I started cross-referencing the payment data with entity data, which identifies the members of each entity that collects subsidies. I found many recipients living quite a distance from the farms they are collecting on. People in Atlanta, New York City and even Saudi Arabia and Japan were collecting subsidies for U.S. farms.\n\nIt became apparent we needed to rework our definition of \"farmer.\" We discovered that college students, churches, prisons, universities, corporate rate pension plans and even a sports bra company collect farm subsidies. Familiar names also popped up in the data, too including Chevron Corp., CNN founder Ted Turner and Georgia Gov. Sonny Perdue.\n\nLate last year after we published our project, FSA released a new data set of subsidy recipients, called 1614, that news organizations have not had until now. The dataset, named for Section 1614 of the 2002 Farm Bill, tracks all subsidy payments to a human being when possible. The new data is only 6 GB and contains 66 million records for payments from Oct. 2002 to June 2006.\n\nPreviously, some payment records stopped at the corporate level and did not disclose the owners cashing the subsidy checks. The 1614 data connects the dots for you. With the old data, I was able to uncover most of the owners behind corporations with a few extra queries.\n\nFor those organizations that don't have access to database management software like SQL Server, The Environmental Working Group, an advocacy group, posts subsidy data online. On its Web site, www.ewg.org, EWG illustrates aggregates and provides a searchable database to look up specific recipients and their entities.\n\nAfter analyzing the payment records, we found:\n• Five percent of eligible farmers collect half of all U.S. farm subsidies.\n• Farmers can easily circumvent USDA limits on subsidy payments. Nearly 200 U.S. farm operations collected more than $1 million each in 2005. That's nearly a quarter of a billion dollars to a group that could fit inside a grade school cafeteria.\n• Subsidies drive up rural land prices and prevent small American growers from farming more land.\n• Wealthy Americans and institutions with no apparent need for government aid, including some who live far from their farmland, collect hundreds of millions of dollars in subsidies. Enforcement of income caps for subsidy recipients is spotty.\n\nThe data also turned up great leads on individual farmers. A top USDA official was the nation's single biggest beneficiary of a disaster relief program for orchard owners. Some farm operators accused of scheming to collect millions they were not entitled to later received millions more under new corporate names. Farmers who hit payment limits organized new partnerships to double or triple their subsidies.\n\nThe findings highlighted a fundamental flaw with current U.S. farm programs: They benefit landowners, sometimes to the detriment of farmers. That's because cropland owners can collect a subsidy even if they grow nothing, or they contract with a farmer to farm the land and split the subsidy.\n\nWe took it a step further and compared subsidy payments with those of another USDA program: food stamps. We found that a third of Georgia's counties collected more in farm subsidies than food stamps.\n\nAgricultural Census data provided an astonishing glimpse at the decline in farms and farmers in Georgia. Compiled by the National Agricultural Statistics Service, the ag census provides state and county-level numbers on everything regarding agriculture and is conducted every five years. NASS will survey farmers in December of this year; the data should be available in February 2009.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-marchapril-farmsubsidyorg-opens-access-to-eu-farm-payments","slug":"2007-marchapril-farmsubsidyorg-opens-access-to-eu-farm-payments","url":"https://dwillis.github.io/uplink/article/2007-marchapril-farmsubsidyorg-opens-access-to-eu-farm-payments.html","year":2007,"month":"March/April","monthNum":3,"headline":"FARMSUBSIDY.ORG OPENS ACCESS TO EU FARM PAYMENTS","kicker":"DOWN ON THE FARM","author_name":"Brigitte Alfter, Tommy Kaas and Nils Mulvad","author_title":"","affiliation":"farmsubsidy.org","summary":"European journalists are increasingly reporting on EU farm subsidies, largely through the work of Farmsubsidy.org, which uses freedom of information laws to expose previously secret subsidy data that comprises nearly half of the EU's annual budget.","keywords":["subsidies","journalism","transparency"],"topics":["Agriculture & Land","Investigative Methods","Business & Economy"],"technologies":[],"full_text":"Each year, the European Union pays more than $100 million to subsidize Arla, the seventh largest dairy corporation in the world. In the accounting year 2004, the sum of the subsidies was larger than the profit declared by the Danish-Swedish corporation.\n\nDuring World Trade Organization negotiations in December 2005, the EU promised to abolish the export subsidies that provide the largest part of Arla's bounty. The consequence, according to the company's deputy director when confronted with the subsidy figures, would be a restructuring, including moving jobs from Denmark and Sweden to a number of lower-cost countries.\n\nThis straightforward business story targeted a Danish audience, but more European journalists now are starting to report in their own language on EU subsidy programs after decades of silence. Almost half of the EU's annual budget is allocated to farm subsidies. In 2007, the amount is €54.9 billion, or $71.2 billion. Yet, until recently, who gets the money has been entirely secret.\n\nFarmsubsidy.org is the driving force in bringing Europe's farm subsidies into the light. This network organization uses Freedom of Information laws and computer-assisted reporting to gather and distribute subsidy data from EU nations. The work to get complete data on the subsidies is vast and far from done, but the potential in the data already gathered is huge. As a result of","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-marchapril-farmsubsidyorg","slug":"2007-marchapril-farmsubsidyorg","url":"https://dwillis.github.io/uplink/article/2007-marchapril-farmsubsidyorg.html","year":2007,"month":"March/April","monthNum":3,"headline":"FARMSUBSIDY.ORG","kicker":"DOWN ON THE FARM","author_name":"Brigitte Alfter, Tommy Kaas and Nils Mulvad","author_title":"Journalists","affiliation":"Farmsubsidy.org","summary":"Journalists pursued freedom of information requests across EU member states to expose farm subsidy recipients, ultimately leading to the creation of Farmsubsidy.org in 2005 and an EU decision to publish all subsidy data starting in 2009. The investigation faced repeated rejections based on privacy and business secrecy arguments but gained support from European Commission officials advocating for transparency.","keywords":["transparency","subsidies","databases"],"topics":["Investigative Methods","Data & Databases","Government & Politics"],"technologies":["Excel"],"full_text":"test case of the EU's young FOI act (http://eur-lex.europa.eu/LexUriServ/site/en/oj/2001/l_145/l_14520010531en00430048.pdf) that only had come into effect in December 2001.\n\nAlfter filed the application in June 2004, and the commission finally rejected it the following September. The core reason for the rejection was the definition of a database. According to the commission, only information retrieved through \"routine operations\" could be released. Another argument was a confidentiality clause for EU member states preventing the commission from revealing the data.\n\nThe case is still pending at the level of the European Ombudsman (www.euro-ombudsman.eu.int/home/en/default.htm). Another option for appeals –although at least as time-consuming as an Ombudsman complaint and far more costly – would have been to go to the European Court of Justice.\n\nThe next step obviously had to be addressing other member states. In spring 2005, Alfter filed applications in Germany, where two cases currently are pending in administrative courts. The cases challenge German legislation and legal practices containing some of Europe's strongest privacy and business secrecy protections.\n\nBritish and Dutch journalists and organisations caught interest, too. British media, including London newspapers The Guardian and the Telegraph, published stories exposing Queen Elizabeth II, her son Prince Charles and other nobility among the largest beneficiaries of what is politically presented as support for the struggling farmers.\n\nIn the Netherlands, the Evert Vermeer Foundation (www.eucoherence.org/renderer.do/clearState/true/menuId/227658/returnPage/227304), which opposes EU policies that harm developing countries, achieved access to national data in late 2005 after a number of court cases.\n\nCreating Farmsubsidy.org\n\nBy May 2005, it was time for a closer network among those interested in transparency, and a handful of people met in Brussels and drafted the idea for the Farmsubsidy.org network, established by Mulvad and British researcher Jack Thurston. It was time to actively include colleagues in other countries, too.\n\nSoon data were released or requested in Sweden, Finland, Poland, Slovenia and a number of other EU member states. When www.farmsubsidy.org went online in late 2005, data of varying quality from a number of EU member states were available, and it became possible to trace subsidies for corporations with subsidiaries in multiple member states.\n\nThe heaviest arguments used for rejections all over the EU were privacy, protection of business secrets and the fundamental question of access to information held in databases. However, parts of the European Commission started their own struggle for transparency of the data with the commissioner for public administration, Estonian Siim Kallas, taking the lead. Late last year, after a long struggle with some opposition within the EU, the EU passed a decision declaring that all subsidy data are to be published, beginning in 2009.\n\nTime to lean back and relax at the Farmsubsidy.org network? Not exactly. Particularly in the countries that have resisted transparency most strongly, Network members already can trace movements warning subsidy recipients of the transparency. This easily could lead to a disguise of the true beneficiaries before 2009. Thus, the work continues.\n\nCAR challenges\n\nWork continues on additional information and additional possibilities for the Web site.\n\nGovernments provide data in many different formats. Some are uploaded to the Web as PDF files – some, such as Spain's, cover more than 5,000 pages, which are filled with spelling errors. Some countries, such as the Netherlands, only give access to a limited search form, and they try to hinder Web scraping, which makes it very difficult to get a copy of all data for analyzing.\n\nOn the other hand, some of the countries are very cooperative and give out data in a standard electronic format (Microsoft Excel or delimited text), which makes it very easy to work with.\n\nNot only the format but also the content varies from country to country. Some countries use the calendar year; others count from the day they open applications for a given program, or they follow the so-called FEOGA year used in the EU budget (an administrative concept similar to a fiscal year in the U.S.).\n\nUnfortunately, the subsidy data is really dirty and changes year after year. For Farmsubsidy.org members, it means trying to create standards and get an overview of the data to understand what to request and what pieces of data are necessary. Mulvad uses his journalism training and CAR experience for scraping, importing and cleaning data. He collaborates with Bob Larkin, a data programmer and project leader who works with MySQL, Drupal (http://drupal.org/about) and PHP.\n\nPreparing for mapping is another major project. The group has geocoded the addresses of subsidy recipients, a process that allows them to be plotted with geographic information system software, and has requested shapefiles of agricultural land that are kept by authorities in each country. This will allow Farmsubsidy.org to geographically display and analyze data about subsidies and other EU statistics.\n\nWe also have rated the score of transparency of government authorities according to how much data they give out and the quality of the data. Now we're close to finishing a process on integrating further indices to prepare it for demands for more data. That will facilitate the process of following the money.\n\nThe main new thing on the Web site for the coming period will be combining documentation with the transparency index. It will involve uploading new data, including subsidy programs and possibly also the main layout in a multilingual system, making it easier for people who speak languages other than English.\n\nThe whole idea is to open opportunities for new types of stories, call it the next wave, in which we look at the aim of the subsidy program and investigate how it actually works in practice. We look into the effects of the schemes that were not taken into consideration, when they were introduced and whether they now might be outdated.\n\nThe entire EU farms subsidy system is going to be submitted to a so-called health check this year and next – even more reason to request the data now – instead of waiting until 2009.\n\nSee data at www.farmsubsidy.org, for the U.S. data, visit the Environmental Working Group's Web site, www.ewg.org.","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}},{"id":"2007-marchapril-reporting-on-migrant-farm-labor","slug":"2007-marchapril-reporting-on-migrant-farm-labor","url":"https://dwillis.github.io/uplink/article/2007-marchapril-reporting-on-migrant-farm-labor.html","year":2007,"month":"March/April","monthNum":3,"headline":"REPORTING ON MIGRANT FARM LABOR","kicker":"DOWN ON THE FARM","author_name":"David Walle","author_title":"","affiliation":"Palm Beach Post","summary":"The Palm Beach Post's investigation into migrant farm labor abuses demonstrates how reporters can access federal and state agency data, records, and reports to uncover stories about modern-day slavery, pesticide dangers, and labor exploitation. The article provides practical guidance and resources for journalists investigating agricultural labor conditions.","keywords":["migrants","investigation","labor"],"topics":["Agriculture & Land","Investigative Methods","Health & Environment"],"technologies":["Internet","World Wide Web"],"full_text":"When a team of reporters for the Palm Beach Post began preparing packages on migrant farm laborers (www.palmbeachpost.com/moderndayslavery/content/moderndayslavery/) and the dangers of pesticide (www.palmbeachpost.com/localnews/content/news/special_reports/carlitos/index.html) in 2003 and 2005, they pursued multiple angles.\n\nThey found instances of modern-day slavery, explored the politics that touch migrant lives and calculated the unseen costs of migrant labor. Covering the story required traveling to its sources: in the crop fields of Florida, to the streets of Guadalupe, Mexico, and the trails taken by smugglers to bring migrants into America. Inquiry into the abuses also meant digging into reports provided by state and federal agencies.\n\nWhenever the federal government contracts a state agency to enforce its laws, the agency must collect data and file reports to the federal government, says Christine Stapleton, a Palm Beach Post reporter who worked on both migrant packages. Stapleton suggests that reporters first read the contract, which explains the type of data and the format in which it's kept. Reporters can then request specific databases or record fields from the state.\n\nFor example, the Palm Beach Post tapped the state health department to check for housing violations caused by migrants living in overcrowded homes. Reporters compiled demographic profiles of labor contractors by getting their phone numbers and birthplaces from the state department of professional registration. While working on the pesticides package, they searched worker compensation data for chemical injury claims filed against agricultural employers. Stapleton shares more advice on investigating agriculture in Tipsheet 2086 (www.ire.org/resourcecenter/viewtipsheets.php?number=2086).\n\nThe following resources also can aid investigations into farm labor conditions:\n\nNational Agricultural Workers Survey\n(www.dol.gov/asp/programs/agworker/naws.htm) – A report of demographic and employment characteristics of the U.S. crop labor force. Released in 2005, the data is taken from face-to-face interviews of farm workers.\n\nMigrant and Seasonal Farmworker Enumeration Profiles Study\n(http://bphc.hrsa.gov/migrant/EnumerationEnumerationStudy.htm) – Profiles of workers in ten states (Alaska, California, Florida, Louisiana, Maryland, North Carolina, Mississippi, Oklahoma, Texas and Washington). It includes county estimates of migrant and seasonal workers and each state's labor demand for crops.\n\nUrban Institute\n(www.urban.org) – Includes various academic articles about immigrants and plenty of statistics.\n\nThe Inter-American Development Bank\n(www.iadb.org) – Has posted two useful papers, \"Data on Immigrant Families\" and \"Remittances 2005,\" which discusses how migrants send money back to families in their native countries.\n\nSarasota/Manatee Farmworker Supporters\n(www.smfws.com) – A collection of national articles about immigration that are updated daily and archived through 2003.\n\nMigrant Farmworker Justice Project\n(www.floridalegal.org/migrant.htm) – Includes a facts section on farm workers in Florida.\n\nIRE Data, stories and tipsheets\nThe Wage and Hour Enforcement Database has information on violations, penalties and employers for more than 700,000 closed cases between the 1980s and 2004 (www.ire.org/datalibrary/databases/viewdatabase.php?dbaseindex=46). The database covers areas including child labor, overtime, minimum wage and medical or family leave.\n\nWINK-Fort Myers, Fla., reported that between 20,000 to 50,000 people are smuggled into the U.S. each year as indentured servants. Some","provenance":{"extracted_by":"anthropic/claude-sonnet-4-6","extracted_at":"2026-07-05","verified":true}}]}